Re: package and testing rant (was Re: package.el, auto-installation, and auto-removal)

[Stefan Monnier]

> I am cross about the way we are building this system, ELPA.

Just to avoid confusion in the discussion: ELPA is the name of the
general package distribution system, IOW the protocol between package.el
and the web servers.

There are several servers which follow this protocol, such as Marmalade,
MELPA, and GNU ELPA.

> Not about package.el, or about what ELPA is.  But about how ELPA (and
> MELPA) work, and to some extent, the provisions we have made for package
> creation.

IIUC by "ELPA" above I guess you mean GNU ELPA (since "ELPA and MELPA"
would otherwise combine with "and" two things of different nature).

> The multi-packages users load in their emacs are tars.  But the packages
> that are checked in to ELPA are directories of files.
           ^^^^^^^^^^^^^^^^^^
So I think here you mean elpa.git (part of the greater GNU ELPA).

> So package authors are not checking in what gets delivered to the
> user.  So there is a magic build step somewhere.

Indeed.  Note that for all packages (even single-file packages) there
are yet more build steps elsewhere:
- the creation of the <foo>-autoloads.el.
- the creation of the *.elc files.

> This discourages testing of packages before they are distributed.

The "magic step" that turns a directory in elpa.git into a tarball is
fairly limited currently.  It is basically limited to "create
<foo>-pkg.el then tar".  Given the desire to limit the number of external
tools required (which is why we distribute *.tar and not *.tar.gz, for
example), I think that "tar+untar" is pretty close to the minimum way to
transfer a directory of files from the developer's machine to the user
who installs the package.

Note that there is a fair bit a pressure to *add* rather than remove
magic steps (the first candidate in the list is to build the *.info
files from the *.texi files).

The way to "test" for elpa.git is "cd .../elpa; make" which should
create the *-autoloads.el and *-pkg.el and *.elc files in the "same" way
as what will happen on the user's machine.

> And I am really starting to think we need better testing.  24.4 looked
> like a slog to release and it still has many bugs.

FWIW, I haven't heard of any case where the issues you're pointing out
have resulted in actual problems.  The only real reliability problem
I've heard about linked to ELPA packages is the issue of
compiling/installing a package when an older version is
already installed.


        Stefan