Re: GNU Guile branch, master, updated. v2.1.0-102-g0f9f51a

[Mark H Weaver]

Andy Wingo <address@hidden> writes:

> On Wed 16 Nov 2011 04:58, Mark H Weaver <address@hidden> writes:
>
>> I think there are better ways to address this problem.  I will explore
>> these in another email.
>
> I look forward to this.  Please be sure to address the following issues:
>
>   * Debian upgrading guile to a newer version, without recompiling
>     guile-foo which depends on a hygienically introduced identifier.
>
>   * A user modifying a Scheme file from Guile, in the spirit of the
>     LGPL, and expecting it to work with program Foo, without recompiling
>     Foo (again, in the spirit of the LGPL).

If we must avoid the recompilations, then I see only one solution:
simply refrain from introducing hygienic top-level identifiers in the
expansions of public interfaces.  IMHO, this isn't so bad.  If a
top-level variable needs to be expanded in user code, then you'd better
explicitly choose a stable name for it.  If you want the name to be
programmatically based on some or all of the macro arguments, this can
already be done using `symbol-append', `datum->syntax' et al.  The point
is, it's better for the user to do this explicitly.  In the general
case, where a macro may have been considerably reworked from one version
to the next, it's _impossible_, even in principle, for the system to
reliably decide the correspondence between top-level gensyms in the new
code vs the old code.  Even your method is only a heuristic that will
often do the wrong thing, in both directions: it will cause unintended
name collisions in R5RS standard code, and it will also sometimes change
the name of a top-level when you didn't want it to.

* * * * *

There's another option: we could properly track the compile-time
dependencies of each module, and automatically consider a .go file stale
if _any_ of its compile-time dependencies are newer than it.

To be more specific: I think we need to record, in every syntax
transformer bound at top-level, the name of the module where it's bound.
Then, within the dynamic extent of `compile-file', (probably using a
fluid) we'd need to accumulate the set of modules whose macro
transformers are used during compilation of the file.  This set of
modules (the compile-time dependencies) would be included in the .go
file.

I think we need this _anyway_.  Right now, if you change the way an
exported macro works (for example a macro that defines a record type),
you must _manually_ force recompilation of other modules that expand
that macro.  I actually ran into this problem while working on adding
new compiler environment types.

* * *

Furthermore, we could provide distros with the necessary infrastructure
to automatically recompile guile modules as needed after package
upgrades.

I know of at least one precedent for this behavior: the emacs packages
in Debian.  Last I checked, Debian had an elaborate system for
automatically recompiling all third-party emacs packages after a new
version/fork of emacs is installed.  Furthermore, when you install a
third-party emacs package, it is compiled separately for each
version/fork of emacs that is currently installed.

The idea is that .elc files are needed for every ordered pair (e,p)
where `e' is a version/fork of emacs, and where `p' is an .el source
file.  Therefore, neither the emacs packages nor the third-party
packages are able to do the right thing on their own.  The emacs-common
handled all of this magic.

Something similar should be done for Guile, and if we provide the right
tools, we could make it relatively easy for distros to do this.

What do you think?

      Mark