Re: Dybvig's module system and interactivity

[Mark Seaborn]

Here are some of my thoughts on the issue of module systems.

I see the evolution of module systems as something like this:

 * We start off with modules as they are in, say, Java or Perl.  They
   are just packages that contain functions or classes.

 * Add parametric modules.  This requires separating the specification 
   of a module (its signature) from its implementation.  This leaves
   you with something like ML's functors.  Unfortunately, this has the 
   disadvantage of disallowing mutual recursion between modules in
   some cases, such as between two modules that take each other as
   parameters.

 * PLT Scheme's module system makes two further changes:

    * Firstly, mutual recursion between modules is allowed by creating 
      a new linking construct which allows circular references, rather 
      than just having functor application as the linking construct.

    * Modules are made first class objects.  This enables dynamic
      linking (and is not, incidentally, a barrier to efficient
      compilation).  More subtly, it removes the universal namespace
      for modules, meaning that modules become more like capabilities
      and can be used to enforce security boundaries (this is
      something Java got wrong -- its universal namespace for packages 
      means it needs to use an ACL scheme for security).

 * The next steps that need to be made are:

    * Export of syntax.

    * A programmatic interface to module linking.  The problem with
      PLT Units is that module linking clauses can get quite big, and
      it becomes a pain maintaining them by hand.  Since the clauses
      are syntax, it is hard to make parts of them conditional.  What
      is needed is an interface via functions, rather than via syntax.
      This will make it easier to link programs automatically, using
      whatever implementations are available (recovering one of the
      advantages of Java/Perl-style package systems -- brevity).


Dybvig's system can handle mutual recursion between modules.  It can
probably handle parametric modules just about.  I am not sure if it
can handle mutual recursion and parametric modules at the same time
(and it can, whether it is efficient, ie. doesn't instantiate
functions more than necessary).

Even if can handle those things, it can't handle modules being first
class objects, dynamic linking, or a programmatic interface to
linking, because Dybvig's system is just a static transformation.

Of course, you could then voice the objection that Dybvig's system
*can* handle these, because you can implement PLT units in it...


Mikael Djurfeldt <address@hidden> writes:

> Dybvig's module system is actually not the usual thing we mean by a
> module system, but rather a tool to control and abstract over scope.

This is very true.  Dybvig's system is basically a sophisticated
library to help in writing macros.

The implementation of PLT core units that Dybvig gives is amazingly
concise, but if you look it at, it's just passing boxes around to
perform the linking.  This is how you can implement it anyway -- it's
not actually using the extra expressive power that Dybvig's system has
over normal macros.

The way Dybig's units implementation links units is analagous to
linking C programs by passing around structs of function pointers --
there are more efficient ways of doing it (such as the way C linkers
link), which could be exploited by the Scheme implementation.

Speaking of C, one thing in favour of units is that the concept
generalises well to other languages.  The PLT team are working on
`Knit', which is basically units for C; there's a paper on it
somewhere.  However, I think it is untyped, doesn't allow units to
export macros, and hasn't been released yet.  Being untyped is a shame
because otherwise it would make it easy to generate bindings for other
languages; and the lack of macros means the same thing, as well as
making it harder to wrap existing programs.  SWIG is too ad-hoc --
units for C should be a cleaner way of doing things.  (I plan to work
on it some time.)  This is quite important since Guile is supposed to
be an extension language.

Lastly, I'll point to my own portable implementation of units: I'll be
uploading it very soon (should be there by the time you read this) in
the file primrose-0.0.3.tar.gz to <http://primrose.sourceforge.net/>
and <http://members.xoom.com/mseaborn/omr/>.  It's part of (or a
diversion from :-) ) a music recognition program I'm writing in
Scheme.  The files of interest are:

 * unit-core.scm:  Core units.
 * unit-sig.scm:  Signed units.
 * unit-dlink.scm:  This is an extension to the interface provided in
   PLT Scheme.  It provides a programmatic interface to module
   linking, which you can see in use in `link.scm'.
 * (Also there's `unit.scm', which is an old and slow implementation
   of core units.)

I'd quite like to see this included in Guile.  Shall I wrap it up in a 
Guile module and submit it?

The main omission is that it doesn't allow modules to export macros
yet.  I hope to work on this soon, but it should just be a case of a
macro-exporting unit providing a function to perform the macro's
transformation along with another unit to link against the free
variables the macro introduced.

So basically I think units are quite a good module system, once some
extras are added. :-)  Although Dybvig's system is quite neat, and it
would be worthwhile implementing it anyway (could his sample
implementation be used?).  In particular, identifier macros are very
useful.

-- 
         Mark Seaborn
   - address@hidden - http://members.xoom.com/mseaborn/ -

  ``Water boils at a lower temperature at high altitude, which partly
     accounts for the nasty taste of coffee on the summit of Mauna Kea''