Re: New module system option :duplicates

[Marius Vollmer]

Mikael Djurfeldt <address@hidden> writes:

> I've just committed the following experimental features.  Please try
> them out and comment on if you they are a good idea.

I can understand the motivation for merging generics very, very well,
but I still am unhappy about its conflation with the module system.  I
think the merging and in general the resolution of duplicates should
be controllable more precisly.  (See below for what I mean by this.)


It is unfortunate that our current module system silently ignores
duplicates, or "collisions".  Sooner or later, we should move to a
more 'disciplined' system.

The core model of such a system (but not its UI) is in the text
workbook/compilation/new-model.text.  Mikael, could you try to
consolidate your duplicate checking with the model laid out in that
text?  I.e., can your :duplicate option be implemented on top of that
model?

I think we should not add features to the module system that can not
be implemented 'naturally' in the idealized compilation framework in
new-model.text.  Maybe the model needs to be changed, or maybe the
:duplicate option needs to be done differently, ut they both should
fit together conceptually.

Thinking superficially about it, I would like the following
alternatives better, since they are more precise.  They are also more
cumbersome, but I do think that it pays to be more long-winded at
times in order to be more robust with regard to a changing external
environment.

For real name collisions that should be resolved, I would prefer
renaming or selective imports over just specifying to take the first.
I don't think we have an elegant way to import all-but-a-selected-few
bindings from a module, but adding it should not be a problem.

So instead of

   (define-module (foo)
     :use-module (blarg)
     :use-module (bar)     ; exports mumble
     :use-module (baz)     ; also exports mumble
     :duplicates first)    ; but we choose the first one

I would prefer

   (define-module (foo)
     :use-module (blarg)
     :use-module (bar)                          ; exports mumble
     :use-module ((baz) :dont-select (mumble))  ; also exports mumble
                                                ; but we don't want it
     :duplicates check)                         ; and we expect no duplicates

where the ":duplicate check" option is ideally the default.

The former variant with ":duplicate first" has the 'race condition'
that (blarg) might at some time start to export mumble as well and we
will silently pick that instead of the one from (bar).  The second
variant will also fail since there are now duplicates, but it will
give an error.

Also (and I think new-model.text will have to change for this), we
might want to have the concept of "potentially imported" bindings:
when importing a set of bindings with unspecified names (such as "all
of (bar)" as opposed to "the binding with name mumble of (bar)"),
these bindings are only potentially visible.  Only when a binding is
looked up in the current module will these bindings be considered and
only then wehn none of the "actually imported" bindings matches.  When
a potentially imported binding is found in this way, it becomes
actually imported.  When more than one potentially imported binding
matches, an error is reported (or the first one is selected in 'legacy
mode').

Such a model can easily support an interface like

   (define-module (foo)
     :use-module (blarg)
     :use-module ((bar) :select (mumble))  ; explicitely import mumble
     :use-module (baz)                     ; also imports mumble
                                           ; but only implicitely and
                                           ; since we have an explicit
                                           ; mumble, we won't see it
     :duplicates check)                    ; we expect no explicit duplicates


So in my view only ":duplicates check" should be added, and ideally be
made the default.  How to orchestrate the transition is another
issue...


For merging generics, I would like to split the task in two: first,
getting the generics into the current module without conflicts; and
second, merging multiple generics into one.

The first part is indepenent from the fact that generics are involved
and deals only with the module system, while the second part has
nothing to do with colliding names, and deals only with generics.

I think this distinction is important, at least conceptually.  Your
:duplicates option respects this distinction to a largish degree, but
it still hooks into the module system at quite a deep level where the
separation of the two issues is not easy to see.  Also, the user has
the same low amount of precision when dealing with it as with the
general collision avoidance.  Just giving the license to merge any and
all generics that have colliding names is probably not what one wants.
One usualy has a specific few in mind that need to be merged, I think.

Given that we have a way to get multiple conflicting symbols into the
current module (via renaming), what about leaving generic merging to a
separate facility like 'merge-prefix-generics'.  Actual code might
look like:

  (define-module (my-module)
    :use-module ((math 2D-vectors) :prefixed 2d)
    :use-module ((math 3D-vectors) :prefixed 3d)
    :duplicates check)

  ;; merge 2d-x and 3d-x into x.  Likewise for y and z.
  (merge-prefix-generics (2d 3d) x y z)

This is not much more tedious than ":duplicates merge-generics" but
much more precise.  No unexpected mergings are done and the remaining
collisions of non-generics are also dealt with in a useful way.

Also, merging of generic functions might be a useful thing to do, even
without using the module system explicitely.

> The major weakness is of course the dynamicity of the merge-generics
> handler, but note that such a handler could very well operate during
> compile-time if module interfaces have typed signatures.

The compiler should likewise be able to handle the
merge-prefix-generics macro.

> * Changes to Scheme functions and syntax
> 
> ** Checking for duplicate bindings in module system
> The module system now can check for duplicate imported bindings.
> The syntax to enable this feature is:
> 
> (define-module (foo)
>   :use-module (bar)
>   :use-module (baz)
>   :duplicates check)
> 
> This will report an error if both (bar) and (baz) exports a binding
> with the same name.

What about adding:

  Eventually, we want to make ":duplicates check" the default.


In summary, I think that ":duplicates check" is a very good thing, but
I'm unconvinced about the rest of the handlers.  I'd rather see
improvements to the selection/renaming mechanism.

Does this make sense, Mikael?



(one more minor comment)

> The detailed rule for method visibility is this:

I think it is wrong to talk about "method visibility".  The visibility
of methods is not an important feature, one might even argue that
methods don't have names at all.

The thing that matters is the set of methods belonging to a certain
generic function.  This set is unaffected by visibility issues.

(I'm not saying that you don't understand methods and generic
functions, Mikael, of course you do, more than anyone else here.  But
still the use of terminology doesn't fit my mental model of generic
functions and their methods.)

-- 
GPG: D5D4E405 - 2F9B BCCC 8527 692A 04E3  331E FAF8 226A D5D4 E405