Re: Trouble with `export'.

[Marius Vollmer]

Dirk Herrmann <address@hidden> writes:

> The possibility to import bindings from several modules in addition to
> the bindings introduced by local definitions leads to the question, which
> binding should take precedence if there are several possible bindings that
> could be taken.  Further, the possibility to add bindings later on in
> arbitrary modules plus the possibility to remove bindings using 'undefine'
> makes it necessary to define these precedence rules in a dynamic
> environment.

Yes.  I would like to define these rules in these fashion: we define a
run-time data structure that represents the state of the module system
from the view of the entity that executes the code (the `code').  The
code will query the data structure to retrieve the thing that belongs
to a symbol, given a specific module (the `current module').
(Usually, the thing will be a variable, but it might also be a syntax
expander, or something else.  Let's concentrate on how to find the
thing, not what it means.)  Thus, we have to specify how this function

    (lookup MODULE SYMBOL)

behaves.  In the real implementation, there will of course be a
caching mechanism, but I think we can ignore it for this discussion.
Also we can ignore how to virtualize this to special purpose modules.

The behavior of `lookup' could then be described by treating
MODULE as a data structure, and our usual module system operations
like `use-modules' and top-level `define' can be described by their
effects they have on this data structure.  (The `code' will then pick
up these changes immediately since conceptually, every time an
identifier is accessed, it is looked up anew.)

Given this framework, what I would specify right now regarding the
relevant data structure of a module is this:

A module consists of

    LOCAL: a set of `bindings' of the form (SYMBOL . THING).

    EXPORTED: a set of SYMBOLs.

    RE-EXPORTED: a set of SYMBOLs.

    IMPORTED: a set of `import-specs' of the form
              (USED-MODULE (OUR-SYMBOL THEIR-SYMBOL FORCE?) ...).

    USED: a set of USED-MODULEs.

The behavior of `(lookup MODULE SYMBOL)' is then defined as

    - find the set of `potential bindings'.

    - if the set is empty
    
      - return #f.

    - if there is only one binding in that set:

      - return the THING of the binding.

    - else 

      - signal a "conflicting definitions" error.

To find the set of potential bindings:

    - start out with the empty set.

    - if LOCAL contains a binding for SYMBOL, include it in the set.

    - for each import-spec of IMPORTED:

      - if there is a entry with OUR-SYMBOL equal to SYMBOL:

        - call (lookup-for-import USED-MODULE THEIR-SYMBOL FORCE?)

        - if that returned a binding:

          - include it in the set.
 
        - else

          - signal a "importing undefined identifier" error.

    - if the set is empty

      - for each USED-MODULE of USED:

        - call (lookup-for-import USED-MODULE SYMBOL #f)

        - if that returned a binding:

          - include it in the set.

The definition of (lookup-for-import MODULE SYMBOL WRITABLE? FORCE?) is

    - if FORCE? is true

      - return (lookup MODULE SYMBOL)
        (but watching out for endless recursion and signalling
         an error in that case.)

    - else

      - if SYMBOL is in both EXPORTED and RE-EXPORTED

        - signal a "identifier both exported and re-exported" error.

      - else if SYMBOL is in EXPORTED

        - if LOCAL contains a binding for SYMBOL:

          - return it

        - else

          - signal a "exporting undefined identifier" error.

      - else

        - return what (lookup MODULE SYMBOL) returns but
          only if it doesn't correspond to a binding from LOCAL.
          Return #f in that case.  Also watch out for endless
          recursion, possibly signalling an error.

[There are certainly some fine points to nail down about this.]

That is, I would make a distinction between implicitly imported
identifiers and explicitly imported ones.  Explicitly imported
identifiers are in the same class as the local ones, causing
conflicts, but implicitly imported identifiers are silently shadowed
by explicitly imported or local ones.

All conflicts would be reported at lookup time, not when they are
effectively introduced.

The behavior of `export' and `re-export' would be to add the symbols
to the respective sets of the current module.

`use-modules' would add to IMPORTED or USED, as appropriate.  When
`:select' is given, it adds to IMPORTED, else it adds to USED.  (Hmm,
what about renaming for USED?)

`define' would create a new LOCAL binding.

`undefine' would remove a LOCAL binding.

We might also want to have `un-use-modules', `unexport' and
`un-re-export'.


How to efficiently implement the specification from above is another
issue...

> Examples for questions that have to be answered:
> 
> Assume that foo and bar are modules which both export a binding for frob:
> 1 When importing foo and bar, which binding for frob takes precedence?

With the specification above, it would depend on whether they are
explicitly or implicitly imported.  If both are imported explicitly or
both implicitly, there would be a conflict.  Else the explicit one
wins.

> 2 When importing foo and bar, and the binding from bar takes precedence,
>   what happens after (undefine frob) in bar?

Since frob must have been imported explicitly, you would get a
"importing undefined identifier" error.

> 3 Assume that there is a rule stating that the definitions of a later
>   imported module take precedence.

There should be no such rule, for the reasons you state.

> 4 When frob is imported from foo, but in the local module frob is defined
>   later on, should the local bindings take precedence?

When frob is imported implicitly, then yes, else there is a conflict.

> 5 Assuming the local definition of frob takes precedence over any
>   imported one, what happens if you do (undefine frob) in the local module
>   again?  Should frob become undefined, or do you want the imported
>   definition to appear again?

If there is no conflict among the imported definitions, the winning
imported definition should appear.

> My position is, that any conflict should be considered an error.  You may
> disagree,

(I would say that certain situations are not a conflict... :-)

> 1 When you import foo and bar with their full set of exported bindings, a
>   conflict is reported:  "frob is imported from several sources".  The
>   user would have to state which frob should be taken.

Yes.

> 2 When frob is imported from bar, and in module bar the definition for
>   frob becomes undefined, it also becomes undefined in the local
>   module.  Still, the information that frob should be taken from bar is
>   still available.  An attempt to define frob locally would lead to an
>   error ("attempt to override an imported binding").

Yes.  With the spec above, you would not get an error at definition
time, but the next time frob is referenced.  At definition time, a
warning might be appropriate.  (You would get the misleading error
message "importing undefined identifier" with my algorithm above,
since that is checked for first...)

> 3 Re-importing a module should not change the origin of bindings.  I. e. 
>   bindings should still be imported from where they were imported
>   before.  If, however, the set of exported bindings of an imported
>   modules has changed in between, it has to be checked that no conflicts
>   occur.

Yes.

> 4 When frob is imported from foo, but in the local module an attempt is
>   made to re-define frob, this should lead to an error (see 2).

Depending on whether it was imported explicitly or implicitly.

> 5 If frob is defined locally, it is not imported (otherwise there would
>   have been a conflict).  Undefining it means that it will be undefined in
>   the local module.

Undefining will be as if frob had never been defined.  Imported
bindings might take effect.

> You could have a system report any conflict, even with the import all
> paradigm.  Assume that foo and bar are imported.  foo exports frob.  
> Later the user enters bar and wants to (define frob ...) (export frob).  
> The system could issue an error message, disallowing to export frob from
> bar in this situation.  This, however, is a strange situation:  A change
> to bar is impossible, because of the situation of imports in a different
> module.

That's why I think errors should be signalled when a identifier is
used, not when it is defined, exported or imported.  Warnings are
nice, but the conflict might go away before the next time a identifier
is referenced.

> I think it is better if conflicts are local to a module and can be solved
> from inside a module.  In the above example this means, that an
> (interactive) change in the export list of some module x should not lead
> to conflicts in another module.  This means, that the set of identifiers
> imported from x in a client module y needs to be constant.

No, if we defer the errors until the last moment, we can live with
temporary potential conflicts.

> A signature can change with a new version of the module.  In an
> interactive system, a signature can even change during the session, but
> this does not affect the set of bindings that have already been
> imported.

Now I get it!

> For example, after executing the command
>   (use-modules (foo) :signature frob)
> in some module bar, the set of bar's imported bindings from foo remains
> the same, even if signature frob changes afterwards.

I see.  However, conceptually I see little difference between changing
a module's signatures interactively, or re-loading a whole system where
some signatures have changed from the previous run.  While developing
the system interactively, the behavior of signatures can cover up for
some conflicts, only to have them popup the next time the system is
loaded from scratch.

If, instead if using signature to avoid conflicts, you would get
warnings about potential conflicts when doing interactive development,
you could fix those conflicts right away, knowing that the system
stays consistent.