Re: Trouble with `export'.

[Dirk Herrmann]

On 9 Jun 2001, Marius Vollmer wrote:

> Dirk Herrmann <address@hidden> writes:
> 
> > On 7 Jun 2001, Marius Vollmer wrote:
> > 
> > > I don't see how this is significantly different from what we have
> > > now.  In my understanding, signatures are a way for a module to
> > > specify multiple lists of exported definitions, while we can only
> > > specify one such list per module.  A signature is not more fixed
> > > than the set of exported bindings in our current system
> > > (i.e. "signature" is just a short name for "set of exported
> > > bindings", and our modules can only have one signature).
> > 
> > Except that a signature is explicit, while the set of imported
> > bindings is implicit.
> 
> In what way is a signature explicit?  Looks like I don't understand
> the concept of signatures.  What I have in mind is that we might say
> 
>     (use-modules ((foo bar) :signature frob))
> 
> to effect that the bindings belonging to the frob signature are
> imported into the current module.  But _which_ bindings this are
> exactly is defined elsewhere, alongside the (foo bar) module.

Since words like explicit and such don't seem to fit to what I try to
express, I will give a more detailed explanation of the issues that I have
in mind.


The possibility to import bindings from several modules in addition to
the bindings introduced by local definitions leads to the question, which
binding should take precedence if there are several possible bindings that
could be taken.  Further, the possibility to add bindings later on in
arbitrary modules plus the possibility to remove bindings using 'undefine'
makes it necessary to define these precedence rules in a dynamic
environment.  Examples for questions that have to be answered:

Assume that foo and bar are modules which both export a binding for frob:
1 When importing foo and bar, which binding for frob takes precedence?
2 When importing foo and bar, and the binding from bar takes precedence,
  what happens after (undefine frob) in bar?
3 Assume that there is a rule stating that the definitions of a later
  imported module take precedence.  You have imported foo and bar, in this
  order.  That would mean that the binding from bar would be seen.  Now,
  you have changed something in module foo.  To make the changes visible,
  you would like to re-import it.  How should this be handled?  (If it 
  was simply imported again, the effect would be that suddenly the   
  bindings of foo take precedence, except you also re-import bar after 
  re-importing foo..  I can imagine that this is something that would
  confuse a lot of users.)
4 When frob is imported from foo, but in the local module frob is defined
  later on, should the local bindings take precedence?
5 Assuming the local definition of frob takes precedence over any
  imported one, what happens if you do (undefine frob) in the local module
  again?  Should frob become undefined, or do you want the imported
  definition to appear again?

My position is, that any conflict should be considered an error.  You may
disagree, but for the rest of this mail I will argue based on the
assumption that we want the system to be designed to detect and report any
conflict as illegal.  I'd like the system to behave as follows in the
above situations:

1 When you import foo and bar with their full set of exported bindings, a
  conflict is reported:  "frob is imported from several sources".  The
  user would have to state which frob should be taken.
2 When frob is imported from bar, and in module bar the definition for
  frob becomes undefined, it also becomes undefined in the local
  module.  Still, the information that frob should be taken from bar is
  still available.  An attempt to define frob locally would lead to an
  error ("attempt to override an imported binding").
3 Re-importing a module should not change the origin of bindings.  I. e. 
  bindings should still be imported from where they were imported
  before.  If, however, the set of exported bindings of an imported
  modules has changed in between, it has to be checked that no conflicts
  occur.
4 When frob is imported from foo, but in the local module an attempt is
  made to re-define frob, this should lead to an error (see 2).
5 If frob is defined locally, it is not imported (otherwise there would
  have been a conflict).  Undefining it means that it will be undefined in
  the local module.

You could have a system report any conflict, even with the import all
paradigm.  Assume that foo and bar are imported.  foo exports frob.  
Later the user enters bar and wants to (define frob ...) (export frob).  
The system could issue an error message, disallowing to export frob from
bar in this situation.  This, however, is a strange situation:  A change
to bar is impossible, because of the situation of imports in a different
module.

I think it is better if conflicts are local to a module and can be solved
from inside a module.  In the above example this means, that an
(interactive) change in the export list of some module x should not lead
to conflicts in another module.  This means, that the set of identifiers
imported from x in a client module y needs to be constant.

* A possible solution to this is, to 'close' the set of imported
identifiers at the time of importing the module.  It is easy to provide
functions to the user to interactively add bindings to that 'closed' set,
if it is desired to have additional bindings imported that were not yet
existent at import-time.  The problem with this approach is, that the set
of exported bindings at any given time does not necessarily reflect what
is considered the 'interface' of the module.  For example, if some module
x is strongly related to another module y, it may be that x exports some
binding foo, that is only to be used internally for communication between
x and y.  It may even be the case that the binding for foo disapplears
from x after some initialization of x and y has been performed.  Further,
it may be that there is a binding for bar in x, that is only exported if x
is loaded in debug mode.  If x is not loaded in debug mode, then bar is
not exported.  Still, it would be nice if code that imports 'all of x'
in non debug mode would be warned if it defines bar, to avoid possible
conflicts when bar instead is once loaded in debug mode.

* Another solution is to have the user state explicitly which identifiers
to import in the import-command.  Again, functions to change this set of
imported bindings can be provided.  The disadvantage is, that naming all
imported bindings explicitly can be tedious.

* A third solution is to use signatures.

What do I understand by the term 'signature':  A signature is a list of
symbols.  A signature is associated with a module and describes a set of
bindings that are provided by the module.

* A signature need not contain all bindings that are exported by a
  module.  For example, a large module like gtk might provide a signature
  that only covers button-related bindings.  That means, however, that a
  module can have several associated signatures.
* Not all symbols in a signature do need to relate to a binding.  This
  could be used to 'reserve' symbols for a module, even if there is no
  implementation yet.  Another use is for definitions that exist only
  conditionally.  In guile, for example, we have some symbols declared as
  deprecated.  This means, the bindings don't necessarily exist.  Never-
  theless, a signature could hold them all.

Signatures can be determined without need to execute any code from a
module.  That is, they can potentially be put into header files that are
separate from the module code itself, and they can be determined by
compilers at compile time (this is one aspect which I meant with saying
that signatures are 'explicit').  This still allows signatures to be
computed, but their computation must be possible (for example) by pure
r5rs code plus the relevant SRFIs (similar to macro expansions that can
also happen at compile time).

Signatures are used by module import commands to specify the set of
bindings that should be imported into the client module.  They are only a
means of convenience to spare the user from having to type in long lists
of symbols to import large sets of identifiers.  For example:  If the
signature 'frob' of module 'foo' is (a b c), it is equivalent to name the
signature or to give the list when importing the module.

A signature can change with a new version of the module.  In an
interactive system, a signature can even change during the session, but
this does not affect the set of bindings that have already been
imported.  For example, after executing the command
  (use-modules (foo) :signature frob)
in some module bar, the set of bar's imported bindings from foo remains
the same, even if signature frob changes afterwards.  In other words, the
module bar has (at the time of the import command) made a local copy of
the contents of the signature (this is a second aspect of what I meant
when I said that signatures are 'explicit').  If however, the user _wants_
to update the set of imported bindings in bar, commands are provided to
update the set of imported bindings interactively.


I hope I was less ambiguous in my explanations this time.  It may be that
my understanding of signatures does not relate to some 'official'
definition.  But, it should be clear now what I understand by the term
signature and that my basic intention is to obtain a system where all
conflicts are reported.  Sorry if my previous mails have led to
confusion.  With these aspects in mind, I will try to answer some further
points in your email:



> > As I said, I consider the 'import-all' paradigm broken.
> 
> I'd say that with signatures, you have the same `import-all'
> semantics.  "Import all of the signature."

I should have better said:  I consider the paradigm broken, that at import
time the set of imported bindings is not fully specified.  With all of the
above solutions ('closing' the set of imported bindings at import time,
using explicit identifier lists, and using signatures) the set of
identifiers is determined at import time.  Since the import lists are
fixed, import conflicts can be detected at import time, and conflicts with
later local defines can also be detected when these definitions are
executed.

> > Thus, the significant difference with signatures is that you can
> > make use of the restrictive paradigm that for every binding it has
> > to be stated from which module the binding comes, while still having
> > a comfortable way of importing large sets of bindings.
> 
> I don't see how you can have a explicit list of imported bindings
> without writing down that explicit list.  Referring to shortcuts
> defined elsewhere doesn't really work, since you don't know for sure
> what they expand into.

Right.  Again, I have to clarify:  You don't know what signatures expand
to, but they don't change dynamically.  At import time their contents are
known.

> > If with a new module version the signature changes, yes, you may run into
> > problems.  For example, module foo's old signature was (a b c) and the new
> > one is (a b c d).  If some module bar had defined its own binding for 'd',
> > this would cause an error with the restrictive solution.  So what?  The
> > conflict can easily be solved,
> 
> Yes, it can be resolved, but I think your argument for signatures is
> that conflicst wouldn't arise in the first place, since the list of
> imported bindings was explicit to be begin with.

It is IMO impossible to define a system where no conflicts occur.  Even if
you state all imported bindings explicitly in the import clause, nothing
could prevent you from importing the same binding twice:
  (use-modules (foo) :signature (frob))
  (use-modules (bar) :signature (frob))
or, importing frob and later do a local (define frob ...).  The question
is, whether you want this to be reported, or whether you prefer implicit
rules to deal with the question which binding is to be taken.  But, again,
signatures are not the only solution to this question.

> In contrast to that, if local bindings would silently shadow imported
> ones, the bar module would simple continue to work, since it is
> unaffected by the new binding appearing in the foo module.

a) It may still lead to confusion.  b) You also have to define what
happens when the local binding is undefined later on.

However, I don't want to be too strict here:  It does not necessarily have
to be rejected if a local binding is created 'above' an imported one in
_interactive_ use of guile.  A warning message should be OK, and it may
even be switched off if it annoys people.  For non-interactive use one can
expect users to have their bindings specified cleanly.

Best regards,
Dirk Herrmann