Re: Move Guile-style modules from scm to scm-modules (issue 567140045 by address@hidden)

[David Kastrup]

Urs Liska <address@hidden> writes:

> Am Freitag, den 31.01.2020, 18:38 +0100 schrieb:
>> Urs Liska <address@hidden> writes:
>> 
>> > Am Mittwoch, den 29.01.2020, 07:01 -0800 schrieb address@hidden:
>> > > On 2020/01/29 12:20:10, mail5 wrote:
>> > > > Unfortunately I haven't set up a build system on my new
>> > > > computer
>> > > > yet,
>> > > > so this
>> > > > patch is not tested locally at all, so I'm humbly waiting for
>> > > > the
>> > > > automated
>> > > > tests to succeed or fail ...
>> > > 
>> > > I don't like the "use-modules clauses adjusted accordingly".  I
>> > > don't
>> > > think it makes sense readjusting use-modules clauses all the time
>> > > while
>> > > we are deciding on the final module organisation, so I'd strongly
>> > > suggest first biting the bullet and deciding on a syntax for a
>> > > user-level command able to load Scheme modules without further
>> > > options,
>> > > and then introduce that command.  In that manner, future
>> > > directory
>> > > organisations (which are almost certain to come) will not affect
>> > > the
>> > > source of user-level documents any more.
>> > > 
>> > > https://codereview.appspot.com/567140045/
>> > 
>> > Maybe I'm missing something, but AFAICS there will always be the
>> > need
>> > for a module path like (ice-9 regex), or (scm display-lily). We
>> > will
>> > have that with *any* user-facing load syntax.
>> 
>> \loadScmPackage display-lily
>> 
>> or even
>> 
>> \loadPackage display-lily.scm
>> 
>> with the last .scm getting interpreted specially.
>> 
>> #(use-modules ...) is not a user-facing load syntax.  We don't want
>> people to have to change their input when an optional package
>> migrates
>> to the system or a local package changes hierarchy.  Specific path
>> components may make sense for disambiguation, but if I take a look at
>> what LaTeX does, a flat hierarchy as first user-level approximation
>> does
>> not appear to have significant scaling problems.
>
> I'm not so sure about that. In the LaTeX world this means that you
> have to come up with file names that are unique to the whole ecosystem

Yes.

> because otherwise you're relying on luck that "your" file is found
> first in case of conflicts.  I had this several times, mostly in the
> form of requests of TeX Live maintainers to rename files which were
> considered dangerously generic.

So?  If you put 5 logical layers before "Schenker", two different
Schenker graph packages will still collide because the logical
categories are still all a match.  You are not even kicking the can down
the road that way, you are just making it bigger.

> The other thing to consider is that path components are (currently)
> not only used for disambiguation but also for the lookup. LilyPond
> starts searching for files at each include path, and it relies on
> relative paths starting from a (any) include path. LaTeX, however,
> recursively searches for files starting from the include paths. I
> think going that route would have several negative impacts
> (disadvantage/risk):
>
>  * The recursive search is potentially expensive

Which is why path databases ultimately exist.

>  * There are actual risks of loops in the search path

Only for symbolic links.

>  * Targets are not really specified, using relative paths is
>  more expressive and explicit.  * kpathsea is pretty complex and has
>  corner cases that don't work well (e.g. it doesn't always follow
>  symlinks when a directory does not contain at least one "real" file),
>  and we would knowingly have to decide creating and maintaining a
>  comparable tool

Eventually.

> So, requiring explicit paths gives clearer interfaces (although more
> typing is required) and reduces searching complexity.

I don't see at all that it is a clearer interface if you have to specify
a detailed path with information that will either be the same for
packages with potential conflicting names due to sharing functionality,
or arbitrary.

Take a look actually more or less at _any_ package system, be it TeX,
Debian, PEP, RPM, Emacs.  The namespace for requesting a package is
flat.  On all of them.  There are search hierarchies containing them,
but they don't make for unique paths in any way.

> In the case of display-lily and all other built-in modules we can
> reduce this by adding the scm directory to the include path. But for
> external packages I am pretty sure that using explicit paths from the
> include paths (like we already have it with \include) makes sense.

I don't think it makes sense to have users exposed to the internal
placement of packages organized in directory trees, and I don't know any
package system that does.

The internal scm files loaded by LilyPond don't need to be visible to
the user.  Those optionally loaded manually (for example, (scm accreg))
are a different game.

> However, I'd like to stress that we are (or at least should be) talking
> about several different things when saying "package" and "loading":
>
> 1)
> Loading a package/module *file*, parsing its code and making elements
> available to clients
>
> 1a)
> One question is how to address these includable files
> 1b)
> The other question is where the elements and which elements of the
> loaded files are visible.
>
> 2)
> Loading/using a package in a more conceptual sense, like "edition-
> engraver" or "scholarLY". Here all the stuff about option handling
> becomes (more) relevant, and the questions of 1) are prerequisites.
>
> ###
>
> So when talking about new commands to "load packages" we are actually
> talking about two different things that *both* have to be done.

For now, we just have to address the case of loading a single scm file
with its own module and exported symbols.

-- 
David Kastrup