Re: address@hidden: dynamic loading of native code modules]

[Rob Browning]

Neil Jerram <address@hidden> writes:

> I'm pretty sure that Thi was wanting to reinstate (use-modules ...)
> support _in addition to_ keeping load-extension and dynamic-call
> etc.

OK, that's somewhat different, though there would still be some issues
to work out like where the libs should go -- i.e. you used to put them
in %load-path, but if we're using libtool, then they have to go in
LD_LIBRARY_PATH or LTDL_LIBRARY_PATH which somewhat complicates the
semantics of use-modules.  This isn't a *big* deal, though, and to
some extent I may just be coming from a perspective of feeling like
shared library loading and modules are better treated more
orthogonally than others might.

> Using this kind of approach, is it always possible to emulate the
> effect of the old use-modules behaviour by installing a .scm file
> that loads the required library?
>
> If it is, then that's probably sufficient for 1.6, and I'll add
> something to the docs to make that clear.

More or less, the primary differences that I can think of are that in
the old approach you would put the shared libs in a /usr/share
directory (which isn't quite right according to the FHS and causes
problems on shared NFS volumes which expect only arch-independent data
in /usr/share), where with the new approach the libs need to be
located somewhere that libltdl can find them (system lib path,
LD_LIBRARY_PATH or LDTL_LIBRARY_PATH).  Also with the old approach,
any functions defined in your shared lib would automatically and
unavoidably be exported from the module that the shared lib
represented -- with the new approach you need to either add an export
for each function to the .scm file by hand, or write some module code
to export all symbols defined in the module after the shared lib has
been initialized -- i.e.

  (for-each
    (lambda (sym) (export sym))
    (module-bindings (current-module)))

or similar...

> Yes, load-extension should be used here.  The difference is that
> load-extension provides a way of handling the case where the LIB is
> already statically linked in.  (And the case where it has previously
> been dynamically linked?)  I think the code is still pretty explicit:
>
> (load-extension "libmylib" "my_init_func")

OK, right -- I just haven't used load-extension much yet, but that
seems good to me.

> Well, I agree with all the examples you give here, but what about the
> following?

OK, here goes nothing :> (Oh, and I am by no means implying here that
I know we haven't ever dropped anything we shouldn't have or that I'm
*certain* that none of these things should have been dropped -- what's
below is just my current impression regarding each.)

> - dropped support for multibyte strings  [unless I'm misunderstanding
>   the old mailing lists, Guile used to have these !]

Hmm.  I not sure about whether or not guile used to have them, but my
main impression is that guile has taken a long time even figuring out
what it thinks a reasonable answer to this problem is, much less
designing a solution with good documentation that is intended to be
supported for the long-haul.  However, I could easily be mistaken
here.

> - dropped/lost support for Tcl/Tk

That strikes me as an "add on" that is only going to stick around for
as long as there are enough people interested in guile's tcl/tk
support to continue maintaining it, and if there aren't enough people
around interested in supporting it, then perhaps it *should* go away.

> - dropped/lost support for Ctax and other things (parser, rx etc.)
>   in the guile-lang-allover package (or guile-rgx-ctax in CVS)

I can't really speak to this having never seen, used, or worked on any
of this -- I like the *idea* of translating multiple languages into
guile, but I doubt that it's likely to happen on a large scale until
guile is a persuasive enough platform with respect to the basics to
attract groups wanting to work on the languages in question.
Otherwise I doubt there are enough people here, just given us, to both
maintain the core of guile well *and* develop and maintain any
significant number of font ends.

I guess in the end I feel that in the absence of infinite resources,
extensions from the guile core that don't have enough demand to create
and sustain the communities that develop and support them often will
(and likely should) fade away.

Things people want and need *will* get worked on.  As an example, I
don't think guile-pg has had a lot of development lately, but recently
we've started using it heavily here, and Dale has also been working
with it.  As a result, I've contacted Ian to see what his current
status is and to see how we might be able to help -- Dale and I have
already fixed it to work at least crudely with the newer autotools and
the stable branch.

> - dropped/lost support for Hobbit compilation

This one I've talked to Marius about off and on at some length, and
have actually worked on a bit myself (Hobbit and CVS in particular).
I think the conclusion was that guile's evaluator needs to be reworked
in some of the ways that Marius, Lynn, and others, including myself
have talked about, or any work on compilation is likely to be way too
much effort for too little gain.  Guile is not currently built to
support compilation cleanly -- the addition of syntax-case and the
ways in which it breaks hobbit (among other things) points this out.

As Marius has mentioned we likely need a cleaner evaluation model
(with clear separations between "read time", "compile time", and
"execution time" before we're going to be able to make a lot of
sustainable headway here.  The good thing is that if we do manage to
seprate these things in a resonable way, we may end up with a lot of
interesting flexibility wrt to offline-compilation, JIT compilation,
byte-compilation, etc.

-- 
Rob Browning
rlb @defaultvalue.org, @linuxdevel.com, and @debian.org
Previously @cs.utexas.edu
GPG=1C58 8B2C FB5E 3F64 EA5C  64AE 78FE E5FE F0CB A0AD