Re: Planning work

[Marius Vollmer]

Rob Browning <address@hidden> writes:

> Of course, whether or not this something we really want as the
> standard model for our system -- i.e. requiring gcc and
> generating/linking code on the fly, I don't know, but it is
> possible.

As a standard, I wouldn't like to involve GCC, i.e., invoking GCC for
every expression you type in the repl, or even for every file you
`load' during interactive development.  As a means for delivering an
application, I think it's a possibility.

> On a related note, at some point, I'd like to have a discussion about
> trying to devise both short and long term plans for speeding up guile.
> Basically, I'd like to revisit the hobbit, vm, lightning, whatever
> else mess and see if we have any further conclusions.  While I don't
> have an infinite amount of time to spend working on that front, I am
> willing to invest substantial resources if we know what we want.

I know what _I_ want! :> Which is a compilation framework for Guile,
with explicit functions `compile-file', `compile-expression' or
similar to shake out the interactions between compiplation and our
module system, and in general to make the possibility of compilation
an integral aspect of Guile.  Code written for a file compiler is
different from code written for an interpreter, I think.  Interactions
with macros and modules are different, for example.

The compiler can usefully be separated into frontend and backend.
Although I ultimately want the backend to generate machine code (on
selected platforms) and bytecodes (on all platforms), I think only
having the frontend as a first step and a primitive backend is a good
start.

The frontend would do macro expansion (with syntax-case), taking care
of module system issues at the same time (by annotating references to
top-level variables with the module they belong to) and generally
reducing the code to a canonical form, like (ice-9 syntax-case) does
now, but maybe using real data structures instead of lists (since
macro expansion will have to use data structures already to represent
the code).

The output of the frontend would still be Scheme with few extensions
for the module system, and we could simply throw this output at our
existing interpreter.

This is what (ice-9 syntax-case) does now (I think), but viewed from a
compiler perspective and not from an interpreter perspective.

The intermediate form is likely not optimal for the interpreter
(having replaced `(let ((a 1)) ....)' with `((lambda (a) ...) 1)', for
example, which the interpreter will create a closure for), but I'm
sure we can improve this.

That is, I would want to have a full blown compilation framework in
Guile, where `eval' first compiles the expression.  To get there, I
want to first concentrate on the frontend and have a simplistic
backend using our current evaluator.

If we feel like it, we can improve the backend incrementally, for
example by recognizing code that can be rewritten into forms that it
has direct support for, like `and', `do', `let*', etc; or by writing
memoized code to disk.

And we can have a bytecode backend.  And one using Hobbit.  And one
with Lighning.  We would know what a backend is supposed to do to get
the interaction with the rest of the system right.  For example, I'm
unsure about the semantics of code compiled by Hobbit, as compared to
the same code interpreted when it comes to macros or the module
system.