Re: [Chicken-users] interpreter/compiler mismatch

[felix]

address@hidden wrote:
> Hi Felix and happy Chicken users,
>
> here is some feedback I had in my hard drive, which was waiting for a 
> favorable moment to be posted ;-). I would be  curious to know what others
> think about the following issues.
>
> 1. First of all, I don't particularly like the fact that the interpreter 
> magically includes modules, so
>
> (pp '(1 2 3))
>
> works in the interpreter but in the compiler I must write
>
> (require 'extras)
> (pp '(1 2 3))
>
> I would force the "require" in the interpreter too. I do realize that
> it is convenient to have everything included by default in the repl:
> but it is easy enough for the user to include what he wants in his
> .csirc file. Magic should not happen under the hood. 

That the interpreter includes some functionality that has to be explicitly
loaded in compiled code is something that bothers me as well, but I haven't
found any fully satisfying solution for this. Depending on a user's .csirc
isn't too good: If one wants to write scripts, it is important that the amount
of work to add the required stuff is kept to a minmum, and scripts are (and 
should be)
self-contained (`-script' implies `-no-init').

A possible option would be to add all the default stuff to compiled code,
but that is unwieldy, for example for code that is to be embedded in another
application. Incidentally, compiling scripts with a "#!/.../csi -script'
header does exactly the right thing and includes all necessary units.

> 2. I also have some complain about the differences between the 
> interpreter and the compiler for what concerns runtime macros. 
> Consider for instance the following:
>
> (define (helper) (print "helper called"))
>
> (define-macro (my-macro)
>   (begin
>     (helper)
>     'dummy))
>
> (my-macro)
>
> This works fine with the interpreter, but does not work with the compiler.
> You (Felix) already explained to me why it is so, and the cure:
>
> (eval-when
>  (compile load eval)
>  (define (helper) (print "helper called")))
>
> (define-macro (my-macro)
>   (begin
>     (helper)
>     'dummy))
>
> (my-macro)
>
> This second version is conceptually cleaner: it makes obvious that
> "helper" must be available at compile time in order to be used in
> the macro. The first version works in the interpreter due to black
> magic: it should not work. So, I would submit to your attention the
> idea of removing some dynamism from the interpreter and force the
> user to use "eval-when" in the interpreter too. 

Hm. I don't know - removing dynamism from the interpreter sounds like
a step back to me. Again, I'm aware of the problem, but I think any attempt
to "solve" it, will reduce functionality.
A (hypothetical) implementation would mean, that at macro-expansion time,
all code executing inside macro-expanders uses a pristine, minimal
environment, right? Unfortunately doing this will IMHO make the macro-expansion
(and the eval-when semantics) rather complicated, for little gain.

> 3. On a similar note, the following code works in the interpreter, but 
> does not work in the compiler, no matter how I put eval-when:
>
> ;; problems with macros at run-time in compiled Chicken
> (define (make-macro name)
>   (eval `(define-macro (,name x) x)))
> (make-macro 'identity-macro)
> (display (identity-macro 1))

(eval-when (compile eval)
  (define (make-macro name)
    (eval `(define-macro (,name x) x)))  ; define make-macro at compile-time
  (make-macro 'identity-macro) )         ; and define another macro with it.

(display (identity-macro 1))

> If this is not valid code for the compiler, it should not work in the
> interpreter!

Admittedly, it is somewhat unelegant.

> I do realize you intended the interpreter mostly for quick and dirty 
> tests in the repl and not for developing programs, still since I don't 
> like to wait for the compiler and I have an interpreter available, so
> I develop using the interpreter. But then I expect a valid interpreted 
> program to be a valid program for the compiler too. Currently, this is
> not the case, so I have to fix by hand my scripts adding "require" and 
> "eval-when" to make the compiler happy.

Yes, this is annoying sometimes (I have the sample problem myself from
time to time). Low-level macros are very easy to implement, and the
implications of the when, what and how of macro-expansion in a system
that provides both a batch-compiler and interpreter are, well, complicated...

How do other implementations handle it?

- PLT: executes macro expanders in a clean, separate environment (with the
  help of a powerful module system)
- Scheme48, Guile, ...: interpreters only
- Larceny, Chez: compile "in-core", no batch-compilers (so no real difference to
  an interpreter)
- Gambit, Bigloo: they probably have the exactly the same problems as Chicken
  (but I'm not sure - if someone knows more about this, I'd be happy to hear it)

> Conclusion: the interpreter shouldn't be too forgiving, otherwise you 
> will get bad habits and you will be beaten by the compiler. It is
> better to be forced to write a valid program from the beginning,
> that to be forced to change an already written program which is
> only apparently valid.

A valid point of view. And I definitely agree with you that it's a problem.
I will think about separate macro-expansion environments, perhaps it's not
going to be too hard to do...
(But be sure that _when_ it is possible, there will be a command-line option
to disable it. ;-)


cheers,
felix