Some more elisp aspects: Reader and documentation

[Daniel Kraft]

Hi Andy and all,

as the main elisp compiler gets more and more complete (well, still a 
lot details missing as well as probably even most built-ins, but 
anyways), I think I'll also work on a real elisp reader (currently, 
Guile's Scheme reader is used) as well as some (internals) documentation 
about what I did, am doing and will do in the future.  I hope this 
sounds like a good plan ;)  BTW, if you want to review the compiler, we 
could agree on a "freeze" to compiler changes on the branch so you can 
do it freely, while I work entirely on the reader or documentation?

Regarding the reader, I think we've got two basic options (this was 
already disussed briefly here at the start of my project, IIRC):

1) Use the existing reader code in C and try to adapt it so that it 
(maybe controlled by some kind of "options") cares correctly about the 
differences between elisp and Scheme.  This might save some work as the 
basic reading of S-expressions, literals and the like can be shared, and 
the code will probably run fast; on the other hand, I'm not sure if we 
might risk messing things up and getting some passages complicated in 
order to have it handle alternatively both Scheme and elisp.  To be 
honest, I've so far not looked at the reader C code thoroughly, so I 
can't judge what this will really lead us to.

2) Write a seperate elisp reader, possibly in Scheme (but could be C as 
well if that's important for performance).  This helps us keep "both" 
readers clean and seperate, but all has to be done from ground up and 
the code is probably slower (when written in Scheme).  But my opinion is 
that performance will not matter that much here anyways (because the 
compiler probably takes most of the time, not the parser), and 
implementing parsing of S-expressions and the like is not really hard, 
so we won't lose much by not sharing existing code here.  So if you 
don't have another opinion because you know the existing code better 
than I do or see problems I don't, I'd go with this approach.

For this and in order to get meaningful debug and error outputs, I guess 
I'll have to find and associate the source code positions myself with 
read objects, right?  It seems that I can use port-line and port-column 
to get the position of something just read, and set-source-properties! 
to associate line/column to the expressions?  Anything else I need to 
care about then?

Documentation:  Currently, I've got the module/language/elisp/README 
file with some notes, mainly about stuff implemented/missing as well as 
the extensions I implemented over original elisp.  Additionally, I tried 
to explain most "internal workings" very briefly directly with comments 
in the source code; but I think it would be nice to have some very basic 
information (like, how dynamic binding or the "void" value of variables 
is implemented, or the different modules for function/value slots) also 
in the real docs.  Where do you suggest I put them 
(chapter/section/subsection of what part of the documentation)?

Yours and have a nice weekend!

Daniel

--
Done:  Arc-Bar-Cav-Ran-Rog-Sam-Tou-Val-Wiz
To go: Hea-Kni-Mon-Pri