Re: Needed: per-port reader options

[Mark H Weaver]

Hi Ludovic!

address@hidden (Ludovic Courtès) writes:
> Mark H Weaver <address@hidden> skribis:
>> address@hidden (Ludovic Courtès) writes:
>>> Mark H Weaver <address@hidden> skribis:
>>>
>>>> I recently tried to implement reader directives, e.g.:
>>>>
>>>>   #!fold-case     (from R7RS)
>>>>   #!no-fold-case  (from R7RS)
>>>>   #!curly-infix   (from SRFI-105)
>>>>
>>>> However, I ran into a rather serious problem.  Guile's reader options
>>>> are global to the entire process, but clearly these reader directives
>>>> should affect only the port they are read from.  So we need to support
>>>> per-port reader options.
>>>
>>> I think we should instead be able to instantiate new readers–i.e., have
>>> a higher-order ‘make-reader’ procedure (that’s how Guile-Reader started,
>>> actually.)
>>>
>>> That, combined with the ‘current-reader’ fluid, should provide good
>>> flexibility.
>>
>> Being able to easily create new readers sounds great to me, but that's
>> not sufficient to implement the reader directives above,
>
> Of course.  I just meant that, if you can call ‘make-reader’ with
> whatever options you’re interested in, instead of globally changing the
> reader’s option as is currently the case, then you’re halfway through.
>
> And in that case, the reader doesn’t need to be associated with the
> port.  Instead, since ‘primitive-load’ honors ‘current-reader’,  it just
> needs to be set when loading a file.  This way, any changes to the
> reader’s options will be local to that file.

I see a few problems with that.

First of all, since the reader directives can be placed anywhere that
comments are permitted, the read options must be changed while the
reader is in the middle of reading a single datum, e.g.:

  (#!no-fold-case BLAH ((BLAH #!fold-case BLAH)) BLAH)
=>
  (BLAH ((BLAH blah)) blah)

This cannot be done by replacing the entire reader, at least not if the
reader is written in a straightforward way.  The reader options must be
changed during a single call to the reader, and then must be honored for
all subsequent tokens until overridden by a future directive.

Another problem is that these reader directives have nothing to do with
evaluation or loading of code.  They affect the reader itself, and
therefore they should work as expected no matter what is doing the
reading.  In other words, if a program uses 'read' on a data file, the
reader directives '#!fold-case' et al should affect all future calls to
'read' on that file.

Fluids cannot solve this problem, because the program might be
performing interleaved reads of multiple files within the same thread.

> Concretely, this would mean changing read.c such that each token reader
> takes the reader options as an additional first parameter.  Instead of
> looking up the global ‘scm_read_opts’, they would look at this explicit
> parameter.

This is almost exactly what my patch does.  I added an explicit
parameter of type 'scm_t_read_opts *' to most of the helper functions in
read.c, and that parameter is consulted instead of the global options.
When reader directives such as '#!fold-case' are encountered, both the
'scm_t_read_opts' struct and the per-port options are mutated.

'scm_read' initializes a local 'scm_t_read_opts' struct based on both
the global read options and the per-port overrides (if any), and a
pointer to that struct is passed down to all the helper functions in
read.c that need it.

What do you think?

      Mark