Re: Needed: per-port reader options

[Mark H Weaver]

address@hidden (Ludovic Courtès) writes:

> Hi Mark!
>
> Mark H Weaver <address@hidden> skribis:
>
>> address@hidden (Ludovic Courtès) writes:
>
> [...]
>
>>> Of course.  I just meant that, if you can call ‘make-reader’ with
>>> whatever options you’re interested in, instead of globally changing the
>>> reader’s option as is currently the case, then you’re halfway through.
>>>
>>> And in that case, the reader doesn’t need to be associated with the
>>> port.  Instead, since ‘primitive-load’ honors ‘current-reader’,  it just
>>> needs to be set when loading a file.  This way, any changes to the
>>> reader’s options will be local to that file.
>>
>> I see a few problems with that.
>>
>> First of all, since the reader directives can be placed anywhere that
>> comments are permitted, the read options must be changed while the
>> reader is in the middle of reading a single datum,
>
> Yes, but the reader can modify its own options data structure.

True, but it cannot arrange for subsequent calls to 'read' on that port
to use the new options data structure without stashing those changed
options somewhere.

Also, modifying its own options data structure cannot work for a reader
directive like #!curly-infix unless curly-infix is supported by the
default reader.

>> In other words, if a program uses 'read' on a data file, the
>> reader directives '#!fold-case' et al should affect all future calls to
>> 'read' on that file.
>
> Just on that file, or on any file subsequently loaded?

Just on that file.  That's the only sane thing.

Think about it.  Suppose you're reading multiple files in an interleaved
fashion (perhaps via a lazy stream that reads the files on demand), and
a #!fold-case in one file changed the way the other files were read.
That would be totally broken, don't you agree?

FWIW, I tested the behavior of Chibi Scheme, and it does the right
thing, exactly as I have described and implemented.

>> Fluids cannot solve this problem, because the program might be
>> performing interleaved reads of multiple files within the same thread.
>
> SRFI-105 reads:
>
>   An implementation of this SRFI MUST accept the marker #!curly-infix
>   followed by a whitespace character in its standard datum readers [...]
>
>   After reading this marker, the reader MUST accept curly-infix
>   expressions in subsequent datums until it reaches an end-of-file [...]
>
> To me, this sounds like global reader options (reset on EOF), not like
> per-port options.

Really?  "until it reaches an end-of-file" sounds like per-port to me.

Not convinced?  Imagine the same thought experiment I proposed above,
with one thread reading multiple files in an interleaved way via lazy
streams.  With your proposal, not only does the appearance of
#!fold-case mysteriously change the way the other files are read at some
random point depending on the interleaving, but now, when the original
file's is EOF is found, the other files mysteriously change back to
case-sensitive mode!

Can you find _any_ scheme implementation that handles reader directives
that way?  If so, please let me know, because I have a strongly worded
bug report to file :)

>>> Concretely, this would mean changing read.c such that each token reader
>>> takes the reader options as an additional first parameter.  Instead of
>>> looking up the global ‘scm_read_opts’, they would look at this explicit
>>> parameter.
>>
>> This is almost exactly what my patch does.  I added an explicit
>> parameter of type 'scm_t_read_opts *' to most of the helper functions in
>> read.c, and that parameter is consulted instead of the global options.
>
> I like it.

Excellent!

>> When reader directives such as '#!fold-case' are encountered, both the
>> 'scm_t_read_opts' struct and the per-port options are mutated.
>>
>> 'scm_read' initializes a local 'scm_t_read_opts' struct based on both
>> the global read options and the per-port overrides (if any), and a
>> pointer to that struct is passed down to all the helper functions in
>> read.c that need it.
>>
>> What do you think?
>
> The patch you posted (“Implement per-port reader options, #!fold-case
> and #!no-fold-case.”) does all three things at once: (1) explicit
> instead of global reader options, (2) per-port reader options, and (3)
> fold-case.
>
> Do you think you could split it into 3 patches?

Fair enough, I agree that this makes sense.  I'll work on it.

> I’m happy with (1) and (3).  I remain skeptical about (2), because of
> the mixture of concerns.

I don't think it's really a mixture of concerns.  The port just provides
an alist for anyone to use, without caring what is put there.  'read'
needs to know about ports, but that's always been the case.

> Sorry for the extra work, but thank you for pushing these things!

No problem, thanks for discussing it :)

      Mark