bug#33403: [Geiser-users] bug#33403: Data length limit in Guile/Geiser/Scheme evaluation

[Mark H Weaver]

Hello all,

"Jose A. Ortega Ruiz" <address@hidden> writes:

> On Fri, Nov 16 2018, Neil Jerram wrote:
>
>> Neil Jerram <address@hidden> writes:
>>
>>> Mark H Weaver <address@hidden> writes:
>>>
>>>> This is a documented limitation in Linux's terminal handling when in
>>>> canonical mode.  See the termios(3) man page, which includes this text:
>>>>
>>>>    Canonical and noncanonical mode
>>>>        
>>>>        The setting of the ICANON canon flag in c_lflag determines
>>>>        whether the terminal is operating in canonical mode (ICANON set)
>>>>        or noncanonical mode (ICANON unset).  By default, ICANON is set.
>>> [...]
>>>>        * The maximum line length is 4096 chars (including the
>>>>          terminating newline character); lines longer than 4096 chars
>>>>          are truncated.  After 4095 characters, input processing (e.g.,
>>>>          ISIG and ECHO* processing) continues, but any input data after
>>>>          4095 characters up to (but not including) any terminating
>>>>          newline is discarded.  This ensures that the terminal can
>>>>          always receive more input until at least one line can be read.
>>>>
>>>> Note that last item above.
>>>
>>> Awesome; thank you Mark.
>>>
>>> So possibly this limit can be removed, in my Org/Geiser context, by
>>> evaluating (system* "stty" "-icanon") when initializing the Geiser-Guile
>>> connection.  I'll try that.  Will the terminal that that 'stty' sees be
>>> the same as Guile's stdin?
>>>
>>> Jao, if that works, I wonder if it should be the default for Geiser?  It
>>> appears to me that Geiser shouldn't ever need the features of canonical
>>> mode.  Is that right?
>>>
>>> Anyway, I'll see first if the stty call is effective.
>>
>> Yes, with this in my ~/.guile-geiser -
>>
>> (system* "stty" "-icanon")
>>
>> - I can do evaluations past the 4K line length limit, and the Org-driven
>> problem that I first reported [1] has disappeared.
>
> Ah, system* is a scheme call! So yeah, maybe we could add that call to
> Geiser's guile initialization... i don't really see how that would cause
> any problem elsewhere.

I think something like this should be done, not only in the Guile
initialization, but ideally in the generic Geiser code that connects to
inferior processes via pseudo-tty.  After the pseudo-tty is allocated
but before launching the inferior Scheme process, something like "stty
--file=/dev/pts/N -icanon" should be run.

However, before doing this, some warnings are in order:

When in noncanonical mode, the normal processing of ERASE (usually DEL
or Ctrl-H) and KILL (usually Ctrl-U) characters are disabled, and input
characters are delivered to the subprocess immediately as they are
typed, rather than waiting for the newline as normally happens in
canonical mode.

At least in the case of the Guile REPL, one notable side effect of
running in noncanonical mode is that when a list is entered at the REPL,
the 'read' returns as soon as the final close parenthesis is entered.
Nothing after that is read, not even the usual newline.  The final
newline is only read if the reader is not yet sure that it has finished
reading the token, e.g. if a number or symbol is entered.  In those
cases, typically any delimiter may be typed to terminate the read,
e.g. space.

To see this, you can try running Guile from a traditional terminal
program (e.g. xterm or GNOME Terminal), and type:

  (system* "stty" "-icanon")

and then:

  (display "hello")

You will see that as soon as you type that close paren, "hello" is
immediately printed, followed by another REPL prompt, all on the same
line.

You might also try (use-modules (ice-9 rdelim)) and then:

  (read-line)

and you'll see that the newline you type at the end of that line is read
by 'read-line', which then immediately returns the empty string.  The
input that you wish for 'read-line' to see must be typed on the same
line, immediately after the close parenthesis.

So, it might be that Geiser needs to be adjusted somewhat to deal with
these differences.

Finally, you might consider the possibility that 'stty' might not be
available in PATH, or fails for some reason, and ideally this case
should be handled as well.

It might be simpler to always use REPL servers over a socket, than to
deal with these headaches, although I don't know if that will be an
option for the other Scheme implementations.

    Regards,
      Mark