bug#65156: 29.1; Reading from pipe with --insert or insert-file-contents no longer supported

[Po Lu]

Eli Zaretskii <eliz@gnu.org> writes:

>> Cc: 65156@debbugs.gnu.org
>> Date: Tue, 08 Aug 2023 21:33:55 +0300
>> From: Eli Zaretskii <eliz@gnu.org>
>> 
>> > Date: Tue, 8 Aug 2023 20:20:23 +0200
>> > From: Lucas Werkmeister <mail@lucaswerkmeister.de>
>> > 
>> > Launch graphical Emacs with standard input attached to a pipe, then
>> > attempt to insert /dev/stdin, for example:
>> > 
>> >      echo test | emacs -Q --insert /dev/stdin
>> >      echo test | emacs -Q --eval '(insert-file-contents "/dev/stdin")'
>> > 
>> > This now results in an error (and nothing inserted into the buffer):
>> > "Maximum buffer size exceeded".
>> 
>> The doc string says:
>> 
>>   When inserting data from a special file (e.g., /dev/urandom), you
>>   can’t specify VISIT or BEG, and END should be specified to avoid
>>   inserting unlimited data into the buffer.
>> 
>> > Previously, this used to work; git bisect identifies cb4579ed6b ("Allow
>> > inserting parts of /dev/urandom with insert-file-contents", bug#18370)
>> > as the first bad commit.
>> > 
>> > Other (non-stdin) pipes are also affected. Testing with a named pipe 
>> > shows that the error only occurs after the pipe is first written to:
>> > 
>> >      # first terminal:
>> >      mkfifo /tmp/fifo
>> >      emacs -Q
>> >      M-x insert-file /tmp/fifo
>> >      # second terminal:
>> >      { echo x; sleep 10; echo y; } > /tmp/fifo
>> > 
>> > Before you run the command in the second terminal, you can observe that 
>> > Emacs is just waiting for input from the pipe; as soon as you run the 
>> > other command, Emacs shows the error before the sleep finishes.
>> 
>> Lars, Paul, any suggestions?
>
> I installed the patch below on the emacs-29 branch; please see if it
> solves your problems with reading from pipes.
>
> Paul, can there be a regular file that is not seekable?  If regular
> files are always seekable, the patch can be simplified.
>
> diff --git a/src/fileio.c b/src/fileio.c
> index 995e414..55132f1 100644
> --- a/src/fileio.c
> +++ b/src/fileio.c
> @@ -4581,7 +4581,7 @@ because (1) it preserves some marker positions (in 
> unchanged portions
>        goto handled;
>      }
>  
> -  if (seekable || !NILP (end))
> +  if ((seekable && regular) || !NILP (end))
>      total = end_offset - beg_offset;
>    else
>      /* For a special file, all we can do is guess.  */
> @@ -4678,7 +4678,7 @@ because (1) it preserves some marker positions (in 
> unchanged portions
>          For a special file, where TOTAL is just a buffer size,
>          so don't bother counting in HOW_MUCH.
>          (INSERTED is where we count the number of characters inserted.)  */
> -     if (seekable || !NILP (end))
> +     if ((seekable && regular) || !NILP (end))
>         how_much += this;
>       inserted += this;
>        }

Everyone, I fixed this on master yesterday, as part of a series of
changes to enable visiting named pipes as files.  The crux of the
problem is this:

      seekable = emacs_fd_lseek (fd, 0, SEEK_CUR) < 0; <----------
      if (!NILP (beg) && !seekable)
        xsignal2 (Qfile_error,

where `seekable' is actually set to 1 if the file is NOT seekable, since
lseek returns 0 upon success and -1 upon failure.

Then, later:

  if (seekable || !NILP (end)) <------------------------
    total = end_offset - beg_offset;
  else
    /* For a special file, all we can do is guess.  */
    total = READ_BUF_SIZE;

total is set to end_offset - beg_offset (both -1 at that point),
resulting in a call to buffer_overflow as the gap is extended beyond
memory.

The change presently in place on emacs-29 should be replaced with a
one-line change that replaces:

  seekable = lseek (fd, 0, SEEK_CUR) < 0;

with

  seekable = lseek (fd, 0, SEEK_CUR) != (off_t) -1;