Re: Using file descriptors in Emacs

[sbaugh]

Eli Zaretskii <address@hidden> writes:
>> From: address@hidden
>> Date: Fri, 09 Sep 2016 16:38:16 -0400
>> 
>> >> Not as far as I know - what in my patches suggests that I need a pipe?
>> >
>> > A pipe is a set of 2 file descriptors between 2 programs, one of the
>> > descriptors is used for reading by one program and writing by the
>> > other, the other descriptor used similarly in the opposite direction.
>> > Isn't that what you set up?
>> >
>> > If not, then I must be missing something, so please describe the data
>> > flow between emacsclient, emacs server, and the external program.
>> 
>> There is not necessarily any external program. Even emacsclient can exit
>> just fine once the file descriptors are passed in and a process object
>> created from them.
>> 
>> Once a process object P is created, the data flow is as follows:
>> 
>> When emacs writes to P, emacs is directly writing to the file descriptor
>> that emacsclient had as its stdout. It is literally calling "write" on
>> the file descriptor that was/is emacsclient's stdout.
>> 
>> When emacs reads from P, emacs is directly reading from the file
>> descriptor that emacsclient had as its stdin. It is literally calling
>> "read" on the file descriptor that was/is emacsclient's stdin.
>
> Emacs doesn't normally read or write from/to file descriptors.  The
> only features that do such things are communications with
> subprocesses, which could be either local subprocesses (in which case
> Emacs talks to them via a pipe) or remote processes (in which case we
> use network or serial connections).  One other kind of feature that
> uses file descriptors is external data sources, such as D-Bus and
> inotify, which are a kind of local process (the OS kernel or some
> other daemon) that doesn't need to be run by Emacs, only communicated
> with.
>
> As we already have a framework for all of the above, what other kinds
> of descriptors, which don't fit into the framework, would we like to
> have?
>
>> For example I would like to bring eshell up to par with other Unix
>> shells, and the ability to manipulate file descriptors directly is
>> pretty much required for that.
>
> Please elaborate: how are file descriptors required for that?  A shell
> needs to pass file descriptors to the subprocesses it invokes, which
> AFAIK we already do in the primitives that invoke subprocesses.  What
> else is needed, and how does the feature you propose fit into that?

Two features that I would like to add to eshell (or some shell in Emacs)
that I believe will require the ability to directly manipulate file
descriptors:

- Piping data to and from processes without that data round-tripping through
Emacs, which is necessary if I want to write a pipeline processing any
significant amount of data (which I frequently do in even casual shell
use)
- Redirecting higher file descriptors than 0/1/2 (that is, when creating
a process, before forking, use dup2 to rearrange what file descriptors are
in what slots)

Of the two, the first is more important, but I think the second is also
necessary.