Re: Any idea about what makes Emacs slow reading on pipes?

[David Kastrup]

address@hidden (Kai Großjohann) writes:

> address@hidden (David Kastrup) writes:
> 
> > Richard Stallman <address@hidden> writes:
> >
> >>     We might change it so that, after stashing the stuff in a
> >>     buffer, Emacs looks again if more data is available from the
> >>     same process.
> >
> > More data will not be available from the same process, since the
> > process has not had any chance to get CPU time again for
> > generating more data.  We are talking about a uniprocessor machine
> > here.
> 
> I'm not familiar with the internals of Unix (-like) systems, but
> maybe it is possible to "look" in such a way that the scheduler has
> a chance to do its thing.  For example, say that using select(2)
> causes the scheduler to run, then we might "look" using select(2).

The scheduler _has_ already made its decision, and the decision is to
have Emacs run rather than the output generating process.

There are various reasons: Emacs has been waiting on that pipe for a
longer time, so it is judged an "interactive program" and not a
calculating one.  It gets an interactivity bonus and gets scheduled
immediately when available.

This interactivity bonus gets lower while busy processing continues.
That is the reason that after a while, a few larger packets make it
through.

> Hm.  But this might make Emacs quite a bit slower.  So maybe it is a
> good idea to only look twice if little data has been received.  If a
> lot of data has been read, then we can just process that
> immediately.

The only way to yield is to actually and voluntarily call a delay.
This should only be done when there is reason to suspect it might be
worth a try, namely when we find ourselves repeatedly working on
small packets in fast succession.  Emacs should just refuse to call a
process filter too fast twice in a row unless it has stuffed the
buffer sufficiently, and it should have a very fast path when it
decides not to run the process filter yet.

Perhaps a better scheme would be to use _blocking_ reads with the full
buffer length and a timeout instead when something is available.  When
the blocking read fails, get just what is there.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum