Re: Understanding filter function calls

[Eli Zaretskii]

> From: Karthik Chikmagalur <karthikchikmagalur@gmail.com>
> Date: Thu, 27 Jul 2023 14:08:23 -0700
> 
> Multiply the above blocks by 600, and that explains the large difference
> in the sizes of the stdout.
> 
> It does not take any longer for the filter function to run on each
> invocation.  The problem is that The filter in Emacs is called with
> about 4KB of new text every time, and these calls are spaced out evenly
> in time, so Emacs takes longer to get around to parsing the stdout with
> TeXLive 2023.
> 
> The dvisvgm process is thus bottlenecked by Emacs.  I can imagine a few
> different ways to fix this, but I don't know how to do them:
> 
> 1.  Reduce the duration between successive calls of the filter function.
> Is this configurable in Emacs?  I don't see anything relevant in the
> manual sections on accepting output from processes or filter functions.

This is not configurable for the simple reason that Emacs checks for
subprocess output every time its main loop gets to that, so basically
Emacs does that as fast as possible, assuming it is idle, i.e. no
other command or timer runs.

> 2.  Enlarge the buffer or "pipe" connecting dvisvgm to Emacs.  This
> stream buffer appears to be set to 4KB.  Since dvisvgm produces far more
> output (to stdout) than this between two successive instances of Emacs
> accepting process output, widening the pipe should relieve this
> pressure.  I tried tweaking `read-process-output-max' but this doesn't
> help.

Which probably means that each time we get to check for subprocess
output, there's less than 4KB of stuff in the pipe ready to be read?
Did you look at the amount of bytes we read each time?  How many bytes
do we read, and does this number change if you change the value of
read-process-output-max?

Also, did you try setting process-adaptive-read-buffering to nil?

> 3.  Get dvisvgm to generate less verbose output.  Unfortunately this is
> not configurable at the level of granularity that we need.  We can't
> turn it off completely either since we rely on certain strings in the
> process stdout to update the LaTeX preview state in Org buffers.
> 
> Any ideas on how to avoid this throttling would be appreciated.

So once again a problem generated by an external program, which cannot
be configured by that program, is blamed on Emacs?  Is this fair?  My
take of this is not that Emacs is "throttling" dvisvgm, but that
dvisvgm is "flooding" Emacs with useless data, and cannot be told to
shut up.  AFAICT, the data it outputs is completely useless in
production use cases, and (with the possible exception of the last
line, which shows the output file name) is basically debug-level info.
Did you try to take this problem up with the dvisvgm developers?

And in any case, we are talking about 0.4 sec delay wrt the older
version of TeXLive, right?  Is this really large enough to worry
about?