bug#66020: (bug#64735 spin-off): regarding the default for read-process-output-max

[Eli Zaretskii]

> Date: Thu, 21 Sep 2023 17:37:23 +0300
> Cc: stefankangas@gmail.com, monnier@iro.umontreal.ca, 66020@debbugs.gnu.org
> From: Dmitry Gutov <dmitry@gutov.dev>
> 
> > How does the throughput depend on this value?  If the dependence curve
> > plateaus at some lower value, we could use that lower value as a
> > "good-enough" default.
> 
> Depends on what we're prepared to call a plateau. Strictly speaking, not 
> really. But we have a "sweet spot": for the process in my original 
> benchmark ('find' with lots of output) it seems to be around 1009600. 
> Here's a table (numbers are different from before because they're 
> results of (benchmark 5 ...) divided by 5, meaning GC is amortized:
> 
> |    4096 | 0.78 |
> |   16368 | 0.69 |
> |   40960 | 0.65 |
> |  409600 | 0.59 |
> | 1009600 | 0.56 |
> | 2009600 | 0.64 |
> | 4009600 | 0.65 |

Not enough data points between 40960 and 409600, IMO.  40960 sounds
like a good spot for the default value.

> >> And I think we should make the process "remember" the value at its
> >> creation either way (something touched on in bug#38561): in bug#55737 we
> >> added an fcntl call to make the larger values take effect. But this call
> >> is in create_process: so any subsequent increase to a large value of
> >> this var won't have effect.
> > 
> > Why would the variable change after create_process?  I'm afraid I
> > don't understand what issue you are trying to deal with here.
> 
> Well, what could we lose by saving the value of read-process-output-max 
> in create_process?

It's already recorded in the size of the pipe, so why would we need to
record it once more?

> Currently I suppose one could vary its value while a process is
> still running, to implement some adaptive behavior or whatnot. But
> that's already semi-broken because fcntl is called in
> create_process.

I see no reason to support such changes during the process run,
indeed.