bug#64735: 29.0.92; find invocations are ~15x slower because of ignores

[Eli Zaretskii]

> Date: Wed, 13 Sep 2023 20:27:09 +0300
> Cc: luangruo@yahoo.com, sbaugh@janestreet.com, yantar92@posteo.net,
>  64735@debbugs.gnu.org
> From: Dmitry Gutov <dmitry@gutov.dev>
> 
> >> Are you sure? IIUC the fact that is passes 'curbuf' as the last argument
> >> to 'decode_coding_c_string' means that decoding happens inside the
> >> buffer. This has been my explanation for the performance improvement 
> >> anyway.
> > 
> > Yes, you are right, sorry.
> 
> So we're not going to try the gap-based approach? Okay.

decode_coding_c_string does that internally.

> >> The would be the safest alternative. OTOH, this way we'd pass up on the
> >> opportunity to make all existing asynchronous processes without custom
> >> filters, a little bit faster in one fell swoop.
> > 
> > We could change the ones we care about, though.
> 
> Which ones do we care about? I've found a bunch of 'make-process' calls 
> without :filter specified (flymake backends, ). Do we upgrade them all?
> 
> The difference is likely not critical in most of them, but the change 
> would likely result in small reduction of GC pressure in the 
> corresponding Emacs sessions.
> 
> We'll also need to version-guard the ones that are in ELPA.
> 
> We don't touch the implementations of functions like start-file-process, 
> right?
> 
> What about the callers of functions like 
> start-file-process-shell-command who want to take advantage of the 
> improvement? Are we okay with them all having to call 
> (set-process-filter proc 'buffer) on the returned process value?

I think these questions are slightly premature.  We should first have
the implementation of that filter, and then look for candidates that
could benefit from it.  My tendency is to change only callers which
are in many cases expected to get a lot of stuff from a subprocess, so
shell buffers are probably out.  But we could discuss that later.

> > We could also discuss changing
> > the default value, but that would require measurements in as many
> > cases as we can afford.
> 
> If you have some particular scenarios in mind, and what to look out for, 
> I could test them out at least on one platform.

Didn't think about that enough to have scenarios.

> I'm not sure what negatives to test for, though. Raising the limit 10x 
> is unlikely to lead to an OOM, but I guess some processes could grow 
> higher latency?..

With a large buffer and small subprocess output we will ask the OS for
a large memory increment for no good reason.  Then the following GC
will want to compact the gap, which means it will be slower.