bug#64735: 29.0.92; find invocations are ~15x slower because of ignores

[Eli Zaretskii]

> Date: Wed, 13 Sep 2023 17:27:49 +0300
> Cc: luangruo@yahoo.com, sbaugh@janestreet.com, yantar92@posteo.net,
>  64735@debbugs.gnu.org
> From: Dmitry Gutov <dmitry@gutov.dev>
> 
> >> Does the patch from my last patch-bearing email look similar enough to
> >> what you're describing?
> >>
> >> The one called read_and_insert_process_output.diff
> > 
> > No, not entirely: it still produces a Lisp string when decoding is
> > needed, and then inserts that string into the buffer.
> 
> Are you sure? IIUC the fact that is passes 'curbuf' as the last argument 
> to 'decode_coding_c_string' means that decoding happens inside the 
> buffer. This has been my explanation for the performance improvement anyway.

Yes, you are right, sorry.

> > Sure, but in this case we don't need any filtering.  It's basically
> > the same idea as internal-default-process-filter: we just need to
> > insert the process output into a buffer, and optionally decode it.
> 
> Pretty much. But that raises the question of what to do with the 
> existing function internal-default-process-filter.

Nothing.  It will remain as the default filter.

> Looking around, it doesn't seem to be used with advice (a good thing: 
> the proposed change would break that), but it is called directly in some 
> packages like magit-blame, org-assistant, with-editor, wisi, sweeprolog, 
> etc. I suppose we'd just keep it around unchanged.

Yes.

> > We can provide a special filter identified by a symbol.  Such a filter
> > will not be Lisp-callable, it will exist for the cases where we need
> > to insert the output into the process buffer.
> 
> The would be the safest alternative. OTOH, this way we'd pass up on the 
> opportunity to make all existing asynchronous processes without custom 
> filters, a little bit faster in one fell swoop.

We could change the ones we care about, though.

> Should we also discuss increasing the default of 
> read-process-output-max? Even increasing it 10x (not necessarily 100x) 
> creates a noticeable difference, especially combined with the proposed 
> change.

That should be limited to specific cases where we expect to see a lot
of stuff coming from the subprocess.  We could also discuss changing
the default value, but that would require measurements in as many
cases as we can afford.