Re: process.c: read_process_output: hard coded 4096 bytes read limit

[Miguel Guedes]

On Sat, 22 Jun 2013 10:27:55 -0700, Paul Eggert wrote:

> I don't know, and that limit has bothered me in the past, too.
> 
> It's a simple matter to increase it to 16 K bytes; does the patch below
> help your performance?  If so, I can install it. Letting the user
> specify a larger limit might make sense, but would take a bit of thought
> and hacking.
> 

Thank you for your reply, Paul!

I am unable to apply and test the patch at the moment but I'm sure it 
will help with performance, though not entirely fix it.  I have this 
minor mode that I'm developing that receives code completion data 
asynchronously from a server bound to localhost.  The actual server side 
processing of code completion always takes less than 100ms and the 
results are sent to the client in full.  It is then ironic (as well as 
frustrating) that the bottleneck (latency lasting many seconds) I'm 
experiencing is being caused by emacs when just receiving the raw data 
before even any processing takes place in the client side (which is very 
fast anyway)!

There are a couple of ways I can use to mitigate the latency issues 
mentioned above so I already have a fix, however IMHO there is a problem 
in the way emacs reads data from process channels that needs to be fixed, 
which I wouldn't mind contributing to.  I assume you're an active emacs 
developer (my own knowledge of emacs' internal code and architecture is 
extremely limited), how would you tackle this, Paul?  

Apart from raising the hardcoded limit, one (perhaps simpler) way would 
be to allow the user to specify the buffer size when spawning a process/
network connection, however that still leaves open the problem of the 
waits between reads.  So perhaps the best strategy to fix this, though 
probably more complicated, would be to tackle the waits between reads 
head on?  How problematic with this be?

Apologies for not understanding but by "I can install it", did you mean 
you can commit it?