Re: [Bug-readline] Pasting large amounts of text into readline-enabled programs truncates parts of the lines of the text being pasted

[Margarita Manterola]

Recap and new findings:

The problem:
Large pastes (5k or more) into a readline enabled program fail when
running kernels larger than 2.6.31-rc5.  "Fail" means that some lines
are incomplete.  From 2.6.39-rc1 onwards, "some" lines become "almost
all lines after the first 4k".

The problematic commits in the kernel tree:
1 - 2009-07-27 (never shipped) -
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=3a54297478e6578f96fd54bf4daa1751130aca86

After this commit, pastes start breaking.  For a 35k file, about 50%
of the times one or two lines are partially incomplete.

2 - 2009-07-29 (v2.6.31-rc5) -
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=e043e42bdb66885b3ac10d27a01ccb9972e2b0a3

This commit reverts the previous one, but adds one extra function that
calls flush_to_ldisc.  Pastes still break, but commenting out the
function call prevents breakage *up to 2.6.39-rc1*.

3 - 2011-03-22 (v2.6.39-rc1) -
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=f23eb2b2b28547fc70df82dd5049eb39bec5ba12

This commit changes many schedule/flush/cancel_delayed_work calls into
schedule/flush/cancel_work.  After this commit, the big breakage
starts: for the 35k example file, it starts breaking at aprox. 4k and
then every line is partially incomplete or directly not there.

Still after this commit, commenting out the tty_flush_to_ldisc(tty)
call added by e043e42bdb66885b3ac10d27a01ccb9972e2b0a3 prevents the
breakage.

4 - 2011-04-04 (v2.6.39-rc2) -
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=a5660b41af6a28f8004e70eb261e1202ad55c5e3

This commit modifies the behaviour of how the ttys are polled.  After
this commit, commenting out the tty_flush_to_ldisc(tty) call still
leads to breakage.

Instead, re-adding the call to schedule_work(&tty->buf.work) that was
removed in this commit, prevents the breakage.

***
Looking at the code in readline, the issue is triggered by these lines
in rltty.c:

tiop->c_lflag &= ~(ICANON | ECHO);
(...)
tiop->c_iflag &= ~(ICRNL | INLCR);

If these two lines are replaced by:

tiop->c_lflag &= ~(ECHO);
(...)
tiop->c_iflag &= ~(INLCR);

Then the pastes work fine: no lines are missing.  Of course, this
means that readline doesn't work properly, but this is just to note
that those are the terminal settings that cause the issue to pop-up.

***
I'll be now mailng the linux kernel mailing list hoping to get their
attention into fixing this.

Credit: this investigation was done together with Maximiliano Curia.

-- 
Regards,
Margarita Manterola