We seem to be losing a TCP_SEG every so often when communicating over a noisy wireless link. Our lwip was taken at some point between stable releases 1.10 and 1.11. Since that time we have installed some of the important fixes to the stack, we are currently working to re-structure our port so we can easily update to
1.2 (and 1.3 when it becomes available). We have examined the source for relevant fixes that have already been addressed in the baseline that might resolve our issue, but we have found none.
Here is what is going on ... this issue exhibits itself in our implementation of an HTTP client, but believe this to be a generic TCP issue. Our lwip is on the client performing a GET operation to a remote server on the other side of a high bit-error rate wireless network interface.
At some point during this scenario we lose a TCP_SEG ...
Client < ---- > Server
SYN Seq 0, Ack 0, Len 0 ------->
<-------- SYN Seq 0, Ack 1, Len 0 PSH,ACK Seq 1, Ack 1, Len 148 (GET) -------> <-------- ACK Seq 1, Ack 149, Len 0
<-------- FIN,ACK Seq 479, Ack 149, Len 0 (OOSEQ) <-------- PSH,ACK Seq 1, Ack 149, Len 478 (OOSEQ AND LOST -- NEVER RECEIVED BY CLIENT)
ACK Seq 149, Ack 1, Len 0 -------> (DUP ACK)
-- 3 second delay as Server times out --
<-------- FIN,PSH,ACK Seq 1, Ack 149, Len 478 (RESEND BY SERVER) ACK Seq 149, Ack 480, Len 0 -------> ACK Seq 149, Ack 480, Win 2919, Len 0 -------> (WINDOW UPDATE)
FIN, ACK Seq 149, Ack 480, Len 0 -------> <-------- ACK Seq 480, Ack 150, Len 0
The significant events that seem to be required in order for us to lose a TCP_SEG are:
Client must lose the data packet back from the server (len 478)
Timeout must occur before data packet resent by server
If client receives the Len 478 packet, but gets it out of sequence the scenario does not leak a TCP_SEG.
Could the following occur?
When the FIN,ACK is received by the client a TCP_SEG is allocated and stored as ooseq
When the FIN,PSH,ACK is received by the client (resend of the data after missing the data in the PSH,ACK from the server, but this time the server added the FIN), lwip pushes the data up to the application and sees the FIN in the packet, but does not dequeue the seg stored with the previous packet containing the FIN.
The socket closes down and the seg is lost (the length 1 seg containing the first FIN attempt).