Re: [lwip-users] TCP Throughput

[David Empson]

Everything looks normal up to packet 39. Between packet 2 and 39 you're 
transferring about 5 MB/s. The two window updates indicate points at which 
the application on the PC read data from the TCP stack, allowing the PC to 
report that it had more space to receive data.

At packet 40 the previous packet was missed. This results in a series of 
acknowledgements from the PC asking for the missing packet, which gets 
resent at packet 46 and all the preceding packets are acknowledged at 47. 
The same thing then happens again several times: each time the missed packet 
is resent and acknowledged, the next packet in sequence has been lost. This 
pattern continues until packet 84, then things return to normal for a while. 
Despite the retransmissions, the average data rate is still 4.8 MB/s for 
this period.

Interesting observation: this period in which the retransmissions occurs has 
sequence numbers in the range 32768 through 65535. As soon as the sequence 
number hits 65536, normal communication resumes.

Packets 85 through 104 are OK again.

Packet 105 is interesting. It appears to be the packet that was mising at 40 
and retransmitted at 46, now completely out of order. Packets 107, 109 and 
111 are also packets that were sent earlier during the 32K-64K region, but 
they don't follow an obvious pattern. Packet 119 is a resend of packet 113 
(which had already been acknowledged), which happens to coincide with the 
end of the burst of out of order packets.

This looks suspiciously like some kind of packet queueing problem where some 
packets are getting mislaid and turn up again later, or perhaps a threading 
issue - is it possible that you have multiple threads active in the LWIP 
core at the same time?

I can't see any sign of that particular problem happening again later, so 
there appears to be something strange going on inside LWIP or your driver 
during the 32K to 64K portion of the data being transmitted.


At packet 195 we get the first of the zero window updates you mentioned. If 
you note the pattern of "Win" values in the preceding packets, you can see 
that they decrease from packets 130 to 182, at which point the window has 
just filled.

Packet 183 indicates that the PC application has read 11680 bytes, but for 
some reason it waited until the window filled up before doing so (probably 
due to the application being blocked elsewhere and then asking to read that 
much data). It then takes a while to read any further data (at packet 196), 
which results in a pause in the data transmission due to the receive window 
filling. The second read has transferred 53855 bytes to the application. 
(Note that 11680+53855 = 65535, which is probably the size of a ring buffer 
in the application.)

The pattern repeats: wait for the receive window to fill, transfer 11680 
bytes, then transfer 53855 bytes in some cases after a delay of a few 
milliseconds. Packet 470 is an example of a case where the 53855 byte read 
happened just before the receive window completely filled.

I can't see any evidence that this bursty read behaviour is anything to do 
with LWIP configuration - it is probably an artifact of the application's 
socket read and data buffering logic combined with taking too long to 
process or display the data, possibly influenced by Windows multitasking. A 
properly designed application should be able to receive the data faster (as 
long as it doesn't have an output or processing bottleneck).

The only issue worth investigating at the LWIP end is what is going on 
between the 32K and 64K point in the data flow.

----- Original Message ----- 
From: "Adam Fullerton" <address@hidden>
To: <address@hidden>
Sent: Friday, June 18, 2010 5:54 AM
Subject: [lwip-users] TCP Throughput


> Hi,
>
> I'm using a 144MHz SH2A with a 72MHz external bus & I'm only seeing TCP 
> throughput of about 2.4MB/S. I am not sure that I have lwIP configured 
> properly.
>
> I am using HyperTerminal on the PC to send a command to the SH to send 
> back a 1MB string. Here is a Wireshark capture of a transfer:
> www.ardware.co.uk/rte/tcpslow.pcap
>
> I am not experienced enough to know if the TCP ZeroWindow message from the 
> PC is an indication that I have configured lwIP wrongly or the PC can't 
> handle the data.
>
> Thanks in advance.
>
> Adam Fullerton.