Re: [lwip-users] Compensating for longer latency connections

[David Empson]

It may be necessary to use Wireshark to see what is actually happening.

 

There are three likely factors with international traffic:

 

1. Latency. Mean round trip time may be in the order of 250 ms or higher. (I see more than this for major sites in some distant countries.)

2. Packet delivery time can fluctuate depending on congestion, alternate routing, etc., resulting in somewhat variable RTT.

3. Packet loss is more likely due to a higher number of hops and routers, and greater chance of congestion.

 

Some things to check:

 

1. How much audio data are you buffering in the application?

 

If a data packet from the server is lost, the server will retransmit based on its retransmit timer. This is dependent on the implementation used by the server, but is typicaly the mean round trip time plus 4 times the standard deviation of recent round trip time measurements. This means the retransmission delay is always at least the mean RTT. If the standard deviation is 25% of the mean RTT (quite possible for international traffic), the retransmission delay may be in the order of (2 * mean_RTT).

 

There will also be interaction with the timer mechanism used by the server, e.g. if it uses fast and slow timers then a retransmission will only happen on the next tick of its fast timer after the retransmission timer expires.

 

This suggests you should be buffering at least (2 * mean_RTT) plus a safety margin. To allow for 250 ms mean RTT and typical fast timer implementations, you probably need to be buffering in the order of 1 second of audio data.

 

2. Do you have TCP_QUEUE_OOSEQ enabled?

 

If not, then a single data packet loss will result in all subsequently received data being discarded until the server retransmits everything starting at the packet which was lost. This is wasteful of bandwidth but should otherwise be dealt with by having a big enough audio buffer.

 

3. Do you have a particularly small TCP_WND (receive window)?

 

If TCP_WND < (4 * actual_MSS) then loss of a single ack packet will cause a delay in the data stream.

 

If TCP_WND < (required_date_rate * mean_RTT) then the server will not be able to send data fast enough. e.g. for 24000 bytes/sec and round trip time of 250 ms then TCP_WND must be greater than 6000.

 

Set TCP_WND well above these thresholds to avoid unnecessary delays.

 

(Comments from others on my analysis are welcome!)

 

----- Original Message -----

From: JM

To: Mailing list for lwIP users

Sent: Wednesday, March 03, 2010 2:30 PM

Subject: Re: [lwip-users] Compensating for longer latency connections

Just to follow up, I've increased pbufs, TCP_SND_QUEUELEN, and MEMP_NUM_TCP_SEG, and looked at memp and etharp stats and am seeing no errors.

--- On Tue, 3/2/10, JM <address@hidden> wrote:

From: JM <address@hidden>
Subject: [lwip-users] Compensating for longer latency connections
To: address@hidden
Date: Tuesday, March 2, 2010, 12:57 PM

I'm using lwIP 1.3.0 for a streaming music player.  I've noticed that I generally have issues when playing higher bitrate streams from servers overseas.  For example, I can play a 192Kbit stream from a domestic host with no problems.  But, when I try playing another 192Kbit stream from across the pond, the audio buffer level is ok for maybe 5-10 seconds, then the buffer level starts dropping, and soon after, it no longer keeps up. 

This occurs in every instance I've tried so I believe it's a latency/distance issue.  Also, I can successfully play all these streams on my computer.  I haven't tried Wiresharking it yet, but I suspect this may not tell me much being sort of an "obvious" problem to those who are well versed on this sort of thing.  Plus, my computer's ethernet hardware doesn't help much by hiding bad packets from me.  What are the likely culprits as far as lwIP configuration?  I have increased the number of pbufs some, but maybe it's not enough, or maybe I'm not addressing the correct parameters.