RE: [lwip-users] tcp_write() errors on snd_queuelen

[Tim Lambrix]

Thanks for your reply.  I am using lwip 1.3.2 version.  I just compared the 
version with 1.4.0 and I see there are many changes.  I will look into the 
option of upgrading but it would be nice to know if there is a bug with my 
application setup first.

I do not see the send queue ever return to zero even when the load is removed. 
For example, I see it start at 0 and I blast a little data where it may climb 
to 50 and then I stop the volume of data.  Usually it returns to 0 within a 
couple seconds.  I blast a little more data and then it climbs again to 40 and 
I stop again.  This time, the value doesn't drop below 6 even after 10 minutes. 
 I blast a little more data and it climbs to 100 and stop again.  This time the 
value won't drop below 63.  It never returns to 0.  If this doesn't return, 
does that number represent data not received by the destination?  It is hard 
for me to verify right now as the tcp_write fail doesn't send data.

So it seems the lwip believes there is data someplace and so I am trying to 
discover the underlying problem.  I could just set the value to 0 but I don't 
know what that would break or why lwip thinks it can't go back down.  I was 
wondering if it was due to a problem with my configuration file with the 
buffers?  

Since my typical data packet is pretty small, about 50 bytes, I figured it 
would be more efficient to combine my packets into a larger TCP packet.  I 
realize this impacts the latency of my data.  What is the best way to configure 
the TCP window for small and large data packets?  Is the window the problem 
with the send queue?

I will figure out what to do with my data when the tcp_write() fails. 



-----Original Message-----
From: address@hidden [mailto:address@hidden On Behalf Of Kieran Mansley
Sent: Monday, March 14, 2011 6:50 AM
To: Mailing list for lwIP users
Subject: Re: [lwip-users] tcp_write() errors on snd_queuelen

On Fri, 2011-03-11 at 22:21 +0000, Tim Lambrix wrote:
> I have a Cortex-M3 device running as a server and I only allow 1
> client to connect.  With a light load, everything performs well.
> However, when I start sending data packets of about 50 bytes every 10
> ms, packets start dropping out and the data is lost.  In most
> configurations I have tried, it gets lost due to the  snd_queuelen
> getting too large.  I can't figure out why it is getting so big.  I
> also notice that if I send data for a brief period at a higher rate,
> that value climbs up and then when I stop sending, it doesn't always
> return back down to 0.  This makes the next blast of data spin the
> value up to and over the limit causing data lost.  Should it return to
> 0?

Packets are added to the send queue whenever the application calls send.

Packets are removed from the send queue whenever the stack is able to
send them; there are lots of factors that limit this for TCP, e.g.
congestion window, available receive window Nagle's algorithm, etc. I
wouldn't expect the send queue to immediately drop back to zero, but it
should after all the sent data have been acknowledged.  

Packets are freed and available for reuse once they are acknowledged by
the other end.  You therefore need enough packet buffers to cover the
rate of sends you want to make times the round trip time of the network
(assuming the other end acknowledges fairly promptly, and there isn't
much loss). 

Note that I don't really count this as packets getting dropped.  You
should get an error from your call to tcp_write() that tells you that it
can't enqueue the data.  It's up to your application to deal with it
then (e.g. by blocking if it wants to preserve the data, or dropping it
if it doesn't).  Fundamentally an application can generate packets
faster than the network can send them, so something has to give.  The
send queue will provide a small amount of smoothing but it's not a
panacea.

> How do I prevent this error from occurring and are there other
> settings in the options file that I should change to obtain the
> performance without lost packets?  

Which version of lwIP are you using?  I would suggest updating to the
current 1.4.0 release candidate as there was a significant re-write of
the tcp_write/tcp_enqueue path recently and if there was a bug there's a
good chance it has been fixed.  There may also be improvements in the
combining of writes together to make a single packet.

There are two limits on the send queue: the number of bytes and the
number of packets.  If you're hitting the packet limit (most likely with
your small sends) you could try increasing that.

Kieran


_______________________________________________
lwip-users mailing list
address@hidden
http://lists.nongnu.org/mailman/listinfo/lwip-users