[lwip-users] Re: Driver Tx queue filling up

[Jeff Barber]

Please ignore this message.  It's the one I originally posted
Thursday.  It seems to have been stuck somewhere in gnu.org's mail
system for a couple of days.

On Thu, Oct 22, 2009 at 8:19 AM, Jeff Barber <address@hidden> wrote:
> I have an lwip-based FTP server built on a nearly stock 1.3.1 lwIP.
> When I do a GET on a large file (resulting in a high-speed
> unidirectional transfer), I'm seeing about every 256th TCP packet
> getting dropped (often the first drop doesn't happen until about 512
> packets into the transfer but then it seems very regular).  TCP
> eventually recovers from this but it takes about a second each time.
> Then a drop happens again 256 packets later.
>
> Now 256 happens to correlate to the size of the Tx and Rx ring buffers
> in my driver so that was the obvious place to look.  I notice that
> sometimes my driver's linkoutput function is being called when its Tx
> queue is full.  I'm thinking that's the proximate cause of the
> problem.  However, shouldn't the TCP_SND_QUEUELEN (32) and TCP_SND_BUF
> (8 * MSS) values be a limit to the maximum number of "outstanding"
> pbufs?  I did turn on TCP_QLEN_DEBUG and according to that info, I
> never get a queue len > 8.  So why do I end up with so many packets
> "in flight"?
>
> And if I'm misunderstanding, what is the intended feedback mechanism
> from the driver?  tcp_output always seems to ignore the return value
> of ip_output.  Hence, if I understand it correctly, that means an
> attempt to send while the Tx queue is full is treated exactly the same
> as if the packet was dropped on the wire: i.e. it will rely on the
> retransmission process to recover.
>
> I'm running with these values in my opt.h:
> #define TCP_MSS                     1460
> #define TCP_SND_BUF                 (8 * TCP_MSS)
> #define TCP_SND_QUEUELEN            (4 * (TCP_SND_BUF/TCP_MSS))
>
> Any ideas on debugging or understanding this would be appreciated.
> Packet capture demonstrating the problem attached (see packet # 399).
>
> Thanks,
> Jeff
>