[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [lwip-users] losing input packets and repeated acks
From: |
Carl D. Blake |
Subject: |
Re: [lwip-users] losing input packets and repeated acks |
Date: |
Mon, 15 Jan 2007 10:05:14 -0700 |
Thank you for your help. I've listed the things I've tried below.
On Fri, 2007-01-12 at 19:53, Jonathan Larmour wrote:
> Carl D. Blake wrote:
> >
> > I've attached my lwipopts.h file and a tcpdump trace.
>
> I just noticed a few things in your config too...
>
> > /* #define TCP_TMR_INTERVAL 250 */
> > #define TCP_TMR_INTERVAL 10
>
> That is very very small. You will probably be too busy processing the TCP
> timer to do any real work. The default should be adequate.
>
I set it back to 250. I noticed that I was getting some 250 ms delays
between packets, so I reduced this to try and improve throughput.
> > #define PBUF_POLL_SIZE 30
>
> This should be PBUF_POOL_SIZE, not POLL. You will therefore have 16
> buffers by default instead.
Oops. I changed the name to PBUF_POOL_SIZE and set it to 90.
>
> > #define PBUF_POOL_BUFSIZE 1536
>
> Are you sure you mean 1536 and not 1500?
>
Per your suggestion below. I changed this to 500. I was trying to set
it to the maximum size of my ethernet packet aligned to 8 bytes.
> > #define PBUF_LINK_HLEN 16
>
> Are you sure you mean 16 and not 14?
>
I definitely mean 16 here. I tried 14, but some combination of lwip,
the GNU C compiler V3.4.4, MEM_ALIGNMENT set to 4, and my processor
(MPC8271) makes the whole thing not work if HLEN is 14. 16 seems to
work.
> > #define TCP_MSS 1476
>
> Again I would have expected this to be 1460 (just like your peer in the
> trace below). Fortunately for you, your peer won't go above 1460, so you
> won't see any problem, but this value still seems wrong, unless there's
> something unconventional about your setup.
>
I changed this to 1460. I saw 1476 in some lwip example somewhere.
> > #define TCP_WND (16 * 1024)
>
> 16K, but you only have 16 pbufs, so it will be easy to run out of pbufs.
> You will use 1 pbuf even if you only receive a tiny packet, and that
> includes acks. It does dependent somewhat on your data pattern, but it is
> usually more efficient to have a larger number of smaller pbufs. It's
> usually better to have 3 500-byte pbufs than 1 1500-byte pbuf. Remember
> that pbufs can be chained.
>
OK. I've changed my setup to have 90 pbufs. Each one is 500 bytes.
> > ------------------------------------------------------------------------
> >
> > 13:53:58.646727 boromir.tucson.boeckeler.com.10001 > 10.0.1.222.4098: P
> > 904882:904904(22) ack 1 win 5840 (DF)
> > 13:53:58.649988 boromir.tucson.boeckeler.com.10001 > 10.0.1.222.4098: .
> > 904904:906364(1460) ack 1 win 5840 (DF)
> > 13:53:58.651482 boromir.tucson.boeckeler.com.10001 > 10.0.1.222.4098: .
> > 906364:907824(1460) ack 1 win 5840 (DF)
> > 13:53:58.654904 10.0.1.222.4098 > boromir.tucson.boeckeler.com.10001: . ack
> > 906364 win 11094 (DF)
> > 13:53:58.654912 boromir.tucson.boeckeler.com.10001 > 10.0.1.222.4098: P
> > 907824:909284(1460) ack 1 win 5840 (DF)
> > 13:53:58.654915 boromir.tucson.boeckeler.com.10001 > 10.0.1.222.4098: P
> > 909284:909926(642) ack 1 win 5840 (DF)
> > 13:53:58.655868 boromir.tucson.boeckeler.com.10001 > 10.0.1.222.4098: .
> > 909926:911386(1460) ack 1 win 5840 (DF)
> > 13:53:58.661166 10.0.1.222.4098 > boromir.tucson.boeckeler.com.10001: . ack
> > 906364 win 12576 (DF)
>
> Nevermind the huge delay later, you are getting problems right here. You
> will probably find you are out of pbufs. You are advertising a larger
> window than you have memory to accept, so the remote size is still
> flinging data at you. If you allow out of sequence packets, this may also
> explain the large delays, as they will use up pbufs waiting for any
> missing data, but there would be no pbufs left to allow the missing data
> to come in!
>
I made the changes you suggested and things are a little better. I'm
still getting the huge delay periodically, but the multiple acks of the
same packet and the same window size are gone. I still get multiple
acks of the same packet, but the system seems to be advertising a larger
window size for each ack. I seem to be getting some corrupt data, but
it's not nearly as bad as it was before.
I enabled LWIP_STATS and printed out the stats before and after the
transfer. TCP stats is reporting several checksum errors. PBUF stats
is reporting several errors. I enabled debug messages for pbufs and I'm
getting the message "pbuf_alloc: Out of pbufs in pool." This happens
quite a bit during the transfer of 1 Mbyte. I hadn't expected this to
be a problem since my pbuf memory is 45Kbytes and my TCP window is
16Kbytes. Any other suggestions?
> Jifl
- [lwip-users] losing input packets and repeated acks, Carl D. Blake, 2007/01/12
- Re: [lwip-users] losing input packets and repeated acks, Jonathan Larmour, 2007/01/12
- Re: [lwip-users] losing input packets and repeated acks, Jonathan Larmour, 2007/01/12
- Re: [lwip-users] losing input packets and repeated acks,
Carl D. Blake <=
- Re: [lwip-users] losing input packets and repeated acks, Jonathan Larmour, 2007/01/15
- Re: [lwip-users] losing input packets and repeated acks, Carl D. Blake, 2007/01/15
- Re: [lwip-users] losing input packets and repeated acks, Carl D. Blake, 2007/01/16
- Re: [lwip-users] losing input packets and repeated acks, Carl D. Blake, 2007/01/16
- Re: [lwip-users] losing input packets and repeated acks, Jonathan Larmour, 2007/01/16
- Re: [lwip-users] losing input packets and repeated acks, Carl D. Blake, 2007/01/16