[lwip-users] Queued segments not being sent

[Bill Auerbach]

With the other problems fixed, which were causing every segment to be flushed, I now see no segments except for the first one being sent:

 

Here’s a cut of my debug serial output:

 

Initializing Ethernet, waiting on link...OK                                    

Ethernet link: 1GbS                                                            

Using DHCP to acquire IP address                                               

Waiting for interface to come up...OK                                          

IP address: 192.168.123.134                                                    

Available dynamic memory: 111MB                                                 

tcp_enqueue(pcb=0x00403284, arg=0x00000000, len=0, flags=12, apiflags=0)       

tcp_enqueue: queueing 6550:6551 (0x12)                                         

tcp_output_segment: 6550:6550                                                   

192.168.123.115 connected to port 58086                                        

tcp_output: nothing to send (0x00000000)                                       

tcp_output: nothing to send (0x00000000)                                        

tcp_output: sending ACK for 3726251674                                         

tcp_output: nothing to send (0x00000000)                                       

calling tcp_write: 0x0108f278, len: 8                                           

tcp_write(pcb=0x00403284, data="" len=8, apiflags=0)                  

tcp_enqueue(pcb=0x00403284, arg=0x0108f278, len=8, flags=0, apiflags=0)        

tcp_enqueue: queueing 6551:6559 (0x0)                                          

tcp_output_segment: 6551:6559                                                  

calling tcp_write: 0x0108fac8, len: 10                                         

tcp_write(pcb=0x00403284, data="" len=10, apiflags=0)                 

tcp_enqueue(pcb=0x00403284, arg=0x0108fac8, len=10, flags=0, apiflags=0)       

tcp_enqueue: queueing 6559:6569 (0x0)                                          

calling tcp_write: 0x010902cc, len: 12                                         

tcp_write(pcb=0x00403284, data="" len=12, apiflags=0)                 

tcp_enqueue(pcb=0x00403284, arg=0x010902cc, len=12, flags=0, apiflags=0)       

tcp_enqueue: queueing 6569:6581 (0x0)                                          

tcp_enqueue: chaining segments, new len 22                                     

calling tcp_write: 0x01090ad0, len: 11                                         

tcp_write(pcb=0x00403284, data="" len=11, apiflags=0)                 

tcp_enqueue(pcb=0x00403284, arg=0x01090ad0, len=11, flags=0, apiflags=0)       

tcp_enqueue: queueing 6581:6592 (0x0)                                          

tcp_enqueue: chaining segments, new len 33                                     

 

<snip lots of small tcp_write calls>

 

calling tcp_write: 0x010c3464, len: 12                                         

tcp_write(pcb=0x00403284, data="" len=12, apiflags=0)                 

tcp_enqueue(pcb=0x00403284, arg=0x010c3464, len=12, flags=0, apiflags=0)       

tcp_enqueue: queueing 7696:7708 (0x0)                                          

tcp_enqueue: chaining segments, new len 1149                                   

calling tcp_write: 0x010c3c68, len: 8                                           

tcp_write(pcb=0x00403284, data="" len=8, apiflags=0)                  

tcp_enqueue(pcb=0x00403284, arg=0x010c3c68, len=8, flags=0, apiflags=0)        

tcp_enqueue: queueing 7708:7716 (0x0)                                          

tcp_enqueue: chaining segments, new len 1157                                   

calling tcp_write: 0x010c446c, len: 13                                         

tcp_write(pcb=0x00403284, data="" len=13, apiflags=0)                 

tcp_enqueue(pcb=0x00403284, arg=0x010c446c, len=13, flags=0, apiflags=0)       

tcp_enqueue: queueing 7716:7729 (0x0)                                          

tcp_enqueue: chaining segments, new len 1170                                   

tcp_output_segment: 6559:7729                                                  

tcp_output: nothing to send (0x00000000)                                       

tcp_output_segment: 6559:7729                                                  

tcp_output: nothing to send (0x00000000)                                       

tcp_output: nothing to send (0x00000000)                                       

tcp_output: nothing to send (0x00000000)                                       

 

The tcp_output_segment at the top is the only thing sent out.  The first tcp_output at the bottom should send the data – there is a pbuf with lots of chained segments.  This line in tcp_output:

 

    /* Stop sending if the nagle algorithm would prevent it

     * Don't stop:

     * - if tcp_enqueue had a memory error before (prevent delayed ACK timeout) or

     * - if FIN was already enqueued for this PCB (SYN is always alone in a segment -

     *   either seg->next != NULL or pcb->unacked == NULL;

     *   RST is no sent using tcp_enqueue/tcp_output.

     */

    if((tcp_do_output_nagle(pcb) == 0) &&

      ((pcb->flags & (TF_NAGLEMEMERR | TF_FIN)) == 0)){

      break;

    }

 

Takes the break out of the loop, and this line following:

 

    pcb->unsent = seg->next;

 

appears to overwrite pcb->unsent, which has the data that has not been sent.

 

This outgoing data is easy to duplicate – it’s the output of stats_display which I’ve directed to an Ethernet connection which is serving as a terminal window.  I have LWIP_PLATFORM_DIAG defined to EthPrintf, which formats to a malloced string and passes it to a function which builds a linked list – if the list is empty, tcp_write is called with the first item to prime pulling the data.  Tcp_sent grabs each item off the list frees it and calls tcp_write with the next item on the list until the list is empty.  The idea is, if I keep printing faster than data sends, I just add to the linked list.  The stats_display is 100+ of these calls, and they are all queued correctly (and above the total length is still < MTU).  I expected to see one burst of all of this data.  I went from seeing every segment (because of the tcp_output_nagle in tcp_sent) to seeing nothing but the first.

 

Bill