Re: [lwip-users] Assertion "pcb->snd_queuelen >= pbuf_clen(next->p)" SOLUTION

[Tomas Mudrunka]

> i have written my own netif driver on FreeRTOS+LWIP.
> Pings work reliably, TCP handshake works reliably,
> but once i start sending data to the established TCP connection it
> crashes on this:
>
> assertion "pcb->snd_queuelen >= pbuf_clen(next->p)" failed: file
> "/opt/Espressif/esp-idf/components/lwip/lwip/src/core/tcp_in.c", line
> 1112, function: tcp_free_acked_segments


OK! Now everything works great. It way my fault, i am sorry to bother 
this mailing list,
but perhaps this thread will be useful for someone triggering this 
assetion in the future.
Perhaps we can make this more clear in documentation and maybe add some 
comment to the assertion to give hints about what might be causing it.

I've figured out what was going on, so i will describe it here.
Despite the assertion failing in tcp_in.c, the bug was actualy in the 
output part of driver.
My driver was corrupting output packet pbufs.

For most protocols this was OK, because LWIP actualy free()s the packet 
immediately and noone knows if something is wrong. But TCP is different. 
It keeps the packet even after being transmited in pcb->snd_buf and it 
holds it in there until the receival is ACKed by the other host.
So everytime packet arrives in tcp_in, the tcp_free_acked_segments() 
does check all packets in that buffer to see if they were succesfully 
transported and therefore can be freed. But i've modified the packets in 
the meantime, so i've broken this mechanism.

This is what i was doing. I've calculated ethernet frame checksum (FCS 
CRC32) and used pbuf_cat() to append it to the end of that pbuf before 
passing it to TX queue, because i didn't wanted the actual communication 
task to loose time calculating CRC32, since it has critical timing 
requirements. I didn't knew that someone was gonna be working with these 
packets further (except for calling pbuf_free() on them)

  low_level_output(struct netif *netif, struct pbuf *p) {

  pbuf_ref(p);
  //p = pbuf_coalesce(p, PBUF_RAW_TX);
  if(p->next != NULL)
    ESP_LOGE(TAG, "Packet pbuf not coalesced before CRC!");
  //4 additional bytes for FCS CRC32
  fcs = pbuf_alloc(PBUF_RAW_TX, FCS_LEN, PBUF_RAM);
  if(!fcs) {
    pbuf_free(p);
    ESP_LOGE(TAG, "No memory for TX FCS!");
    return ERR_MEM;
  }
  //Calculate CRC32
  *((uint32_t *)(fcs->payload)) = ethernet_crc32(p->payload, p->len, 0);
  pbuf_cat(p, fcs);
  xQueueSendToBack(mynetif->tx_queue, &p, 0);

  ... }

I have completely fixed this code by using pbuf_copy() to create my own 
pbuf which i can modify all i want without causing any troubles to TCP 
stack. I don't call pbuf_ref() on original packet, because i don't 
reference original pbuf anymore.

  //Alloc with 4 additional bytes for FCS CRC32
  fcs = pbuf_alloc(PBUF_RAW_TX, p->tot_len+FCS_LEN, PBUF_RAM);
  if(!fcs) {
    ESP_LOGE(TAG, "No memory for TX FCS!");
    return ERR_MEM;
  }
  if(pbuf_copy(fcs, p) != ERR_OK) {
    ESP_LOGE(TAG, "Cannot copy TX!");
    return ERR_MEM;
  }
  if(fcs->next != NULL)
    ESP_LOGE(TAG, "Copied packet pbuf not coalesced before CRC!");
  //Calculate CRC32
  *((uint32_t *)(fcs->payload+fcs->tot_len-FCS_LEN)) =
    ethernet_crc32(fcs->payload, fcs->len-FCS_LEN, 0);
  xQueueSendToBack(mynetif->tx_queue, &fcs, 0);

I wonder if there is way to append (or prepend) data to the packet pbuf
without having to ditch the zero copy API using pbuf_ref(),
so i can save some RAM. To implement my original logic correctly.

Anyway... This seems to work quite well for me now.
Thank you guys for checking this with me!

--
S pozdravem
Best regards
     Tomáš Mudruňka - SPOJE.NET s.r.o.