[lwip-users] Zero copy receive buffers

[Marc Wells]

I have been working on a project which integrates the 
lwIP code into an embedded system.  The target system 
consists of a Power PC and memory, a Gigabit Ethernet 
controller and some proprietary logic. 

In this application, maximizing throughput and 
minimizing latency are critical.  I was able to use 
most of the lwIP code unmodified but in order to 
take advantage of support provided by the Gigabit 
Ethernet controller some modifications were needed.  
I apologize for the length of this posting but I 
wanted to cover the issue in sufficient detail to 
allow lwIP implementors to take advantage of this 
information.


Packet Buffer Modifications

When an Ethernet packet arrives, lwIP expects the 
Interrupt Service Routine (ISR) to copy the data 
from the Ethernet hardware to a packet buffer (pbuf) 
of type PBUF_POOL or PBUF_RAM.  In the target system, 
however, the Ethernet controller copies the packet 
data to a user specified buffer then initiates an 
interrupt.  By the time the ISR runs, the data has 
already been copied to memory.  In order to minimize 
latency, the data should not be copied again, which 
implies a zero copy input strategy.  A packet buffer 
of type PBUF_ROM could theoretically be used except 
for the function pbuf_header.

Pbuf_header is used to move the 'payload' member of 
the packet buffer structure to selectively include 
or exclude various headers (Ethernet, IP, UDP, TCP, 
etc.) as the packet data moves up or down the stack.  
Since, presumably, pbufs of type ROM contain a 
payload that is Read Only, the pbuf_header function 
returns immediately if it is passed a ROM type pbuf.

To get around this I added a new pbuf type, PBUF_EXT 
(to indicate that the data is external to the packet 
buffer), and modified both pbuf_alloc and pbuf_header.
I also added a new member to the pbuf structure, 
'buffer', which pbuf_alloc sets.

The new enumeration of PBUF_EXT was added to the 
pbuf_type definition and a new flag, PBUF_FLAG_EXT, 
was added to the list of defined flags.  It must be 
noted that the flag must be defined as a unique bit 
to avoid problems when checking the flag.  
Specifically, defining the flag value as 0x03 would 
make it look as if it is both a ROM and POOL type 
pbuf.  All these additions and modifications are 
made in pbuf.h.

Inside the pbuf_alloc function, the new PBUF_EXT 
type is handled identically to the PBUF_ROM type 
except for the setting of the 'flags' member of 
the pbuf.  The new 'buffer' member of the pbuf 
structure is set to the address of the beginning 
of the payload section of RAM and POOL type pbufs.
This value is used by the pbuf_header function for 
error checking.

The changes to pbuf_header are straightforward also.  
Pbufs of type ROM are still rejected.  All other 
types have the payload pointer moved by the amount 
specified in the 'header_size' argument.  

In the original implementation, the modified 
payload pointer was checked to make sure that it 
didn't collide with the pbuf structure.  This was 
based on the fact that the payload sections of RAM 
and POOL pbufs are contiguous with the associated 
PBUF structure.  If the pbuf_header function moved 
the payload pointer too far, it would wind up 
somewhere before the end of the pbuf struct thus 
indicating an error.

Since the external data pbuf can have the payload 
allocated from anywhere in memory, it is no longer 
valid to merely check for the modified payload 
pointer being less than the end of the pbuf struct.

The buffer member of the pbuf struct was added to 
facilitate this error check.  When the payload 
is initially set, either during pbuf allocation or,
as in the case of ROM and EXT type pbufs, after 
the fact, the buffer member is set pointing to 
the start of the payload region.  Subsequent calls 
to pbuf_header will compare the modified payload 
pointer to the buffer pointer.  If the payload 
is ever less than the buffer value, the payload 
pointer has been backed up too far and an error 
has occurred.  

It can be argued that the payload pointer should 
also be checked to make sure it hasn't been set 
beyond the end of the payload:

(u8_t*)(pbuf->payload) >= (u8_t*)(pbuf->buffer) + pbuf->len

However, since the original pbuf_haeder didn't 
contain this check, I didn't add it.

Previously, the payload member of ROM type pbufs 
were set by directly accessing the structure 
member.  Since the new structure requires that two 
members be set, I added a function, 

void pbuf_set_payload (struct pbuf* p, void* buf);

to do this.  The function checks for a pbuf of 
type ROM or EXT and sets both the payload and 
buffer members of the pbuf struct to be the value 
passed in (buf).

The pbuf_realloc and pbuf_free functions were 
updated to check for EXT type pbufs which are 
treated identically to ROM type pbufs.


Application Memory Management

No memory management is required for ROM type 
pbufs since the application memory is assumed 
to be static, read only memory.  The memory 
for the new external pbufs, however, needs to 
be managed by the application.  Since the header 
information is contained in the application 
buffer, it seemed safest to me to manage the 
application buffer and the pbuf structure 
together.

To do this, I increment the pbuf reference count 
immediately when it is allocated by the ISR and 
add the pbuf to a list maintained by the driver 
software.  Since I'm using the raw API, the pbuf 
gets passed to the application code, not just the 
input data.  When the application is finished with 
the pbuf, it calls pbuf_free which decrements the 
reference count but the count should never reach 
zero.  

After each input packet has been processed, I
call a function which scans the list of input pbufs.  
Whenever a pbuf is found with a reference count of 
one, it is assumed that the application is done with 
the data associated with the pbuf and it is finally 
freed by a call to pbuf_free.  If the application is 
very slow in processing the input data, this could 
result in lost packets since there may not be a 
pbuf available when a new packet comes in.  This is 
not expected to be a problem in the target system, 
however.

As has been discussed on the mail list recently, 
specifically with respect to UDP packets, there 
are similar issues associated with zero copy output.

Pbufs of type ROM should be adequate for managing 
the output data but some type of coordination is 
needed between the the application code and the 
the driver code to be sure that pbufs are freed 
appropriately and application memory leaks are 
prevented.  Again, using the raw API facilitates 
this since the application code deals directly 
with the pbufs.


     Marc Wells
     address@hidden
     503-466-7607

marc_wells.vcf
Description: Card for Marc Wells