Re: [lwip-users] lwIP reentrancy issues and TCP

[AjK]

Richard,

>  (although I would pay a small amount for somebody to provide one,
> as per previous discussion).

You probably won't have to pay anyone. I've just completed a new "zero-copy" 
emac/dma driver for the LPC1768. It's "tightly bound" to LWIP (in that it uses 
pbuf_pool for the descriptor target buffers).

There are two compile modes configured by macros.

1) Classic Lwip: user responsible for calling lwip to process queues and timers 
in it's "single thread" user context process.

2) FreeRTOS: uses just the technique you describe. The queues are handled by 
queue messages from the ISR to the lwip task. Timers simply by the lwip task.

Still in beta at the moment but when I am happy I'll be publishing it all as 
foss.

But while I have your attention I'd like to ask a question just so I can get 
this spot on and usable as a good reference design for others.

First up, a brief description of "Classic Lwip" mode (no rtos) which was where 
I 
started out from (I decided to make it FreeRTOS friendly later which means a 
re-factor, it's this re-factor I want to get right to make it truly useful for 
others).

In Classic Lwip mode, lwip pbuf_pool is setup to provide "many small pbufs". In 
my case at the moment, 38 of them all payload sized at 256 bytes. When the emac 
driver inits the rx descriptor arrays (there are 18 rx descriptors currently) 
are each assigned a pbuf_pool pbuf. Once all is setup, the emac and isr is 
enabled.

Now, here's where things get interesting. Although the emac driver is tightly 
bound to lwip (in that is uses lwip memory manager) I wanted my driver to be 
able to handle Ethernet protocols other than just IP and ARP. So my emac driver 
after init actually drops all packets. It's not until a "protocol handler" 
registers itself as "I want these etherType packets" that the driver actually 
starts doing anything useful (the register is done during init).

So, for example, during low_level_init() in the Lwip code it calls a function 
to 
register 0800 and 0806 etherType packets and provides a C function pointer to 
callback to. The callback functions simply take a structpbuf * arg which points 
at the head pbuf in a pbuf chain. The callback then places that into a queue. 
It's the "user process" that then empties this queue into lwip during it's 
normal single user thread mode of operation. Likewise, other protocol handlers 
do the same (in my case I also have an LLDP protocol handler which plays nice 
with lwip pbufs). If the driver has no etherType callbacks registered it drops 
the packet.

So this design means there is "some minimal intelligence" built into the 
emacISR 
handler.

So, moving onto FreeRTOS. Instead of registering a C function callback to call 
in ISR context, the protocol handler (lwip netif, or LLDP, etc) registers an 
etherType number and a Queue into which it just injects the pbuf chain.

Additionally, when the emacISR forms pbuf chains from the descriptor array, 
each 
descriptor is now void of a payload taget. So the ISR makes an attempt to 
replace this lost pbuf with a new one using pbuf_alloc() (using the usual lwip 
protection mechanisms in sys_arch). It may not succeed if all pbufs are in use 
in which case it attempts to "keep up" when other events trigger. 


Anyway, coming onto my question. You've said that in FreeRTOS the ISR should 
"just send a message" and the task should then do the processing. However, that 
really doesn't fit my design as it assumes that the task is going to manage the 
emac descriptor array, forming pbuf_chains, etc etc. So I have two choices:-

1) Leave a minimal amount of intelligence in the ISR to manage things at the 
hardware level and chain forming, queue injection, etc.

2) Create a gatekeeper task that sits behind the ISR that is then responsible 
managing the hardware, dispatching packets into queues, etc etc.

Which is the "preferable design" given that the amount of work that's actually 
being done is quite minimal. Option 2 seems like the "most logical solution" 
but 
given the small amount of code involved option 1 seems like the "most sensible 
solution" given the tight constraints of the uCenv, memory, etc. In addition, 
option 2 probably means more work for me in the Classic Lwip (non-rtos) design. 
Whereas keeping a small but efficient ISR means both designs become simpler to 
implement for both Classic and RTOS versions.

Although I'm fairly new to lwip-users mailing list I have noticed that the work 
I'm doing and going to publish addresses many of the questions I've seen on the 
list. So from a "reference" point of view (ie telling people "go look at this 
for an example of how to do that") I'd like to get it spot on and an example of 
best practice. As, I'm not just going to publish the code, I intend to to use 
the Lwip Wiki to fully document from hardware to examples of Lwip use the whole 
process.

Sorry for the long and boring post, I'm just seeking to make sure I get the 
best 
possible implementation and thereby best possible reference for others to look 
at and pick apart to see how things can be done.

regards,
--Andy