Re: [lwip-users] Stack corruption with high load tcp raw api

[Erik Ekman]

On 11/14/07, Kieran Mansley <address@hidden> wrote:
> On Wed, 2007-11-14 at 11:22 +0100, Erik Ekman wrote:
> > Any ideas? I am thinking about trying the CVS HEAD, but I think it
> > just is a error in my raw api usage.
>
> The most common explanation for errors of this kind is that there is > 1
> thread concurrently accessing the core of lwIP.  This can include
> application threads, TCP timer threads, and receive packet processing
> driven by interrupts.  lwIP isn't terribly thread safe, so you need to
> be careful about protecting it.  This is improving (and CVS is better
> than 1.2.0 in this respect).  There is some documentation on the wiki
> about what exactly needs to be protected: http://lwip.scribblewiki.com/
>
> Kieran
>
>
>

Hi

I will try to explain how my os works and how I handle interrupts to
hope to prove that I do not need locks :) If I have missed something,
please tell me!

The system (incorrectly named "nosys") is made up of messages, tasks
with queues, a timer service and a scheduler.

A message has a type, a word for data, and a pointer. It also contains
a pointer to the replyqueue if a reply is to be sent. Messages are
preallocated in an array so alloc/free is constant time and can be
done in interrupt context.

Tasks are registered with a function pointer and each given a queue.
The functions should take no arguments and return void. The task is
only called when it has an message in its queue. The message is
retrieved with get_msg() and returned with del_msg().

The timer can send messages to tasks. It is configured with a interval
in ms and is implemented with the system timer on target and with a
thread on Linux.

The scheduler is a very simple loop shown below which calls each task
if it has a message queued. Then after the task has completed the next
task is called, and so on. This means that if the signal_send() (shown
in the first mail) is running, no other lwIP stuff runs. No task is
interrupted and message delivery is asynchronous.

 while(1) {

    for(t = nosys_tasks; t; t = t->next) {
      if(t && t->inq.msg_cnt > 0) {
        if(t->fn) {
          self = t;
          t->fn();
        } else {
          DBG(1, "task %08x has no function!", t);
          ASSERT(0);
        }
      }
    }
    NOSYS_SLEEP;
  }

In Linux, a thread monitors the tap device with a select() call, and
runs low_level_input if a packet shows up. That function calls
pbuf_alloc (this might be risky) and then mempcy()s the packet there.
Then a message with a pointer to the pbuf is queued for the lwip task.

On target (LPC236x), an interrupt detects when a packet has been DMAd
to the ethernet RAM. All the interrupt handler does is send a message
for the lwip task with a pointer to the correct netif interface. The
task will then copy it to a pbuf and give it to the stack.

Both timer interrupts and new packets will be handled the next time
the scheduler comes to the lwip task, so the plan is that no
concurrency will occur.

Hope this makes any sense.

Thanks,

/Erik