[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[lwip-users] Non-blocking netconn callback and TCP accept backlog
From: |
Nick Brereton |
Subject: |
[lwip-users] Non-blocking netconn callback and TCP accept backlog |
Date: |
Sat, 14 Jul 2012 17:24:47 +0100 |
Hi,
I'm currently attempting to use lwIP's non-blocking netconn interface to
implement an http based service. I am having trouble with new incoming
connections, where if I delay accepting the connection from my listening
connection, I seem to lose some of the data sent. I won't claim to be an expert
at networking, or of lwIP so I may well have mis-configured something or have
otherwise misunderstood how things should work. I would appreciate any hints or
help that you can put my way about this problem.
To give a little background, I'm currently attempting to build a SOAP based
server for a device that uses lwIP for networking (and, probably not relevant,
but using FreeRTOS). I need to be able to handle multiple connection
concurrently, some of which can take a significant amount of time to process;
because of memory constraints I can only handle a small number of active
connections at any one time (where small is less than the number of requests
that could be outstanding). As alluded to, I have a listening connection (port
80) that has been set to non-blocking mode, and I have a callback that is used
to notify the main event loop of events that occur from the netconn events. I
have configured the TCP_LISTEN_BACKLOG option.
When I get get a NETCONN_EVT_RECVPLUS event with 0 length for the listening
connection I add this to a queue of accept event, which I then drain whenever I
have memory available to process the connection. As I have understood it data
should not start flowing until I have accepted the connection (presumably
completion of the handshake should be delayed until the netconn_accept call,
thus preventing the client from sending any data?) and the connection should be
sitting in the tcp listen queue.
What happens is that the connection is immediately accepted (presumably when
the callback occurs?) and the client starts sending data immediately. But when
I do accept the connection, there is never any data to retrieve with the
netconn_recv call (no events & conn->recv_avail == 0) -- ie the data has
effectively been lost. Of course at this point my server stops doing anything
useful as it is expecting to receive data before it can do anything.
Poking through the lwip_stats structure, after I seem to have lost the data,
the tcp stats reports that all of the data packets I can see with wireshark
have been received without error (in this case recv=52, sent=42) and the memp
stats tell me that there are no allocated buffers (i.e. there are no orphan
packets that might contain my missing data). The tcp_active_pcb list contains
established connections for all current and outstanding (backlogged)
connections on my listening connection.
I have attached a wireshark pcap log of the network traffic (where the lwip
server is at 192.168.7.1 and the client is 192.168.7.2); connections from
client's port 57221, 57222, 57228, 57229 complete correctly; connections from
client ports 57230 and 57231 have been netconn_accept'ed only after 57228 and
57229 have been closed and think that they are waiting for the first data
packet (i.e. a HTTP request line) and the connection from client port 57232 has
never had netconn_accept called for it. As noted lwip_stats suggests that all
of the packets have been accepted, but where there is no application data
response packets the incoming data seems to have been 'lost' by lwIP, although
no error is indicated.
My code looks something like this (callback & main loop); simplified of course
but should give the key points:
void callback(struct netconn *pconn, enum netconn_evt event, u16_t len)
{
switch(event) {
case NETCONN_EVT_RCVPLUS:
if(len > 0) {
HTTPD_POST_MESSAGE(pconn, len, HTTPD_CMD_READRDY);
} else if(pconn == httpd) {
/* queue message noting that a connection is requested, will accept
when ready */
} /* otherwise this is caused by closing the socket ... */
break;
case NETCONN_EVT_RCVMINUS: break; /* not useful? */
case NETCONN_EVT_SENDPLUS:
if(len > 0) /* otherwise doesn't seem to be useful */
HTTPD_POST_MESSAGE(pconn, len, HTTPD_CMD_TXDONE);
break;
case NETCONN_EVT_SENDMINUS: break; /* not useful? */
case NETCONN_EVT_ERROR: HTTPD_POST_MESSAGE(pconn, len, HTTPD_CMD_ERROR);
break;
}
}
void main_loop(void* param)
{
/* setup a non-blocking listening connection on port 80 */
httpd = netconn_new_with_callback(NETCONN_TCP, callback);
netconn_set_nonblocking(httpd, 1);
netconn_bind(httpd, 0, HTTP_PORT);
netconn_listen(httpd);
while(1) {
/* get events (network, timers etc) */
switch(event) {
case HTTPD_CMD_READRDY:
netconn_recv(...);
/* do stuff with data, free buffers etc.*/
break;
case HTTPD_CMD_READRDY:
/* send more data on connection, or close */
break;
case HTTPD_CMD_TIMER:
if(memory_available() && accept_pending()) {
netconn_accept(httpd, &new_connection);
/* associate new_state & new_connection, assume that we will
get a NETCONN_EVT_RECVPLUS event to kick us off */
}
}
}
}
My lwipopts.h file contains this:
#define BYTE_ORDER LITTLE_ENDIAN
#define TCPIP_THREAD_NAME "tcpip"
#define LWIP_HTTPD_MAX_TAG_NAME_LEN 20
#define LWIP_HTTPD_MAX_TAG_INSERT_LEN 1500
#define TCPIP_THREAD_PRIO (tskIDLE_PRIORITY + 1)
#define TCPIP_THREAD_STACKSIZE configMINIMAL_STACK_SIZE * 4
#define DEFAULT_TCP_RECVMBOX_SIZE 10
#define DEFAULT_ACCEPTMBOX_SIZE 10
#define TCPIP_MBOX_SIZE 10
#define NO_SYS 0
#define LWIP_SOCKET 0
#define LWIP_NETCONN 1
#define RECV_BUFSIZE_DEFAULT 1536
#define LWIP_SNMP 0
#define LWIP_IGMP 0
#define LWIP_ICMP 1
#define LWIP_DNS 0
#define LWIP_HAVE_LOOPIF 0
#define TCP_LISTEN_BACKLOG 1
#define LWIP_SO_RCVTIMEO 1
#define LWIP_SO_RCVBUF 1
#define MEM_ALIGNMENT 4
#define MEM_SIZE 10240
#define MEMP_NUM_PBUF 20
#define LWIP_RAW 0
#define MEMP_NUM_RAW_PCB 0
#define MEMP_NUM_UDP_PCB 2
#define MEMP_NUM_TCP_PCB 40
#define MEMP_NUM_TCP_PCB_LISTEN 2
#define MEMP_NUM_TCP_SEG 20
#define DEFAULT_UDP_RECVMBOX_SIZE 10
#define MEMP_NUM_SYS_TIMEOUT 15
#define MEMP_NUM_NETBUF 10
#define MEMP_NUM_NETCONN 10
#define MEMP_NUM_TCPIP_MSG_API 6
#define MEMP_NUM_TCPIP_MSG_INPKT 6
#define MEMP_NUM_ARP_QUEUE 5
#define PBUF_POOL_SIZE 10
#define PBUF_POOL_BUFSIZE 375
#define PBUF_LINK_HLEN 16
#define SYS_LIGHTWEIGHT_PROT (NO_SYS==0)
#define LWIP_TCP 1
#define TCP_TTL 255
#define TCP_QUEUE_OOSEQ 1
#define TCP_MSS 1460
#define TCP_SND_BUF ( TCP_MSS * 2 )
#define TCP_SND_QUEUELEN (4 * TCP_SND_BUF/TCP_MSS)
#define TCP_SNDLOWAT (TCP_SND_BUF/2)
#define TCP_WND ( PBUF_POOL_SIZE *
(PBUF_POOL_BUFSIZE - (PBUF_LINK_HLEN + PBUF_IP_HLEN + PBUF_TRANSPORT_HLEN)))
#define TCP_MAXRTX 12
#define TCP_SYNMAXRTX 4
#define LWIP_ARP 1
#define ARP_TABLE_SIZE 10
#define ARP_QUEUEING 1
#define IP_FORWARD 0
#define IP_REASSEMBLY 0
#define IP_REASS_MAX_PBUFS 10
#define MEMP_NUM_REASSDATA 10
#define IP_FRAG 0
#define ICMP_TTL 255
#define LWIP_DHCP 0
#define DHCP_DOES_ARP_CHECK (LWIP_DHCP)
#define LWIP_AUTOIP 0
#define LWIP_DHCP_AUTOIP_COOP (LWIP_DHCP && LWIP_AUTOIP)
#define LWIP_UDP 1
#define LWIP_UDPLITE 1
#define UDP_TTL 255
#define LWIP_STATS 1
#define LWIP_STATS_DISPLAY 0
#define PPP_SUPPORT 0
I can't say that I have fully understood how to configure lwIP, so this is
lightly modified from a FreeRTOS example and may not be quite what I need but
it seems to have worked so far.
I should also note that when I have tried using a single loop with a blocking
server I have not lost data, although a simple loop & blocking netconns is not
practical for my use-case.
Many thanks for any help,
Nick
httpd_accept_breaks.pcap
Description: httpd_accept_breaks.pcap
- [lwip-users] Non-blocking netconn callback and TCP accept backlog,
Nick Brereton <=