[lwip-users] SO_REUSE_RXTOALL doesn't respect bind()

[Jakub Šebek]

Hello,

I am not well versed in lwIP internals, and I figured I would ask here 
instead of having to decipher the source.

The issue is simple. I enable lwIP's SO_REUSE + SO_REUSE_RXTOALL 
options, expecting to reach behaviour that is the norm of UNIX/Linux, 
that is: (from UNIX Network Programming, Volume 1, Page 267)

"SO_REUSEADDR allows completely duplicate bindings: a `bind` of an IP 
address and port, when that same IP address and port are already bound 
to another socket, if the transport protocol supports it. Normally this 
feature is supported only for UDP sockets.

This feature is used with multicasting to allow the same application to 
be run multiple times on the same host. When a UDP datagram is received 
for one of these multiply bound sockets, the rule is that if the 
datagram is destined for either a broadcast address or a multicast 
address, one copy of the datagram is delivered to each matching socket. 
But if the datagram is destined for a unicast address, the datagram is 
delivered to only one socket. [...]"

Of course, I also empirically verified this behaviour on recent Debian 
and FreeBSD virtual machines.

Instead, however, lwIP with these options acts somewhat like this (what 
I was able to observe):

1) Incoming undirected broadcast (255.255.255.255) packets are spread 
across ALL sockets that listen on the port (by virtue of SO_REUSE), and 
not just those bound to the broadcast address. I.e. the broadcast packet 
nonchalantly reaches a socket bound to the unicast (interface) address 
as well as a socket bound to a multicast address.
2) Incoming directed broadcast (192.168.0.255 in my case) packets are 
spread just like in 1) except that they now respect sockets bound to a 
multicast address, and don't reach those.
3) Multicast is well-behaved and works perfectly. (Packets are only 
copied to sockets bound to the corresponding multicast address.)

After first observing just 1), I thought there could be an argument made 
for this behaviour, based on e.g. the often-cited link 
http://www.kohala.com/start/mcast.api.txt, which only mentions ports and 
not bound addresses ("[...] In this case, every incoming multicast or 
broadcast UDP datagram destined to the shared port is delivered to all 
sockets bound to the port. [...]"). But after observing 2) and 3), this 
really just seems like buggy behaviour. One thing I didn't find/look for 
was a corresponding RFC, but I consider "UNIX Network Programming", 
FreeBSD and Linux enough of an authority. (It doesn't help my situation 
that searching the web makes me feel like I'm the first person in 20 
years to attempt using broadcast/multicast functionality.)

So I came to ask, what is happening here? Am I observing bugs, or 
unprecedented features of lwIP?

Thanks everyone
Jakub