Re: [lwip-users] multiple network interfaces

[Leon Woestenberg]

Hello Jim

Jim Gibbons wrote:

> Thank you for your prompt reply. 
>
> I agree with your assessment that my problem(s) could be resolved by 
> inclusion of the netif as part of the pcb.  It would also appear that an 
> some kind of ioctl call would be required for this purpose, so that the 
> netif could be associated with the pcb prior to its first use for 
> outbound transmission.  Another possibility would be to (optionally) 
> perform the netif association in the connect call.
I would like to see the semantics of the calls being equal to that of
the well-known BSD approach. I think a socket ioctl call works best,
also keeping in mind future work.

> Kieran Mansley also wrote quite promptly, suggesting that an "add route" 
> call, and a related routing table might provide a solution to some of my 
> problems, though I don't think it would address the DHCP issue.  I would 
> be interested to hear your thoughts on his suggestion (and his thoughts 
> on your suggestions).
Excellent suggestion by Kieran.

What I would like to add, is that the routing table not only must 
specify the destination gateway address, but also the destination
network interface (optionally).

The routing code should only select the netif when not specified in
the routing table.

This then, works hand-in-hand with attaching PCB's to a particular netif.

Correct, this does not solve the DHCP issue. But the socket (or
pcb) ioctl does solve that.

> Your point about possibly looking for a larger stack is also well 
> taken.  I'm emotionally invested in lwIP, but that is a lousy way to 
> make such a decision.  In the short term, however, I really don't have 
> the option of making such a change.
No problem. It is just that it is a tiny piece of work to be done. I
am sure some developers want to assist in adding new features that
make sense. I do.

> I didn't understand the point about netif in ARP.  Was this to avoid 
> possible ambiguity in the event of connection to two separate private 
> subnets?  Or is there some other issue?
Yes, strictly speaking an ARP cache is network interface specific and
there is one for each network interface.

Currently, we have one ARP cache table and this works because the 
hardware addresses are supposed to be unique. Also, we do not expect
people to hook up two identical subnetwork ranges on two netifs.

Table lookups become twice as slow when we keep a single table for
two network interface.

> To summarize, I think it would be good (for me) to have all of the 
> following:
>
> 1. Netif pointer in the udp/tcp pcb, NULL initially.
> 2. ip_route called only if netif pointer in pcb is NULL.
> 3. udp_connect_if and tcp_connect_if (or ioctl) to specify netif for a 
> connection

This must be  udp_pcb_ioctl() as the operation is an argument to ioctl.
(i.e. "connect to link" in this case).

> 4. ability to listen for udp or tcp (prior to connection) on all netif's
>
Please elaborate. Do you mean match incoming packets with destination 
address IP_ADDR_ANY when pcb->netif == NULL? Then yes, I agree.

Also, match incoming packets from netif x *only* with destination 
address IP_ADDR_ANY iff pcb->netif == x.

> 5. fill in netif upon connection when listening as described in 4. above
> 6. netif record keeping in ARP to avoid ambiguity
>
> This does not go as far as the routing table that Kieran has suggested, 
> but I think it would deal with the problems I face currently.  Do you 
> think that this would be of general value, or should I consider this to 
> be a private need?
Of general value, but I think it must be another LWIP_PCB_IOCTL option,
i.e. coditionally compiled code.

If you plan working on this, you are welcome to join the CVS team.

Regards,

Leon Woestenberg.

> Leon Woestenberg wrote:
>
> > Hello Jim,
> >
> > thanks for your input.
> >
> > Yes, I think you are right that lwIP needs enhancements in these
> > areas. It should not be hard to add this though, the structures
> > are in place and the code is there (tm).
> >
> > ip_route() wants to see the destination address on any of the local
> > networks, or otherwise will select the default gateway.
> >
> > The routing code currently has no interfaces to add or remove routes
> > with their own gateway addresses. So this is work to be done.
> >
> > Regarding sending DHCP requests: You want to direct them to a single
> > interface (which is not configured yet, so non-selectable by its
> > IP address). This would need extra info, probably in the UDP PCB's
> > to be carried. We probably then would have a udp_ioctl() call that
> > attaches the PCB to a particular network link.
> >
> > Also the ARP table must be enhanced to also store the interface in
> > the table. (Or, have a per-interface table).
> >
> > Other areas in which lwIP may scale upwards is support for bonding,
> > bridging, firewalling, IGMP, multicast routing etc.
> >
> > However, there is a point where you are better of selecting a full
> > BSD (based) TCP/IP stack over lwIP, as the goals of the stacks are
> > different.
> >
> > Regards,
> >
> > Leon. 
> >
> >
> > On Wed, 2004-06-09 at 01:49, Jim Gibbons wrote:
> >  
> >
> > > We are about to add a enable a second interface on a system running
> > > lwIP.  I am concerned about DHCP and udp/tcp routing issues.
> > >
> > > The system in question is a security product that will sit between an
> > > unsecured external network having public servers and a secured 
> > > internal network.  The internal network may be of significant size,
> > > and may itself contain multiple subnets connected by routers.  The
> > > external network is exposed to the internet as a whole.
> > >
> > > My concern focuses on ip_route in ip.c, and upon the protocols that
> > > use it.  ip_route looks for a netif with a subnet that encompasses the
> > > destination IP address.  If it finds one, then that one is returned. 
> > > If it doesn't, then netif_default is returned.  This obviously works
> > > well for a single netif, but I'm not so sure about multiple
> > > interfaces. 
> > >
> > > For example, DHCP uses udp_send to transmit its discovery packet.  The
> > > destination address is the IP broadcast address, 255.255.255.255. 
> > > udp_send in its turn relies on ip_route to find the interface for
> > > sending the packet.  Since the convention for unassigned netif's is to
> > > set the address, gateway and mask to 0, any of the netif's with
> > > unassigned IP addresses could be selected by ip_route.  Thus the first
> > > netif in the list with an unassigned address will be returned.  I
> > > think that this would then lead to all of the DHCP discovery packets
> > > going out the same interface.
> > >
> > > It seems that related problems could also occur for UDP and TCP
> > > transmissions.  In general, one would have to have routing tables of
> > > some kind to do a really good job.  I certainly wouldn't want lwIP to
> > > go that far.  In our case, though,  we are always responding to
> > > external transmissions.  For example, we listen on a TCP port and only
> > > respond after a connection is initiated from outside our box.  So it
> > > would seem to make sense to just use the netif that carried the
> > > inbound traffic.   But this doesn't appear to be the way that UDP and
> > > TCP transmissions work.  Instead, they seem to be using ip_route to
> > > find the destination netif, and ip_route looks only at the destination
> > > IP address.  If it isn't on the subnet for one of the netif's then
> > > netif_default will be used.  I guess this is the same as saying that
> > > lwIP can only use one gateway.  I hope to use more than one.
> > >
> > > Again, I may be missing something in the code.  I hope I am.  Are my
> > > concerns about multiple netif's groundless, or is there really
> > > something to be worried about?
> > >
> > >
> > > --
> > > Jim Gibbons
> > > address@hidden
> > > Gibbons and Associates, Inc.
> > > TEL: (408) 984-1441
> > > 900 Lafayette, Suite 704, Santa
> > > Clara, CA
> > > FAX: (408) 247-6395
> > >
> > >
> > >
> > > ______________________________________________________________________
> > > _______________________________________________
> > > lwip-users mailing list
> > > address@hidden
> > > http://lists.nongnu.org/mailman/listinfo/lwip-users
> > >    
>
> --
> Jim Gibbons
>         address@hidden
> Gibbons and Associates, Inc.
>         TEL: (408) 984-1441
> 900 Lafayette, Suite 704, Santa Clara, CA
>         FAX: (408) 247-6395