We have a problem where a computer connected to one node cannot connect to another node on the mesh, or any nodes "behind" that node.
The details: The lan port of node A is connected to a router and then to a local lan: 192.168.4.0. Hosts on this lan can see node A and connect (I.e. pull up their status pages) to all other nodes on the mesh EXCEPT node B and any other nodes whose primary route passes through node B.
Here is the routing table from node A:
Destination Gateway Genmask Flags Metric Ref Use Iface
default 1998-Router 0.0.0.0 UG 0 0 0 eth0
10.0.0.0 * 255.0.0.0 U 0 0 0 eth0.2
10.0.0.0 * 255.0.0.0 U 0 0 0 wlan0
10.224.74.96 * 255.255.255.240 U 0 0 0 eth0
192.162.4.0 1998-Router 255.255.255.0 UG 0 0 0 ethic
and here is the routing table from node B
Destination Gateway Genmask Flags Metric Ref Use Iface
default 192.168.3.23 0.0.0.0 UG 0 0 0 eth0.1
10.0.0.0 * 255.0.0.0 U 0 0 0 eth0.2
10.0.0.0 * 255.0.0.0 U 0 0 0 wlan0
10.16.166.176 * 255.255.255.248 U 0 0 0 eth0
192.168.3.0 * 255.255.255.0 U 0 0 0 eth0.1
Note node A is not configured as a gateway, node B is configured as a gateway. Nodes, other than node B and those behind it, can ping 192.168.4.1.
Installing a static route for 192.168.4.0 from node B to node A's wifi or lan interface does nothing. Turning off the default route on node B does nothing.
Is this problem a by-product of node B offering a gateway? If so, what can be done to avoid this problem? If that is not the issue, where should we look?
Need support data from the Administration page on both A and B nodes to remotely diagnose. Take a look at "cat /tmp/run/hosts_olsr" to see if the route is there. This is the dynamically olsr maintained route table in use.
Update thought: In linux, "policy routing" is in use on the AREDN node. This means there are more routing tables that apply. A given route table can be assigned based on criteria, e.g. what interface the packet is coming in on. To see what routing is actually occurring requires navigation of the policies through the various tables to find the right route that applies.
Q: did you use any other routes? and were did you apply them.
Q: on Node A in the Basic setup / Lan , did you select NAT and set it to the 192.168.4.0.block? (Ive had problems not using the 1, 5 ,and 13host settings) I dont know what setting NAT dose on the back end It might not be producing the proper Routes.
I am not doing NAT on the node, it is set to 13 local lan addresses. That network connects to a router where it is routed to 192.168.4.0. So, the pac kets show a source IP of 192.168.4.x.
"The lan port of node A is connected to a router and then to a local lan: 192.168.4.0"
wb6tae, I missed this first round. This isn't going to work without a lot of very custom routing table changes, olsr custom advertisements, and DHCP changes. The out-of-box firmware does not support connecting the LAN port of a node to another network with a different address space. This is in conflict, e.g. which network would a device get an address from as both your 192.x.x.x network and the mesh node LAN network are trying to issue and both have DHCP servers responding (or is the 'router' here trying to bridge the gap and it's not just a switch?). Regardless, the mesh routing mechanizm doesn't know how to route to this other network--custom "HNA" routes and non-trivial firewall forward settings would have to be made.
In the current AREDN design, another network only connects on the WAN interface of the mesh node. The mesh node then expects to obtain an IP address from this other network and it is configure to advertise this 'gateway' so other mesh nodes can select it, if lowest cost ETX path.
If you want to connect a router to the LAN port it needs to be doing NAT.
Basically it sound like your throwing packets out into the mesh with a source of 192.168.4.0 and the mesh (rightfully so as this is OUTSIDE our address scope) doesn't know how to get it back to you.
Sounds like we might need to lock down the firewall more in the future to make sure packets on the LAN port are sources from the LAN subnet.... So far we have had an "anything on the lan port is permitted to go to WAN or MESH" policy might have to consider tighting that to keep stray packets off the mesh. Though that might be overkill.
Joe, Conrad: what both of you write makes complete sense, and is how I would assume it should operate. However, my setup, as described, works for all but one node on this mesh. I have been able to connect to approximately 30 nodes via this setup. (And, for some reason, from all of those nodes, I can ping hosts on the 192.168.4.0 net) It is just the one node, and those that route through it I cannot reach. In this case, that seems to be the anomaly.
So, yes, I can, should, and ultimately will, switch to using NAT on the lan port. But, I would really like to understand what is different about that one node and why it works at all on the other nodes.
If I had to guess. It's only worked so long as Node B was a Gateway and that detail was propagated across the network. What you were likely seeing is the remote nodes trying to route back to node B via node A and I'm betting because of the changes you made NODE A routed it back to your PC. Aka it was just dumb luck that Node A happend to be in the path to get to Node B and that it never really actually worked.
Thanks Conrad. Your explanation makes sense, more-or-less. Though, I'd swear it worked before node B was installed. Oh well.
Anyway, the resolution. I had not really wanted to use NAT on the lan port because that makes it a more difficult to add local services. The ease of the sub-net+DHCP is nice. So, I did what I should have done at the outset: On the router I just turned on NAT for the port connecting to the local node's network. I am using a "real" router (Ubiquity Edgerouter-PoE) so it was pretty easy to setup.
Everything is now working exactly as I wanted. I can access the entire mesh from any computer on my local (home) lan. But, I am still able to have NAT+DHCP on the node's lan port and do not have to offer a gateway to the mesh
I'm still looking how to "merge" the entire "local.lan" mesh inside my home LAN configured as "192.168.1.0/24" so to be able to reach every mesh node from every device.
I am using a "real" router (Ubiquity Edgerouter-PoE) so it was pretty easy to setup.
Everything is now working exactly as I wanted. I can access the entire mesh from any computer on my local (home) lan. But, I am still able to have NAT+DHCP on the node's lan port and do not have to offer a gateway to the mesh.
This is a pretty good news!!!
Can you write an HOW-TO about this useful configuration ???
WB6TAE, While designing a way to connect another network to the LAN of a mesh node, I'm wondering if we've missed the boat with what it is you are trying to do to begin with--there may be an alternative design that works better for the services you intend to deploy, e.g. VOIP has challenges through a NAT.
Taking a guess, is the original problem such that you want the devices to have 1) access to the mesh; 2) access to your home network; and 3) access to the Internet, but 4) you do not want to give others on the mesh ability to use your internet gateway?
You can meet these requirements by plugging these devices into the LAN of a mesh node, connect your home network on the WAN port of the mesh node and do NOT check the advertise gateway box. The devices still have access to internet, with no RF, and others can not use your internet gateway.
The trade off is where you want the NAT to be; A) between your devices and the mesh or B) between your devices and the home network/internet. Could do both options depending on what services a given computer is providing/using. Just make sure if there is a NAT on the LAN side of a mesh node that it is doing IP masq (or SNAT) so that all reply traffic from a mesh devices goes back to the 10.x.x.x address and not the 192.168.x.x. address.
The solution I found is basically what you describe in the last paragraph: I want to be able to access any node or service on the mesh network (10.0.0.0/8) from my home lan (192.168.4.0/24). I also want to offer services to the mesh from my node's lan port (10.x.x.x/ 28) and have dhcp on that subnet. My solution was to NAT/Masq traffic from my home lan to the mesh lan subnet).
So, to give a picture to others... on my home gateway router I have three interfaces: 1) My ISP; 2) my home LAN; 3) the mesh lan (local lan subnet) - this router interface has an IP address from the node's lan subnet. I essentially treat both interface 1 and 3 as wide-area links and NAT between them and my local lan. I then have a route for 10.0.0.0/8 pointing to interface 3 and a default route (0.0.0.0) pointing to interface 1. Most home routers only have 2 ports, wan and lan and, therefore, this approach would not work.
As I noted above, for offering local services to the mesh, there are advantages to not using NAT on the node's lan interface. I could have used the node's wan interface, but that would require setting up VLAN support, and would not have provided any benefit.
I do have one question on your post, in reference to the functionality of the "advertise gateway" box. I thought that box controlled whether the lan dhcp service included the lan port's IP address as the default gateway setting to dhcp clients.
The 'advertise gateway' does not impact the default route to LAN devices (otherwise your devices could not have an option to access other internet gateways on the mesh network). There is a separate check box under the LAN section in setup to "disable default route" (or not). This is where you indicate if the LAN device should receive a default route or not (back to the mesh node it is attached to). The default set of IP addresses is basically any address that is not in the OLSR advertised hosts table (the IPs of the mesh nodes on the network) or HNA subnets (all the LAN subnets across the mesh) advertised. This default is back to the mesh node the device is attached to, then the mesh node routes to whoever it selects as the default in the direction of the lowest cost gateway (or itself if WAN port is live).
We have a problem where a computer connected to one node cannot connect to another node on the mesh, or any nodes "behind" that node.
The details: The lan port of node A is connected to a router and then to a local lan: 192.168.4.0. Hosts on this lan can see node A and connect (I.e. pull up their status pages) to all other nodes on the mesh EXCEPT node B and any other nodes whose primary route passes through node B.
Here is the routing table from node A:
and here is the routing table from node B
Note node A is not configured as a gateway, node B is configured as a gateway. Nodes, other than node B and those behind it, can ping 192.168.4.1.
Installing a static route for 192.168.4.0 from node B to node A's wifi or lan interface does nothing. Turning off the default route on node B does nothing.
Is this problem a by-product of node B offering a gateway? If so, what can be done to avoid this problem? If that is not the issue, where should we look?
Need support data from the Administration page on both A and B nodes to remotely diagnose. Take a look at "cat /tmp/run/hosts_olsr" to see if the route is there. This is the dynamically olsr maintained route table in use.
Update thought: In linux, "policy routing" is in use on the AREDN node. This means there are more routing tables that apply. A given route table can be assigned based on criteria, e.g. what interface the packet is coming in on. To see what routing is actually occurring requires navigation of the policies through the various tables to find the right route that applies.
The problem is probably not in olsr. If I connect a computer to node A directly (I.e. on the nodes local lan) I can connect to and through node B.
I agree on the "policy routing." Do you know the command to display the tables that are present?
Q: did you use any other routes? and were did you apply them.
Q: on Node A in the Basic setup / Lan , did you select NAT and set it to the 192.168.4.0.block? (Ive had problems not using the 1, 5 ,and 13host settings) I dont know what setting NAT dose on the back end It might not be producing the proper Routes.
I am not doing NAT on the node, it is set to 13 local lan addresses. That network connects to a router where it is routed to 192.168.4.0. So, the pac kets show a source IP of 192.168.4.x.
"The lan port of node A is connected to a router and then to a local lan: 192.168.4.0"
wb6tae, I missed this first round. This isn't going to work without a lot of very custom routing table changes, olsr custom advertisements, and DHCP changes. The out-of-box firmware does not support connecting the LAN port of a node to another network with a different address space. This is in conflict, e.g. which network would a device get an address from as both your 192.x.x.x network and the mesh node LAN network are trying to issue and both have DHCP servers responding (or is the 'router' here trying to bridge the gap and it's not just a switch?). Regardless, the mesh routing mechanizm doesn't know how to route to this other network--custom "HNA" routes and non-trivial firewall forward settings would have to be made.
In the current AREDN design, another network only connects on the WAN interface of the mesh node. The mesh node then expects to obtain an IP address from this other network and it is configure to advertise this 'gateway' so other mesh nodes can select it, if lowest cost ETX path.
Joe AE6XE
If you want to connect a router to the LAN port it needs to be doing NAT.
Basically it sound like your throwing packets out into the mesh with a source of 192.168.4.0 and the mesh (rightfully so as this is OUTSIDE our address scope) doesn't know how to get it back to you.
Sounds like we might need to lock down the firewall more in the future to make sure packets on the LAN port are sources from the LAN subnet.... So far we have had an "anything on the lan port is permitted to go to WAN or MESH" policy might have to consider tighting that to keep stray packets off the mesh. Though that might be overkill.
Joe, Conrad: what both of you write makes complete sense, and is how I would assume it should operate. However, my setup, as described, works for all but one node on this mesh. I have been able to connect to approximately 30 nodes via this setup. (And, for some reason, from all of those nodes, I can ping hosts on the 192.168.4.0 net) It is just the one node, and those that route through it I cannot reach. In this case, that seems to be the anomaly.
So, yes, I can, should, and ultimately will, switch to using NAT on the lan port. But, I would really like to understand what is different about that one node and why it works at all on the other nodes.
If I had to guess. It's only worked so long as Node B was a Gateway and that detail was propagated across the network. What you were likely seeing is the remote nodes trying to route back to node B via node A and I'm betting because of the changes you made NODE A routed it back to your PC. Aka it was just dumb luck that Node A happend to be in the path to get to Node B and that it never really actually worked.
Thanks Conrad. Your explanation makes sense, more-or-less. Though, I'd swear it worked before node B was installed. Oh well.
Anyway, the resolution. I had not really wanted to use NAT on the lan port because that makes it a more difficult to add local services. The ease of the sub-net+DHCP is nice. So, I did what I should have done at the outset: On the router I just turned on NAT for the port connecting to the local node's network. I am using a "real" router (Ubiquity Edgerouter-PoE) so it was pretty easy to setup.
Everything is now working exactly as I wanted. I can access the entire mesh from any computer on my local (home) lan. But, I am still able to have NAT+DHCP on the node's lan port and do not have to offer a gateway to the mesh
I'm still looking how to "merge" the entire "local.lan" mesh inside my home LAN configured as "192.168.1.0/24" so to be able to reach every mesh node from every device.
This is a pretty good news!!!
Can you write an HOW-TO about this useful configuration ???
WB6TAE, While designing a way to connect another network to the LAN of a mesh node, I'm wondering if we've missed the boat with what it is you are trying to do to begin with--there may be an alternative design that works better for the services you intend to deploy, e.g. VOIP has challenges through a NAT.
Taking a guess, is the original problem such that you want the devices to have 1) access to the mesh; 2) access to your home network; and 3) access to the Internet, but 4) you do not want to give others on the mesh ability to use your internet gateway?
You can meet these requirements by plugging these devices into the LAN of a mesh node, connect your home network on the WAN port of the mesh node and do NOT check the advertise gateway box. The devices still have access to internet, with no RF, and others can not use your internet gateway.
The trade off is where you want the NAT to be; A) between your devices and the mesh or B) between your devices and the home network/internet. Could do both options depending on what services a given computer is providing/using. Just make sure if there is a NAT on the LAN side of a mesh node that it is doing IP masq (or SNAT) so that all reply traffic from a mesh devices goes back to the 10.x.x.x address and not the 192.168.x.x. address.
Joe AE6XE
Joe:
The solution I found is basically what you describe in the last paragraph: I want to be able to access any node or service on the mesh network (10.0.0.0/8) from my home lan (192.168.4.0/24). I also want to offer services to the mesh from my node's lan port (10.x.x.x/ 28) and have dhcp on that subnet. My solution was to NAT/Masq traffic from my home lan to the mesh lan subnet).
So, to give a picture to others... on my home gateway router I have three interfaces: 1) My ISP; 2) my home LAN; 3) the mesh lan (local lan subnet) - this router interface has an IP address from the node's lan subnet. I essentially treat both interface 1 and 3 as wide-area links and NAT between them and my local lan. I then have a route for 10.0.0.0/8 pointing to interface 3 and a default route (0.0.0.0) pointing to interface 1. Most home routers only have 2 ports, wan and lan and, therefore, this approach would not work.
As I noted above, for offering local services to the mesh, there are advantages to not using NAT on the node's lan interface. I could have used the node's wan interface, but that would require setting up VLAN support, and would not have provided any benefit.
I do have one question on your post, in reference to the functionality of the "advertise gateway" box. I thought that box controlled whether the lan dhcp service included the lan port's IP address as the default gateway setting to dhcp clients.
Richard
The 'advertise gateway' does not impact the default route to LAN devices (otherwise your devices could not have an option to access other internet gateways on the mesh network). There is a separate check box under the LAN section in setup to "disable default route" (or not). This is where you indicate if the LAN device should receive a default route or not (back to the mesh node it is attached to). The default set of IP addresses is basically any address that is not in the OLSR advertised hosts table (the IPs of the mesh nodes on the network) or HNA subnets (all the LAN subnets across the mesh) advertised. This default is back to the mesh node the device is attached to, then the mesh node routes to whoever it selects as the default in the direction of the lowest cost gateway (or itself if WAN port is live).
Right. Yup, I confused the two settings. Thanks for clarifying.