I've read up on using VLANs for segmenting the ethernet connection from a mesh node. I don't have a VLAN capable switch yet. I was looking through the help text and read that if the LAN connection has access to the internet, and that if Mesh Gateway is checked on, that the node will advertise to the mesh that it has internet access.
I tried this but was unsuccessful. Here's what I've got:
Node A is a mesh node with a Direct-5 lan.
Node B is a mesh node with a NAT LAN, ip of 192.168.1.150, mask of 255.255.255.0, and gateway of 192.168.1.1. WAN is disabled, and Mesh Gateway is checked on.
From a workstation on the LAN of Node A, I get a mesh ip address and can ping Node B. But Node A doesn't know how to forward anything out the internet...it doesn't think it has a default route. A traceroute from the workstation stops at Node A.
Is it only possible to share an internet connection using VLAN segmentation?
correct. On Ubiquiti devices, you *MUST* have a VLAN capable switch configured properly to get access to the "WAN"/internet port. The WAN port is virtual only (VLAN1).
so, when you enabled the "mesh gateway" option, you are advertising that this node a gateway to the internet, BUT, you not provided a VLAN1 for it to use. Therefore, you have no internet.
You can pick up the Netgear GS105E for around $45 on eBay and they are very good.
Also, internet sharing is not a "normal" thing to do. While it's possible, it's not normal. None of my nodes have the "mesh gateway" option enabled. If your node has a VLAN1 internet connection, *THAT NODE* can use the internet, but, will advertise it and provide the "internet" to other nodes on the mesh. This is how my tunnel servers and clients operate. They have VLAN1 internet, but, are NOT a gateway.
Darryl
After a little more playing and verifying that I did have things configured and wired correctly, I was able to get Node B to advertise that it had internet access, and I can see that it is able to talk to the internet because it updated its time. On Node A, the default gateway on the status page is automatically pointing to Node B (which is good!). I can now traceroute and get back to Node B, but then traffic stops. So Node B knows how to get to the internet, is using the internet, but it will not route traffic out for me.
I don't mind picking up the switch - I was probably going to need it anyway, just was wanting to experiment a bit here since it appeared from the help text that this was possible without using VLANs.
When you use the WAN interface via that VLAN to connect to the internet, does that mesh node then NAT all traffic from the MESH out the WAN interface? Or do I need to configure the router that sits between the mesh node and the internet to know about routes in the MESH?
My wireless internet provider uses 10.x addresses, which I then NAT to 192.168.1.x inside my network. If I give the WAN interface a 192.168.1.x address, will it NAT traffic from the MESH to my firewall...if it doesn't and I need to tell my firewall to route any 10.x addresses back to the mesh node, I'm in trouble.
All systems leaving the mesh node (From the WLAN Mesh or from the node LAN) is nat masqurated to use the mesh node WAN IP address so all traffic will appear to come from the mesh node.
"On Ubiquiti devices, you *MUST* have a VLAN capable switch configured properly to get access to the "WAN"/internet port."
Do you need a VLAN switch on each end, and should they be managed or unmanaged? Thanks.
...so that must be the problem I'm seeing. Since the WAN port is disabled in my configuration to enable the node to use the LAN's default gateway, NAT isn't working. Ok, this is making more sense now. I think some diagrams will be helpful. After my switch arrives this Thursday I'll be ready to rock.
The hardware for my third node arrived today and the tower sites are being confirmed. We should have the backbone ready in just a few weeks here in Central IL.
So my experiment didn't work based on the aforementioned reasons. Using a vlan capable switch, and following these instructions, I was able to configure a switch and share the internet connection. As a matter of fact, I'm typing this response using the mesh from a remote node. Yay!
http://ae5ca.com/?p=49
awesome! Glad you are up and running.
Here is how I am set-up...
Node A is at my home.
Node B is 1 km from my home.
Node A & B are linked together with two ubiquiti radios over standard wifi ... both location have a TP-link switch which support VLANs.
In MESH Status, I see Node A and B connected as dtdlink.
There is 2 nodes connected behind node A which are OTA. Let name them C & D.
There is 3 nodes connected behind Node B, OTA (Over the Air), let name them E & F & G
On the network, any nodes can talk to any nodes... they can ping each other, there is no routing problems,
Node A and Node B can have a WAN ... they are both on the same LAN and have the same gateway.
Node A LAN IP is 172.20.0.100 and Node B LAN IP is 172.20.0.200 and the gateway is 172.20.0.1 ... since they are connected with the ubiquiti link they are basically on same network ... and that explain the DTDLink. Geographically it is not possible to have a air link over mesh between these two nodes and that's why I have a private link between them to connect two mesh network together.
now ... each node get the nearest WAN gateway properly. But they can't ping on the internet ! I can't use the time server or any function ( like uploading position to AREDN network ) because it say "ERROR: Cannot update online map. Please ensure this node has access to the internet.".
Any clue ? Is it something to do with VLAN or is it something else I've missed ?
I have "Disable Default Route" activated...
Thanks for help !
VE9MDB
(I do recall that there was at least one TP-Link switch that could not use VLAN1 (WAN) due to internal restrictions of the switch. I think it used VLAN1 as the management VLAN and you could not change that setting.)
The TP-Link switch aren't configured ... but once I installed them, I was able to have DTD working between the two sites.
However, you might be right, I can't tag VLAN1 on any ports.
The hardware version of my tp-link is : TL-SG108E 2.0
Is there a way I can bypass this ... somehow ?
This should be fixable, we just need a few more pieces of information!
Yes, the ping command and traceroute command works on both node perfectly ... !
Here is a copy/paste :
root@VE9MDB-FN76NE-MAARC:~# traceroute 8.8.8.8
traceroute to 8.8.8.8 (8.8.8.8), 30 hops max, 38 byte packets
1 172.20.0.1 (172.20.0.1) 13.429 ms 10.219 ms 4.381 ms
2 192.168.0.1 (192.168.0.1) 19.177 ms 12.616 ms 9.114 ms
3 * * *
4 69.63.254.209 (69.63.254.209) 20.123 ms 44.295 ms 26.071 ms
5 24.156.147.33 (24.156.147.33) 18.301 ms 13.121 ms 27.486 ms
6 van58-9-229-241.dynamic.rogerstelecom.net (209.148.229.241) 64.568 ms 54.574 ms 53.479 ms
7 van58-9-229-225.dynamic.rogerstelecom.net (209.148.229.225) 42.891 ms 53.156 ms 55.869 ms
8 van58-9-230-14.dynamic.rogerstelecom.net (209.148.230.14) 43.838 ms 39.032 ms 39.712 ms
9 72.14.222.87 (72.14.222.87) 50.370 ms 51.397 ms 54.021 ms
10 216.239.43.231 (216.239.43.231) 39.128 ms 209.85.255.197 (209.85.255.197) 65.594 ms 44.015 ms
11 108.170.234.15 (108.170.234.15) 41.762 ms 108.170.234.33 (108.170.234.33) 65.977 ms 108.170.234.37 (108.170.234.37) 52.734 ms
12 google-public-dns-a.google.com (8.8.8.8) 47.977 ms 49.479 ms 37.564 ms
So, my current gateway is 172.20.0.1 and it is a router located 1 km from my qth, where the physical router ( 192.168.0.1 ) is located. I get my internet from there and send it to my QTH....
73
Matt - VE9MDB
root@VE9UDM-MAARC-Nano:~# traceroute 8.8.8.8
traceroute to 8.8.8.8 (8.8.8.8), 30 hops max, 38 byte packets
1 VE9SHM-FN76ND-TCARC.local.mesh (10.0.0.10) 6.175 ms 2.316 ms 2.504 ms
2 VE9SHM-FN76ND-TCARC.local.mesh (10.0.0.10) 2.652 ms 1.530 ms 1.405 ms
as you can see, it stop at one of the Mesh internet gateway ( VE9SHM ) ... and doesn't get out to Internet ( next expected hop should be 172.20.0.200 and then 172.20.0.1 and then 192.168.0.1 IMO )...
Matt
Please attach a support data file from the gateway nodes so that the members of the forum can better provide answers to this question.
Here is the support data....
I would suggest you open a bloodhound ticket (http://bloodhound.aredn.org) on it so it can be moved into an official track for a resolution decision.
Dev notes:
No masquerade is currently occurring from MESH to LAN when destination is Internet. Without masquerade or configuration on gateway router it's likely that the packets are not making the correct return path because the home router is likely routing the packets the wrong way not knowing that 10.0.0.0/8 and 172.16.0.0/12 need to go to the mesh node.
[root @ Router] ~ # route
Table de routage IP du noyau
Destination Passerelle Genmask Indic Metric Ref Use Iface
10.0.0.0 172.20.0.100 255.0.0.0 UG 0 0 0 eth0
172.20.0.0 * 255.255.0.0 U 0 0 0 eth0
172.20.1.0 * 255.255.255.0 U 0 0 0 eth0
172.20.5.0 * 255.255.255.0 U 0 0 0 eth0
172.20.10.0 * 255.255.255.0 U 0 0 0 eth0
172.20.20.0 * 255.255.255.0 U 0 0 0 eth0
172.20.255.0 * 255.255.255.0 U 0 0 0 eth0
192.0.2.0 * 255.255.255.0 U 0 0 0 utun
192.168.0.0 * 255.255.255.0 U 0 0 0 eth1
192.168.0.1 * 255.255.255.255 UH 0 0 0 eth1
The router is an untangle box but I did deactivate the dhcp and dns server from it to run my own ...
My first idea was to have VLAN on it but then I realized that it would be too complicated since I have multiples clients running from multiple points.
Can I do something on the router side to make it working ?
Matt - VE9MDB
[root @ Router] ~ # iptables -t nat -L
Chain PREROUTING (policy ACCEPT)
target prot opt source destination
uvm-tcp-redirect tcp -- anywhere anywhere [goto] /* Redirect utun traffic to untangle-vm */
port-forward-rules all -- anywhere anywhere ctstate NEW /* Port Forward rules */
Chain INPUT (policy ACCEPT)
target prot opt source destination
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
Chain POSTROUTING (policy ACCEPT)
target prot opt source destination
nat-rules all -- anywhere anywhere ctstate NEW /* SNAT rules */
Chain l2tp-forward-rules (1 references)
target prot opt source destination
Chain nat-rules (1 references)
target prot opt source destination
MASQUERADE all -- anywhere anywhere mark match 0x1fa/0xffff /* NAT WAN-bound openvpn traffic */
MASQUERADE all -- anywhere anywhere connmark match 0xfb/0xff /* NAT l2tp traffic */
RETURN all -- anywhere anywhere policy match dir out pol ipsec /* dont NAT IPsec traffic */
RETURN all -- anywhere anywhere /* Never NAT loopback traffic */
RETURN all -- anywhere anywhere /* Never NAT loopback traffic */
MASQUERADE all -- anywhere anywhere connmark match 0x102/0xffff /* NAT traffic, 2 -> 1 (egress setting) */
MASQUERADE all -- anywhere anywhere connmark match 0x102/0xffff /* NAT traffic, 2 -> 1 (ingress setting) */
MASQUERADE all -- anywhere anywhere ctstate DNAT connmark match 0x101/0xffff /* NAT all port forwards (hairpin) (interface 1) */
MASQUERADE all -- anywhere anywhere ctstate DNAT connmark match 0x202/0xffff /* NAT all port forwards (hairpin) (interface 2) */
Chain port-forward-rules (1 references)
target prot opt source destination
l2tp-forward-rules all -- anywhere anywhere /* Port forward jump for L2TP */
DNAT tcp -- anywhere 172.20.0.1 tcp dpt:10001 /* Send 172.20.0.1:10001 to Apache */ to:172.20.0.1:80
DNAT tcp -- anywhere 172.20.0.1 tcp dpt:https /* Send 172.20.0.1:443 to Apache */ to:172.20.0.1:443
DNAT tcp -- anywhere 192.168.0.100 tcp dpt:https /* Send 192.168.0.100:443 to Apache */ to:192.168.0.100:443
DNAT tcp -- anywhere anywhere /* Port Forward Rule #1 */ connmark match 0x1/0xff multiport dports http to:172.20.0.5:80
DNAT tcp -- anywhere anywhere /* Port Forward Rule #2 */ connmark match 0x1/0xff multiport dports 5525 to:172.20.0.100:5525
DNAT udp -- anywhere anywhere /* Port Forward Rule #5 */ multiport dports 8443 to:172.20.0.5:8443
DNAT tcp -- anywhere anywhere /* Port Forward Rule #5 */ multiport dports 8443 to:172.20.0.5:8443
DNAT udp -- anywhere anywhere /* Port Forward Rule #6 */ multiport dports 8888 to:172.20.1.10:80
DNAT tcp -- anywhere anywhere /* Port Forward Rule #6 */ multiport dports 8888 to:172.20.1.10:80
DNAT udp -- anywhere anywhere /* Port Forward Rule #7 */ multiport dports 8889 to:172.20.1.20:80
DNAT tcp -- anywhere anywhere /* Port Forward Rule #7 */ multiport dports 8889 to:172.20.1.20:80
DNAT udp -- anywhere anywhere /* Port Forward Rule #9 */ multiport dports 30053 to:172.20.0.25:30053
DNAT tcp -- anywhere anywhere /* Port Forward Rule #9 */ multiport dports 30053 to:172.20.0.25:30053
REDIRECT tcp -- anywhere anywhere ADDRTYPE match dst-type LOCAL tcp dpt:https /* Drop local HTTPS traffic that hasn't been handled earlier in chain */ redir ports 0
REDIRECT tcp -- anywhere anywhere ADDRTYPE match dst-type LOCAL tcp dpt:http /* Drop local HTTP traffic that hasn't been handled earlier in chain */ redir ports 0
Chain uvm-tcp-redirect (1 references)
target prot opt source destination
DNAT tcp -- anywhere 192.168.0.100 /* Redirect reinjected packets to 192.168.0.100 to the untangle-vm */ to:192.168.0.100:9500-9627
DNAT tcp -- anywhere 172.20.0.1 /* Redirect reinjected packets to 172.20.0.1 to the untangle-vm */ to:172.20.0.1:9500-9627
DNAT tcp -- anywhere 172.20.1.1 /* Redirect reinjected packets to 172.20.1.1 to the untangle-vm */ to:172.20.1.1:9500-9627
DNAT tcp -- anywhere 172.20.10.1 /* Redirect reinjected packets to 172.20.10.1 to the untangle-vm */ to:172.20.10.1:9500-9627
DNAT tcp -- anywhere 172.20.5.1 /* Redirect reinjected packets to 172.20.5.1 to the untangle-vm */ to:172.20.5.1:9500-9627
DNAT tcp -- anywhere 172.20.20.2 /* Redirect reinjected packets to 172.20.20.2 to the untangle-vm */ to:172.20.20.2:9500-9627
DNAT tcp -- anywhere 172.20.255.1 /* Redirect reinjected packets to 172.20.255.1 to the untangle-vm */ to:172.20.255.1:9500-9627
REDIRECT tcp -- anywhere anywhere /* Redirect reinjected packets to the untangle-vm */ redir ports 9500-9627
Here is what I did. On the Mesh Gateway ( WAN ) I did use the following commands to get it working :
- iptables -t nat -F
- iptables -t mangle -F
- iptables -F
- iptables -X
Thus allowing all forwarding to occurs ...Of course .. it's not the way it should be working but I'll write a script and have it handy on the node in case it is rebooting and that I need to enable the gateway again...
73 for now ... Oh, and by the way, I opened a ticket ... #184.
Matt - VE9MDB
So just in case somebody want to do the same thing ...
I have to say that I've added a route back to the 10.x.x.x network from the router ...
When I browse my internal network from the mesh network, I see myself with the IP of the mesh node I'm from ... so that's why we know mesquarade is not working. However, with the proper route set into the router, the network know how to answer back to the local node and back on the mesh.
Also ... I've not used the WAN port at all ... just to make it clear, I've just set up a gateway onto the LAN configuration of the mesh node .... and then activated the mesh gateway so it is advertised onto the mesh network.
The problem is at the way it is handling mesh traffic to the default route. If you use the commands I posted, you will clean the iptables and make the ip forwarding working. However, the IP seen outside will remain 10.x.x.x so to have it working you need the router to know how to answer it back through the mesh node gateway.
This could cause a problem if you're trying to use an internal DNS .. and at the moment we are using an external DNS ( 8.8.8.8 & 8.8.4.4 ) as the 10.x.x.x is not allowed into my own DNS server ( I haven't configured it yet to allow this traffic ).
This was a hard learning .... :) But I'm so happy it is finally ready for our mesh demo which will be held in October !
73 and again thanks for your help and tips ... without knowing there was a problem with mesquarade, I wouldn't have looked further !
Matt - VE9MDB
To sit back and design what you are trying to, we'd unlikely engineer it this way. Right now the LAN configured in NAT mode is masquerading the IP address in the wrong direction that you are going--you are breaking these firewall rules to get it to work in the opposite direction. Masquerading basically means we add an SNAT rule. The current design just does this in the opposite direction.
I think this is an enhancement request to add to the LAN configuration to have 3 options instead of 2:
1) Mesh mode: standard 10.x.x.x devices directly on the mesh <- we have today with the 1, 5, 13 address allocations
2) Protected from the Mesh mode: private address space 172.16.x.x or 192.168.x.x and devices are protected from the mesh <- today's 'NAT' option
3) Connect to and protected from the Internet mode: private address space, fully and directly routable from the mesh, but NAT'd and protected from the internet <- new option (and could still work with a live WAN interface to yet another foreign network)
We just have to setup the appropriate and right firewall rules to meet these 3 very different requirements. So I think this is both a bug and an enhancement request...
Joe AE6XE