Being a newby (or noob) to VLANs, I'm looking to set up the network shown in the picture below while using the VLAN configuration recommended on the forum. After much research, I still have quite a few questions:
- Should both NSM2 units be running DHCP?
- If only one NSM2 is running DHCP, will it assign an IP to both the laptop and the RPi?
- If both NSM2 units are running NAT on the same LAN subnet, can I port forward from the mesh on both of them to the same resource, namely the RPi?
- Is it necessary to have the laptop on Port 1, or could I just join the home network and have access to both NSM2s as well as the RPi?
- If the laptop is connected to Port 1, will it have access to the NSM2s, RPi, home network, and the Internet?
- If I disconnect NSM2-B from the switch, will traffic route between the RPi and NSM1?
- Are NSM2-A and NSM2-B assigned IP addresses from the Comcast router over VLAN1?
- If I removed VLAN20 and connected Port 4 to VLAN10, would NSM2-B still have access to the RPi? Is there really any reason to have VLAN20 in this configuration?
- With the configuration shown, is there a way to advertise a resource located on the home network to the mesh, or can only resources that are issued an IP from one of the NSM2s able to be advertised?
- If I have MeshChat running on the RPi and wish to store and retrieve files on the RPi through the mesh with NAT running on the NSM2s, if I open up ports 80, 20, and 21, should that allow file upload and download? I tried an experiment with this and could not retrieve files stored in MeshChat through the node firewall, so I’m thinking I missed a port.
Thanks,
Dave K3GX
1) the mesh nodes have several networks that are isolated and each network is a specific vlan. This translates to different network interfaces (physical or virtual) on the mesh node. These vlans, when defined correctly on the switch, correctly separate the traffic to each of these networks. Traffic 'routes' through a mesh node to go from one network to the other.
2) let's define a network as a given set of IP addresses, that is of size 2^x -- 2, 4, 8, 16, ... (the max device count is less than this.)
3) generally we talk about these networks by name, the mesh node's LAN, WAN, DtDLink, RF Link, etc.
1) Should both NSM2 units be running DHCP?
Yes, if the switch is configured accordingly (and is best practice). Having devices with the LAN DHCP turned on or off, makes it difficult to re-purpose or take to an incident site. The goal is to have all the devices interoperable with consistent configuration, so you can power on however they are connected in, and "it just works".
2) If only one NSM2 is running DHCP, will it assign an IP to both the laptop and the RPi?
This is only used when a vlan capable switch is not available and the LAN networks of 2 nodes are munged together and add complexity to get things working. I always avoid this situation. With a vlan switch, the switch would be configured so a device is on the LAN network of only a given mesh node, there is no confusion where it should get an address from.
3) If both NSM2 units are running NAT on the same LAN subnet, can I port forward from the mesh on both of them to the same resource, namely the RPi?
The assumption is that the switch is configured to have different vlans for the LAN of each mesh node. Then we have 2 distinct 'networks' and the traffic is not intermixed or seen by devices on one network directly from the other. These 2 distinct networks (the LAN of both NSM2s) can have the same IP allocation, e.g. 192.168.x.x range if using NAT mode. But to route between them or across the mesh network, the mesh node (the RPi is getting an IP address from) masqurades or hides its 192.168.x.x. address. The mesh node has to be configured to advertise the access to the RPi and tells everyone else, use my 10.x.x.x address instead, and when I get traffic on the defined port, I'll forward to the 192.168.x.x address of the RPi. Only the mesh node knows the address of the attached RPi.
4) Is it necessary to have the laptop on Port 1, or could I just join the home network and have access to both NSM2s as well as the RPi?
This WAN port as configured on the switch is logically an extension of both the 2 mesh nodes' WAN port. The purpose and design is to be a cat5 connected to your home network. Attaching a wifi client in bridge mode to your home network's access point works too. Devices on this WAN network (the 2 NSM2s) are configured to obtain an IP address from your home network's DHCP server and would both be on this same network. At present as configured, the laptop doesn't have a DHCP server, so nothing on this network will get an IP address. This is the vlan '1' network inside the switch. Your laptop and devices should all be on a LAN network of one of the NSM2s, then you can access across the mesh and the internet at the same time (routing though the mesh node to get from LAN to WAN/mesh/etc.).
5), 6), and 7) hopefully the response above addresses these questions?
8) If I removed VLAN20 and connected Port 4 to VLAN10, would NSM2-B still have access to the RPi? Is there really any reason to have VLAN20 in this configuration?
vlan 10 and 20 are intended to be the LAN networks of the respective mesh node. If you consolidated to one vlan, which can be done, this would be assigned on the LAN of only one mesh node -- all devices would be on the LAN of this one node and traffic routing through this node to reach elsewhere. That's OK, it just means we won't use the LAN of the other mesh node. It will work both ways.
9) With the configuration shown, is there a way to advertise a resource located on the home network to the mesh, or can only resources that are issued an IP from one of the NSM2s able to be advertised?
All the resources (laptops, RPi, etc.) on this switch have to be connected in to the LAN port (which is configured to be the LAN network of a given NSM2). On this NSM2, is where you advertise the service available across the mesh. The mesh node doesn't advertise services external, or on your home network or the internet. However, keep in mind if your home network and DNS server is configured (meaning the hostname can be resolved), then the mesh network has access. From a mesh devcie, you can go to www.google.com or a hostname on your home network if youf home DNS server is configured and knows about the device. The mesh network as a whole can only get to your home network devices if you check the box to advertise your node as a gateway. But your LAN devices on this mesh node gateway always have access by default to your internet connection. The internet and your home network can not reach into the mesh network though the gateway by design (anyone could be on part97 links). The mesh node gateway, can port forward in the UI from your home network to it's LAN devices (no part 97 involved). You'd have to setup in your home network a host name for the mesh node's IP address to use.
10) If I have MeshChat running on the RPi and wish to store and retrieve files on the RPi through the mesh with NAT running on the NSM2s, if I open up ports 80, 20, and 21, should that allow file upload and download? I tried an experiment with this and could not retrieve files stored in MeshChat through the node firewall, so I’m thinking I missed a port.
I'm not sure all the ports that need to be open on meshchat and if it can work though a NAT -- guessing no, it wasn't designed to do so, or you'd need to do some manual surgery to get it working. The complexity of running NAT on the mesh node's LAN offer's some protection from virus threats, however you can also run firewalls and antivirus on the LAN devices to be protected too, then the NAT becomes another layer. Do you need 2 locks or is one sufficient?
Joe AE6XE
Your answers brought about a few questions organized by question number:
1) OK.
2) OK.
3) Yep, that's what I would expect with NAT. The LAN10 address doled out by the node is in the 172.xxx subnet rather than 192.xxx, but that's immaterial to the way NAT works.
4) Your comment that the laptop per my diagram would not have a way to get an IP address on VLAN10 is puzzling. Since it would be connected to NSM2-A in the same way the RPi would be connected to NSM2-B over VLAN20, wouldn't it get an IP address?
5) 6) 7) Yes, a bit help.
8) OK.
9) I don't think the NSM2 is getting an address from VLAN1, so I've got to find out why. As I mentioned earlier, I tested a laptop on VLAN1 and it successfully got an IP assigned from the home router.
10) Regarding MeshChat, I'm able to do the regular chat stuff with just port 80 opened up while running NAT. It's just the file storage and retrieval that isn't working. I thought I would run NAT, because I don't want the mesh to have full access to my home network as well as the Internet. I would rather open up a port or two so the rest of the mesh has NTP and maybe node firmware update access. Otherwise, how would I keep the commercial encrypted Internet off of the mesh? I also noticed that you can open up ports to two entities... I'm not at my mesh location, so I can't remember exactly the two choices, but it appeared to be the mesh and/or LAN1. When in NAT mode, do I have to open up ports in both directions? It would be nice if the LAN1 side was totally open so the node could easily access the Internet for updating or for NTP, but could remain protected on the RF mesh side. Is that possible?
Thanks again for such a detailed post!
Dave K3GX
4) Your comment that the laptop per my diagram would not have a way to get an IP address on VLAN10 is puzzling. Since it would be connected to NSM2-A in the same way the RPi would be connected to NSM2-B over VLAN20, wouldn't it get an IP address?
Edit: I see in the color coding how you have it configured, the WAN and LAN on port 1 and 5 need to be reversed to be consistent with the vlan tags. So the below is not correct assumption. If the switch is configured per the color coding it should be correct. You might turn off NAT mode on the LANs to test out everything is working as expected first, then make this transition. The laptop will get an IP address from NSM2 A, and the RPi will get an IP address from NSM2 B. The status screen on each mesh ndoe, should show an IP address for the WAN from your home network. Ignore the below, if you saw it already.
Maybe some confusion of how port 1 is configured? If it is on vlan 1 untagged, then there is no device on this network running DHCP. vlan 1 is the 'WAN' network of the NSM2's, consistent with the label in the diagram. They both would expect to get an address from your home network on this port. (The NSM2's have DHCP running on their respective LAN networks -- vlan 10 and 20 respectively -- different networks).
Guessing a little here, but I suspect you need to move the TP-link router (in bridge mode) from port 5 to 1. Then the "TBD" wireless bridge AP would move to port 5 and serve up IP addresses from the NSM2's DHCP server for the vlan configured on this port.
Joe
No. The RPi is on the LAN of NSM2-B. If NSM2-B is offline, then the RPi can't communicate to anyone. It is on NSM2-B's LAN and all trafic routes through NSM2-B to any other network.
Joe, sorry for the confusion on the diagram. When I posed the additional questions, I configured the switch with only VLAN1 and VLAN10, just to make sure I could get the node configured correctly and understood how it all worked. The attached diagram is how I configured the switch and where I had the laptop connected. The "LAN" label I put on the TP-Link router was not meant to imply the LAN port on the NSM2, but instead the LAN port on the router. Same for the WAN label I put on the TBD router in the previous diagram.
So far, the TP-Link router seems to be in the correct configuration in that if I connect something to the switch ports 1 or 3, they are assigned an IP 10.0.0.X through my Comcast router. I can also ping a device that is connected to the switch from my home network. However, the NSM2 is not getting an IP assigned, and I'm not sure why. When I look on the node home page, it says there is no WAN address and the default gateway is none. Just for kicks, I tried a couple of things with the node to see if an IP would be assigned on the WAN.
1) I changed the LAN to Direct mode instead of NAT. That allowed me to turn on the Mesh Gateway. The RPi worked fine, but there was still no IP assigned to the NSM2 WAN port on VLAN1.
2) I have an XW version of the NSM2, and I was running a nightly build "develop-176-8c5be0cf." Since there's a new nightly build out there, I thought I would upgrade the firmware and try it. I upgraded to "154-32b6fbc." The node came up fine, and I could see that it was different firmware by looking at the setup pages. I kept the Mesh Gateway enabled in setup, but still see WAN address = none and default gateway = none on the home page.
Is the mesh gateway not working as expected on these two nightly builds?
Thanks, Dave K3GX
"they are assigned an IP 10.0.0.X through my Comcast router"
This could be problematic. If there are any mesh nodes with LAN subnets or IP addresses on this 10.0.0.x network, then there will be a conflict. it's maybe a small chance of this conflict, but it can happen, so best to review the mesh host file periodically to see if there is a conflict. Alternative, change your home network in one of the following ranges:
172.16.0.0 – 172.31.255.255
192.168.0.0 – 192.168.255.255
On the mesh node, you can review the mesh wide assigned IP addresses in "/tmp/run/hosts_olsr", but need command line knowledge to do so, to determine if there is a conflict.
"I have an XW version of the NSM2"
The XW Nanostations have an odd vlan port configuration. Something we are working to better document on the website. These NS XW devices have an internal switch chip that other devices don't have, except the AirRouter. Unfortunately, there is an openwrt/linux design or bug issue with the vlan implementation with internal switches. It fails to work when there is untagged and tagged packets on the same port. Everyone gets bit by the issue, and it still happens to me once in a while, easy to forget. The main port is LAN only. The secondary port is WAN and DtDLink. This means you're going to have to use 2 x cat5 cables from the NS XW node to the switch, and configure the switch accordingly. The WAN and DtDlink paths won't work as currently configured :( .
Joe AE6XE
So, now that the gateway is up and running, what is the best way to restrict traffic from the RF mesh through to the Internet? Should this be done on my home router by fixing the IP on the node's WAN port and then including only the services that should be used over the mesh? If so, has someone generated a list of services or websites that are needed in order to maintain the nodes (i.e., NTP, location upload to the map) and keep the traffic from going to encrypted / commercial sites? I noticed that the mesh-to-WAN connection does not allow someone to see what's in my home network - it just accesses the Internet as far as I can tell.
Good point on the home IP range - I thought about that as well and wondered if it would cause problems. I did check the /tmp/run/hosts_olsr file on the node and didn't see a conflict, but as the mesh grows, this could be a problem. I have a few RPi's running fixed IPs in the house that I'll need to change in the process.
Dave K3GX
By default, no one across the mesh network (this includes all nodes via dtdlink, RF links, and tunnels) can use your internet gateway. If you want to allow this access, then in setup, check the box "mesh gateway" under the WAN section.
"does not allow someone to see what's in my home network"
Technically, if someone knows the IP addresses of your home network AND you had 'mesh gateway' box checked, AND OLSR didn't find a lower cost mesh gateway to route them to, then they could type this IP address to get to your home network. If your home network DNS has an entry, and you point the node-gateway's WAN DNS settings to it, then they could also use a hostname.
The LAN devices of a mesh gateway node, can always access the home network or internet. To turn this off, it's the "disable default route" checkbox under the LAN section. (I want to change this terminology to be more descriptive and less technical...)
If the greater mesh network can use your internet gateway, then yes, one can put in firewall rules on a home router to restrict traffic accordingly.
Dave K3GX
Here are the options currently available to further restrict the network traffic between a mesh node and your home network:
1) There is a package that can be installed, called "blockknownencryption". I've not yet heard of anyone testing this on the nightly builds and inspecting the rules, I don't see any reason why it would not still work (and need someone to confirm :) ). This will block the standard ports for https, ssh/scp, smtp over ssl, imap over ssl, and pop3 over ssl -- only on the RF link. This means your LAN devices can still do all these things routing through the node to your home network with no RF.
2) It is possible to write your own "iptables" commands to add custom rules to the mesh nodes. This is done by creating a file under /etc/local/mesh-firewall on the node. There are other forum posts of a couple example scripts. These are not preserved between firmware installs, so best to keep an offline copy somewhere. This is advanced linux stuff to get these scripts correct.
3) A home router with sufficient ability to add the rules you want.
Joe AE6XE
I'm assuming that with the https restrictions in the blockknownencryption package, no nodes on the mesh would be able to do firmware updates through arednmesh.org or uploading of data to the AREDN servers (i.e., map locations)?
I will look into the iptables - that sounds like a good way to restrict from my home lan IPs. I ran the iptables -S command and saw what was already there.
Thanks, Dave K3GX
The nightly build packages currently have signatures as you've determined. There's some consideration to turn off signatures for a nightly build because of this issue. You're back a nightly build, so unable to refresh and pull down. If you do a quick sysupgrade to the latest build it will work, until the next build occurs. Alternatively, yes, can manually copy down to your computer and upload in the UI, but you have to figure out any dependent packages (from the errors), and get those manually too. (not any dependent packages with the block encryption package.) The structure of the firewall rule chains is a little different between 3.16.1.1 and what we have today, so different packages.
The nodes use http to refresh and download data to avoid any part 97 concerns -- http://downloads.arednmesh.org .
There are several tables in use to inspect with the rule chains. There's a lot of information on the internet to build a mental map of the built in flow across the various tables. Openwrt also has a template of rule chains they create to auto add from a standard configuration file definitions.
Joe,
I've made some progress in the last couple of weeks and now have the architecture shown here. One RPi is used as a MeshChat/Web Page server on the mesh while the other is a firewall that protects the mesh from encrypted traffic as well as isolates it from my home network. I just want to check and make sure my iptables on the firewall look right to you and cover all scenarios. The firewall is acting as a DHCP server to the node's WAN connection on a different subnet than my home network. BTW, I've also opened up a port for tunneling on my home router as well as on iptables as you'll see in the list below.
sudo iptables -t nat -A POSTROUTING -o wlan0 -j MASQUERADE
sudo iptables -A FORWARD -i eth0 -o wlan0 -m state --state RELATED,ESTABLISHED -j ACCEPT
sudo iptables -A FORWARD -i wlan0 -o eth0 -j ACCEPT
sudo iptables -I FORWARD -i eth0 -o wlan0 -m iprange --dst-range 192.168.0.2-192.168.0.254 -j DROP
sudo iptables -I FORWARD -i eth0 -p tcp --dport 22 -o wlan0 -j DROP
sudo iptables -I FORWARD -i eth0 -p tcp --dport 443 -o wlan0 -j DROP
sudo iptables -I FORWARD -i eth0 -p tcp --dport 465 -o wlan0 -j DROP
sudo iptables -I FORWARD -i eth0 -p tcp --dport 993 -o wlan0 -j DROP
sudo iptables -I FORWARD -i eth0 -p tcp --dport 995 -o wlan0 -j DROP
sudo iptables -I FORWARD -i wlan0 -p tcp --dport 5525 -o eth0 -j ACCEPT
Thanks,
Dave K3GX
sudo iptables -A FORWARD -i eth0 -o wlan0 -m state --state RELATED,ESTABLISHED -j ACCEPT
This kind of rule is to allow the return traffic coming back--from server back to client. When a mesh client goes to make a connect to, e.g. www.google.com, the internet server responds back and that return traffic needs a rule to allow the packets back though. The state identifies if it is returning traffic to say it is OK. You would need a rule like this from -i wlan0 coming back to -o eth0 retuning back to the client on the mesh for the www.google.com example.
sudo iptables -A FORWARD -i wlan0 -o eth0 -j ACCEPT
wlan0 to eth0 is the path of doing a port forward into the mesh node (1 of the 2 NATs). For a client on your home network or the internet, it is trying to reach a mesh device behind the RasPi NAT. The networks and IP addresses on the other side of this NAT are by design unknown. This is the whitelist rule to allow very specific traffic though and to which specific hidden device/IP address it can go to. What you'd have, from you home network's perspective on 192.168.0.x computer, is a packet addressed to the RasPi IP 192.168.0.205, an IP address known/routeable on your home network. You would need the 2 or 3 iptable port forward/DNAT/SNAT rules on the rasPi that says the incoming wlan0 packet should be forwarded and translated to go to the mesh node's IP address and intended port number, e.g. 192.168.1.?:8080. This IP address:port number translation is hardcoded in the port forward iptable rules. This way your home computers on 192.168.0.x could reach the mesh node by typing, e.g., into a browser "http://192.168.0.205:8088". The rules translate it to http://192.168.1.?:8080 and knows how to route it over eth0 to the mesh node.
Got a ways to go to get all the rules shaped up. Hopefully these pointers will take it another few steps to get there.
Joe AE6XE
https://www.arednmesh.org/content/gs105e-1-lan-4-node
Given that, if you have at least one NS, I suggest you connect your computer and to, and provide POE to, the primary port. Then connect the secondary port to the switch with a VLAN2 tagged configuration. Configure the rest of the ports as VLAN2 tagged as well. If connecting a Bullet or a Rocket, just connect its one Ethernet port to one of the remaining switch ports. If connecting NS nodes to the remaining ports, provide POE to the node's secondary port and connect the secondary port to the switch... just remember to not put power on the Ethernet connection to the switch (use the LAN connection out of the POE power supply to connect).
Hope this isn't too confusing!
73's, Dave K3GX
Only put vlan 2 on the ports of mesh nodes, leave tagged.
Put vlan 1 on the ports of the mesh nodes, leave tagged. The only other port for vlan 1 is the port with the cat5 to your home network, and is untagged and PVID vlan 1 on this port.
For ports with LAN devices, they only have a single vlan ID assigned to this port for the LAN of the mesh node you want to get IP addresses from. untagged and PVID for the respective mesh node's LAN vlan #.
Joe AE6XE