I do volunteer work for the American Red Cross, New Jersey Region, on the Disaster Services Technology (DST) team. About 3 weeks before the recent visit of Pope Francis to the United States, I was contacted by Fred, WW2VEH, to discuss a backup plan to cover communications in cities in several states: New York City, Fairfield, NJ, Tinton Falls, NJ, and Philadelphia. What made this discussion interesting was that 3 Red Cross regions needed to communicate for one event, a situation not normally encountered. Fred asked me if I could put together some sort of phone system that could be deployed to these 3 cities as well as several field locations. Using elements of technology from amateur mesh operations and the AllStar Link Network, I was able to deploy a system across those cities using voip phones, an AllStar repeater node and remote base station, and mesh tunnels. Tying this all together was an Asterisk PBX.
Each city had one or two phones tied back to the main AllStar pbx. A tunnel server was established at the pbx end on a Ubiquity Nano station and tunnel clients using Linksys routers were established at each of the locations with phone(s). A dial plan was established enabling the phones to directly dial each other. Behind each phone there was voice mail as well a conference bridge available to all users. In the background, a number of test extensions were configured on the pbx to assist with provisioning the network. At Tinton Falls, 3 phones were set up: one directly off the tunnel client and two others via mesh on 2.4 GHz. Philadelphia had one phone, Fairfield had two phones and New York City had two phones.
Midway through the operation, a remote base station was established on the NJ OEM 222 MHz repeater. This was made available to all users through a special extension number.
All but one of the tunnel clients came up without incident. The one that did not come up was blocked by corporate firewall rules which did not allow unusual ports to leave the premises. All the phones worked well, except for one phone via mesh. The remote base station worked well, even though it was a 1 watt portable talking to a repeater 40 miles away with an attic mounted antenna.
None of the field locations were activated. However, in preparation for activating one location, a ViaSat satellite link was set up in the parking lot of Tinton Falls. A tunnel client was established over this link and the voip phone came up fine. Several calls were placed over this phone, and other than a long turn-around time, voice quality was good.
Some takeaways:
1. The phone that did not work via mesh was on a Part 15 channel. During the operation, Red Cross personnel did all their work via wireless laptops, between 20 and 30 laptops. In the future, phones connected this way will need to be using ham band only channels. 2. To remedy the problem of the client node being blocked by a firewall, perhaps a design change could be made allowing port 80 or maybe 443 to be used for tunnels.
Some pictures can be seen on the mesh at
http://n2mh-meshphone2/redcross/pope
Weblinks
AllStarLink Network
AllStar on a BeagleBone Black or Raspberry Pi 2
http://crompton.com/hamradio/BeagleBoneBlackAllstar/
73, Mark, N2MH
Very nice and thorough writeup Mark. thanks for the detail!
"2. To remedy the problem of the client node being blocked by a firewall, perhaps a design change could be made allowing port 80 or maybe 443 to be used for tunnels."
One may find this actually creates more issues than it solves. Firewalls are quickly moving towards identifying protocols and when the protocols do NOT match they drop the connections. More and more in my day job I'm deploying firewalls with these features enabled and we are much more likely to enable them on port 80 and 443 to block based on protocol not matching because they are abused so often by programs I'm much more likely to deploy less security on unknown ports by request of the customer than on the known ports.
On top of that we have the issue with that lots of web filters in line on port 80 and 443 will go crazy (not understand the protocol) if they see the VPN traffic and are VERY likely to block them and these are everywhere in corporate networks already.
That's very interesting but not surprising. For someone wishing to establish an ad-hoc tunnel, is there any solution?
This really will depend on the environment one is working in. As a security guy I'm suppose to say "no, if the admin does not want you going out you will not be able to get out" however as a security guy I also know "there are always ways data is allowed out and some of these methods could be exploited" however those methods require some time to either map out the situation, and some resources to bypass and often will be specific to one environment.
Your suggest of port 80 and 443 (443 has a higher chance of being successful since the connection is already mostly encrypted when used for https traffic) is one method that works on some networks that only do port based security. I've used this method on many occasions in my life to bypass some security, yet as I mentioned prior on some networks this method would fail in a heartbeat as the security measures detect it, I've even had my computer automatically kicked off a network once by the security defenses when it detected a tunnel over HTTPS (as a security guy I really like that persons network, as a user who was trying to get around a filter block, I really hated that persons network)
As your own testing probably showed however you had a significant number of networks that did not block the traffic at all because it was on a "random high level port" but that is little consolation for the one deployment that didn't work.
The possibility to fail during a disaster is the largest reason why the AREDN group often states that tunnels are intended to be used for joining mesh islands, however firewall security and network volatility like this are reasons why it may not work in an emergency situation as well (I'm aware of one group who touted tunnels at one time as being the save all, all you had to do was find a network connection)
If I were in a disaster situation I would probably try the router first and after that If it was an absolute need I probably would go as far as to force my way through the network (eg basically I unplug all hardware and plug right into the network line and setup the connection) obviously not the case here.
I've seen one group test with using MIFI routers with good success, I do wonder how well it will work when towers start becoming loaded down during a disaster however. I've seen another group that has a donated satellite uplink that could be used to run off of (though the couple seconds latancy of a cell jump often makes VOIP a bit hard, fully doable just delayed)
I should also take this time to mention the tunnel technology used for linking mesh nodes is very lightweight (in comparison to other technologies.) I was involved in discussions when the question came up "which tunnel method do we support" and various well known, and very good methods, were evaluated. One method being very common (ipsec) was known to be very picky (home user firewalls didn't handle it well but corporate ones would) and had reports of at least one ISP blocking the connections. Another tunnel technology (OpenVPN) was known to be very reliable, able to get through in some situations with additional programming and was well known by at least one of the dev members (myself) but was fairly heavy on resources on a node, in addition at the time tunnel selection was ongoing no GUI was planned to be in the works and setup would of been much more complicated with this solution. Another solution is the one we ended up settling on is the current tunnel technology, its lightweight, and does the job of connecting two networks with a bit of authentication. Its not DoD grade like the other solutions could be, but it fulfills the goal of linking networks together and gets through most home owner firewalls (where we figured it would likely be used most often)
What all that means is that its best to be able to work with the firewall admin to get access, if that is not available than there may be some other service you could put in front of the mesh node (acting like a router) that can better process packets (so resource consumption is not an issue) and can have a lot more technology options but it is not the sort of item I would want to be fumbling around with unless I had to and it is unlikely such a deployment could be made easy, it would require infrastructure to be placed before hand, and a method for trying (and integrating) potentially dozens of methods to bypass security. One could fill a node with just VPN software and have no re sources for anything else and still not have a way through a network firewall.
nice and thourough response Conrad.
I thing I'll add... due to the above, we are not committing to making the port changes, however, if you still have access to that site/network, I can walk you thru a few config changes to allow whatever port that you want. This would be for the purpose of testing whether or not that site has a more advanced content filter or not (as Conrad mentioned).
K5DLQ,
Are we talking about Port 5525?
I have lost my Tunneling capability, I can see but can't connect to anything.
A Port Scan shows my Home Network has recently had Port 5525 Blocked from the outside.
What are the alternatives so we can continue testing?
Thank You,
Rich W4NMH
Hey Mark,
That first link (http://n2mh-meshphone2/redcross/pope) doesn't look correct... Thanks!
--Chris
Yeah. you gotta be on the mesh to get to that. Mark, do you have a public link?
Actually, here... I pulled them from the mesh for you... (N2MH, if you prefer to host them, just let me know).
https://drive.google.com/folderview?id=0B24aGNNGNT3pTlcxc1Fpb3o2eVk&usp=...
I was actually trying to post several of them here but they are too big for this forum software. I got busy playing with a new radio and never got to shrinking them down....
A little description of the pictures...
The first several are the ViaSat installation in the parking lot. This is probably very similar to one of the scenarios Conrad spoke about above. What you see is the satellite dish, its "portable" carrying case, and on top of the carrying case, the tunnel client and the voip phone. The whole dish assembly and modem breaks down and fits exactly into the carrying case. The next series shows the tunnel client inside the building sitting next to the TV. On the desk below is the voip phone. The final pictures are the NanoStation tunnel server at my home QTH. That is being moved outside the shack shortlly so that it looks outside.
Daryl, thanks for posting them.