AREDN at the 2016 NYC Marathon

The New York City – Tata Consultancy Services (NYC TCS) Marathon is a hugely visible annual event. This early November road race draws runners from around the world. A record 51,388 finishers crossed the finish line on Sunday, November 6, 2016, making it the largest marathon in the world in history.

More than one million spectators and thousands of volunteers line the city streets in support of the runners, while millions more watch the television broadcast in 175 countries and territories, including viewers in the New York area on WABC-TV, Channel 7, nationally on ESPN2, and via various international broadcast partners.

A vast cadre of amateur radio operators provide essential communications to support the safety of race participants and organizers.  An AREDN mesh network and associated Voice over IP telephony services provided the reliable communications infrastructure necessary for this important event.  The story of this deployment in a very challenging environment is told by Gordon W2TTT.
 

2016 New York City Marathon Recap

J. Gordon Beattie, Jr., W2TTT

Introduction

Earlier this summer, Mike Hoeft, K2MPH and Deb Kerr, KC2GPV reached out to me to inquire whether we could use the AREDN Mesh equipment to address the networking problems of the Amateur Radio net control operators for the New York City Marathon in November.  We discussed the notion of a simple link between the “Ham Trailer” and the Race Control Center over a distance of less than 500 yards.  The link was to support a variety of net control positions for this 55,000+ participant race and allow them to update status information in real-time in Google Docs.  This information would not only be shared within the finish line area, but also from other points including One Police Plaza at the NYPD Command Center downtown.  Additionally, there was a request to support the operating positions with Voice over IP (VoIP) phones to reduce the noise level across the trailer and RCC and to enable “off air” conversations among all the operating positions inside and between these locations.

The finish line area in a very challenging RF environment, where 900 MHz, as well as 2.4 and 5 GHz are overwhelmed by a variety of Wi-Fi and other in-band signals overlaying a stew of IMD that is present in the finish line area for hours at a time during this event.     We decided that the use of 3 GHz Ubquiti M3 Rockets and sector antennas borrowed from Randy, WU2S would be the most reliable answer. 

Our next step was a series of phone calls to marshal the essential equipment and expertise from those who had planned to volunteer for the Marathon and to “adjust” their assignments to enable them to help make the operation a success.  This led to Dave, N3UXK and Mark, N2MH joining the team to help, plan, build, test and execute the key parts which included redundant All-Star PBX Switches, the aforementioned M3 Rockets, a pile of Grandstream GXP1450 VoIP phones, a slew of other Ubiquiti nodes, Netgear Smart Switches, power supplies and support components.  It all had to be integrated, tested and packaged before the first weekend in November – and it was, thanks to a great crew!  J
 

Preparation

We set about refining what Mike and Deborah needed and also tracked the requirements as they evolved.  Mark, configured two Beaglebone All-Star PBX switches using the approach that the two line phones would have the same numbers on two different PBX switches, and that the two switches would have a non-looping trunk between them to allow all calls to ring on both lines no matter which line and switch was used to originate the call.  After some testing in mock setups at the Red Cross and at Mark’s house, this and all the other issues were resolved.  Some further challenges were that the AREDN nodes only allow for a maximum of 13 IP addresses from their DHCP server, so we had to split the network into several VLANs to create different DHCP pools.  We also had a last minute request for secure Wi-Fi access support, which we immediately put on channel 9 to get it away from our backup 2.4 GHz AREDN links on channel -2. 

As part of this effort, we were also asked to update the Google Doc with status information derived from APRS.  APRS in New York City is a challenge, again due to the RF congestion issues on 144.39 MHz.  Even the alternative of using APRSDroid and comparable cell phone and tablet apps for iPhones and iPads had issues due to the cell network congestion at some points of the Marathon course.  We decided to use both technologies and to collect what we could on a “best effort” basis, but to later analyze both the radio and phone/tablet reports for quality and reliability.  We had several operators moving on the course who have provided detailed information as to their setups and networks.  We plan to conduct that analysis of participating stations early in the New Year, but early data show that the RF links were surprisingly good.   

We also received a request for integration into the VoIP telephone network that the New York City Roadrunners Club had established, but their interface specifications were never made available, so that will be a “to do” for next year.  Another future objective is to set up support in additional medical and control points in Central Park but away from the Finish Line area.  This will be investigated for next year as well.  In the end, Internet gateways, Asterisk PBX switches, Smart (VLAN) Switches, VoIP phones, laptops, tablets and phones were all combined to make thorough use of our four DHCP pools.   
 

Conclusion

We had a large number of changes to absorb during the process of getting this support activity done, but sometimes I think we take for granted all that is involved and the breadth and depth of knowledge that is required to make these operations run smoothly.  There is a reason Amateur Radio Operators use the ARRL-inspired phrase, “When all else fails” and why we often get asked to solve problems that are not strictly communications or at least Amateur Radio communications.  Simply put, our role depended upon at least the following key elements of knowledge and expertise:

1. Amateur Radio Emergency Data Network Mesh software,
2. DHCP server and VLAN switch configuration,
3. Propagation at UHF and SHF frequencies and likely IMD and RF congestion issues,
4. AC mains and 12 VDC power systems including battery backups,
5. VoIP phone and Asterisk All-Star switch configuration, 
6. Data, RF, Power cable fabrication,
7. Portable field deployment under unstable conditions skills,
8. Unit and system testing,
9. Packaging and Logistics,
10. Effective interpersonal and written communication, and finally,
11. Leadership skills.  

 
In order to make this happen, we kept a full set of documentation of the end user requirements, our evolving plans, assumptions and status.  These kept us from failing from an organizational viewpoint, but it was the folks who contributed to the effort with their expertise, equipment and time who made the difference.  Enclosed is a PowerPoint slide deck of the work that was done along with a photo gallery.  This is the summary slide deck used to recap the essential elements of the system and how we got there, but there were lots of other slides that we generated along the way, which captured design concepts, questions, issues and progress reports on specific topics of interest to Mike, Deb and other members of the leadership team.  Configuration files and help are available for any of our components.  We would be happy to help anyone with the planning of their event, so feel free to ask!
 

Presentation slides are found here