You are here

crawler and visualizations for AREDN mesh

5 posts / 0 new
Last post
KK7LZM
KK7LZM's picture
crawler and visualizations for AREDN mesh
Our club is building out an AREDN mesh in Seattle, WA and to help us plan our deployment I put together a little crawler and network/map visualization for our mesh in python.

https://github.com/gerner/aredncrawl

It's a quick and simple tool. I'm still learning AREDN. There's probably bugs/issues.

The tool can produce two visuals (right now): 

A network graph (using graphviz):
https://drive.google.com/file/d/1Vv7_DbaxTrAeFWTOGYOCA-0MPRsvDM4o/view?usp=sharing

A map (this is post processed into KML, but it natively produces geojson):
https://www.google.com/maps/d/u/0/edit?hl=en&mid=1HVMqIbpGF-_S0uNXjV1F1TclkIVBcXM&ll=40.60170599060227%2C-98.72273750000002&z=5

It uses sysinfo.json to get link information and saves the responses as it crawls. I'm happy to answer questions, consider feature requests, collaborate or support another more mature effort.

I was inspired by https://github.com/kn6plv/MeshInfo which is in node.js. But I'm not so good with node.js and, sadly for me, this was quicker to bang out in python in a night than to figure out how npm works :P

Cheers!
nc8q
nc8q's picture
gelmce@HP-Laptop:~/Downloads$ echo "10.66.157.253" |python3 craw
gelmce@HP-Laptop:~/Downloads$ echo "10.66.157.253" |python3 crawl.py

2023-09-16 14:28:23,645 root         INFO     crawling 10.66.157.253
2023-09-16 14:28:23,645 root         WARNING  10.66.157.253 failed to get sysinfo Missing dependencies for SOCKS support.

 
KK7LZM
KK7LZM's picture
Ah, yes, I have it set up to
Ah, yes, I have it set up to run through an ssh tunnel socks proxy to a node connected to my home network.

The issue is that the crawler needs to be able to route to all the nodes. If you're using your AREDN node as your gateway, that's easy. If you're not (as in my topology), you need some way to route to all the other nodes. I do that with an ssh tunnel SOCKS proxy.

Sorry I didn't include that info in the documentation. I will update it and make it configurable shortly.

And thanks for taking a look!
nc8q
nc8q's picture
Is this appropriate for python developers or python users?
gelmce@HP-Laptop:~/Downloads$ python -mvenv .venv
Command 'python' not found, did you mean:
  command 'python3' from deb python3
  command 'python' from deb python-is-python3
gelmce@HP-Laptop:~/Downloads$ python3 -mvenv .venv
The virtual environment was not created successfully because ensurepip is not
available.  On Debian/Ubuntu systems, you need to install the python3-venv
package using the following command.

    apt install python3.10-venv

You may need to use sudo with that command.  After installing the python3-venv
package, recreate your virtual environment.

Failing command: /home/gelmce/Downloads/.venv/bin/python3

gelmce@HP-Laptop:~/Downloads$

73, Chuck

 
KK7LZM
KK7LZM's picture
I updated documentation
I updated documentation regarding proxies and clarifying prerequisites.

Using this tool right now definitely requires comfort with the command line and python tooling, e.g. pip.

Theme by Danetsoft and Danang Probo Sayekti inspired by Maksimer