Good morning,
It is my first post on your site and I wasn't too sure what forum was the most appropriate for it.
I have expert knowledge of building and managing linux software, several programming languages, have built custom images based on OpenWRT and have contributed to other open source projects.
I was hoping you could shed some light on a few things related to building aredn.
First of all, I have the following clones:
- git://git.aredn.org/aredn_ar71xx (hash:0560ee3d6696fcb4e9a4d3c40e138a70b3b775fc branch:release-1.15.1.0) at ~/aredn_ar71xx
- git://git.openwrt.org/openwrt.git (hash:8acab6fe26e524d0486bd19b238efc1cbb5bf1ef branch:master) at ~/openwrt-trunk
- https://github.com/koalaman/shellcheck (compiled, installed and in PATH)
Now for the issues...
I have compiled ~/openwrt-trunk without a problem but I noticed that your makeimage.sh script takes care of adding/updating feeds and compiling so I started with a clean clone and issued BUILD_SET_VERSION=999 ../aredn_ar71xx/build/makeimage.sh from ~/openwrt-trunk/. To accelerate the build process I modified ~/aredn_ar71xx/build/buildsteps/003-buildimage.sh to take advantage of the 32 core machine I am using.
By looking at the output/logs I noticed a few errors related to -z tests in the .sh scripts and found a couple of places with
if ( -z "$BUILD_SET_VERSION" ); then
Normally with a shebang of #!/bin/sh the ( ) would be replaced by [ ] so I did that and no more error is thrown there however shellcheck is very unhappy, as shown in this snippet here:
+++ Running case: 08-usrlocalbinshell_olsrd-namechange-loop
+++ working dir: /home/marc/openwrt-trunk
+++ command: shellcheck "/home/marc/aredn_ar71xx/build/../files/usr/local/bin/olsrd-namechange-loop"In /home/marc/aredn_ar71xx/build/../files/usr/local/bin/olsrd-namechange-loop line 6:
while [ ! -f /tmp/namechange -a $N -lt 60 ]; do
^-- SC2166: Prefer [ p ] && [ q ] as [ p -a q ] is not well defined.
In /home/marc/aredn_ar71xx/build/../files/usr/local/bin/olsrd-namechange-loop line 8:
N=$(($N + 5))
^-- SC2004: $/${} is unnecessary on arithmetic variables.+++ exit code: 1
+++ error: 1+++ Running case: 09-usrlocalbinshell_olsrd-watchdog
+++ working dir: /home/marc/openwrt-trunk
+++ command: shellcheck "/home/marc/aredn_ar71xx/build/../files/usr/local/bin/olsrd-watchdog"In /home/marc/aredn_ar71xx/build/../files/usr/local/bin/olsrd-watchdog line 36:
uptime=`cat /proc/uptime | cut -d' ' -f1`
^-- SC2006: Use $(..) instead of legacy `..`.
^-- SC2002: Useless cat. Consider 'cmd < file | ..' or 'cmd file | ..' instead.
In /home/marc/aredn_ar71xx/build/../files/usr/local/bin/olsrd-watchdog line 37:
date=`date`
^-- SC2006: Use $(..) instead of legacy `..`.+++ exit code: 1
+++ error: 1In /home/marc/aredn_ar71xx/build/../files/usr/local/bin/vtun_up line 125:
ip rule del pref 20099 iif $interface unreachable
^-- SC2086: Double quote to prevent globbing and word splitting.+++ exit code: 1
+++ error: 1
I simply ignored that for now and later-on I got a few issues related to finding the "files" folder... I modified the makeimage.sh script and the like to fix that but my biggest concern is that maybe aredn_ar71xx should be cloned somewhere inside the openwrt-tree and that I am not invoking makeimage.sh correctly. I looked at the commit logs and saw:
Some ENV variables needs to be set before executing these scripts.
at http://bloodhound.aredn.org/products/AREDN/changeset/885e93b241cc62c4c483c972a81184f21e391a62/aredn_ar71xx and I was wondering what are the expected variables/values?
In the end I managed to generate files like so:
-rw-r--r-- 1 marc marc 5243292 Sep 12 18:29 AREDN----ubnt-air-gateway-squashfs-factory.bin
-rw-r--r-- 1 marc marc 5242884 Sep 12 18:29 AREDN----ubnt-air-gateway-squashfs-sysupgrade.bin
-rw-r--r-- 1 marc marc 5243292 Sep 12 18:29 AREDN----ubnt-airrouter-squashfs-factory.bin
-rw-r--r-- 1 marc marc 5242884 Sep 12 18:29 AREDN----ubnt-airrouter-squashfs-sysupgrade.bin
-rw-r--r-- 1 marc marc 5243292 Sep 12 18:29 AREDN----ubnt-bullet-m-squashfs-factory.bin
-rw-r--r-- 1 marc marc 5242884 Sep 12 18:29 AREDN----ubnt-bullet-m-squashfs-sysupgrade.bin
-rw-r--r-- 1 marc marc 5243292 Sep 12 18:29 AREDN----ubnt-loco-m-xw-squashfs-factory.bin
-rw-r--r-- 1 marc marc 5242884 Sep 12 18:29 AREDN----ubnt-loco-m-xw-squashfs-sysupgrade.bin
The files there are a good start but as I started troubleshooting the "----" (why did it ignore my BUILD_SET_VERSION value?) I figured I should check with you on
- what are typical invocations for makeimage.sh
- how should the repositories look on disk (do I need symlinks somewhere?)
- how does the content of aredn_ar71xx get into the resulting image? (a etc/mesh-release file was genererated by 003-buildimage.sh but I see no trace of those files in openwrt-trunk/bin, openwrt-trunk/build_dir, etc)
I know this is a long post but I really thank you for taking the time to read it. I would hope to contribute to the project in the future and I think I did a good part of my homework asking questions.
Regards,
VA2MVC
VA2MVC, shot me an email at <callsign>@arrl.net . We're excited to see this level of knowledge and always eager for others that wish to contribute. Have you played with Luci/Lua stuff yet? I'll defer the script feedback to Conrad, who maintains the build automation and chief guru developer. We're running jenkins. The key linkage between the 2 repos is: ...../arednbase/files -> ......./aredn_ar71xx/files (see http://wiki.openwrt.org/doc/howto/build , custom files section). because 95% of the changes are in these files, a full build of all source code packages is not needed very frequently, the images can be put together with commands that build from scratch or simply reassemble the compiled packages with updates from the files.
Joe AE6XE
So first a couple of notes: The build script is not classified as production ready. The build script is as the commit says its intended to be run a Jenkins continuous integration server, it is a grab (with some re-writes to make it long term more flexible because my static config method is NOT flexible) from the code currently being executed by a Jenkins server I configured earlier this year. The build script is currently NOT utilized by any build systems and may have a bug or two in it and is more for documentation of the current steps usually taken to build an image (Every time I've made a change to the Jenkins server I've put an equivalent change into the build system to make sure we can swap over to the script after release 3.15.1.0 when all branches will have a copy of the script). The script is not necessarily currently a replacement from normal building skills with OpenWRT nor does it document everything needed in the backend before hand. The script also currently assumes it is starting from a BLANK system (the build server wipes the workspace before every build) so purging the base was a good decision on your part.
So now that those are out of the ways, aredn_ar71xx/files is usually copied to arednbase/files (per OpenWRT buildroot expectations and Joe's comment ). There are expectations by the scripts that some variables will be set by Jenkins though if they are not set as long as BUILD_SET_VERSION is (and its working which again this script is still not in production nor fully tested since the script is not yet utilized there could be an issue or two and on quick glance it does look like you caught an issue that occurred in the translating of the script) that you should be able to (currently) get away w/o the Jenkins variables as long as you copy the files into position
Usually I just build by hand when I'm doing personal testing (if needed) most code is testable w/o a recompile (copy into position on node) and the buildroot rebuilding is more about caputring it and making sure it doesn't interact in an unexpected manner later.
As for shellcheck reports those are known and expected right now, those are coming from the basic unit-check precheck scripts which are intended to be monitored for code errors in the various AREDN files. Its the first step in a sub project now that we have CI servers to begin code cleanup and capture programming flaws early. A lot of the BBHN code base which AREDN started with has known non posix compatibility or other non best practice code but is still 100% working inside of the known, controlled, OpenWRT base and as such do not pose a critical issue but are something we wish to capture, monitor and resolve.