Last week I said I was going to spend this week on fixing the 5GHz
wireless suport and then put the GL-AR750 away and start working on
some other device. Some of that happened ...
Radio
Getting the ath10k driver running was mostly a similar process to the
ath9k that provides 2.4GHz wifi: I'd already done the work for
building and loading modules from the backports tree, so just had to
load more of them. The only extra fillip is that the ath10k driver
requires "calibration data" (for the full handwavey explanation see
my NixWRT blog
on the subject) which lives in a separate partition of the flash.
I don't want to extract this binary blob from my device and check it
into the Liminix source repo because (1) it's not source, and (2)
probably that ends up with having to do the same for everybody's
device. So I wrote a service to extract it at boot time and made the
module depend on it.
This and some reflection led to a rearrangement of responsibilities in
the module structure. Given that the function, whereabouts and
labelling of network ports on a particular hardware device are
properties of that device, Liminix device modules are now expected to
provide config.hardware.networkInterfaces.{lan,wan,wlan,whatever}
services that correspond to whatever actual connections the board
offers. For hardware ethernet the name should follow whatever's
printed on the case, or whatever the device manufacturer calls that
port; for wireless I don't have a good pattern yet so we still need to
use judgement.
The other thing that ath10k made apparent is that an insmod call can
return before the device is ready for use, so my insmod ath10k_pci && ip link set up dev wlan was failing because the second command ran
too soon. To accommodate this I have evolved (and simplified) waitup
into ifwait, which doesn't just wait for a device to be running
but can also wait for it to exist.
Everything else
Before I put this device down and start work on another, I want it to
boot unattended and without a serial connection. This meant:
adding outputs.flashable target to make a flashable image. This has
slightly different kernel config from the TFTP-bootable image we're using
in development, because it needs to have its kernel command line
hardcoded into the image (actually the DTB) instead of getting it from
the bootloader.
enabling ssh service,
provided by dropbear. This needs to be cleaned up and moved from
rotuer.nix into Liminix proper.
defining the config.users type properly so that an end-user
configuration.nix can set the root password
Sorting these little niggles out took most of the rest of the
week. Maybe next week I'll get to Hardware Device Number Two.
This week I have the MT-300A running in what I am pleased to call
"production", providing Wifi
access
in the outbuilding I use as an office, and an MT-300N-V2 almost in
production as my backup
host,
except that for reasons I haven't yet been able to ascertain it won't
mount its USB disk at boot time (though it works fine if I ssh in and
start that service after boot). It's a big lump of spinning rust and
it might just need more time to reach its target angular velocity.
Highlights
added the "swconfig" package, and a "vlan" network interface
type. This is for configuring the network switches that are built into
these devices - the device has two RJ45 sockets but only one ethernet
device visible to the kernel, so if you want to treat the LAN and WAN
ports separately then you do this with VLAN tags.
worked a bit on reducing image size, mostly by configuring hostap and
nftables with fewer options, and removing a copy of bash that had
crept in by thoughtless use of writeShellScript.
created an examples folder and moved rotuer.nix into it, along
with the configs for arhcive and extneder.
add hostname as a configuration option
add configuration options for Busybox, so we can start with a
minimal-ish Busybox that has only the bare minimum applets, and
then modules can specify the additional applets or features.
It would be absolutely lovely if packages could specify the
applets they need from busybox just as they do for
package-level dependencies. But I haven't figured out a nice
way to do that yet. I don't want an image with ten different
busybox packages that all have different config, I want one
busybox with the union of all those config.
small amounts of tidying up so we have slightly less copy-paste code.
What's next
I'm pretty happy with where we've got to: there are only a couple of
gaps between where we are now and NixWRT feature parity
find whatever's stopping arhcive from mounting its disk at boot
add NTP or some other way to get accurate time and date
That done, I need to turn my attention to the User Manual, which is
currently completely silent on how to flash the image onto the device.
This week: I made a video, then I went on a tangent.
The things I said last week that I was going to do this week were
fixing a bug in the backup server, and adding NTP - but all the
devices I need to do them to are deployed to "in the field" and I
would rather not have to unplug them again and reattach the serial
console wires to update them.
How can we make this better? After a bit of thought I have a new
approach, which works in principle but needs a bit of polish and more
testing. We use a combination of
kexec and
the watchdog timer to allow a new configuration that runs entirely in
RAM to be booted from a running system. If the new configuration is
doesn't work right, the watchdog will reboot the device - and because
the new system hasn't been written to flash, then it'll boot to the
previous configuration instead of bootlooping the new one.
The new configuration is built using outputs.kexecboot, which
generates a vmlinux, a dtb and a squashfs that can be
transferred to the device e.g. via ssh:
$ nix-build --show-trace -I liminix-config=./examples/arhcive.nix \
--arg device "import ./devices/gl-ar750" -A outputs.kexecboot \
&& (tar chf - result | ssh root@192.168.8.185 tar -C /run -xvf -)
(the magic numbers in this command will be generated appropriately by
tooling) to map the new root filesystem into a "phram" fake MTD device
and then boot the new kernel.
This involved
a patch to kexec-tools
to add the --map-file option,
which I hope to be able to upstream after cleaning it up. My use case
(create an MTD for the new system) is probably quite niche, but the
affordance in kexec-tools (map any file anywhere you like!) is pretty generic
so perhaps there are other uses it could be put to as well.
a small change to the kernel so that kexec provides the DTB address to the
kernel being exec'ed instead of just copying it into a place where it
might find it if MIPS_RAW_APPENDED_DTB is set.
Other fixes this week:
add kernel config to make USB host work on gl-ar750
make ssh key authentication work for root
extract repeated code for applying openwrt kernel patches into a
derivation
Next week ... the things we were originally going to do this week? I'd
like also to set up a mailing list for the project, but am trying to
find something more usable/searchable for the archives than Pipermail.
[ Reminder: more regular updates on what I'm spending most of my time
on lately are at https://www.liminix.org/ ]
I had occasion recently to set up some mailing lists and although the subject
matter for those lists is Liminix-relevant, the route to their
existence really isn't. So, some notes before I forget:
the NixOS Manual has a section on Mailman
which is a good start - though I didn't copy the ssl-related
directives because I already have an SSL cert for my mailserver
but it broke my mail delivery. It turns out that this is because
my Postfix configuration is a few years old and doesn't set
local_recipient_maps - although the default behaviour works OK,
setting it explicitly for Mailman overrides that default instead
of ignoring it. So we need to add them back:
next look at Configure Web Frontend - again, most of this is already done except for mailman-web createsuperuser, which has to be done as the mailman-web user.
to get HTTPS working on the mailman web interface, the nginx virtualHost needs enableACMEandforceSSL
now we point a web browser at lists.example.org/admin and login as the
superuser. Per the instructions in the FAQ we had to change (not delete! deleting will break everything! don't ask me how I know that :sad-angry-face:) the domain name and display name to something real.
now point a web browser at lists.example.org/ and we can start
adding the actual lists
Anyway, that's where we are. I'm quite certain I've done something wrong, but I'm yet to discover what.
As you'll see if you're reading this on www.liminix.org (but not if you're
using the RSS feed) we're now on version 0.2 of the web site. I've
starting to add a bit more structure beyond the blog entries and would
draw your attention particularly to:
Mailing Lists: one for
announcements, one for discussions, and one for techy stuff. Please
sign up, it's a bit lonely being the only subscriber.
Continuous Build
service, which has been running for a while and invaluable for most of
that while - but only this week have I made it available outside my
home network. The web site has some
notes on
what it tests: for more information read the source :-)
In development news - yes, there was also a little bit of development -
got "kexecboot" into a usable state, and wrote an explanation of how to use it.
wrote a proof-of-concept watchdog service. This is only in the
arhcive example right now and hasn't made it to Liminix "core"
removed the second copy of Lua I had accidentally introduced into the
image
unified kernel parameter-passing behaviour. I'm slightly proud of this...
At your command
The problem is this: many devices have hacked-up or restricted versions of
U-boot installed which hardcode the kernel command-line parameters
instead of letting the user configure them. These hardcoded parameters
are most likely wrong for Liminix, but luckily we can configure the
kernel to ignore the bootloader command line and use a builtin
command line or refer to the embedded DTB.
However, this creates another problem: when we are hacking away on an
open device with a serial cable, or when we are booting with
kexecboot, or when running emulated Liminix with QEMU, we want to
change the bootloader kernel command line.
So we needed one kernel with CONFIG_MIPS_CMDLINE_FROM_BOOTLOADER and one with CONFIG_MIPS_CMDLINE_FROM_DTB. Because
kernel configuration is a property of the configuration not of
the "output", this meant editing the configuration.nix to comment-out
or uncomment the kexecboot or flashable module depending on which
output was needed - a reliable source of strange behaviour whenever I forgot to.
Our solution is to add an option for "ignore the bootloader command
line unless it contains a magic string", and then ensure that the
tooling for tftpboot and kexecboot adds that string. The bootloader
hardcoded command line doesn't, so we have the desired behaviour in
both use cases.
This does require a
small kernel patch
but hopefully that's not a huge maintenance burden to carry.
(Fixing vendor U-boot on all devices is most certainly not within
scope for Liminix.)
Next week
Next week we are moving on to "Task 2":
NixWRT used a self-designed module system (instead of Nixos modules)
to allow us to easily express the configuration of multiple
instances of the same module – for example when two differently
configured copies of the same service are running on different
network interfaces. But it had no validity checking or type checking
of module options, making it painfully easy to generate and install
a bad image. In this phase we revisit this decision, and figure out
how to gain the advantages of the Nixos options/type system while
still allowing multiple instances of the same module