My New Year's Resolution is to blog something every Tuesday (shut up
at the back there, I haven't been to bed yet so it's still nominally
Tuesday in my personal timezone) whenever I haven't posted in the
preceding week.
Recently I had the idea of repurposing my previous wireless router (a
TL-WR842ND ) as the
brain for a backup server in my study, by plugging a USB disk into it
and installing rsync. In order to fulfill my yak shaving quota, I decided to
do this using Nixpkgs/NixOS instead of just doing the sensible thing and
installing the relevant OpenWRT package.
Story so far:
In the pursuit of getting a serial console on it, I have probably
burnt out the
UART by
bad soldering and/or inadvertently connecting TX to the 5V rail.
But I also have an Arduino Yun lying around which has an Atheros
AR9331 MIPS 32 bit SoC - more or less the same hardware as most
consumer broadband routers - with an Arduino microcontroller stuck to
it that can be persuaded into a role as a USB/serial
converter - so I
have a console and no soldering required. This felt like a waste of
an Atmega, but clearly in a good cause so I pressed on.
Right now I'm at the stage where I can build a bootable kernel and an
unuseably large filesystem for it. Here are some things I have
learned:
Nix finds runtime
dependencies
by grepping the generated package output for the store path for each
build-time dependency and discarding the ones it can't find. I am in
awe that this works. I know this because I was getting huge
filesystem image sizes, because the "strip" invocation in package
postinstallation was failing silently so the debugging symbols which
had not been expunged were causing it to hang on to references to
three different kinds of gcc and glibc and and Various Other Random Stuff.
Nix cross-compilation support is in a state of flux right now. If
"state of flux" is not an oxymoron, anyway. At one point I got stuck
and employed the "let's just upgrade everything" fault resolution
technique, which gave me a new and different fault that I opened an
issue about , and learned
that I should probably not expect anything to work as expected until
PR #26805 is merged.
Currently I am working from the branch named in that pull request: ar
works and strip too,
but I still have huge image sizes, now because it has decided that glibc
depends on gcc and the kernel headers - this seems to be a problem
with cross-compilation generally and not with MIPS specifically,
because ARM has the same issue.
It is entirely possible, of course, that I will never get a
GLibc-based system into 8MiB even when it's not dragging in the
kitchen sink, the plumber that installed it, and the staff and plant
of the factory that made the plumber's van (er, figuratively) and I
should switch to uclibc or musl, but in the meantime this is all
educational.
I have some clearly still very work-in-progress code at
https://github.com/obsidiansystems/nixpkgs/compare/02726a2...telent:nixwrt-cross-elegant
for anyone who wants to see it.
In other news, I've also been addressing my apparent need to solder
stuff by having Fun With Arduinos and Neopixels
[ Postemporaneous edit: the next thrilling installment in this series is now up at https://ww.telent.net/2018/1/2/gehen_sie_bitte_mit_hier_ist_nix_zu_sehen ]
[ Meta: I don't actually speak German. I hope the pun works, but I
have no particular reason to suppose it should do. ]
Happy New Year, if you observe the Gregorian Calendar. This week in
NixWRT was typified by lots of beating head on brick wall followed by
an unexpected achievement: I have a working rootfs in qemu!
Look, isn't it cool?
[nix-shell:~/src/nixwrt]$ qemu-system-mipsel -M malta -m 64 -nographic -kernel
linux-*/vmlinux -append 'root=/dev/sr0 console=ttyS0 init=/bin/sh' -blockdev d
river=file,node-name=squashed,read-only=on,filename=tftproot/rootfs.image -block
dev driver=raw,node-name=rootfs,file=squashed,read-only=on -device ide-cd,drive=
rootfs -nographic
Linux version 4.14.1 (dan@loaclhost) (gcc version 6.4.0 (GCC)) #2 SMP Tue Jan 2
14:58:10 UTC 2018
[...]
BusyBox v1.27.2 () built-in shell (ash)
# LD_TRACE_LOADED_OBJECTS=1 /nix/store/*-rsync*/bin/rsync --version
linux-vdso.so.1 (0x77cc8000)
libpopt.so.0 => /nix/store/79ffdcjvk5bpbm1vgrxii935vhjbdg5p-popt-1.16-mi
psel-unknown-linux-gnu/lib/libpopt.so.0 (0x77c70000)
libc.so.6 => /nix/store/7njknf9mhcj7jd3l0axlq8ql0x7396pk-glibc-2.26-75-m
ipsel-unknown-linux-gnu-mipsel-unknown-linux-gnu/lib/libc.so.6 (0x77ad4000)
/nix/store/7njknf9mhcj7jd3l0axlq8ql0x7396pk-glibc-2.26-75-mipsel-unknown
-linux-gnu-mipsel-unknown-linux-gnu/lib/ld.so.1 (0x77c98000)
Points of note here:
I fixed the stupid-huge image size I talked about last week
by removing dontStrip in
the glibc
derivation
. Although I don't know if this is correct, it Seems To Work. My
hypothesis is that the derivation was previously running an x86 strip
on target (in this case, MIPS) binaries and trashing them, and so
dontStrip was added to stop it doing that. Now it's using a strip
that understands MIPS, so it can be re-enabled. Is my guess.
@mksquashfs@ is super fussy about trailing slashes on filenames when
you use the -root-becomes option to regraft directories. For best results, don't use any
(-> head wall)
I spent a lot of time, with no actual result yet, on getting the Yun
to tftp its kernel and rootfs and run them in-place without having to
write anything to flash. Motivation here is: it's not my Yun, it
belongs to my employer who will
probably want it back next time we do a hackathon or something. So I
don't want to brick the device accidentally, nor use all the flash
erase cycles, and anyway it's probably slower than running from RAM.
This is my theory which almost works but for some reason not quite: we
should be able to tftp the root fs into RAM then use the MTD "phram"
driver to emulate an MTD device at that address, and the memmap
option to hide that region of memory from the Linux system (so it
doesn't overwrite it)
Here's where it gets weird. With those options, it rus most of the
way through boot then hangs after printing NET: Registered protocol
family 17 (that's netlink, if you were wondering). If I misspell the
console device name, though, it gets slightly further. wat?
NET: Registered protocol family 17
Warning: unable to open an initial console.
VFS: Mounted root (squashfs filesystem) readonly on device 31:0.
Freeing unused kernel memory: 208K
This architecture does not have kernel memory protection.
Kernel panic - not syncing: Attempted to kill init! exitcode=0x00000000
So it's identified that there is a squashfs filesystem there, which is
a positive sign, but it's not going to run init without a console.
Also falling into the "known unknowns" quadrant: you will note that we
randomly set and unset the high bit on some of our addresses there:
this is because the same physical RAM is mapped into more than one
place in the MIPS address space
and I sort of think I have a handle on how it works but not really.
[ Postemporaneous edit: the next thrilling installment in this series is now up at https://ww.telent.net/2018/1/7/baud_games ]