diary @ telent

All MIPSy were the borogroves#

Tue Dec 26 23:14:32 2017

Topics: nixwrt nix

My New Year's Resolution is to blog something every Tuesday (shut up at the back there, I haven't been to bed yet so it's still nominally Tuesday in my personal timezone) whenever I haven't posted in the preceding week.

Recently I had the idea of repurposing my previous wireless router (a TL-WR842ND ) as the brain for a backup server in my study, by plugging a USB disk into it and installing rsync. In order to fulfill my yak shaving quota, I decided to do this using Nixpkgs/NixOS instead of just doing the sensible thing and installing the relevant OpenWRT package.

Story so far:

In the pursuit of getting a serial console on it, I have probably burnt out the UART by bad soldering and/or inadvertently connecting TX to the 5V rail.

But I also have an Arduino Yun lying around which has an Atheros AR9331 MIPS 32 bit SoC - more or less the same hardware as most consumer broadband routers - with an Arduino microcontroller stuck to it that can be persuaded into a role as a USB/serial converter - so I have a console and no soldering required. This felt like a waste of an Atmega, but clearly in a good cause so I pressed on.

Right now I'm at the stage where I can build a bootable kernel and an unuseably large filesystem for it. Here are some things I have learned:

Currently I am working from the branch named in that pull request: ar works and strip too, but I still have huge image sizes, now because it has decided that glibc depends on gcc and the kernel headers - this seems to be a problem with cross-compilation generally and not with MIPS specifically, because ARM has the same issue.

[dan@loaclhost:~/src/nixwrt]$ nix-store -q --references /nix/store/pf047ij2z1bfzlkkyf0v7m4p273713d6-glibc-2.26-75-armv6l-unknown-linux-gnueabihf-armv6l-unknown-linux-gnueabihf/
/nix/store/3x1wd17r8fg3zhasljxdm6vabyn7qr5y-gcc-6.4.0-armv6l-unknown-linux-gnuea
/nix/store/xh15qw8k4za1va29ks7z3kjbjlcfb15v-linux-headers-4.4.10-armv6l-unknown-
/nix/store/pf047ij2z1bfzlkkyf0v7m4p273713d6-glibc-2.26-75-armv6l-unknown-linux-g

It is entirely possible, of course, that I will never get a GLibc-based system into 8MiB even when it's not dragging in the kitchen sink, the plumber that installed it, and the staff and plant of the factory that made the plumber's van (er, figuratively) and I should switch to uclibc or musl, but in the meantime this is all educational.

I have some clearly still very work-in-progress code at https://github.com/obsidiansystems/nixpkgs/compare/02726a2...telent:nixwrt-cross-elegant for anyone who wants to see it.

In other news, I've also been addressing my apparent need to solder stuff by having Fun With Arduinos and Neopixels

[ Postemporaneous edit: the next thrilling installment in this series is now up at https://ww.telent.net/2018/1/2/gehen_sie_bitte_mit_hier_ist_nix_zu_sehen ]

gehen Sie bitte mit, hier ist Nix zu sehen#

Tue Jan 2 11:08:23 2018

Topics: nixwrt nix

[ Meta: I don't actually speak German. I hope the pun works, but I have no particular reason to suppose it should do. ]

Happy New Year, if you observe the Gregorian Calendar. This week in NixWRT was typified by lots of beating head on brick wall followed by an unexpected achievement: I have a working rootfs in qemu!

Look, isn't it cool?

[nix-shell:~/src/nixwrt]$ qemu-system-mipsel  -M malta -m 64 -nographic -kernel 
linux-*/vmlinux   -append 'root=/dev/sr0 console=ttyS0 init=/bin/sh' -blockdev d
river=file,node-name=squashed,read-only=on,filename=tftproot/rootfs.image -block
dev driver=raw,node-name=rootfs,file=squashed,read-only=on -device ide-cd,drive=
rootfs -nographic                                                               
Linux version 4.14.1 (dan@loaclhost) (gcc version 6.4.0 (GCC)) #2 SMP Tue Jan 2 
14:58:10 UTC 2018                                                               
[...]
BusyBox v1.27.2 () built-in shell (ash)

# LD_TRACE_LOADED_OBJECTS=1 /nix/store/*-rsync*/bin/rsync --version
        linux-vdso.so.1 (0x77cc8000)
        libpopt.so.0 => /nix/store/79ffdcjvk5bpbm1vgrxii935vhjbdg5p-popt-1.16-mi
psel-unknown-linux-gnu/lib/libpopt.so.0 (0x77c70000)
        libc.so.6 => /nix/store/7njknf9mhcj7jd3l0axlq8ql0x7396pk-glibc-2.26-75-m
ipsel-unknown-linux-gnu-mipsel-unknown-linux-gnu/lib/libc.so.6 (0x77ad4000)
        /nix/store/7njknf9mhcj7jd3l0axlq8ql0x7396pk-glibc-2.26-75-mipsel-unknown
-linux-gnu-mipsel-unknown-linux-gnu/lib/ld.so.1 (0x77c98000)

Points of note here:

(-> head wall)

I spent a lot of time, with no actual result yet, on getting the Yun to tftp its kernel and rootfs and run them in-place without having to write anything to flash. Motivation here is: it's not my Yun, it belongs to my employer who will probably want it back next time we do a hackathon or something. So I don't want to brick the device accidentally, nor use all the flash erase cycles, and anyway it's probably slower than running from RAM.

This is my theory which almost works but for some reason not quite: we should be able to tftp the root fs into RAM then use the MTD "phram" driver to emulate an MTD device at that address, and the memmap option to hide that region of memory from the Linux system (so it doesn't overwrite it)

ar7240> setenv kernaddr 0x81000000
ar7240> setenv rootaddr 1178000
ar7240> setenv rootaddr_useg 0x$rootaddr
ar7240> setenv rootaddr_ks0 0x8$rootaddr
ar7240> setenv bootargs keep_bootcon console=ttyATH0,250000 panic=10 oops=panic init=/bin/sh
phram.phram=rootfs,$rootaddr_ks0,9Mi root=/dev/mtdblock0 memmap=10M\$$rootaddr_useg
ar7240> setenv bootn "tftp $kernaddr /tftp/kernel.image ; tftp $rootaddr_ks0 
/tftp/rootfs.image; bootm  $kernaddr"
ar7240> run bootn

Here's where it gets weird. With those options, it rus most of the way through boot then hangs after printing NET: Registered protocol family 17 (that's netlink, if you were wondering). If I misspell the console device name, though, it gets slightly further. wat?

NET: Registered protocol family 17
Warning: unable to open an initial console.
VFS: Mounted root (squashfs filesystem) readonly on device 31:0.
Freeing unused kernel memory: 208K
This architecture does not have kernel memory protection.
Kernel panic - not syncing: Attempted to kill init! exitcode=0x00000000

So it's identified that there is a squashfs filesystem there, which is a positive sign, but it's not going to run init without a console.

Also falling into the "known unknowns" quadrant: you will note that we randomly set and unset the high bit on some of our addresses there: this is because the same physical RAM is mapped into more than one place in the MIPS address space and I sort of think I have a handle on how it works but not really.

[ Postemporaneous edit: the next thrilling installment in this series is now up at https://ww.telent.net/2018/1/7/baud_games ]