Argumentum ad arborem fabrica
Wed, 25 Jul 2018 22:54:16 +0000
What I'd like you to take away from this post title is that I speak about as much Latin as I do German.
What I'd like you to take away from the post body is (i) that I have a solution for the problem it describes, and (ii) that it required a tonne more of reading code and adding debugging statements and experimenting than I think it reasonably ought to have done, so look upon my words ye mighty and despair. The picture is a Google search result for "flattened tree", if you were wondering.
So, as I said previously we have (now, "had") a problem with kexec and specifying the command line arguments to the kernel: on the one hand we want to ignore any arguments that the bootloader provides, because generally they're probably wrong, but on the other we want to pay attention to the command line when booted by kexec, because the appropriate parameters for booting from flash are not also appropriate when booting from RAM. I'm going to skip over the voyage of discovery here because it's almost as tedious to relate as it was to, er, discover. So here are the highlights:
Kexec on MIPS (for ELF) provides two ways to supply the kernel command line.
The first option is that you add a segment which starts with the magic
"kexec " to the list of segments that you call
with, and then the pre-reboot kernel kexec code
machine_kexec_init_argv) iterates through the segments, finds the
one with the right magic prefix, and parses it into
Then after the reboot, code in
relocate_kernel.S loads the argument
vector into register
a1 before it calls into the new
head.S in the new kernel then copies the pointer into
fw_arg1, and then some board-specific code is responsible for what
happens next. For the ralink case, this is
ralink/prom.c which copies the argument vector back into a single
arcs_cmdline. After that, the next point of interest is in
kernel/setup.c which tests a complex combination of kernel config options
to decide which of
builtin_cmdline (gotta love that consistent use of abbreviations) are used and in what combination to form the
command line that the kernel will actually see.
There is a comment in the kexec source to say that this only works on an Octeon. Now that I trace the entire execution path I no longer understand why it only works on Octeon, but I will note that it didn't work for me. And, incidentally, wouldn't solve the problem if it did as we can't identify whether the command line came from kexec or u-boot. Anyway, taking inspiration from the said comment that this is "legacy", I decided to go with the second way.
The second way is to pass a DTB (a compiled device tree) from kexec,
and embed a command line in there. There's a branch of the tree
chosen and within that is a leaf called
that's where you find the command line. As a string, not an array of
strings, please note.
In stock Linux there are two defined ways tell your kernel where its
DTB is (in addition to anything your bootloader might do, if your
bootloader is an accomplice rather than an adversary). The first
option is to include it in the kernel as a special ELF section, or the
second is to append it (using
cat or similar) to a raw kernel image.
It should be noted that the first approach only works if your kernel
image is ELF (ours isn't) and the second - aside from being somewhat
brittle if you ever boot a kernel where you forgot to concatenate the
DTB - only works if your "raw image" is a zImage (ours isn't). So
it's probably not at all surprising that OpenWRT have added a third
kernel/head.S they've added 16kB of zeroes preceded by the
magic string "OWRTDTB:", then provided a utility called
run at kernel build time, that looks through the kernel image for this
string and patches a provided DTB into place. This location is
__image_dtb, and for ralink boards there's some code in
plat_mem_setup to call
__dt_setup_arch on it.
(You will observe, if you're following all this, that this code is unconditional, so the third option is not so much an option as an override)
early_init_dt_scan which calls
early_init_dt_scan_nodes which calls
After that, we're back to
kernel/setup.c and the same complex
combination of kernel config options we already saw, to decide which
builtin_cmdline are used
- except that this time the answer we want is
So: how do we get kexec to inject a different DTB (which will in
practice be a very similar DTB except for the
into this sequence? Three things.
First, the userland side . This turns out to be pretty simple if you have the ELF code to crib from - we read or create the DTB in
RAM, and then add it as a segment to the segment list that
kexec_load is called with.
Second, the pre-reboot kernel side. There is existing support in the
MIPS "generic" kernel for finding this segment by stepping through
the list looking for an fdt
. We're not running the generic kernel - we're running the ralink
kernel - so we moved that code into
and made it conditional on
CONFIG_USE_OF. The effect of this code,
if it finds a DTB, is to put its address into
kexec_args and set
kexec_args to -2.
Third, after reboot the
kexec_args ends up in register
then gets copied to
fw_passed_dtb. All that remains after that is to
change the code that hardcodes using
image_dtb into code that
image_dtb if we didn't get a DTB some other way -
- and we're basically good to go. One yak successfully popped off the shaving stack.