diary @ telent

Time to tidy up#

Sun 09 Jul 2023 03:25:12 PM UTC

Topics: liminix

Last week I wrote:

The focus of development this week has been "CPE wifi router by any means possible", with the intention that we will extract modules and services from the running system once we have ... a running system.

Well, good news is that we have a CPE wifi router. I'm not writing this while connected to it, but I could be if I weren't sitting at the other end of the garden. It has PPPoE to its upstream, IPv4 connectivity through NAT and an IPv6 /64 network. Packet routing, DHCP (v4 and stateless v6), DNS service and accurate time. Not good but also not news is that "any means possible" has meant that overall it's less than polished.

So, victory of sorts. The next step is to make it look less like a horrible mess - I have some ideas about how that can be done but they will have to wait until daylight.

Illogical Volume Management#

Sun 16 Jul 2023 11:29:18 AM UTC

Topics: nix hardware

I bought a new SSD for my primary desktop system, because the spinning rust storage I originally built it with is not keeping up with the all the new demands I'm making of it lately: when I sit down in front of it in the morning and wave the mouse around, I have to sit listening to rattly disk sounds for tens of seconds while it pages my desktop session back in. For reasons I can no longer remember, the primary system partition /dev/sda3 was originally set up as a LVM PV/VG/VL with a bcache layered on top of that. I took the small SSD out to put the big SSD in, so this seemed like a good time to straighten that all out.

Removing bcache

Happily, a bcache backing device is still readable even after the cache device has been removed.

echo eb99feda-fac7-43dc-b89d-18765e9febb6 > /sys/block/bcache0/bcache/detach

where the value of the uuid eb99...ebb6 is determined by looking in /sys/fs/bcache/ (h/t DanielSmedegaardBuus on Stack Overflow )

It took either a couple of attempts or some elapsed time for this to work, but eventually resulted in

# cat /sys/block/bcache0/bcache/state
no cache

so I was able to boot the computer from the old HDD without the old SSD present

Where is my mind?

At this time I did a fresh barebones NixOS 23.05 install onto the new SSD from an ISO image on a USB stick. Then I tried mounting the old disk to copy user files across, but it wouldn't. Even, for some reason, after I did modprobe bcache. Maybe weird implicit module dependencies?

The internet says that you can mount a bcache backing device even without bcache kernel support, using a loop device with an offset:

If bcache is not available in the kernel, a filesystem on the backing device is still available at an 8KiB offset.

... but, that didn't work either? binwalk will save us:

$ nix-shell -p binwalk --run "sudo binwalk /dev/backing/nixos"|head

DECIMAL       HEXADECIMAL     DESCRIPTION
--------------------------------------------------------------------------------
41943040      0x400000        Linux EXT filesystem, blocks count: 730466304, image size: 747997495296, rev 1.0, ext4 filesystem data, UUID=37659245-3dd8-4c60-8aec-cdbddcb4dcb4, volume name "nixos"

The offset is not 8K, it's 8K * 512. Don't ask me why, I only work here. So we can get to the data using

$ sudo mount /dev/backing/nixos /mnt -o loop,offset=4194304

and copy across the important stuff like /home/dan/src and my .emacs. But I'd rather like a more permanent solution as I want to carry on using the HDD for archival (it's perfectly fast enough for my music, TV shows, Linux ISOs etc) and nixos-generate-config gets confused by loop devices with offsets.

If it were an ordinary partition I'd simply edit the partition table to add 8192 sectors to the start address of sda3, but I don't see a straightforward way to do the analogous thing with a logical volume.

Resolution

Courtesy of Andy Smith's helpful blog post (you should read it and not rely on my summary) and a large degree of luck, I was able to remove the LV completely and turn sda3 back into a plain ext4 partition. We follow the steps in his blog post to find out how many sectors at the start of sda3 are reserved for metadata (8192) and how big each extent is (8192 sectors again, or 4MiB). Then when I looked at the mappings:

sudo pvdisplay --maps /dev/sda3
  --- Physical volume ---
  PV Name               /dev/sda3
  VG Name               backing
  PV Size               2.72 TiB / not usable 7.44 MiB
  Allocatable           yes (but full)
  PE Size               4.00 MiB
  Total PE              713347
  Free PE               0
  Allocated PE          713347
  PV UUID               7ec302-b413-8611-ea89-ed1c-1b0d-9c392d

  --- Physical Segments ---
  Physical extent 0 to 713344:
    Logical volume	/dev/backing/nixos
    Logical extents	2 to 713346
  Physical extent 713345 to 713346:
    Logical volume	/dev/backing/nixos
    Logical extents	0 to 1

It's very nearly a continuous run, except that the first two 4MiB chunks are at the end. But ... we know there's a 4MiB offset from the start of the LV to the ext4 filesystem (because of bcache). Do the numbers match up? Yes!

Physical extent 713345 to 713346 are the first two 4MiB chunks of /dev/backing/nixos. 0-4MiB is bcache junk, 4-8MiB is the beginning of the ext4 filesystem, all we need to do is copy that chunk into the gap at the start of sda3 which was reserved for PV metadata:

# check we've done the calculation correctly
# (extent 713346 + 4MiB for PV metadata)
$ sudo dd if=/dev/sda3 bs=4M skip=713347  count=1 | file -
/dev/stdin: Linux rev 1.0 ext4 filesystem data, UUID=37659245-3dd8-4c60-8aec-cdbddcb4e3c8, volume name "nixos" (extents) (64bit) (large files) (huge files)
# save the data
$ sudo dd if=/dev/sda3 bs=4M skip=713347  count=1 of=ext4-header

# backup the start of the disk, in case we got it wrong
$ sudo dd if=/dev/sda3 bs=4M  count=4 of=sda3-head

# deep breath, in through nose
# exhale
# at your own risk, don't try this at home, etc etc
$ sudo dd bs=4M count=1 conv=nocreat,notrunc,fsync if=ext4-header of=/dev/sda3
$

It remains only to fsck /dev/sda3, just in case, and then it can be mounted somewhere useful.

With hindsight, the maths is too neat to be a coincidence, so I think I must have used some kind of "make-your-file-system-into-a-bcache-device tool" to set it all up in the first place. I have absolutely no recollection of doing any such thing, but Firefox does say I've visited that repo before ...

Full service wash#

Sun 23 Jul 2023 08:21:00 AM UTC

Topics: liminix

Once upon a time I wrote

With that in mind, I’ve decided to defer the redesign of modules until later down the line when I also need to do some serious thinking about services. There’s no point considering one without the other.

and today I'm pleased to say I think I have a plan. To recap the problem:

So how do we resolve this? We're taking the obvious-in-hindsight path: do both! Where a service depends on global state (e.g. firewall support depends on kernel nftables options) then instead of making that service globally accessible in pkgs, we define it in a module that specifies the required state. Including the module does not create any services in itself, but makes the service definition available in config.system.service.firewall. So you do

  imports = [
    ../modules/firewall
  ];

  services.firewall = config.system.service.firewall {
    ruleset = import ./my-firewall-ruleset.nix;
  };

If you don't add the import, there is no config.system.service.firewall, so no risk of accidentally adding the service without its prerequisite global config.

The other half of making this user-friendly is that we typeckeck the service definition parameters, using the NixOS module type system. For example, in the dnsmasq service we see

  t = {
    user = mkOption {
      type = types.str;
      default = "dnsmasq";
    };
    group = mkOption {
      type = types.str;
      default = "dnsmasq";
    };
    resolvconf = mkOption {
      type = types.nullOr liminix.lib.types.service;
      default = null;
    };
    interface = mkOption {
      type = liminix.lib.types.service;
      default = null;
    };
    upstreams = mkOption {
      type = types.listOf types.str;
      default = [];
    };
    ranges = mkOption {
      type = types.listOf types.str;
    };
    domain = mkOption {
      type = types.str;
    };
  };

(Some of these types are still a bit "loose": we could use a type for IP addresses/networks, and a type for "s6 service that describes a network interface" instead of "any s6 service". And I've been very slack about adding description fields, but they will be super-useful too when we get to the point of producing documentation from these definitions.)

As a related cleanup, we moved config.outputs to config.system.outputs, because it's not right to be spattering derivations and build products all over the "config" namespace that rightfully ought to be for configuration. (By that token config.system is still philosophically a misuse of config, but it's the only misuse of config we're going to have: everything we want to carry around in our modules is going to live under that prefix). We also now use the module type system to specify the outputs by name instead of just having a grab bag that any module can add new names to. (I am grateful to Samuel Dionne-Riel here for the description and reasoning in mobile-nixos#406 describing the Mobile NixOS switch from system.build to mobile.outputs)

Current status: this is in progress

⟪ Jun 2023 Aug 2023 ⟫