synaesthesia

One of the great things about Debian is the ease of migrating an installation through multiple releases via the dist-upgrade facility. However, regardless of the technical design, over time opinions change and begin to collide with those prior intentions.

One such example is the practice of placing /usr on a separate partition. Previously, doing so was the default approach of the Debian installer, but it has fallen out of favor due to the declining utility and popularity of remote-mounting /usr. Hidden boot-time dependencies on /usr have crept in as a result, in turn making a separate /usr mount increasingly impractical to maintain.

Upon upgrading a system with such a configuration to Debian 9 “Buster”, the user will be confronted an unbootable system and the following console gibberish:

Gave up waiting for /usr device. Common problems:
 - Boot args (cat /proc/cmdline)
   - Check rootdelay= (did the system wait long enough?)
 - Missing modules (cat /proc/modules; ls /dev)
ALERT! /dev/mapper/debian-usr does not exist. Dropping to shell!

BusyBox ........
Enter 'help' for a list of built-in commands.

/bin/sh: can't access tty; job control turned off
(initramfs)

What’s going on here? Even the initramfs init script seems to accommodate a separate /usr, searching for and mounting it:

if read_fstab_entry /usr; then
        log_begin_msg "Mounting /usr file system"
        mountfs /usr
        log_end_msg
fi

The actual problem is that on a LVM system, the block device (volume) containing /usr is not accessible because it hasn’t been activated. The reason is because the LVM initramfs scripts only activate two volumes: the volume designated as the root filesystem, and the volume containing a swap area designated as the resume device.

/usr/share/initramfs-tools/scripts/local-top/lvm2:

[..]

activate "$ROOT"
activate "$resume"

exit 0

A manual workaround to get the system booting from the initramfs prompt:

(initramfs) lvm vgchange -a y
  5 logical volume(s) in volume group "debian" now active
(initramfs) exit

To fix it permanently, add a custom script to /etc/initramfs-tools – the description of how to do this is in Debian bug #980021.

Happy hacking!

This entry was posted on Wednesday, January 13th, 2021 at 12:54 am and is filed under Embedded, Linux/UNIX/Open Source. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.