My sysadmin notes

Linux from scratch... in a qcow2 disk

Posted on Sep 25, 2024
Table of contents:

Linux From Scratch is a book which provides step-by-step instructions for building a GNU/Linux system entirely from source code.

I’ve always wanted to build my own LFS, to improve my understanding of GNU/Linux system architectures, but at the same time, I don’t want to break my laptop. I’ll describe in this post all the deviations from the book to build and boot a LFS virtual machine.

Let’s start with LFS v12.2-systemd.

I’ve chosen the systemd variant to learn new things about… systemd, which is now the industry standard in the Linux world. The main release of LFS is based on System V init scripts. You may consider it if you want to develop your shell scripting skills, and acquire knowledge which is still relevant, at least in the BSD world.

Pre-requisites

I work on Archlinux, I’ve only installed qemu-base, which includes the package qemu-img.

There are several ways of mounting a qcow2 file, I’ll use qemu-nbd because it works perfectly for this task. I’ve ruled out libguestfs (command guestmount), which uses fuse (file system in user space) because it seems to have difficulties to preserve file ownership information.

Creating a New Partition - Section 2.4

As root, create the qcow2 disk file lfs.qcow2.

qemu-img create -f qcow2 /root/lfs.qcow2 20G
Formatting 'lfs.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=21474836480 lazy_refcounts=off refcount_bits=16

Load the nbd kernel module (“network block device”), and bind the disk file to /dev/nbd0.

modprobe nbd max_part=8
qemu-nbd --connect /dev/nbd0 /root/lfs.qcow2

Verify that the size is consistent

lsblk /dev/nbd0
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
nbd0  43:0    0  20G  0 disk

Let’s create the partition layout with fdisk:

  • DOS (MBR)
  • root partition: 18GB, ext4
  • swap: 2GB
fdisk /dev/nbd0

Welcome to fdisk (util-linux 2.40.2).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.

Device does not contain a recognized partition table.
Created a new DOS (MBR) disklabel with disk identifier 0x05fd5918.

Command (m for help): p
Disk /dev/nbd0: 20 GiB, 21474836480 bytes, 41943040 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x05fd5918

Command (m for help): o
Created a new DOS (MBR) disklabel with disk identifier 0x178d7b95.

Command (m for help): n
Partition type
   p   primary (0 primary, 0 extended, 4 free)
   e   extended (container for logical partitions)
Select (default p):

Using default response p.
Partition number (1-4, default 1):
First sector (2048-41943039, default 2048):
Last sector, +/-sectors or +/-size{K,M,G,T,P} (2048-41943039, default 41943039): +18G

Created a new partition 1 of type 'Linux' and of size 18 GiB.

Command (m for help): n
Partition type
   p   primary (1 primary, 0 extended, 3 free)
   e   extended (container for logical partitions)
Select (default p):

Using default response p.
Partition number (2-4, default 2):
First sector (37750784-41943039, default 37750784):
Last sector, +/-sectors or +/-size{K,M,G,T,P} (37750784-41943039, default 41943039):

Created a new partition 2 of type 'Linux' and of size 2 GiB.

Command (m for help): t
Partition number (1,2, default 2):
Hex code or alias (type L to list all): 82

Changed type of partition 'unknown' to 'Linux swap / Solaris'.

Command (m for help): p
Disk /dev/nbd0: 20 GiB, 21474836480 bytes, 41943040 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x178d7b95

Device      Boot    Start      End  Sectors Size Id Type
/dev/nbd0p1          2048 37750783 37748736  18G 83 Linux
/dev/nbd0p2      37750784 41943039  4192256   2G 82 Linux swap / Solaris

Command (m for help): w
The partition table has been altered.
Calling ioctl() to re-read partition table.
Syncing disks.
⚠ If you want to configure the system for UEFI with a GPT-formatted disk, the recommended size of the EFI partition (1MB) is way too small, it will cause issues later when installing the boot loader. Choose 100MB instead

Creating a File System on the Partition - Section 2.5

Note that the root partition is /dev/nbd0p1, and the swap partition is /dev/nbd0p2.

mkfs.ext4 /dev/nbd0p1
mke2fs 1.47.1 (20-May-2024)
Discarding device blocks: done
Creating filesystem with 4718592 4k blocks and 1179648 inodes
Filesystem UUID: 61312e87-a920-49f0-8391-042532d7dd8e
Superblock backups stored on blocks:
        32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
        4096000

Allocating group tables: done
Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done

# mkswap /dev/nbd0p2
Setting up swapspace version 1, size = 2 GiB (2146430976 bytes)
no label, UUID=c187d56b-873e-47d6-b793-9f5578275ed4

Setting the $LFS variable - Section 2.6

This is a paraphrase, but do not forget this variable!

Set the $LFS variable, we will work exclusively in this directory.

export LFS=/mnt/lfs

Executing the next commands without setting the $LFS variable will likely destroy the host system sooner or later.

Mounting the New Partition - Section 2.7

mkdir -pv $LFS
mount -v -t ext4 /dev/nbd0p1 $LFS

At this point, you can proceed to build the cross toolchain and temporary tools until Section 7.3.

Preparing Virtual Kernel File Systems - Section 7.3

This is a copy paste from the documentation.

mount -v --bind /dev $LFS/dev
mount -vt devpts devpts -o gid=5,mode=0620 $LFS/dev/pts
mount -vt proc proc $LFS/proc
mount -vt sysfs sysfs $LFS/sys
mount -vt tmpfs tmpfs $LFS/run
if [ -h $LFS/dev/shm ]; then
  install -v -d -m 1777 $LFS$(realpath /dev/shm)
else
  mount -vt tmpfs -o nosuid,nodev tmpfs $LFS/dev/shm
fi

Because we use qemu-nbd, if you interrupt the build process and suspend/power off your laptop, partitions must be properly unmounted and the /dev/nbd0 block device must be disconnected, like in Section 7.13. To resume, the qcow2 file must be reconnected to /dev/nbd0 block device like in Section 2.4, the root partitions must be remounted like in Section 2.7, and the virtual file systems remounted as described above.

Entering the Chroot Environment - Section 7.4

At this point, the environment can be chrooted, which is an important milestone :)

chroot "$LFS" /usr/bin/env -i   \
    HOME=/root                  \
    TERM="$TERM"                \
    PS1='(lfs chroot) \u:\w\$ ' \
    PATH=/usr/bin:/usr/sbin     \
    MAKEFLAGS="-j$(nproc)"      \
    TESTSUITEFLAGS="-j$(nproc)" \
    /bin/bash --login

Cleaning up and Saving the Temporary System - Section 7.13

We can backup the qcow2 image, so that it is easy to rollback in the event of a catastrophic failure in the next sections. Unmount all the file systems and disconnect the /dev/nbd0 block device.

mountpoint -q $LFS/dev/shm && umount $LFS/dev/shm
umount $LFS/dev/pts
umount $LFS/{sys,proc,run,dev}
umount $LFS
qemu-nbd -d /dev/nbd0
/dev/nbd0 disconnected

The qcow2 file can now be copied safely.

cp /root/lfs.qcow2 /root/lfs.bkp.qcow2

Beginning Section 8

Re-enter the chroot, follow the commands given in Sections 2.4, 2.7, 7.3 and 7.4.

Creating the /etc/fstab file - Section 10.2

Because we use virtio with qemu, the local disk will be known as /dev/vda after boot. But if you use other settings or another hypervisor, it may be known as /dev/sda or maybe something else.

So instead of hard coding the block device name in the next configuration files, we will refer to the partitions using their UUID or PARTUUID, given by the command blkid.

blkid /dev/nbd0p1
/dev/nbd0p1: UUID="61312e87-a920-49f0-8391-042532d7dd8e" BLOCK_SIZE="4096" TYPE="ext4" PARTUUID="178d7b95-01"
blkid /dev/nbd0p2
/dev/nbd0p2: UUID="c187d56b-873e-47d6-b793-9f5578275ed4" TYPE="swap" PARTUUID="178d7b95-02"

The file /etc/fstab must be updated accordingly.

# Begin /etc/fstab

# file system                              mount-point  type     options             dump  fsck
#                                                                                          order

UUID=61312e87-a920-49f0-8391-042532d7dd8e  /            ext4     defaults            1     1
UUID=c187d56b-873e-47d6-b793-9f5578275ed4  swap         swap     pri=1               0     0

# End /etc/fstab

Using GRUB to Set Up the Boot Process - Section 10.4

Setting Up the Configuration - Section 10.4.3

Install to /dev/nbd0, do not touch your own hard disk as suggested in the documentation!

grub-install /dev/nbd0

Creating the GRUB Configuration File - Section 10.4.4

Note that I’ve updated the root variable (hd0,1) to point to the first partition of the first disk, and the root parameter (root=/dev/vda1 or better, root=PARTUUID=178d7b95-01) in the linux command line. The root partition is indicated by its PARTUUID, given above by blkid.

cat > /boot/grub/grub.cfg << "EOF"
# Begin /boot/grub/grub.cfg
set default=0
set timeout=5

insmod part_gpt
insmod ext2
set root=(hd0,1)

menuentry "GNU/Linux, Linux 6.10.5-lfs-12.2-systemd" {
        linux   /boot/vmlinuz-6.10.5-lfs-12.2-systemd root=PARTUUID=178d7b95-01 ro
}
EOF

Rebooting - Section 11.3

Exit the chroot, unmount the file systems.

umount -v $LFS/{dev/pts,dev/shm,sys,proc,run,dev,}

The nbd driver won’t be needed anymore.

qemu-nbd -d /dev/nbd0
modprobe -rv nbd

Conclusion

Boot the system using qemu

qemu-system-x86_64 -cpu host -machine type=q35,accel=kvm -m 2048 -snapshot -drive if=virtio,format=qcow2,file=lfs.qcow2

Et voilà!

GRUB menu in QEMU TTY in QEMU

It has been a very smooth experience, smoother than I expected. Note that the content of the root partition could just be rsync’ed to a real partition, the only requirement would be to adapt the fstab (Section 10.2) and GRUB configuration (Section 10.4.3).

I now have a functional LFS virtual machine, which is easy to copy and snapshot before experimenting and breaking everything :)