How to enlarge RAIDframe

Sometime one has to resize a filesystem. For FFS, we have resize_fss(8), but things are more complicated if the FFS filesystem is inside a RAIDframe set. This documents show how to fo deal with such a setup.

The use case is a RAID-1 RAIDframe set of two 6 TB hard disks that were formatted the old way, using MBR and disklabel. This only enables using the first 2 TB of the disks, and migrating to GPT is required to use the full 6 TB. We will also need to enlarge the RAID, a tricky operation that is documented here.

Our setup has two disks, wd0 and wd1, and raid0 is a RAID-1 set based on wd0a and wd1a. Here is what disklabel says about our two disks before the migration:

# disklabel wd0
 a: 4261410815      2048       RAID
 b:  33554432 4261412863       swapo

After migration we will have:

# gpt show -l wd0 
        start         size  index  contents
            0            1         PMBR (active)
            1            1         Pri GPT header
            2           32         Pri GPT table
           34         1980      1  GPT part - efi@wd0
         2014  11687488689      2  GPT part - raid0@wd0
  11687490703     33554432      3  GPT part - swap@wd0
  11721045135           32         Sec GPT table
  11721045167            1         Sec GPT header

# gpt show -l raid0
        start         size  index  contents
            0            1         PMBR
            1            1         Pri GPT header
            2           32         Pri GPT table
           34  11687488445      1  GPT part - root@raid0
  11687488479           32         Sec GPT table
  11687488511            1         Sec GPT header

There are two ways of resizing the RAIDframe. First here is Greg Oster's recommanded method:

This approach is long, as it requires copying all the data. It requires a few reboots, but it has the advantage that the data is always available during the migration.

A faster but less secure approach will be documented here, all the operations are done booting on an INSTALL kernel, using the ramdisk's shell. It takes aboit 10 minutes to complete, most of the time being spent in fsck and resize_ffs.

The approach is fast, but make a mistake on a partition start block and you data is gone. Note that rewriting RAID parity is fast, because it just reads the disks to check data is in sync. No copy is done. The longest operations are fsck and resize_ffs.

Here are the commands. Note that if you have other GPT-enabled disks, the dk devices unit numbers way vary. We install both EFI and BIOS boostraps. The latter requires a primary bootstrap from 2023-07-01 or later (nebsd-9 and netbsd-10 branches) to boot a GPT/RAID-1/GPT/FFS setup.

raidctl -G raid0 > raid0.conf.orig
raidctl -u raid0

gpt create -Af wd0
gpt create -Af wd1

gpt add -t efi -l efi@wd0 -b 34 -s $(( 2048 - 34 - 34 )) wd0
gpt add -t raid -l raid0@wd0 -b $(( 2048 - 34 )) -s 11687488689 wd0
gpt add -t swap -l swap@wd0 wd0
gpt biosboot -A -i 2 wd0

gpt add -t efi -l efi@wd1 -b 34 -s $(( 2048 - 34 - 34 )) wd1
gpt add -t raid -l raid0@wd1 -b $(( 2048 - 34 )) -s 11687488689 wd1
gpt add -t swap -l swap@wd1 wd1
gpt biosboot -A -i 2 wd1

# Read the dk unit numbers from dmesg

newfs_msdos /dev/r${efi0}
mount -t msdos /dev/${efi0} /mnt && mkdir -p /mnt/EFI/BOOT  && \
        cp /usr/mdec/bootx64.efi /mnt/EFI/BOOT && umount /mnt

newfs_msdos /dev/r${efi1}
mount -t msdos /dev/${efi1} /mnt && mkdir -p /mnt/EFI/BOOT && \
        cp /usr/mdec/bootx64.efi /mnt/EFI/BOOT && umount /mnt

sed "s/wd0a/${raid0}/; s/wd1a/${raid1}/;" raid0.conf.orig > raid0.conf
raidctl -C raid0.conf raid0
raidctl -A root raid0
raidctl -I 20230710 raid0   # Unique serial number is advised!
raidctl -i raid0

gpt create -Af raid0
gpt add -t ffs -l root@raid0 raid0
gpt set -a bootme -i 1 raid0

# check the dk unit numbe from dmesg

# INSTALL ramdisk lacks resize_ffs, copy it from the target filesystem
# This is the opportunity to check that eveyrthing went fine.
mount -o rw,log  /dev/${root} /mnt
cp /mnt/libexec/ld.elf_so /libexec/
cp /mnt/sbin/resize_ffs /tmp/
cp /mnt/lib/ /tmp/

# Copy secondary bootstrap
cp /usr/mdec/boot /mnt/boot

# Adjust fstab
cp /mnt/etc/fstab  /mnt/etc/fstab.orig
sed '  
' /mnt/etc/fstab.orig > /mnt/etc/fstab

umount /mnt

# Adjust console settings, this is for 115200 bps serial console
# and FFSv2 filesystem. As noted earlier, you need a recent bootxx_ffsv2
installboot -o console=com0,speed=115200 /dev/r${raid0} /usr/mdec/bootxx_ffsv2
installboot -o console=com0,speed=115200 /dev/r${raid1} /usr/mdec/bootxx_ffsv2

fsck -fy /dev/r${root}
# Make sure resize_ffs finds
export LD_LIBRARY_PATH=/tmp/
/tmp/resize_ffs -y /dev/r${root}

# Check new size
mount /dev/${root} /mnt
df -h /mnt
umount /mnt

# Reboot into normal operations