How to enlarge RAIDframe
Sometime one has to resize a filesystem. For FFS, we have resize_fss(8), but things are more complicated if the FFS filesystem is inside a RAIDframe set. This documents show how to fo deal with such a setup.
The use case is a RAID-1 RAIDframe set of two 6 TB hard disks that were formatted the old way, using MBR and disklabel. This only enables using the first 2 TB of the disks, and migrating to GPT is required to use the full 6 TB. We will also need to enlarge the RAID, a tricky operation that is documented here.
Our setup has two disks, wd0 and wd1, and raid0 is a RAID-1 set based on wd0a and wd1a. Here is what disklabel says about our two disks before the migration:
# disklabel wd0
(...)
a: 4261410815 2048 RAID
b: 33554432 4261412863 swapo
After migration we will have:
# gpt show -l wd0
start size index contents
0 1 PMBR (active)
1 1 Pri GPT header
2 32 Pri GPT table
34 1980 1 GPT part - efi@wd0
2014 11687488689 2 GPT part - raid0@wd0
11687490703 33554432 3 GPT part - swap@wd0
11721045135 32 Sec GPT table
11721045167 1 Sec GPT header
# gpt show -l raid0
start size index contents
0 1 PMBR
1 1 Pri GPT header
2 32 Pri GPT table
34 11687488445 1 GPT part - root@raid0
11687488479 32 Sec GPT table
11687488511 1 Sec GPT header
There are two ways of resizing the RAIDframe. First here is Greg Oster's recommanded method:
make sure you have a backup of your valuable data
fail wd1 in raid0 using raidctl -f /dev/wd0a raid0
migrate wd1 to GPT and resize the RAID partition
create a new raid1 with compenents NAME=raid0@wd1 and "absent"
creata a GPT and a FFS partition inside raid1
format the FFS filesystem inside raid1
copy data from raid0 FFS partiton to raid1 FFS partition
unconfigure raid0
migrate wd0 to GPT and resize the RAID partition
add NAME=raid@wd0 as a spare to raid1
reconstruct composent absent on spare NAME=raid@wd0
optionally renamae raid1 to raidà using raictl -U 0 raid1
This approach is long, as it requires copying all the data. It requires a few reboots, but it has the advantage that the data is always available during the migration.
A faster but less secure approach will be documented here, all the operations are done booting on an INSTALL kernel, using the ramdisk's shell. It takes aboit 10 minutes to complete, most of the time being spent in fsck and resize_ffs.
make sure you have a backup of your valuable data
unconfigure raid0
migrate wd0 and wd1 to GPT and enlarge the RAID partiton, lower the RAID partition start of 34 blocks, so that once a GPT is added in the RAID, the FFS partition star block in unchanged.
configure a new raid0 with components NAME=raid0@wd0 and NAME=raid0@wd1
create a GPT inside raidà with an FFS partition
DO NOT format the FFS partition, as you created it at the exact place where the pre-migration FFS partition was. You data is still there, unchanged.
You can now remount the FFS partition. It works without a hitch if you did not screw up things.
Rewrite the new RAID parity.
unmount the FFS partition and run fsck and resize_ffs to enlarge it.
The approach is fast, but make a mistake on a partition start block and you data is gone. Note that rewriting RAID parity is fast, because it just reads the disks to check data is in sync. No copy is done. The longest operations are fsck and resize_ffs.
Here are the commands. Note that if you have other GPT-enabled disks, the dk devices unit numbers way vary. We install both EFI and BIOS boostraps. The latter requires a primary bootstrap from 2023-07-01 or later (nebsd-9 and netbsd-10 branches) to boot a GPT/RAID-1/GPT/FFS setup.
raidctl -G raid0 > raid0.conf.orig
raidctl -u raid0
gpt create -Af wd0
gpt create -Af wd1
gpt add -t efi -l efi@wd0 -b 34 -s $(( 2048 - 34 - 34 )) wd0
gpt add -t raid -l raid0@wd0 -b $(( 2048 - 34 )) -s 11687488689 wd0
gpt add -t swap -l swap@wd0 wd0
gpt biosboot -A -i 2 wd0
gpt add -t efi -l efi@wd1 -b 34 -s $(( 2048 - 34 - 34 )) wd1
gpt add -t raid -l raid0@wd1 -b $(( 2048 - 34 )) -s 11687488689 wd1
gpt add -t swap -l swap@wd1 wd1
gpt biosboot -A -i 2 wd1
# Read the dk unit numbers from dmesg
efi0=dk0
efi1=dk3
raid0=dk1
raid1=dk4
newfs_msdos /dev/r${efi0}
mount -t msdos /dev/${efi0} /mnt && mkdir -p /mnt/EFI/BOOT && \
cp /usr/mdec/bootx64.efi /mnt/EFI/BOOT && umount /mnt
newfs_msdos /dev/r${efi1}
mount -t msdos /dev/${efi1} /mnt && mkdir -p /mnt/EFI/BOOT && \
cp /usr/mdec/bootx64.efi /mnt/EFI/BOOT && umount /mnt
sed "s/wd0a/${raid0}/; s/wd1a/${raid1}/;" raid0.conf.orig > raid0.conf
raidctl -C raid0.conf raid0
raidctl -A root raid0
raidctl -I 20230710 raid0 # Unique serial number is advised!
raidctl -i raid0
gpt create -Af raid0
gpt add -t ffs -l root@raid0 raid0
gpt set -a bootme -i 1 raid0
# check the dk unit numbe from dmesg
root=dk6
# INSTALL ramdisk lacks resize_ffs, copy it from the target filesystem
# This is the opportunity to check that eveyrthing went fine.
mount -o rw,log /dev/${root} /mnt
cp /mnt/libexec/ld.elf_so /libexec/
cp /mnt/sbin/resize_ffs /tmp/
cp /mnt/lib/libc.so.12 /tmp/
# Copy secondary bootstrap
cp /usr/mdec/boot /mnt/boot
# Adjust fstab
cp /mnt/etc/fstab /mnt/etc/fstab.orig
sed '
s|/dev/raid0a|NAME=root@raid0|;
s|/dev/wd0b|NAME=swap@wd0|;
s|/dev/wd1b|NAME=swap@wd1|;
' /mnt/etc/fstab.orig > /mnt/etc/fstab
umount /mnt
# Adjust console settings, this is for 115200 bps serial console
# and FFSv2 filesystem. As noted earlier, you need a recent bootxx_ffsv2
installboot -o console=com0,speed=115200 /dev/r${raid0} /usr/mdec/bootxx_ffsv2
installboot -o console=com0,speed=115200 /dev/r${raid1} /usr/mdec/bootxx_ffsv2
fsck -fy /dev/r${root}
# Make sure resize_ffs finds libc.so.12
export LD_LIBRARY_PATH=/tmp/
/tmp/resize_ffs -y /dev/r${root}
# Check new size
mount /dev/${root} /mnt
df -h /mnt
umount /mnt
# Reboot into normal operations
reboot