NetBSD-9 gained much improved ZFS support. However, one feature it's still missing is the ability to have your system root on ZFS. For that, we need to teach our boot loader about ZFS,xi but until then we can work around this limitation by using a FFS boot partition and a ZFS root ramdisk. You'll also need to use NetBSD-9.99.48 or newer.

The idea is simple - the boot loader loads the NetBSD kernel and the required modules (solaris and zfs) from FFS and the ZFS root ramdisk. The ramdisk then imports the ZFS root pool called rpool, mounts rpool/ROOT to /altroot and finally instructs the kernel to chroot to it. Because the initial root is on a ramdisk, we can then load the boot partition from inside the ZFS chroot to maintain it; for example to update the kernel and modules.

Assumptions

The ramdisk does make some assumptions as there is no way to pass variables from boot.cfg:

• The kernel has matching modules installed in the same partition; you only need solaris and zfs.
• The ZFS root pool is called rpool and the root filesystem is called ROOT.

Once inside the chroot the kernel will load modules from the ZFS filesystem because there is a symlink to it from the ramdisk. The same is true for any firmware needed after boot.

Generic setup

Until the NetBSD installer can be updated, here are the manual steps from fresh:

• Build the ramdisk (cd src/distrib/amd64/ramdisks/ramdisk-zfsroot; nbmake-amd64)
• Boot the NetBSD installer
• Create a small (I used 2G) FFS partition and a normal swap partition
• Create a partition for ZFS
• Finish the NetBSD installation - install the etc, base and maybe rescue sets
• Reboot into your minimal NetBSD install

ZFS Setup

So far, so good. Now we need to make the needed adjustments to change root:

• Create a ZFS pool called rpool on your ZFS partition
• Create a ZFS filesystem called ROOT in rpool
• Set the mountpoint of rpool/ROOT to legacy so that the mount command can deal with it
• Create any other ZFS filesystems you want to - if you create /usr or /var or anything in these they must to be legacy mount points as well
• zfs umount rpool/ROOT
• Add rpool/ROOT /altroot zfs rw to /etc/fstab
• mount /altroot
• Extract the sets you need to /altroot. At a minimum you will need etc and base.
• cd /altroot/dev; ./MAKEDEV all
• cd /altroot; mkdir kern proc
• Copy /etc/wscons.conf, /etc/fstab and any other config files sysinst might have created to /etc/rc.conf to /altroot/etc
• Set zfs=YES in /altroot/etc/rc.conf
• Remove rpool/ROOT from /altroot/etc/fstab
• Add name=boot /altroot ffs rw,noauto to /altroot/etc/fstab along with any ZFS filesytems you created in '/usr' and '/var'
• Add the filesystems as well to critical_filesystems_local in /altroot/etc/rc.conf - such as critical_filesystems_local="/usr /var /var/log"
• Copy the ramdisk-zfsroot.fs to /
• Edit /boot.cfg and add menu=Boot ZFS Root:load solaris;load zfs;fs /ramdisk-zfsroot.fs;boot

And done! You can now reboot and enjoy your root on ZFS and all the benefits it brings. When updating the kernel, remember to mount /altroot and update it there as well as the modules.

Future work

All these steps could be done in the installer. This is probably an easier task than adding ZFS support to the bootloader. But don't let that stop you if you are stuck for something to do!