[[!meta title="Root On ZFS"]]

[[!template id=needs-update reason="""
the `fs ramdisk-zfsroot.fs` in [[!template id=man name="boot.cfg" section="5"]] obviates the need for a custom kernel module with the ramdisk embedded
"""]]

NetBSD-9 gained much improved ZFS support.
However, one feature it's still missing is the ability to have your system
root on ZFS.
For that, we need to teach our boot loader about ZFS,xi
but until then we can work around this limitation by using a FFS boot
partition and a ZFS root ramdisk.
You'll also need to use NetBSD-9.99.48 or newer.

The idea is simple - the boot loader loads the NetBSD kernel and the
required modules *(solaris and zfs)* from FFS and the ZFS root ramdisk.
The ramdisk then imports the ZFS root pool called `rpool`,
mounts `rpool/ROOT` to `/altroot` and finally instructs the kernel to
chroot to it.
Because the initial root is on a ramdisk,
we can then load the boot partition from inside the ZFS chroot to maintain it;
for example to update the kernel and modules.

## Assumptions

The ramdisk does make some assumptions as there is no way to pass variables
from boot.cfg:

  *  The kernel has matching modules installed in the same partition; you only need `solaris` and `zfs`.
  *  The ZFS root pool is called `rpool` and the root filesystem is called `ROOT`.

Once inside the chroot the kernel will load modules from the ZFS filesystem
because there is a symlink to it from the ramdisk.
The same is true for any firmware needed after boot.

## Generic setup

Until the NetBSD installer can be updated, here are the manual steps from fresh:

  *  Build the ramdisk (`cd src/distrib/amd64/ramdisks/ramdisk-zfsroot; nbmake-amd64`)
  *  Boot the NetBSD installer
  *  Create a small (I used 2G) FFS partition and a normal swap partition
  *  Create a partition for ZFS
  *  Finish the NetBSD installation - install the etc, base and maybe rescue sets
  *  Reboot into your minimal NetBSD install

## ZFS Setup

So far, so good. Now we need to make the needed adjustments to change root:

  *  Create a ZFS pool called `rpool` on your ZFS partition
  *  Create a ZFS filesystem called `ROOT` in `rpool`
  *  Set the mountpoint of `rpool/ROOT` to legacy so that the mount command can deal with it
  *  Create any other ZFS filesystems you want to - if you create `/usr` or `/var` or anything in these they **must** to be legacy mount points as well
  *  `zfs umount rpool/ROOT`
  *  Add `rpool/ROOT /altroot zfs rw` to /etc/fstab
  *  `mount /altroot`
  *  Extract the sets you need to `/altroot`. At a minimum you will need etc and base.
  *  `cd /altroot/dev; ./MAKEDEV all`
  *  `cd /altroot; mkdir kern proc`
  *  Copy `/etc/wscons.conf`, `/etc/fstab` and any other config files sysinst might have created to `/etc/rc.conf` to `/altroot/etc`
  *  Set `zfs=YES` in `/altroot/etc/rc.conf`
  *  Remove `rpool/ROOT` from `/altroot/etc/fstab`
  *  Add `name=boot /altroot ffs rw,noauto` to `/altroot/etc/fstab` along with any ZFS filesytems you created in '/usr' and '/var'
  *  Add the filesystems as well to `critical_filesystems_local` in `/altroot/etc/rc.conf` - such as `critical_filesystems_local="/usr /var /var/log"`
  *  Copy the `ramdisk-zfsroot.fs` to `/`
  *  Edit `/boot.cfg` and add `menu=Boot ZFS Root:load solaris;load zfs;fs /ramdisk-zfsroot.fs;boot`

And done!
You can now reboot and enjoy your root on ZFS and all the benefits it brings.
When updating the kernel, remember to `mount /altroot` and update it there as
well as the modules.

## Future work

All these steps could be done in the installer.
This is probably an easier task than adding ZFS support to the bootloader.
But don't let that stop you if you are stuck for something to do!