--- wikisrc/wiki/RootOnZFS.mdwn 2020/02/22 19:17:04 1.1 +++ wikisrc/wiki/RootOnZFS.mdwn 2020/02/25 19:56:20 1.2 @@ -1,26 +1,34 @@ # Root On ZFS NetBSD-9 gained much improved ZFS support. -However, one feature it's still missing is the ability to have your system root on ZFS. -For that, we need to teach our boot loader about ZFS, but until then we can work around this limitation by using a FFS boot partition and a ZFS root ramdisk. -You'll also need to use NetBSD-9.99.47 or newer. - -The idea is simple - the boot loader loads the NetBSD kernel from FFS and the ZFS root ramdisk. -The ramdisk then mounts the boot partition and copies the requires ZFS modules from it into the ramdisk. -ZFS is then initialised to load the modules and then umounts the boot partition. -This step saves you from having to load the modules in boot.cfg. -Because the initial root is on a ramdisk, we can then load the boot partition from inside the ZFS root to maintain it. - -Once ZFS is initialised, the ZFS root is then mounted and sysctl then instructs init to chroot to it and the normal boot procedure then starts. -Once started, the kernel and modules can updated by mounting the boot partition to /altroot. +However, one feature it's still missing is the ability to have your system +root on ZFS. +For that, we need to teach our boot loader about ZFS,xi +but until then we can work around this limitation by using a FFS boot +partition and a ZFS root ramdisk. +You'll also need to use NetBSD-9.99.48 or newer. + +The idea is simple - the boot loader loads the NetBSD kernel and the +required modules *(solaris and zfs)* from FFS and the ZFS root ramdisk. +The ramdisk then imports the ZFS root pool called `rpool`, +mounts `rpool/ROOT` to `/altroot` and finally instructs the kernel to +chroot to it. +Because the initial root is on a ramdisk, +we can then load the boot partition from inside the ZFS chroot to maintain it; +for example to update the kernel and modules. ## Assumptions -The ramdisk does make some assumptions as there is no way to pass variables from boot.cfg: +The ramdisk does make some assumptions as there is no way to pass variables +from boot.cfg: - * The boot partition can be referenced as `NAME=boot` - for GPT this is as simple as giving it a label. + * The kernel has matching modules installed in the same partition; you only need `solaris` and `zfs`. * The ZFS root pool is called `rpool` and the root filesystem is called `ROOT`. +Once inside the chroot the kernel will load modules from the ZFS filesystem +because there is a symlink to it from the ramdisk. +The same is true for any firmware needed after boot. + ## Generic setup Until the NetBSD installer can be updated, here are the manual steps from fresh: @@ -31,7 +39,6 @@ Until the NetBSD installer can be update * Create a partition for ZFS * Finish the NetBSD installation - install the etc, base and maybe rescue sets * Reboot into your minimal NetBSD install - * Label the boot partition boot - if you don't do this then the ramdisk will moan it can't load modules ## ZFS Setup @@ -51,19 +58,15 @@ So far, so good. Now we need to make the * Add `name=boot /altroot ffs rw,noauto` to `/altroot/etc/fstab` along with any ZFS filesytems you created in '/usr' and '/var' * Add the filesystems as well to `critical_filesystems_local` in `/altroot/etc/rc.conf` - such as `critical_filesystems_local="/usr /var /var/log"` * Copy the `ramdisk-zfsroot.fs` to `/` - * Edit `/boot.cfg` and add `menu=Boot ZFS Root:fs /ramdisk-zfsroot.fs;boot` + * Edit `/boot.cfg` and add `menu=Boot ZFS Root:load solaris;load zfs;fs /ramdisk-zfsroot.fs;boot` -And done! You can now reboot and enjoy your root on ZFS and all the benefits it brings. -When updating the kernel, remember to `mount /altroot` and update it there as well as the modules. +And done! +You can now reboot and enjoy your root on ZFS and all the benefits it brings. +When updating the kernel, remember to `mount /altroot` and update it there as +well as the modules. ## Future work All these steps could be done in the installer. -This is probably an easier task than adding ZFS support to the bootloader. But don't let that stop you if you are stuck for something to do! - -# DANGER NETBSD ROOT ON ZFS USER, DANGER! - -There seems to be an issue accessing `/dev` nodes on ZFS. -While -current has enough fixes to work somewhat, accessing the boot partition whilst while root is on ZFS will hang, panic and maybe even corrupt your boot partition. -As such, you'll have to reboot into single user. From here you can mount the ZFS partition to `/altroot` and copy the updated kernel and modules from there to the boot partition. -Ironically, this is what the ramdisk approach was designed to actively avoid. Hopefully we can get it resolved soon. +This is probably an easier task than adding ZFS support to the bootloader. +But don't let that stop you if you are stuck for something to do!