File:  [NetBSD Developer Wiki] / wikisrc / wiki / RootOnZFS.mdwn
Revision 1.3: download - view: text, annotated - select for diffs
Tue Apr 14 12:04:37 2020 UTC (2 years, 2 months ago) by wiki
Branches: MAIN
CVS tags: HEAD
web commit by roy

    1: # Root On ZFS
    2: 
    3: NetBSD-9 gained much improved ZFS support.
    4: However, one feature it's still missing is the ability to have your system
    5: root on ZFS.
    6: For that, we need to teach our boot loader about ZFS,xi
    7: but until then we can work around this limitation by using a FFS boot
    8: partition and a ZFS root ramdisk.
    9: You'll also need to use NetBSD-9.99.48 or newer.
   10: 
   11: The idea is simple - the boot loader loads the NetBSD kernel and the
   12: required modules *(solaris and zfs)* from FFS and the ZFS root ramdisk.
   13: The ramdisk then imports the ZFS root pool called `rpool`,
   14: mounts `rpool/ROOT` to `/altroot` and finally instructs the kernel to
   15: chroot to it.
   16: Because the initial root is on a ramdisk,
   17: we can then load the boot partition from inside the ZFS chroot to maintain it;
   18: for example to update the kernel and modules.
   19: 
   20: ## Assumptions
   21: 
   22: The ramdisk does make some assumptions as there is no way to pass variables
   23: from boot.cfg:
   24: 
   25:   *  The kernel has matching modules installed in the same partition; you only need `solaris` and `zfs`.
   26:   *  The ZFS root pool is called `rpool` and the root filesystem is called `ROOT`.
   27: 
   28: Once inside the chroot the kernel will load modules from the ZFS filesystem
   29: because there is a symlink to it from the ramdisk.
   30: The same is true for any firmware needed after boot.
   31: 
   32: ## Generic setup
   33: 
   34: Until the NetBSD installer can be updated, here are the manual steps from fresh:
   35: 
   36:   *  Build the ramdisk (`cd src/distrib/amd64/ramdisks/ramdisk-zfsroot; nbmake-amd64`)
   37:   *  Boot the NetBSD installer
   38:   *  Create a small (I used 2G) FFS partition and a normal swap partition
   39:   *  Create a partition for ZFS
   40:   *  Finish the NetBSD installation - install the etc, base and maybe rescue sets
   41:   *  Reboot into your minimal NetBSD install
   42: 
   43: ## ZFS Setup
   44: 
   45: So far, so good. Now we need to make the needed adjustments to change root:
   46: 
   47:   *  Create a ZFS pool called `rpool` on your ZFS partition
   48:   *  Create a ZFS filesystem called `ROOT` in `rpool`
   49:   *  Set the mountpoint of `rpool/ROOT` to legacy so that the mount command can deal with it
   50:   *  Create any other ZFS filesystems you want to - if you create `/usr` or `/var` or anything in these they **must** to be legacy mount points as well
   51:   *  `zfs umount rpool/ROOT`
   52:   *  Add `rpool/ROOT /altroot zfs rw` to /etc/fstab
   53:   *  `mount /altroot`
   54:   *  Extract the sets you need to `/altroot`. At a minimum you will need etc and base.
   55:   *  `cd /altroot/dev; ./MAKEDEV all`
   56:   *  `cd /altroot; mkdir kern proc`
   57:   *  Copy `/etc/wscons.conf`, `/etc/fstab` and any other config files sysinst might have created to `/etc/rc.conf` to `/altroot/etc`
   58:   *  Set `zfs=YES` in `/altroot/etc/rc.conf`
   59:   *  Remove `rpool/ROOT` from `/altroot/etc/fstab`
   60:   *  Add `name=boot /altroot ffs rw,noauto` to `/altroot/etc/fstab` along with any ZFS filesytems you created in '/usr' and '/var'
   61:   *  Add the filesystems as well to `critical_filesystems_local` in `/altroot/etc/rc.conf` - such as `critical_filesystems_local="/usr /var /var/log"`
   62:   *  Copy the `ramdisk-zfsroot.fs` to `/`
   63:   *  Edit `/boot.cfg` and add `menu=Boot ZFS Root:load solaris;load zfs;fs /ramdisk-zfsroot.fs;boot`
   64: 
   65: And done!
   66: You can now reboot and enjoy your root on ZFS and all the benefits it brings.
   67: When updating the kernel, remember to `mount /altroot` and update it there as
   68: well as the modules.
   69: 
   70: ## Future work
   71: 
   72: All these steps could be done in the installer.
   73: This is probably an easier task than adding ZFS support to the bootloader.
   74: But don't let that stop you if you are stuck for something to do!

CVSweb for NetBSD wikisrc <wikimaster@NetBSD.org> software: FreeBSD-CVSweb