--- wikisrc/ports/xen/howto.mdwn 2021/02/26 18:39:20 1.190 +++ wikisrc/ports/xen/howto.mdwn 2021/02/26 19:03:20 1.191 @@ -107,19 +107,21 @@ NetBSD 8 and up support PV and HVM modes Support for PVHVM and PVH is available only in NetBSD-current. -NetBSD as a dom0 does not run SMP, because some drivers are not yet -safe for this. \todo Link to more information about what needs work. +NetBSD up to and including NetBSD 9 as a dom0 does not run SMP, +because some drivers are not yet safe for this. NetBSD-current +supports SMP in dom0. NetBSD, when run as a domU, can and does typically run SMP. -Note: NetBSD support is called XEN3. However, it does support Xen 4, -because the hypercall interface has remained identical. +Note that while Xen 4.13 is current, the kernel support is still +called XEN3, because the hypercall interface has not changed +significantly. # Creating a NetBSD dom0 -In order to install a NetBSD as a dom0, one must first install a normal -NetBSD system, and then pivot the install to a dom0 install by changing -the kernel and boot configuration. +In order to install a NetBSD as a dom0, one first installs a normal +NetBSD system, and then pivot the install to a dom0 install by +changing the kernel and boot configuration. In 2018-05, trouble booting a dom0 was reported with 256M of RAM: with 512M it worked reliably. This does not make sense, but if you see @@ -127,12 +129,16 @@ In 2018-05, trouble booting a dom0 was r ## Installation of NetBSD -[Install NetBSD/amd64](/guide/inst/) -just as you would if you were not using Xen. -Therefore, use the most recent release, or a build from the most recent stable branch. +[Install NetBSD/amd64](/guide/inst/) just as you would if you were not +using Xen. Therefore, use the most recent release, or a build from +the most recent stable branch. Alternatively, use -current, being +mindful of all the usual caveats of lower stability of current, and +likely a bit more so. ## Installation of Xen +### Building Xen + Use the most recent version of Xen in pkgsrc, unless the DESCR says that it is not suitable. Therefore, choose 4.13. In the dom0, install xenkernel413 and xentools413 from pkgsrc. @@ -144,28 +150,49 @@ where the boot process will be able to f # cp -p /usr/pkg/xen413-kernel/xen.gz / """]] -Then, place a NetBSD XEN3_DOM0 kernel in the `/` directory. Such kernel -can either be compiled manually, or downloaded from the NetBSD FTP, for -example at: +Then, place a NetBSD XEN3_DOM0 kernel in the `/` directory. Such +kernel can either be taken from a local release build.sh run, compiled +manually, or downloaded from the NetBSD FTP, for example at: [[!template id=programlisting text=""" ftp.netbsd.org/pub/NetBSD/NetBSD-9.1/amd64/binary/kernel/netbsd-XEN3_DOM0.gz """]] -Add a line to /boot.cfg to boot Xen: +### Configuring booting + +Read boot.cfg(8) carefully. Add lines to /boot.cfg to boot Xen: [[!template id=filecontent name="/boot.cfg" text=""" menu=Xen:load /netbsd-XEN3_DOM0.gz console=pc;multiboot /xen.gz dom0_mem=512M +menu=Xen single user:load /netbsd-XEN3_DOM0.gz console=pc -s;multiboot /xen.gz dom0_mem=512M """]] +\todo Explain why console=pc is given. Perhaps there is some notion +of a separate console instead of xencons. + +\todo Check the single user line. + This specifies that the dom0 should have 512MB of ram, leaving the rest to be allocated for domUs. -NB: This says add, not replace, so that you will be able to boot a -NetBSD kernel without Xen. Once Xen boots ok, you may want to set it -as default. +NB: This says add, not replace, so that you will be able to more +easily boot a NetBSD kernel without Xen. Once Xen boots ok, you may +want to set it as default. It is highly likely that you will have +trouble at some point, and keeping an up-to-date GENERIC for use in +fixing problems is the standard prudent approach. + +### Console selection + +See boot_console(8). Understand that you should start from a place of +having console setup correct for booting GENERIC before trying to +configure Xen. + +By default, the hypervisor (Xen itself) will use \todo as the console -To use a serial console, add settings as follows: +By default, the NetBSD dom0 kernel will use xencons(4) (the man page +does not exist) as console. + +To use a serial console for the Xen hypervisor, add settings as follows: [[!template id=filecontent name="/boot.cfg" text=""" menu=Xen:load /netbsd-XEN3_DOM0.gz;multiboot /xen.gz dom0_mem=512M console=com1 com1=9600,8n1 @@ -178,12 +205,21 @@ speed/parity. Because the NetBSD comman which directs the console I/O through Xen to the same console device Xen itself uses (in this case, the serial port). +\todo Explain why boot.cfg shows forcing the dom0 console also, +separately from the hypervisor console. + +### Tuning + In an attempt to add performance, one can also add `dom0_max_vcpus=1 dom0_vcpus_pin`, to force only one vcpu to be provided (since NetBSD dom0 can't use more) and to pin that vcpu to a physical CPU. Xen has [many boot options](http://xenbits.xenproject.org/docs/4.13-testing/misc/xen-command-line.html), and other than dom0 memory and max_vcpus, they are generally not necessary. +\todo Revisit this advice with current. +\todo Explain if anyone has ever actually measured that this helps. + +### rc.conf Ensure that the boot scripts installed in `/usr/pkg/share/examples/rc.d` are in `/etc/rc.d`, either because you @@ -198,6 +234,8 @@ xencommons=YES \todo Recommend for/against xen-watchdog. +### Testing + Now, reboot so that you are running a DOM0 kernel under Xen, rather than GENERIC without Xen. @@ -234,9 +272,9 @@ this will get fixed any time soon. There are (at least) two additional things different about NetBSD as a dom0 kernel compared to hardware. -One is that the module ABI is different because some of the #defines -change, so one must build modules for Xen. As of netbsd-7, the build -system does this automatically. +One is that through NetBSD 9 the module ABI is different because some +of the #defines change, so there are separate sets of modules in +/stand. In NetBSD-current, there is only one set of modules. The other difference is that XEN3_DOM0 does not have exactly the same options as GENERIC. While it is debatable whether or not this is a @@ -260,8 +298,8 @@ version supports the version of Xen you replaces the kernel and reboots, and then overlays userland binaries and adjusts `/etc`. -Note that one must update both the non-Xen kernel typically used for -rescue purposes and the DOM0 kernel used with Xen. +Note that one should update both the non-Xen kernel typically used for +rescue purposes, as well as the DOM0 kernel used with Xen. ## anita (for testing NetBSD) @@ -561,13 +599,11 @@ Use type='pvh'. \todo Explain where the kernel comes from. - ## Creating a Solaris domU See possibly outdated [Solaris domU instructions](/ports/xen/howto-solaris/). - ## PCI passthrough: Using PCI devices in guest domains NB: PCI passthrough only works on some Xen versions and as of 2020 it