--- wikisrc/ports/xen/howto.mdwn 2015/01/04 02:15:34 1.72 +++ wikisrc/ports/xen/howto.mdwn 2015/01/17 13:04:01 1.80 @@ -100,10 +100,11 @@ Note that NetBSD support is called XEN3. Xen command program ------------------- -Early Xen used a program called "xm" to manipulate the system from the +Early Xen used a program called xm to manipulate the system from the dom0. Starting in 4.1, a replacement program with similar behavior -called "xl" is provided. In 4.2 and later, "xl" is preferred. 4.4 is -the last version that has "xm". +called xl is provided, but it does not work well in 4.1. In 4.2, both +xm and xl work fine. 4.4 is the last version that has xm. You must +choose one or the other, because it affects which daemons you run. NetBSD ------ @@ -274,36 +275,34 @@ For debugging, one may copy xen-debug.gz to DIAGNOSTIC and DEBUG in NetBSD. xen-debug.gz is basically only useful with a serial console. Then, place a NetBSD XEN3_DOM0 kernel in /, copied from releasedir/amd64/binary/kernel/netbsd-XEN3_DOM0.gz -of a NetBSD build. Both xen and NetBSD may be left compressed. (If -using i386, use releasedir/i386/binary/kernel/netbsd-XEN3PAE_DOM0.gz.) - -With Xen as the kernel, you must provide a dom0 NetBSD kernel to be -used as a module; place this in /. Suitable kernels are provided in -releasedir/binary/kernel: - - i386 XEN3_DOM0 - i386 XEN3PAE_DOM0 - amd64 XEN3_DOM0 - -The first one is only for use with Xen 3.1 and i386-mode Xen (and you -should not do this). Current Xen always uses PAE on i386, but you -should generally use amd64 for the dom0. In a dom0 kernel, kernfs is -mandatory for xend to comunicate with the kernel, so ensure that /kern -is in fstab. TODO: Say this is default, or file a PR and give a -reference. +of a NetBSD build. If using i386, use +releasedir/i386/binary/kernel/netbsd-XEN3PAE_DOM0.gz. (If using Xen +3.1 and i386, you may use XEN3_DOM0 with the non-PAE Xen. But you +should not use Xen 3.1.) Both xen and the NetBSD kernel may be (and +typically are) left compressed. + +In a dom0 kernel, kernfs is mandatory for xend to comunicate with the +kernel, so ensure that /kern is in fstab. TODO: Say this is default, +or file a PR and give a reference. Because you already installed NetBSD, you have a working boot setup with an MBR bootblock, either bootxx_ffsv1 or bootxx_ffsv2 at the beginning of your root filesystem, /boot present, and likely /boot.cfg. (If not, fix before continuing!) -See boot.cfg(5) for an example. The basic line is +Add a line to to /boot.cfg to boot Xen. See boot.cfg(5) for an +example. The basic line is menu=Xen:load /netbsd-XEN3_DOM0.gz console=pc;multiboot /xen.gz dom0_mem=256M which specifies that the dom0 should have 256M, leaving the rest to be -allocated for domUs. In an attempt to add performance, one can also -add +allocated for domUs. To use a serial console, use + + menu=Xen:load /netbsd-XEN3_DOM0.gz console=com0;multiboot /xen.gz dom0_mem=256M console=com1 com1=9600,8n1 + +which will use the first serial port for Xen (which counts starting +from 1), forcing speed/parity, and also for NetBSD (which counts +starting at 0). In an attempt to add performance, one can also add dom0_max_vcpus=1 dom0_vcpus_pin @@ -314,6 +313,9 @@ As with non-Xen systems, you should have kernel that works without Xen) and fallback versions of the non-Xen kernel, Xen, and the dom0 kernel. +Now, reboot so that you are running a DOM0 kernel under Xen, rather +than GENERIC without Xen. + Using grub (historic) --------------------- @@ -332,13 +334,14 @@ Configuring Xen Xen logs will be in /var/log/xen. -Now, you have a system that will boot Xen and the dom0 kernel, and -just run the dom0 kernel. There will be no domUs, and none can be -started because you still have to configure the dom0 tools. The -daemons which should be run vary with Xen version and with whether one -is using xm or xl. Note that xend is for supporting "xm", and should -only be used if you plan on using "xm". Do NOT enable xend if you -plan on using "xl" as it will cause problems. +Now, you have a system that will boot Xen and the dom0 kernel, but not +do anything else special. Make sure that you have rebooted into Xen. +There will be no domUs, and none can be started because you still have +to configure the dom0 tools. The daemons which should be run vary +with Xen version and with whether one is using xm or xl. Note that +xend is for supporting "xm", and should only be used if you plan on +using "xm". Do NOT enable xend if you plan on using "xl" as it will +cause problems. The installation of NetBSD should already have created devices for xen (xencons, xenevt), but if they are not present, create them: @@ -366,7 +369,7 @@ For 4.2 with xm, add to rc.conf xencommons=YES xend=YES -For 4.2 with xl (preferred), add to rc.conf: +For 4.2 with xl, add to rc.conf: xencommons=YES TODO: explain if there is a xend replacement @@ -374,8 +377,8 @@ For 4.2 with xl (preferred), add to rc.c TODO: Recommend for/against xen-watchdog. After you have configured the daemons and either started them (in the -order given) or rebooted, run the following (or use xl) to inspect -Xen's boot messages, available resources, and running domains: +order given) or rebooted, use xm or xl to inspect Xen's boot messages, +available resources, and running domains. An example with xm follows: # xm dmesg [xen's boot info] @@ -385,17 +388,21 @@ Xen's boot messages, available resources Name Id Mem(MB) CPU State Time(s) Console Domain-0 0 64 0 r---- 58.1 +With xl, the commands are the same, and the output may be slightly +different. TODO: add example output for xl, after confirming on 4.2 +and resolving the TODO about rc.conf. + anita (for testing NetBSD) -------------------------- With the setup so far, one should be able to run anita (see -pkgsrc/sysutils/py-anita) to test NetBSD releases, by doing (as root, +pkgsrc/misc/py-anita) to test NetBSD releases, by doing (as root, because anita must create a domU): anita --vmm=xm test file:///usr/obj/i386/ Alternatively, one can use --vmm=xl to use xl-based domU creation instead. -TODO: check this. +TODO: check this, and make the example use xl when confirmed. Xen-specific NetBSD issues -------------------------- @@ -883,7 +890,6 @@ configuration. The big issue is usually is booted, because the kernel is nominally in the dom0 filesystem (to which VPS users do not normally have acesss). A second issue is how to install NetBSD. - A VPS user may want to compile a kernel for security updates, to run npf, run IPsec, or any other reason why someone would want to change their kernel. @@ -894,6 +900,10 @@ or to select from a prepopulated list. kernel from the domU filesystem. This is closer to a regular physical computer, where someone who controls a machine can replace the kernel. +A second issue is multiple CPUs. With NetBSD 6, domUs support +multiple vcpus, and it is typical for VPS providers to enable multiple +CPUs for NetBSD domUs. + pygrub ------- @@ -913,11 +923,12 @@ calls. It is booted from the dom0 as th that pvgrub works with FFsv2 with 16K/2K and 32K/4K block/frag sizes (and hence with defaults from "newfs -O 2"). See [Panix's pvgrub page](http://www.panix.com/v-colo/grub.html), which describes only -Linux but should be updated to cover NetBSD :-) +Linux but should be updated to cover NetBSD :-). [prgmr.com](http://prgmr.com/) also lets users with pvgrub to boot their own kernel. See then [prgmr.com NetBSD -HOWTO](http://wiki.prgmr.com/mediawiki/index.php/NetBSD_as_a_DomU). +HOWTO](http://wiki.prgmr.com/mediawiki/index.php/NetBSD_as_a_DomU) +(which is in need of updating). It appears that [grub's FFS code](http://xenbits.xensource.com/hg/xen-unstable.hg/file/bca284f67702/tools/libfsimage/ufs/fsys_ufs.c)