--- wikisrc/ports/xen/howto.mdwn 2017/12/15 16:40:59 1.141 +++ wikisrc/ports/xen/howto.mdwn 2018/07/26 10:49:59 1.148 @@ -1,3 +1,5 @@ +[[!meta title="Xen HowTo"]] + Introduction ============ @@ -51,7 +53,7 @@ attempts to address both the case of run and running domUs under it (NetBSD and other), and also running NetBSD as a domU in a VPS. -Xen 3.1 in pkgsrc supports "PCI passthrough", which means that +Xen 3.1 in pkgsrc used to support "PCI passthrough", which means that specific PCI devices can be made available to a specific domU instead of the dom0. This can be useful to let a domU run X11, or access some network interface or other peripheral. @@ -86,60 +88,28 @@ hardware architecture on which it runs. both Intel and AMD, and in 2016 a normal PC has this CPU architecture.) -Xen ---- +Xen versions +------------ In NetBSD, Xen is provided in pkgsrc, via matching pairs of packages xenkernel and xentools. We will refer only to the kernel versions, but note that both packages must be installed together and must have matching versions. -xenkernel3 provides Xen 3.1. It is no longer maintained by Xen, and -the last applied security patch was in 2011. Thus, it should not be -used. It supports PCI passthrough, which is why people use it anyway. -Xen 3.1 runs on i386 (both non-PAE and PAE) and amd64 hardware. - -xenkernel33 provides Xen 3.3. It is no longer maintained by Xen, and -the last applied security patch was in 2012. Thus, it should not be -used. Xen 3.3 runs on i386 PAE and amd64 hardware. There are no good -reasons to run this version. - -xenkernel41 provides Xen 4.1. It is no longer maintained by Xen, but -as of 2016-12 received backported security patches. Xen 4.1 runs on -i386 PAE and amd64 hardware. There are no good reasons to run this -version. - -Note that 3.1, 3.3 and 4.1 have been removed from pkgsrc-current, but -are in 2016Q4. They will be removed from this HOWTO sometime after -2017Q1. - -xenkernel42 provides Xen 4.2. It is no longer maintained by Xen, but -as of 2016-12 received backported security patches. Xen 4.2 runs on -i386 PAE and amd64 hardware. The only reason to run this is if you -need to use xm instead of xl, or if you need to run on hardware that -supports i386 but not amd64. (This might also be useful if you need -an i386 dom0, if it turns out that an amd64 Xen kernel and an i386 -dom0 is problematic.) - -xenkernel45 provides Xen 4.5. As of 2016-12, security patches were -released by Xen and applied to pkgsrc. Xen 4.5 runs on amd64 hardware -only. While slightly old, 4.5 has been tested and run by others, so -it is the conservative choice. - -xenkernel46 provides Xen 4.6. It is new to pkgsrc as of 2016-05. As -of 2016-12, security patches were released by Xen and applied to -pkgsrc. Xen 4.6 runs on amd64 hardware only For new installations, -4.6 is probably the appropriate choice and it will likely soon be the -standard approach. (If using Ubuntu guests, be sure to have the -xentools46 from December, 2016). +Versions available in pkgsrc: -Xen 4.7 (released 2016-06) and 4.8 (released 2016-12) are not yet in -pkgsrc. +[[!table data=""" +Xen Version |Package Name |Xen CPU Support |EOL'ed By Upstream +4.2 |xenkernel42 |32bit, 64bit |Yes +4.5 |xenkernel45 |64bit |Yes +4.6 |xenkernel46 |64bit |Partially +4.8 |xenkernel48 |64bit |No +4.11 |xenkernel411 |64bit |No +"""]] See also the [Xen Security Advisory page](http://xenbits.xen.org/xsa/). -Note that NetBSD support is called XEN3. It works with Xen 3 and Xen -4 because the hypercall interface has been stable. +Note: Xen 4.2 was the last version to support 32bit CPUs. Xen command program ------------------- @@ -152,7 +122,7 @@ xm and xl work fine. 4.4 is the last ve You must make a global choice to use xm or xl, because it affects not only which command you use, but the command used by rc.d scripts (specifically xendomains) and which daemons should be run. The -xentools packages provide xm for 3.1, 3.3 and 4.1 and xl for 4.2 and up. +xentools packages provide xl for 4.2 and up. In 4.2, you can choose to use xm by simply changing the ctl_command variable and setting xend=YES in rc.conf. @@ -165,27 +135,24 @@ xbd, where a vnd must be allocated). Bu been adequately tested for a complex custom setup with a large number of interfaces. -NetBSD ------- +NetBSD versions +--------------- -The netbsd-6, netbsd-7, and -current branches are all reasonable +The netbsd-7, netbsd-8, and -current branches are all reasonable choices, with more or less the same considerations for non-Xen use. -Therefore, netbsd-7 is recommended as the stable version of the most -recent release for production use. In addition, netbsd-7 and -current -have a important scheduler fix (in November of 2015) affecting -contention between dom0 and domUs; see -https://releng.netbsd.org/cgi-bin/req-7.cgi?show=1040 for a -description. For those wanting to learn Xen or without production -stability concerns, netbsd-7 is still likely most appropriate, but --current is also a reasonable choice. (Xen runs ok on netbsd-5, but -the xentools packages are likely difficult to build, and netbsd-5 is -not supported.) +NetBSD 8 is recommended as the stable version of the most recent +release for production use. + +For developing Xen, netbsd-current may be appropriate. As of NetBSD 6, a NetBSD domU will support multiple vcpus. There is no SMP support for NetBSD as dom0. (The dom0 itself doesn't really need SMP for dom0 functions; the lack of support is really a problem when using a dom0 as a normal computer.) +Note: NetBSD support is called XEN3. However, it does support Xen 4, +because the hypercall interface has remained identical. + Architecture ------------ @@ -227,25 +194,15 @@ caution that the total situation is comp understood. On top of that caution, the post is about Linux, not NetBSD. TODO: Include link to benchmarks, if someone posts them. -Stability ---------- - -Mostly, NetBSD as a dom0 or domU is quite stable. -However, there are some open PRs indicating problems. - - - [PR 48125](http://gnats.netbsd.org/48125) - -Note also that there are issues with sparse vnd(4) instances, but -these are not about Xen -- they just are noticed with sparse vnd(4) -instances in support of virtual disks in a dom0. - Recommendation -------------- -Therefore, this HOWTO recommends running xenkernel45 or xenkernel46, -xl, the NetBSD 7 stable branch, and to use an amd64 kernel as the -dom0. Either the i386PAE or amd64 version of NetBSD may be used as -domUs. +Therefore, this HOWTO recommends running xenkernel46, xl, the NetBSD 7 +stable branch, and therefore to use an amd64 kernel as the dom0. +Either the i386PAE or amd64 version of NetBSD may be used as domUs. + +A tentative replacement recommendation is xenkernel48, xl, and NetBSD +8. Because bugs are fixed quite often, and because of Xen security advisories, it is good to stay up to date with NetBSD (tracking a @@ -255,65 +212,6 @@ pkgsrc), and with the Xen tools. Specif November, 2015, and xentools46 got a fix to enable Ubuntu guests to boot in December, 2016. -Status ------- - -Ideally, all versions of Xen in pkgsrc would build on all supported -versions of NetBSD/amd64, to the point where this section would be -silly. However, that has not always been the case. Besides aging -code and aging compilers, qemu (included in xentools for HVM support) -is difficult to build. Note that there is intentionally no data for -4.5+ up for i386, and often omits xentools info if the corresponding -kernel fails. - -The following table gives status, with the date last checked -(generally on the most recent quarterly branch). The first code is -"builds" if it builds ok, and "FAIL" for a failure to build. The -second code/date only appears for xenkernel* and is "works" if it runs -ok as a dom0 and can support a domU, and "FAIL" if it won't boot or -run a domU. - - xenkernel3 netbsd-6 i386 FAIL 201612 - xenkernel33 netbsd-6 i386 FAIL 201612 - xenkernel41 netbsd-6 i386 builds 201612 - xenkernel42 netbsd-6 i386 builds 201612 - xentools3 netbsd-6 i386 FAIL 201612 - xentools33 netbsd-6 i386 FAIL 201612 - xentools41 netbsd-6 i386 builds 201612 - xentools42 netbsd-6 i386 FAIL 201612 - - xenkernel3 netbsd-7 i386 FAIL 201412 - xenkernel33 netbsd-7 i386 FAIL 201412 - xenkernel41 netbsd-7 i386 builds 201412 - xenkernel42 netbsd-7 i386 builds 201412 - xentools41 netbsd-7 i386 builds 201412 - xentools42 netbsd-7 i386 ??FAIL 201412 - - xenkernel3 netbsd-6 amd64 FAIL 201612 - xenkernel33 netbsd-6 amd64 FAIL 201612 - xenkernel41 netbsd-6 amd64 builds 201612 works 201612 - xenkernel42 netbsd-6 amd64 builds 201612 works 201612 - xenkernel45 netbsd-6 amd64 builds 201612 - xenkernel46 netbsd-6 amd64 builds 201612 - xentools41 netbsd-6 amd64 builds 201612 - xentools42 netbsd-6 amd64 builds 201612 - xentools45 netbsd-6 amd64 builds 201612 - xentools46 netbsd-6 amd64 FAIL 201612 - - xenkernel3 netbsd-7 amd64 builds 201612 - xenkernel33 netbsd-7 amd64 builds 201612 - xenkernel41 netbsd-7 amd64 builds 201612 - xenkernel42 netbsd-7 amd64 builds 201612 - xenkernel45 netbsd-7 amd64 builds 201612 - xenkernel46 netbsd-7 amd64 builds 201612 - xentools3 netbsd-7 amd64 builds 201612 - xentools3-hvm netbsd-7 amd64 builds 201612 - xentools33 netbsd-7 amd64 FAIL 201612 - xentools41 netbsd-7 amd64 builds 201612 - xentools42 netbsd-7 amd64 builds 201612 - xentools45 netbsd-7 amd64 builds 201612 - xentools46 netbsd-7 amd64 builds 201612 - NetBSD as a dom0 ================ @@ -332,6 +230,10 @@ half-dozen domUs of 512M and 32G each. have to be bigger than the sum of the RAM/disk needs of the dom0 and all the domUs. +In 2018-05, trouble booting a dom0 was reported with 256M of RAM: with +512M it worked reliably. This does not make sense, but if you see +"not ELF" after Xen boots, try increasing dom0 RAM. + Styles of dom0 operation ------------------------ @@ -429,12 +331,12 @@ beginning of your root file system, have Add a line to to /boot.cfg to boot Xen. See boot.cfg(5) for an example. The basic line is - menu=Xen:load /netbsd-XEN3_DOM0.gz console=pc;multiboot /xen.gz dom0_mem=256M + menu=Xen:load /netbsd-XEN3_DOM0.gz console=pc;multiboot /xen.gz dom0_mem=512M -which specifies that the dom0 should have 256M, leaving the rest to be +which specifies that the dom0 should have 512M, leaving the rest to be allocated for domUs. To use a serial console, use - menu=Xen:load /netbsd-XEN3_DOM0.gz;multiboot /xen.gz dom0_mem=256M console=com1 com1=9600,8n1 + menu=Xen:load /netbsd-XEN3_DOM0.gz;multiboot /xen.gz dom0_mem=512M console=com1 com1=9600,8n1 which will use the first serial port for Xen (which counts starting from 1, unlike NetBSD which counts starting from 0), forcing @@ -602,8 +504,8 @@ section. # Install secondary boot loader cp -p /usr/mdec/boot / # Create boot.cfg following earlier guidance: - menu=Xen:load /netbsd-XEN3PAE_DOM0.gz console=pc;multiboot /xen.gz dom0_mem=256M - menu=Xen.ok:load /netbsd-XEN3PAE_DOM0.ok.gz console=pc;multiboot /xen.ok.gz dom0_mem=256M + menu=Xen:load /netbsd-XEN3PAE_DOM0.gz console=pc;multiboot /xen.gz dom0_mem=512M + menu=Xen.ok:load /netbsd-XEN3PAE_DOM0.ok.gz console=pc;multiboot /xen.ok.gz dom0_mem=512M menu=GENERIC:boot menu=GENERIC single-user:boot -s menu=GENERIC.ok:boot netbsd.ok @@ -872,7 +774,7 @@ Sizing domains Modern x86 hardware has vast amounts of resources. However, many virtual servers can function just fine on far less. A system with -256M of RAM and a 4G disk can be a reasonable choice. Note that it is +512M of RAM and a 4G disk can be a reasonable choice. Note that it is far easier to adjust virtual resources than physical ones. For memory, it's just a config file edit and a reboot. For disk, one can create a new file and vnconfig it (or lvm), and then dump/restore, @@ -1160,18 +1062,7 @@ to update the special boot partition. Amazon ------ -See the [Amazon EC2 page](../amazon_ec2/). - -Using npf ---------- - -In standard kernels, npf is a module, and thus cannot be loaded in a -DOMU kernel. - -TODO: Explain how to compile npf into a custom kernel, answering (but -note that the problem was caused by not booting the right kernel) -[this email to -netbsd-users](https://mail-index.netbsd.org/netbsd-users/2014/12/26/msg015576.html). +See the [Amazon EC2 page](/amazon_ec2/). TODO items for improving NetBSD/xen =================================== @@ -1191,8 +1082,6 @@ TODO items for improving NetBSD/xen * Get UFS2 patches into pvgrub. * Add support for PV ops to a version of /boot, and make it usable as a kernel in Xen, similar to pvgrub. -* Solve somehow the issue with modules for GENERIC not being loadable - in a Xen dom0 or domU kernel. Random pointers ===============