--- wikisrc/ports/xen/howto.mdwn 2015/01/28 19:27:11 1.87 +++ wikisrc/ports/xen/howto.mdwn 2015/03/05 14:04:55 1.95 @@ -142,6 +142,18 @@ running an i386 NetBSD kernel as a domU, (Note that emacs (at least) fails if run on i386 with PAE when built without, and vice versa, presumably due to bugs in the undump code.) +Stability +--------- + +Mostly, NetBSD as a dom0 or domU is quite stable. +However, there are some open PRs indicating problems. + + - [PR 48125](http://gnats.netbsd.org/48125) + - [PR 47720](http://gnats.netbsd.org/47720) + +Note also that there are issues with sparse vnd(4) instances, but +these are not about Xen. + Recommendation -------------- @@ -214,7 +226,7 @@ dom0 is what the computer would have bee desktop or laptop. Then, one can run domUs at will. Purists will deride this as less secure than the previous approach, and for a computer whose purpose is to run domUs, they are right. But Xen and a -dom0 (without domUs) is not meaingfully less secure than the same +dom0 (without domUs) is not meaningfully less secure than the same things running without Xen. One can boot Xen or boot regular NetBSD alternately with little problems, simply refraining from starting the Xen daemons when not running Xen. @@ -239,7 +251,7 @@ over a RAID1 header to find /boot from a partition; this is no different when booting Xen. There are 4 styles of providing backing storage for the virtual disks -used by domUs: raw partitions, LVM, file-backed vnd(4), and SAN, +used by domUs: raw partitions, LVM, file-backed vnd(4), and SAN. With raw partitions, one has a disklabel (or gpt) partition sized for each virtual disk to be used by the domU. (If you are able to predict @@ -312,6 +324,11 @@ starting at 0). In an attempt to add pe to force only one vcpu to be provided (since NetBSD dom0 can't use more) and to pin that vcpu to a physical cpu. TODO: benchmark this. +Xen has [many boot +options](http://xenbits.xenproject.org/docs/4.5-testing/misc/xen-command-line.html), +and other tham dom0 memory and max_vcpus, they are generally not +necessary. + As with non-Xen systems, you should have a line to boot /netbsd (a kernel that works without Xen) and fallback versions of the non-Xen kernel, Xen, and the dom0 kernel. @@ -401,6 +418,18 @@ With xl, the commands are the same, and different. TODO: add example output for xl before the xm example, after confirming on 4.2 and resolving the TODO about rc.conf. +### Issues with xencommons + +xencommons starts xenstored, which stores data on behalf of dom0 and +domUs. It does not currently work to stop and start xenstored. +Certainly all domUs should be shutdown first, following the sort order +of the rc.d scripts. However, the dom0 sets up state with xenstored, +and is not notified when xenstored exits, leading to not recreating +the state when the new xenstored starts. Until there's a mechanism to +make this work, one should not expect to be able to restart xenstored +(and thus xencommons). There is currently no reason to expect that +this will get fixed any time soon. + anita (for testing NetBSD) -------------------------- @@ -502,7 +531,8 @@ In 2015-01, the following combination wa dom0 kernel: NetBSD/amd64 6.1.5 Xen tools: xentools42-4.2.5 from pkgsrc -See http://gnats.netbsd.org/47720 for a problem with dom0 shutdown. +See [PR 47720](http://gnats.netbsd.org/47720) for a problem with dom0 +shutdown. Unprivileged domains (domU) =========================== @@ -564,7 +594,7 @@ domain, and see if it has finished stopp xm console foo xm create -c foo xm shutdown foo - xm list + xm list Typing ^] will exit the console session. Shutting down a domain is equivalent to pushing the power button; a NetBSD domU will receive a @@ -624,6 +654,10 @@ create an empty 4G virtual disk, simply dd if=/dev/zero of=foo-xbd0 bs=1m count=4096 +Do not use qemu-img-xen, because this will create sparse file. There +have been recent (2015) reports of sparse vnd(4) devices causing +lockups, but there is apparently no PR. + With the lvm style, one creates logical devices. They are then used similarly to vnds. TODO: Add an example with lvm. @@ -727,7 +761,7 @@ i386 and amd64 provide the following ker i386 XEN3_DOMU i386 XEN3PAE_DOMU - amd64 XEN3_DOMU + amd64 XEN3_DOMU Unless using Xen 3.1 (and you shouldn't) with i386-mode Xen, you must use the PAE version of the i386 kernel. @@ -780,7 +814,13 @@ It is also desirable to add in rc.conf. This way, the domain will be properly shut down if `xm shutdown -R` or `xm shutdown -H` is used on the dom0. -Your domain should be now ready to work, enjoy. +It is not strictly necessary to have a kernel (as /netbsd) in the domU +filesystem. However, various programs (e.g. netstat) will use that +kernel to look up symbols to read from kernel virtual memory. If +/netbsd is not the running kernel, those lookups will fail. (This is +not really a Xen-specific issue, but because the domU kernel is +obtained from the dom0, it is far more likely to be out of sync or +missing with Xen.) Creating an unprivileged Linux domain (domU) -------------------------------------------- @@ -968,7 +1008,7 @@ to update the special boot partiion. Amazon ------ -TODO: add link to NetBSD amazon howto. +See the [Amazon EC2 page](../amazon_ec2/). Using npf --------- @@ -976,15 +1016,18 @@ Using npf In standard kernels, npf is a module, and thus cannot be loaded in a DOMU kernel. -TODO: explain how to compile npf into a custom kernel, answering (but -note that the problem was caused by not booting the right kernel): -http://mail-index.netbsd.org/netbsd-users/2014/12/26/msg015576.html +TODO: Explain how to compile npf into a custom kernel, answering (but +note that the problem was caused by not booting the right kernel) +[this email to +netbsd-users](http://mail-index.netbsd.org/netbsd-users/2014/12/26/msg015576.html). TODO items for improving NetBSD/xen =================================== -* Package Xen 4.4. -* Get PCI passthrough working on Xen 4.2 (or 4.4). +* Make the NetBSD dom0 kernel work with SMP. +* Test the Xen 4.5 packages adequately to be able to recommend them as + the standard approach. +* Get PCI passthrough working on Xen 4.5 * Get pvgrub into pkgsrc, either via xentools or separately. * grub * Check/add support to pkgsrc grub2 for UFS2 and arbitrary @@ -996,3 +1039,14 @@ TODO items for improving NetBSD/xen * Get UFS2 patches into pvgrub. * Add support for PV ops to a version of /boot, and make it usable as a kernel in Xen, similar to pvgrub. +* Solve somehow the issue with modules for GENERIC not being loadable + in a Xen dom0 or domU kernel. + +Random pointers +=============== + +TODO: This section contains links from elsewhere not yet integrated +into the HOWTO. + +* http://www.lumbercartel.ca/library/xen/ +* http://pbraun.nethence.com/doc/sysutils/xen_netbsd_dom0.html