version 1.57, 2014/12/27 00:33:45
|
version 1.75, 2015/01/17 01:32:12
|
Line 4 Introduction
|
Line 4 Introduction
|
[![[Xen |
[![[Xen |
screenshot]](http://www.netbsd.org/gallery/in-Action/hubertf-xens.png)](http://www.netbsd.org/gallery/in-Action/hubertf-xen.png) |
screenshot]](http://www.netbsd.org/gallery/in-Action/hubertf-xens.png)](http://www.netbsd.org/gallery/in-Action/hubertf-xen.png) |
|
|
Xen is a virtual machine monitor or hypervisor for x86 hardware |
Xen is a hypervisor (or virtual machine monitor) for x86 hardware |
(i686-class or higher), which supports running multiple guest |
(i686-class or higher), which supports running multiple guest |
operating systems on a single physical machine. With Xen, one uses |
operating systems on a single physical machine. Xen is a Type 1 or |
the Xen kernel to control the CPU, memory and console, a dom0 |
bare-metal hypervisor; one uses the Xen kernel to control the CPU, |
operating system which mediates access to other hardware (e.g., disks, |
memory and console, a dom0 operating system which mediates access to |
network, USB), and one or more domU operating systems which operate in |
other hardware (e.g., disks, network, USB), and one or more domU |
an unprivileged virtualized environment. IO requests from the domU |
operating systems which operate in an unprivileged virtualized |
systems are forwarded by the hypervisor (Xen) to the dom0 to be |
environment. IO requests from the domU systems are forwarded by the |
fulfilled. |
hypervisor (Xen) to the dom0 to be fulfilled. |
|
|
Xen supports two styles of guests. The original is Para-Virtualized |
Xen supports two styles of guests. The original is Para-Virtualized |
(PV) which means that the guest OS does not attempt to access hardware |
(PV) which means that the guest OS does not attempt to access hardware |
Line 151 Build problems
|
Line 151 Build problems
|
Ideally, all versions of Xen in pkgsrc would build on all versions of |
Ideally, all versions of Xen in pkgsrc would build on all versions of |
NetBSD on both i386 and amd64. However, that isn't the case. Besides |
NetBSD on both i386 and amd64. However, that isn't the case. Besides |
aging code and aging compilers, qemu (included in xentools for HVM |
aging code and aging compilers, qemu (included in xentools for HVM |
support) is difficult to build. The following are known to fail: |
support) is difficult to build. The following are known to work or FAIL: |
|
|
xenkernel3 netbsd-6 i386 |
|
xentools42 netbsd-6 i386 |
|
|
|
The following are known to work: |
|
|
|
|
xenkernel3 netbsd-5 amd64 |
|
xentools3 netbsd-5 amd64 |
|
xentools3=hvm netbsd-5 amd64 ???? |
|
xenkernel33 netbsd-5 amd64 |
|
xentools33 netbsd-5 amd64 |
xenkernel41 netbsd-5 amd64 |
xenkernel41 netbsd-5 amd64 |
xentools41 netbsd-5 amd64 |
xentools41 netbsd-5 amd64 |
|
xenkernel42 netbsd-5 amd64 |
|
xentools42 netbsd-5 amd64 |
|
|
|
xenkernel3 netbsd-6 i386 FAIL |
|
xentools3 netbsd-6 i386 |
|
xentools3-hvm netbsd-6 i386 FAIL (dependencies fail) |
|
xenkernel33 netbsd-6 i386 |
|
xentools33 netbsd-6 i386 |
xenkernel41 netbsd-6 i386 |
xenkernel41 netbsd-6 i386 |
xentools41 netbsd-6 i386 |
xentools41 netbsd-6 i386 |
|
xenkernel42 netbsd-6 i386 |
|
xentools42 netbsd-6 i386 *MIXED |
|
|
|
(all 3 and 33 seem to FAIL) |
|
xenkernel41 netbsd-7 i386 |
|
xentools41 netbsd-7 i386 |
|
xenkernel42 netbsd-7 i386 |
|
xentools42 netbsd-7 i386 ??FAIL |
|
|
|
(*On netbsd-6 i386, there is a xentools42 in the 2014Q3 official builds, |
|
but it does not build for gdt.) |
|
|
NetBSD as a dom0 |
NetBSD as a dom0 |
================ |
================ |
Line 255 For debugging, one may copy xen-debug.gz
|
Line 274 For debugging, one may copy xen-debug.gz
|
to DIAGNOSTIC and DEBUG in NetBSD. xen-debug.gz is basically only |
to DIAGNOSTIC and DEBUG in NetBSD. xen-debug.gz is basically only |
useful with a serial console. Then, place a NetBSD XEN3_DOM0 kernel |
useful with a serial console. Then, place a NetBSD XEN3_DOM0 kernel |
in /, copied from releasedir/amd64/binary/kernel/netbsd-XEN3_DOM0.gz |
in /, copied from releasedir/amd64/binary/kernel/netbsd-XEN3_DOM0.gz |
of a NetBSD build. Both xen and NetBSD may be left compressed. (If |
of a NetBSD build. If using i386, use |
using i386, use releasedir/i386/binary/kernel/netbsd-XEN3PAE_DOM0.gz.) |
releasedir/i386/binary/kernel/netbsd-XEN3PAE_DOM0.gz. (If using Xen |
|
3.1 and i386, you may use XEN3_DOM0 with the non-PAE Xen. But you |
With Xen as the kernel, you must provide a dom0 NetBSD kernel to be |
should not use Xen 3.1.) Both xen and the NetBSD kernel may be (and |
used as a module; place this in /. Suitable kernels are provided in |
typically are) left compressed. |
releasedir/binary/kernel: |
|
|
In a dom0 kernel, kernfs is mandatory for xend to comunicate with the |
i386 XEN3_DOM0 |
kernel, so ensure that /kern is in fstab. TODO: Say this is default, |
i386 XEN3PAE_DOM0 |
or file a PR and give a reference. |
amd64 XEN3_DOM0 |
|
|
|
The first one is only for use with Xen 3.1 and i386-mode Xen (and you |
|
should not do this). Current Xen always uses PAE on i386, but you |
|
should generally use amd64 for the dom0. In a dom0 kernel, kernfs is |
|
mandatory for xend to comunicate with the kernel, so ensure that /kern |
|
is in fstab. TODO: Say this is default, or file a PR and give a |
|
reference. |
|
|
|
Because you already installed NetBSD, you have a working boot setup |
Because you already installed NetBSD, you have a working boot setup |
with an MBR bootblock, either bootxx_ffsv1 or bootxx_ffsv2 at the |
with an MBR bootblock, either bootxx_ffsv1 or bootxx_ffsv2 at the |
Line 283 See boot.cfg(5) for an example. The bas
|
Line 294 See boot.cfg(5) for an example. The bas
|
menu=Xen:load /netbsd-XEN3_DOM0.gz console=pc;multiboot /xen.gz dom0_mem=256M |
menu=Xen:load /netbsd-XEN3_DOM0.gz console=pc;multiboot /xen.gz dom0_mem=256M |
|
|
which specifies that the dom0 should have 256M, leaving the rest to be |
which specifies that the dom0 should have 256M, leaving the rest to be |
allocated for domUs. In an attempt to add performance, one can also |
allocated for domUs. To use In an attempt to add performance, one can |
add |
also add |
|
|
dom0_max_vcpus=1 dom0_vcpus_pin |
dom0_max_vcpus=1 dom0_vcpus_pin |
|
|
Line 457 Unprivileged domains (domU)
|
Line 468 Unprivileged domains (domU)
|
This section describes general concepts about domUs. It does not |
This section describes general concepts about domUs. It does not |
address specific domU operating systems or how to install them. The |
address specific domU operating systems or how to install them. The |
config files for domUs are typically in /usr/pkg/etc/xen, and are |
config files for domUs are typically in /usr/pkg/etc/xen, and are |
typically named so that the file anme, domU name and the domU's host |
typically named so that the file name, domU name and the domU's host |
name match. |
name match. |
|
|
The domU is provided with cpu and memory by Xen, configured by the |
The domU is provided with cpu and memory by Xen, configured by the |
Line 531 anyplace, reasonable places to store dom
|
Line 542 anyplace, reasonable places to store dom
|
(so they are near the dom0 kernel), in /usr/pkg/etc/xen (near the |
(so they are near the dom0 kernel), in /usr/pkg/etc/xen (near the |
config files), or in /u0/xen (where the vdisks are). |
config files), or in /u0/xen (where the vdisks are). |
|
|
|
Note that loading the domU kernel from the dom0 implies that boot |
|
blocks, /boot, /boot.cfg, and so on are all ignored in the domU. |
See the VPS section near the end for discussion of alternate ways to |
See the VPS section near the end for discussion of alternate ways to |
obtain domU kernels. |
obtain domU kernels. |
|
|
Line 854 NetBSD as a domU in a VPS
|
Line 867 NetBSD as a domU in a VPS
|
The bulk of the HOWTO is about using NetBSD as a dom0 on your own |
The bulk of the HOWTO is about using NetBSD as a dom0 on your own |
hardware. This section explains how to deal with Xen in a domU as a |
hardware. This section explains how to deal with Xen in a domU as a |
virtual private server where you do not control or have access to the |
virtual private server where you do not control or have access to the |
dom0. |
dom0. This is not intended to be an exhaustive list of VPS providers; |
|
only a few are mentioned that specifically support NetBSD. |
|
|
VPS operators provide varying degrees of access and mechanisms for |
VPS operators provide varying degrees of access and mechanisms for |
configuration. The big issue is usually how one controls which kernel |
configuration. The big issue is usually how one controls which kernel |
is booted, because the kernel is nominally in the dom0 filesystem (to |
is booted, because the kernel is nominally in the dom0 filesystem (to |
which VPS users do not normally have acesss). |
which VPS users do not normally have acesss). A second issue is how |
|
to install NetBSD. |
A VPS user may want to compile a kernel for security updates, to run |
A VPS user may want to compile a kernel for security updates, to run |
npf, run IPsec, or any other reason why someone would want to change |
npf, run IPsec, or any other reason why someone would want to change |
their kernel. |
their kernel. |
|
|
One approach is to have an adminstrative interface to upload a kernel, |
One approach is to have an adminstrative interface to upload a kernel, |
or to select from a prepopulated list. |
or to select from a prepopulated list. Other approaches are pygrub |
|
(deprecated) and pvgrub, which are ways to have a bootloader obtain a |
|
kernel from the domU filesystem. This is closer to a regular physical |
|
computer, where someone who controls a machine can replace the kernel. |
|
|
|
A second issue is multiple CPUs. With NetBSD 6, domUs support |
|
multiple vcpus, and it is typical for VPS providers to enable multiple |
|
CPUs for NetBSD domUs. |
|
|
|
pygrub |
|
------- |
|
|
|
pygrub runs in the dom0 and looks into the domU filesystem. This |
|
implies that the domU must have a kernel in a filesystem in a format |
|
known to pygrub. As of 2014, pygrub seems to be of mostly historical |
|
interest. |
|
|
|
pvgrub |
|
------ |
|
|
Otehr approaches are pvgrub and py-grub, which are ways to start a |
pvgrub is a version of grub that uses PV operations instead of BIOS |
bootloader from the dom0 instead of the actual domU kernel, and for |
calls. It is booted from the dom0 as the domU kernel, and then reads |
that loader to then load a kernel from the domU filesystem. This is |
/grub/menu.lst and loads a kernel from the domU filesystem. |
closer to a regular physical computer, where someone who controls a |
|
machine can replace the kernel. |
[Panix](http://www.panix.com/) lets users use pvgrub. Panix reports |
|
that pvgrub works with FFsv2 with 16K/2K and 32K/4K block/frag sizes |
|
(and hence with defaults from "newfs -O 2"). See [Panix's pvgrub |
|
page](http://www.panix.com/v-colo/grub.html), which describes only |
|
Linux but should be updated to cover NetBSD :-). |
|
|
|
[prgmr.com](http://prgmr.com/) also lets users with pvgrub to boot |
|
their own kernel. See then [prgmr.com NetBSD |
|
HOWTO](http://wiki.prgmr.com/mediawiki/index.php/NetBSD_as_a_DomU) |
|
(which is in need of updating). |
|
|
|
It appears that [grub's FFS |
|
code](http://xenbits.xensource.com/hg/xen-unstable.hg/file/bca284f67702/tools/libfsimage/ufs/fsys_ufs.c) |
|
does not support all aspects of modern FFS, but there are also reports |
|
that FFSv2 works fine. At prgmr, typically one has an ext2 or FAT |
|
partition for the kernel with the intent that grub can understand it, |
|
which leads to /netbsd not being the actual kernel. One must remember |
|
to update the special boot partiion. |
|
|
prmgr and pvgrub |
Amazon |
---------------- |
------ |
|
|
TODO: Perhaps reference panix, prmgr, amazon as interesting examples. |
TODO: add link to NetBSD amazon howto. |
Explain what prmgr does. |
|
|
|
Using npf |
Using npf |
--------- |
--------- |
Line 886 Using npf
|
Line 934 Using npf
|
In standard kernels, npf is a module, and thus cannot be loadeed in a |
In standard kernels, npf is a module, and thus cannot be loadeed in a |
DOMU kernel. |
DOMU kernel. |
|
|
TODO: explain how to compile npf into a custom kernel, answering: |
TODO: explain how to compile npf into a custom kernel, answering (but |
|
note that the problem was caused by not booting the right kernel): |
http://mail-index.netbsd.org/netbsd-users/2014/12/26/msg015576.html |
http://mail-index.netbsd.org/netbsd-users/2014/12/26/msg015576.html |
|
|
|
TODO items for improving NetBSD/xen |
|
=================================== |
|
|
|
* Package Xen 4.4. |
|
* Get PCI passthrough working on Xen 4.2 (or 4.4). |
|
* Get pvgrub into pkgsrc, either via xentools or separately. |
|
* grub |
|
* Check/add support to pkgsrc grub2 for UFS2 and arbitrary |
|
fragsize/blocksize (UFS2 support may be present; the point is to |
|
make it so that with any UFS1/UFS2 filesystem setup that works |
|
with NetBSD grub will also work). |
|
See [pkg/40258](http://gnats.netbsd.org/40258). |
|
* Push patches upstream. |
|
* Get UFS2 patches into pvgrub. |
|
* Add support for PV ops to a version of /boot, and make it usable as |
|
a kernel in Xen, similar to pvgrub. |