version 1.154, 2018/07/26 16:29:45
|
version 1.155, 2018/08/01 13:19:43
|
Line 3
|
Line 3
|
Introduction |
Introduction |
============ |
============ |
|
|
[![[Xen |
|
screenshot]](https://www.netbsd.org/gallery/in-Action/hubertf-xens.png)](https://www.netbsd.org/gallery/in-Action/hubertf-xen.png) |
|
|
|
Xen is a hypervisor for x86 hardware, which supports running multiple guest |
Xen is a hypervisor for x86 hardware, which supports running multiple guest |
operating systems on a single physical machine. Xen is a Type 1 or |
operating systems on a single physical machine. Xen is a Type 1 or |
bare-metal hypervisor; one uses the Xen kernel to control the CPU, |
bare-metal hypervisor; one uses the Xen kernel to control the CPU, |
Line 31 guests must be specifically coded for Xe
|
Line 28 guests must be specifically coded for Xe
|
modification is required; however, hardware support is required, such |
modification is required; however, hardware support is required, such |
as VT-x on Intel CPUs and SVM on AMD CPUs. |
as VT-x on Intel CPUs and SVM on AMD CPUs. |
|
|
There are further features for IOMMU virtualization, Intel's VT-d and |
|
AMD's AMD-Vi. TODO: Explain whether Xen on NetBSD makes use of these |
|
features. TODO: Review by someone who really understands this. |
|
|
|
At boot, the dom0 kernel is loaded as a module with Xen as the kernel. |
At boot, the dom0 kernel is loaded as a module with Xen as the kernel. |
The dom0 can start one or more domUs. (Booting is explained in detail |
The dom0 can start one or more domUs. (Booting is explained in detail |
in the dom0 section.) |
in the dom0 section.) |
Line 83 See also the [Xen Security Advisory page
|
Line 76 See also the [Xen Security Advisory page
|
|
|
Note: Xen 4.2 was the last version to support 32bit CPUs. |
Note: Xen 4.2 was the last version to support 32bit CPUs. |
|
|
Xen command program |
|
------------------- |
|
|
|
Early Xen used a program called xm to manipulate the system from the |
|
dom0. Starting in 4.1, a replacement program with similar behavior |
|
called xl is provided, but it does not work well in 4.1. In 4.2, both |
|
xm and xl work fine. 4.4 is the last version that has xm. |
|
|
|
You must make a global choice to use xm or xl, because it affects not |
|
only which command you use, but the command used by rc.d scripts |
|
(specifically xendomains) and which daemons should be run. The |
|
xentools packages provide xl for 4.2 and up. |
|
|
|
In 4.2, you can choose to use xm by simply changing the ctl_command |
|
variable and setting xend=YES in rc.conf. |
|
|
|
With xl, virtual devices are configured in parallel, which can cause |
|
problems if they are written assuming serial operation (e.g., updating |
|
firewall rules without explicit locking). There is now locking for |
|
the provided scripts, which works for normal casses (e.g, file-backed |
|
xbd, where a vnd must be allocated). But, as of 201612, it has not |
|
been adequately tested for a complex custom setup with a large number |
|
of interfaces. |
|
|
|
NetBSD versions |
NetBSD versions |
--------------- |
--------------- |
|
|
The netbsd-7, netbsd-8, and -current branches are all reasonable |
|
choices, with more or less the same considerations for non-Xen use. |
|
NetBSD 8 is recommended as the stable version of the most recent |
NetBSD 8 is recommended as the stable version of the most recent |
release for production use. |
release for production use. |
|
|
Line 128 because the hypercall interface has rema
|
Line 95 because the hypercall interface has rema
|
Architecture |
Architecture |
------------ |
------------ |
|
|
Xen itself can run on i386 (Xen < 4.2) or amd64 hardware (all Xen |
Xen itself runs on amd64 hardware. Practically, almost any computer |
versions). Practically, almost any computer where one would want to |
where one would want to run Xen today supports amd64. |
run Xen today supports amd64. |
|
|
The dom0 system, plus each domU, can be either i386PAE or amd64. |
Xen, the dom0 system, and each domU system can be either i386 or |
i386 without PAE is not supported. |
amd64. When building a xenkernel package, one obtains an i386 Xen |
|
kernel on an i386 host, and an amd64 Xen kernel on an amd64 host. If |
The standard approach is to use NetBSD/amd64 for the dom0. For domUs, |
the Xen kernel is i386, then the dom0 kernel and all domU kernels must |
NetBSD/i386 (PAE) and NetBSD/amd64 are in widespread use, and there is |
be i386. With an amd64 Xen kernel, an amd64 dom0 kernel is known to |
|
work, and an i386 dom0 kernel should in theory work. An amd64 |
|
Xen/dom0 is known to support both i386 and amd64 domUs. |
|
|
|
i386 dom0 and domU kernels must be PAE. PAE kernels are included in |
|
the NetBSD default build. |
|
|
|
Because of the above, the standard approach is to use an amd64 Xen |
|
kernel and NetBSD/amd64 for the dom0. For domUs, NetBSD/i386 (PAE) and |
|
NetBSD/amd64 are in widespread use, and there is |
|
little to no Xen-specific reason to prefer one over the other. |
little to no Xen-specific reason to prefer one over the other. |
|
|
Note that to use an i386 dom0 with Xen 4.5 or higher, one must build |
Note that to use an i386 dom0 with Xen 4.5 or higher, one must build |
Line 159 explained that PV system call overhead w
|
Line 116 explained that PV system call overhead w
|
there is some notion that i386 guests are faster. It goes on to |
there is some notion that i386 guests are faster. It goes on to |
caution that the total situation is complex and not entirely |
caution that the total situation is complex and not entirely |
understood. On top of that caution, the post is about Linux, not |
understood. On top of that caution, the post is about Linux, not |
NetBSD. TODO: Include link to benchmarks, if someone posts them. |
NetBSD. |
|
|
NetBSD as a dom0 |
NetBSD as a dom0 |
================ |
================ |
Line 245 HOWTO page.)
|
Line 202 HOWTO page.)
|
Installation of Xen |
Installation of Xen |
------------------- |
------------------- |
|
|
In the dom0, install sysutils/xenkernel42 and sysutils/xentools42 from |
We will consider that you chose to use Xen 4.8, with NetBSD/amd64 as |
pkgsrc (or another matching pair). See [the pkgsrc |
dom0. In the dom0, install xenkernel48 and xentools48 from pkgsrc. |
documentation](https://www.NetBSD.org/docs/pkgsrc/) for help with |
Ensure that your packages are recent. |
pkgsrc. Ensure that your packages are recent; the HOWTO does not |
|
contemplate old builds. |
Once this is done, install the Xen kernel itself: |
|
|
Next you need to install the selected Xen kernel itself, which is |
[[!template id=programlisting text=""" |
installed by pkgsrc as "/usr/pkg/xen*-kernel/xen.gz". Copy it to /. |
# cp /usr/pkg/xen48-kernel/xen.gz / |
For debugging, one may copy xen-debug.gz; this is conceptually similar |
"""]] |
to DIAGNOSTIC and DEBUG in NetBSD. xen-debug.gz is basically only |
|
useful with a serial console. Then, place a NetBSD XEN3_DOM0 kernel |
Then, place a NetBSD XEN3_DOM0 kernel in /, copied from |
in /, copied from releasedir/amd64/binary/kernel/netbsd-XEN3_DOM0.gz |
releasedir/amd64/binary/kernel/netbsd-XEN3_DOM0.gz of a NetBSD build. |
of a NetBSD build. If using i386, use |
|
releasedir/i386/binary/kernel/netbsd-XEN3PAE_DOM0.gz. Both xen and |
|
the NetBSD kernel may be (and typically are) left compressed. |
|
|
|
In a dom0, kernfs is mandatory for xend to communicate with the |
|
kernel, so ensure that /kern is in fstab. (A standard NetBSD install |
|
should already mount /kern.) |
|
|
|
Because you already installed NetBSD, you have a working boot setup |
|
with an MBR bootblock, either bootxx_ffsv1 or bootxx_ffsv2 at the |
|
beginning of your root file system, have /boot, and likely also |
|
/boot.cfg. (If not, fix before continuing!) |
|
|
|
Add a line to /boot.cfg to boot Xen. See boot.cfg(5) for an |
Add a line to /boot.cfg to boot Xen: |
example. The basic line is: |
|
|
|
[[!template id=programlisting text=""" |
[[!template id=programlisting text=""" |
menu=Xen:load /netbsd-XEN3_DOM0.gz console=pc;multiboot /xen.gz dom0_mem=512M |
menu=Xen:load /netbsd-XEN3_DOM0.gz console=pc;multiboot /xen.gz dom0_mem=512M |
"""]] |
"""]] |
|
|
which specifies that the dom0 should have 512M, leaving the rest to be |
which specifies that the dom0 should have 512MB of ram, leaving the rest |
allocated for domUs. To use a serial console, use |
to be allocated for domUs. To use a serial console, use |
|
|
[[!template id=programlisting text=""" |
[[!template id=programlisting text=""" |
menu=Xen:load /netbsd-XEN3_DOM0.gz;multiboot /xen.gz dom0_mem=512M console=com1 com1=9600,8n1 |
menu=Xen:load /netbsd-XEN3_DOM0.gz;multiboot /xen.gz dom0_mem=512M console=com1 com1=9600,8n1 |
Line 298 dom0_max_vcpus=1 dom0_vcpus_pin
|
Line 242 dom0_max_vcpus=1 dom0_vcpus_pin
|
"""]] |
"""]] |
|
|
to force only one vcpu to be provided (since NetBSD dom0 can't use |
to force only one vcpu to be provided (since NetBSD dom0 can't use |
more) and to pin that vcpu to a physical CPU. TODO: benchmark this. |
more) and to pin that vcpu to a physical CPU. |
|
|
Xen has [many boot |
Xen has [many boot |
options](http://xenbits.xenproject.org/docs/4.5-testing/misc/xen-command-line.html), |
options](http://xenbits.xenproject.org/docs/4.5-testing/misc/xen-command-line.html), |
Line 306 and other than dom0 memory and max_vcpus
|
Line 250 and other than dom0 memory and max_vcpus
|
necessary. |
necessary. |
|
|
As with non-Xen systems, you should have a line to boot /netbsd (a |
As with non-Xen systems, you should have a line to boot /netbsd (a |
kernel that works without Xen). Consider a line to boot /netbsd.ok (a |
kernel that works without Xen). |
fallback version of the non-Xen kernel, updated manually when you are |
|
sure /netbsd is ok). Consider also a line to boot fallback versions |
|
of Xen and the dom0 kernel, but note that non-Xen NetBSD can be used |
|
to resolve Xen booting issues. |
|
|
|
Now, reboot so that you are running a DOM0 kernel under Xen, rather |
Now, reboot so that you are running a DOM0 kernel under Xen, rather |
than GENERIC without Xen. |
than GENERIC without Xen. |