version 1.112, 2016/12/20 12:53:06
|
version 1.147, 2018/07/26 10:47:25
|
Line 1
|
Line 1
|
|
[[!meta title="Xen HowTo"]] |
|
|
Introduction |
Introduction |
============ |
============ |
|
|
[![[Xen |
[![[Xen |
screenshot]](http://www.netbsd.org/gallery/in-Action/hubertf-xens.png)](http://www.netbsd.org/gallery/in-Action/hubertf-xen.png) |
screenshot]](https://www.netbsd.org/gallery/in-Action/hubertf-xens.png)](https://www.netbsd.org/gallery/in-Action/hubertf-xen.png) |
|
|
Xen is a hypervisor (or virtual machine monitor) for x86 hardware |
Xen is a hypervisor (or virtual machine monitor) for x86 hardware |
(i686-class or higher), which supports running multiple guest |
(i686-class or higher), which supports running multiple guest |
Line 27 code for Xen and need not be aware that
|
Line 29 code for Xen and need not be aware that
|
Attempts to access hardware registers are trapped and emulated. This |
Attempts to access hardware registers are trapped and emulated. This |
style is less efficient but can run unmodified guests. |
style is less efficient but can run unmodified guests. |
|
|
Generally any amd64 machine will work with Xen and PV guests. In |
Generally any machine that runs NetBSD/amd64 will work with Xen and PV |
theory i386 computers without amd64 support can be used for Xen <= |
guests. In theory i386 computers (without x86_64/amd64 support) can |
4.2, but we have no recent reports of this working (this is a hint). |
be used for Xen <= 4.2, but we have no recent reports of this working |
For HVM guests, the VT or VMX CPU feature (Intel) or SVM/HVM/VT |
(this is a hint). For HVM guests, hardware support is needed, but it |
(amd64) is needed; "cpuctl identify 0" will show this. TODO: Clean up |
is common on recent machines. For Intel CPUs, one needs the VT-x |
and check the above features. |
extension, shown in "cpuctl identify 0" as VMX. For AMD CPUs, one |
|
needs the AMD-V extensions, shown in "cpuctl identify 0" as SVM. |
|
There are further features for IOMMU virtualization, Intel's VT-d and |
|
AMD's AMD-Vi. TODO: Explain whether Xen on NetBSD makes use of these |
|
features. TODO: Review by someone who really understands this. |
|
|
|
Note that a FreeBSD dom0 requires VT-x and VT-d (or equivalent); this |
|
is because the FreeBSD dom0 does not run in PV mode. |
|
|
At boot, the dom0 kernel is loaded as a module with Xen as the kernel. |
At boot, the dom0 kernel is loaded as a module with Xen as the kernel. |
The dom0 can start one or more domUs. (Booting is explained in detail |
The dom0 can start one or more domUs. (Booting is explained in detail |
Line 44 attempts to address both the case of run
|
Line 53 attempts to address both the case of run
|
and running domUs under it (NetBSD and other), and also running NetBSD |
and running domUs under it (NetBSD and other), and also running NetBSD |
as a domU in a VPS. |
as a domU in a VPS. |
|
|
Some versions of Xen support "PCI passthrough", which means that |
Xen 3.1 in pkgsrc used to support "PCI passthrough", which means that |
specific PCI devices can be made available to a specific domU instead |
specific PCI devices can be made available to a specific domU instead |
of the dom0. This can be useful to let a domU run X11, or access some |
of the dom0. This can be useful to let a domU run X11, or access some |
network interface or other peripheral. |
network interface or other peripheral. |
Line 62 things must be done, guiding the reader
|
Line 71 things must be done, guiding the reader
|
path when there are no known good reasons to stray. |
path when there are no known good reasons to stray. |
|
|
This HOWTO presumes a basic familiarity with the Xen system |
This HOWTO presumes a basic familiarity with the Xen system |
architecture. This HOWTO presumes familiarity with installing NetBSD |
architecture, with installing NetBSD on i386/amd64 hardware, and with |
on i386/amd64 hardware and installing software from pkgsrc. |
installing software from pkgsrc. See also the [Xen |
See also the [Xen website](http://www.xenproject.org/). |
website](http://www.xenproject.org/). |
|
|
Versions of Xen and NetBSD |
Versions of Xen and NetBSD |
========================== |
========================== |
Line 74 of Xen version and NetBSD version. This
|
Line 83 of Xen version and NetBSD version. This
|
which version to choose. Versions not in pkgsrc and older unsupported |
which version to choose. Versions not in pkgsrc and older unsupported |
versions of NetBSD are intentionally ignored. |
versions of NetBSD are intentionally ignored. |
|
|
Xen |
The term "amd64" is used to refer to both the NetBSD port and to the |
--- |
hardware architecture on which it runs. (Such hardware is made by |
|
both Intel and AMD, and in 2016 a normal PC has this CPU |
|
architecture.) |
|
|
|
Xen versions |
|
------------ |
|
|
In NetBSD, Xen is provided in pkgsrc, via matching pairs of packages |
In NetBSD, Xen is provided in pkgsrc, via matching pairs of packages |
xenkernel and xentools. We will refer only to the kernel versions, |
xenkernel and xentools. We will refer only to the kernel versions, |
but note that both packages must be installed together and must have |
but note that both packages must be installed together and must have |
matching versions. |
matching versions. |
|
|
xenkernel3 provides Xen 3.1. It is no longer maintained by Xen, and the last applied security patch was in |
Versions available in pkgsrc: |
2011. Thus, it should not be used. It supports PCI passthrough, |
|
which is why people use it anyway. Xen 3.1 supports i386, both PAE and |
|
non-PAE. |
|
|
|
xenkernel33 provides Xen 3.3. It is no longer maintained by Xen, and |
|
the last applied security patch was in 2012. Thus, it should not be |
|
used. Xen 3.3 supports i386, but only in PAE mode. There are no good |
|
reasons to run this version. |
|
|
|
xenkernel41 provides Xen 4.1. It is no longer maintained by Xen, but |
|
as of 2016-12 received backported security patches. Xen 4.1 supports |
|
i386, but only in PAE mode. There are no good reasons to run this |
|
version. |
|
|
|
xenkernel42 provides Xen 4.2. It is no longer maintained by Xen, but |
|
as of 2016-12 received backported security patches. Xen 4.2 supports |
|
i386, but only in PAE mode. The only reason to run this is if you |
|
need to use xm instead of xl, or if you need to run an i386 dom0 |
|
(because your hardware is i386 only). |
|
|
|
xenkernel45 provides Xen 4.5. It is no longer maintained by Xen, but |
|
as of 2016-12 it received security patches. Xen 4.5 requires an amd64 |
|
dom0, but domUs can be amd64 or i386 PAE. TODO: It is either a |
|
conservative choice or somewhat old. |
|
|
|
xenkernel45 provides Xen 4.6. It is new to pkgsrc in 2016-05. It is |
|
no longer maintained by Xen, but as of 2016-12 it received security |
|
patches. Xen 4.6 requires an amd64 dom0, but domUs can be amd64 or |
|
i386 PAE. TODO: It is either a somewhat aggressive choice or the |
|
standard choice |
|
|
|
See also the [Xen Security Advisory page](http://xenbits.xen.org/xsa/). |
[[!table data=""" |
|
Xen Version |Package Name |Xen CPU Support |EOL'ed By Upstream |
|
4.2 |xenkernel42 |32bit, 64bit |Yes |
|
4.5 |xenkernel45 |64bit |Yes |
|
4.6 |xenkernel46 |64bit |Partially |
|
4.8 |xenkernel48 |64bit |No |
|
4.11 |xenkernel411 |64bit |No |
|
"""]] |
|
|
Ideally newer versions of Xen will be added to pkgsrc. |
See also the [Xen Security Advisory page](http://xenbits.xen.org/xsa/). |
|
|
Note that NetBSD support is called XEN3. It works with Xen 3 and Xen |
Note: Xen 4.2 was the last version to support 32bit CPUs. |
4 because the hypercall interface has been stable. |
|
|
|
Xen command program |
Xen command program |
------------------- |
------------------- |
Line 127 Xen command program
|
Line 117 Xen command program
|
Early Xen used a program called xm to manipulate the system from the |
Early Xen used a program called xm to manipulate the system from the |
dom0. Starting in 4.1, a replacement program with similar behavior |
dom0. Starting in 4.1, a replacement program with similar behavior |
called xl is provided, but it does not work well in 4.1. In 4.2, both |
called xl is provided, but it does not work well in 4.1. In 4.2, both |
xm and xl work fine. 4.4 is the last version that has xm. You must |
xm and xl work fine. 4.4 is the last version that has xm. |
choose one or the other, because it affects which daemons you run. |
|
However, the rc.d scripts provided by xentools packages expect a |
|
particular version, and you should use the version used by the |
|
scripts. |
|
|
|
NetBSD |
You must make a global choice to use xm or xl, because it affects not |
------ |
only which command you use, but the command used by rc.d scripts |
|
(specifically xendomains) and which daemons should be run. The |
|
xentools packages provide xl for 4.2 and up. |
|
|
|
In 4.2, you can choose to use xm by simply changing the ctl_command |
|
variable and setting xend=YES in rc.conf. |
|
|
|
With xl, virtual devices are configured in parallel, which can cause |
|
problems if they are written assuming serial operation (e.g., updating |
|
firewall rules without explicit locking). There is now locking for |
|
the provided scripts, which works for normal casses (e.g, file-backed |
|
xbd, where a vnd must be allocated). But, as of 201612, it has not |
|
been adequately tested for a complex custom setup with a large number |
|
of interfaces. |
|
|
The netbsd-6, netbsd-7, and -current branches are all reasonable |
NetBSD versions |
|
--------------- |
|
|
|
The netbsd-7, netbsd-8, and -current branches are all reasonable |
choices, with more or less the same considerations for non-Xen use. |
choices, with more or less the same considerations for non-Xen use. |
Therefore, netbsd-7 is recommended as the stable version of the most |
NetBSD 8 is recommended as the stable version of the most recent |
recent release for production use. For those wanting to learn Xen or |
release for production use. |
without production stability concerns, netbsd-7 is still likely most |
|
appropriate, but -current is also a reasonable choice. Xen runs fine |
For developing Xen, netbsd-current may be appropriate. |
on netbsd-5, but the xentools packages are likely difficult to build. |
|
|
|
As of NetBSD 6, a NetBSD domU will support multiple vcpus. There is |
As of NetBSD 6, a NetBSD domU will support multiple vcpus. There is |
no SMP support for NetBSD as dom0. (The dom0 itself doesn't really |
no SMP support for NetBSD as dom0. (The dom0 itself doesn't really |
need SMP for dom0 functions; the lack of support is really a problem |
need SMP for dom0 functions; the lack of support is really a problem |
when using a dom0 as a normal computer.) |
when using a dom0 as a normal computer.) |
|
|
|
Note: NetBSD support is called XEN3. However, it does support Xen 4, |
|
because the hypercall interface has remained identical. |
|
|
Architecture |
Architecture |
------------ |
------------ |
|
|
Xen itself can run on i386 (Xen < 3.1) or amd64 machines (all Xen |
Xen itself can run on i386 (Xen < 4.2) or amd64 hardware (all Xen |
versions). (Practically, almost any computer where one would want to |
versions). (Practically, almost any computer where one would want to |
run Xen today supports amd64.) |
run Xen today supports amd64.) |
|
|
Xen, the dom0 kernel, and each domU kernel can be either i386 or |
Xen, the dom0 system, and each domU system can be either i386 or |
amd64. When building a xenkernel package, one obtains i386 on an i386 |
amd64. When building a xenkernel package, one obtains an i386 Xen |
host, and amd64 on an amd64 host. If the Xen kernel is i386, then the |
kernel on an i386 host, and an amd64 Xen kernel on an amd64 host. If |
dom0 kernel and all domU kernels must be i386. With an amd64 Xen |
the Xen kernel is i386, then the dom0 kernel and all domU kernels must |
kernel, an amd64 dom0 kernel is known to work, and an i386PAE dom0 |
be i386. With an amd64 Xen kernel, an amd64 dom0 kernel is known to |
kernel should in theory work. An amd64 Xen/dom0 is known to support |
work, and an i386 dom0 kernel should in theory work. An amd64 |
both i386PAE and amd64 domUs. |
Xen/dom0 is known to support both i386 and amd64 domUs. |
|
|
i386 dom0 and domU kernels must be PAE (except for Xen 3.1); these are |
i386 dom0 and domU kernels must be PAE (except for an i386 Xen 3.1 |
built by default. (Note that emacs (at least) fails if run on i386 |
kernel, where one can use non-PAE for dom0 and all domUs); PAE kernels |
with PAE when built without, and vice versa, presumably due to bugs in |
are included in the NetBSD default build. (Note that emacs (at least) |
the undump code.) |
fails if run on i386 with PAE when built without, and vice versa, |
|
presumably due to bugs in the undump code.) |
Because of the above, the standard approach is to use amd64 for the |
|
dom0. |
Because of the above, the standard approach is to use an amd64 Xen |
|
kernel and NetBSD/amd64 for the dom0. For domUs, NetBSD/i386 (with |
Xen 4.2 is the last version to support i386 as a host. TODO: Clarify |
the PAE kernel) and NetBSD/amd64 are in widespread use, and there is |
if this is about the CPU, the Xen kernel, or the dom0 kernel having to |
little to no Xen-specific reason to prefer one over the other. |
be amd64. |
|
|
Note that to use an i386 dom0 with Xen 4.5 or higher, one must build |
|
(or obtain from pre-built packages) an amd64 Xen kernel and install |
|
that on the system. (One must also use a PAE i386 kernel, but this is |
|
also required with an i386 Xen kernel.). Almost no one in the |
|
NetBSD/Xen community does this, and the standard, well-tested, |
|
approach is to use an amd64 dom0. |
|
|
|
A [posting on |
|
xen-devel](https://lists.xen.org/archives/html/xen-devel/2012-07/msg00085.html) |
|
explained that PV system call overhead was higher on amd64, and thus |
|
there is some notion that i386 guests are faster. It goes on to |
|
caution that the total situation is complex and not entirely |
|
understood. On top of that caution, the post is about Linux, not |
|
NetBSD. TODO: Include link to benchmarks, if someone posts them. |
|
|
Stability |
Stability |
--------- |
--------- |
|
|
Mostly, NetBSD as a dom0 or domU is quite stable. |
Mostly, NetBSD as a dom0 or domU is quite stable. However, just like every |
However, there are some open PRs indicating problems. |
other architecture, there are some open PRs indicating problems. |
|
|
- [PR 48125](http://gnats.netbsd.org/48125) |
|
- [PR 47720](http://gnats.netbsd.org/47720) |
|
|
|
Note also that there are issues with sparse vnd(4) instances, but |
Note also that there are issues with sparse vnd(4) instances, but |
these are not about Xen -- they just are noticed with sparse vnd(4) |
these are not about Xen -- they just are noticed with sparse vnd(4) |
Line 193 instances in support of virtual disks in
|
Line 207 instances in support of virtual disks in
|
Recommendation |
Recommendation |
-------------- |
-------------- |
|
|
Therefore, this HOWTO recommends running xenkernel45 or xenkernel46, |
Therefore, this HOWTO recommends running xenkernel46, xl, the NetBSD 7 |
xl, the NetBSD 7 stable branch, and to use an amd64 kernel as the |
stable branch, and therefore to use an amd64 kernel as the dom0. |
dom0. Either the i386PAE or amd64 version of NetBSD may be used as |
Either the i386PAE or amd64 version of NetBSD may be used as domUs. |
domUs. |
|
|
A tentative replacement recommendation is xenkernel48, xl, and NetBSD |
Build problems |
8. |
-------------- |
|
|
Because bugs are fixed quite often, and because of Xen security |
Ideally, all versions of Xen in pkgsrc would build on all versions of |
advisories, it is good to stay up to date with NetBSD (tracking a |
NetBSD on both i386 and amd64. However, that isn't the case. Besides |
stable branch), with the Xen kernel (tracking a Xen version via |
aging code and aging compilers, qemu (included in xentools for HVM |
pkgsrc), and with the Xen tools. Specifically, NetBSD (-7 and |
support) is difficult to build. The following are known to work or FAIL: |
-current) got an important fix affecting dom0/domU timesharing in |
|
November, 2015, and xentools46 got a fix to enable Ubuntu guests to |
xenkernel3 netbsd-5 amd64 |
boot in December, 2016. |
xentools3 netbsd-5 amd64 |
|
xentools3=hvm netbsd-5 amd64 ???? |
|
xenkernel33 netbsd-5 amd64 |
|
xentools33 netbsd-5 amd64 |
|
xenkernel41 netbsd-5 amd64 |
|
xentools41 netbsd-5 amd64 |
|
xenkernel42 netbsd-5 amd64 |
|
xentools42 netbsd-5 amd64 |
|
|
|
xenkernel3 netbsd-6 i386 FAIL |
|
xentools3 netbsd-6 i386 |
|
xentools3-hvm netbsd-6 i386 FAIL (dependencies fail) |
|
xenkernel33 netbsd-6 i386 |
|
xentools33 netbsd-6 i386 |
|
xenkernel41 netbsd-6 i386 |
|
xentools41 netbsd-6 i386 |
|
xenkernel42 netbsd-6 i386 |
|
xentools42 netbsd-6 i386 *MIXED |
|
|
|
(all 3 and 33 seem to FAIL) |
|
xenkernel41 netbsd-7 i386 |
|
xentools41 netbsd-7 i386 |
|
xenkernel42 netbsd-7 i386 |
|
xentools42 netbsd-7 i386 ??FAIL |
|
|
|
(*On netbsd-6 i386, there is a xentools42 in the 2014Q3 official builds, |
|
but it does not build for gdt.) |
|
|
|
NetBSD as a dom0 |
NetBSD as a dom0 |
================ |
================ |
Line 248 configuration.
|
Line 235 configuration.
|
|
|
For experimenting with Xen, a machine with as little as 1G of RAM and |
For experimenting with Xen, a machine with as little as 1G of RAM and |
100G of disk can work. For running many domUs in productions, far |
100G of disk can work. For running many domUs in productions, far |
more will be needed. |
more will be needed; e.g. 4-8G and 1T of disk is reasonable for a |
|
half-dozen domUs of 512M and 32G each. Basically, the RAM and disk |
|
have to be bigger than the sum of the RAM/disk needs of the dom0 and |
|
all the domUs. |
|
|
|
In 2018-05, trouble booting a dom0 was reported with 256M of RAM: with |
|
512M it worked reliably. This does not make sense, but if you see |
|
"not ELF" after Xen boots, try increasing dom0 RAM. |
|
|
Styles of dom0 operation |
Styles of dom0 operation |
------------------------ |
------------------------ |
Line 271 Xen daemons when not running Xen.
|
Line 265 Xen daemons when not running Xen.
|
Note that NetBSD as dom0 does not support multiple CPUs. This will |
Note that NetBSD as dom0 does not support multiple CPUs. This will |
limit the performance of the Xen/dom0 workstation approach. In theory |
limit the performance of the Xen/dom0 workstation approach. In theory |
the only issue is that the "backend drivers" are not yet MPSAFE: |
the only issue is that the "backend drivers" are not yet MPSAFE: |
http://mail-index.netbsd.org/netbsd-users/2014/08/29/msg015195.html |
https://mail-index.netbsd.org/netbsd-users/2014/08/29/msg015195.html |
|
|
Installation of NetBSD |
Installation of NetBSD |
---------------------- |
---------------------- |
Line 313 Installation of Xen
|
Line 307 Installation of Xen
|
------------------- |
------------------- |
|
|
In the dom0, install sysutils/xenkernel42 and sysutils/xentools42 from |
In the dom0, install sysutils/xenkernel42 and sysutils/xentools42 from |
pkgsrc (or another matching pair). |
pkgsrc (or another matching pair). See [the pkgsrc |
See [the pkgsrc |
documentation](https://www.NetBSD.org/docs/pkgsrc/) for help with |
documentation](http://www.NetBSD.org/docs/pkgsrc/) for help with pkgsrc. |
pkgsrc. Ensure that your packages are recent; the HOWTO does not |
|
contemplate old builds. |
|
|
|
|
For Xen 3.1, support for HVM guests is in sysutils/xentool3-hvm. More |
For Xen 3.1, support for HVM guests is in sysutils/xentool3-hvm. More |
recent versions have HVM support integrated in the main xentools |
recent versions have HVM support integrated in the main xentools |
Line 333 releasedir/i386/binary/kernel/netbsd-XEN
|
Line 329 releasedir/i386/binary/kernel/netbsd-XEN
|
should not use Xen 3.1.) Both xen and the NetBSD kernel may be (and |
should not use Xen 3.1.) Both xen and the NetBSD kernel may be (and |
typically are) left compressed. |
typically are) left compressed. |
|
|
In a dom0 kernel, kernfs is mandatory for xend to communicate with the |
In a dom0, kernfs is mandatory for xend to communicate with the |
kernel, so ensure that /kern is in fstab. TODO: Say this is default, |
kernel, so ensure that /kern is in fstab. (A standard NetBSD install |
or file a PR and give a reference. |
should already mount /kern.) |
|
|
Because you already installed NetBSD, you have a working boot setup |
Because you already installed NetBSD, you have a working boot setup |
with an MBR bootblock, either bootxx_ffsv1 or bootxx_ffsv2 at the |
with an MBR bootblock, either bootxx_ffsv1 or bootxx_ffsv2 at the |
beginning of your root file system, /boot present, and likely |
beginning of your root file system, have /boot, and likely also |
/boot.cfg. (If not, fix before continuing!) |
/boot.cfg. (If not, fix before continuing!) |
|
|
Add a line to to /boot.cfg to boot Xen. See boot.cfg(5) for an |
Add a line to to /boot.cfg to boot Xen. See boot.cfg(5) for an |
example. The basic line is |
example. The basic line is |
|
|
menu=Xen:load /netbsd-XEN3_DOM0.gz console=pc;multiboot /xen.gz dom0_mem=256M |
menu=Xen:load /netbsd-XEN3_DOM0.gz console=pc;multiboot /xen.gz dom0_mem=512M |
|
|
which specifies that the dom0 should have 256M, leaving the rest to be |
which specifies that the dom0 should have 512M, leaving the rest to be |
allocated for domUs. To use a serial console, use |
allocated for domUs. To use a serial console, use |
|
|
menu=Xen:load /netbsd-XEN3_DOM0.gz console=com0;multiboot /xen.gz dom0_mem=256M console=com1 com1=9600,8n1 |
menu=Xen:load /netbsd-XEN3_DOM0.gz;multiboot /xen.gz dom0_mem=512M console=com1 com1=9600,8n1 |
|
|
which will use the first serial port for Xen (which counts starting |
which will use the first serial port for Xen (which counts starting |
from 1), forcing speed/parity, and also for NetBSD (which counts |
from 1, unlike NetBSD which counts starting from 0), forcing |
starting at 0). In an attempt to add performance, one can also add |
speed/parity. Because the NetBSD command line lacks a |
|
"console=pc" argument, it will use the default "xencons" console device, |
|
which directs the console I/O through Xen to the same console device Xen |
|
itself uses (in this case, the serial port). |
|
|
|
In an attempt to add performance, one can also add |
|
|
dom0_max_vcpus=1 dom0_vcpus_pin |
dom0_max_vcpus=1 dom0_vcpus_pin |
|
|
Line 367 and other than dom0 memory and max_vcpus
|
Line 368 and other than dom0 memory and max_vcpus
|
necessary. |
necessary. |
|
|
As with non-Xen systems, you should have a line to boot /netbsd (a |
As with non-Xen systems, you should have a line to boot /netbsd (a |
kernel that works without Xen) and fallback versions of the non-Xen |
kernel that works without Xen). Consider a line to boot /netbsd.ok (a |
kernel, Xen, and the dom0 kernel. |
fallback version of the non-Xen kernel, updated manually when you are |
|
sure /netbsd is ok). Consider also a line to boot fallback versions |
|
of Xen and the dom0 kernel, but note that non-Xen NetBSD can be used |
|
to resolve Xen booting issues. |
|
|
|
Probably you want a default=N line to choose Xen in the absence of |
|
intervention. |
|
|
Now, reboot so that you are running a DOM0 kernel under Xen, rather |
Now, reboot so that you are running a DOM0 kernel under Xen, rather |
than GENERIC without Xen. |
than GENERIC without Xen. |
Line 378 Using grub (historic)
|
Line 385 Using grub (historic)
|
|
|
Before NetBSD's native bootloader could support Xen, the use of |
Before NetBSD's native bootloader could support Xen, the use of |
grub was recommended. If necessary, see the |
grub was recommended. If necessary, see the |
[old grub information](/ports/xen/howto-grub/). |
[old grub information](/ports/xen/howto-grub). |
|
|
The [HowTo on Installing into |
The [HowTo on Installing into |
RAID-1](http://mail-index.NetBSD.org/port-xen/2006/03/01/0010.html) |
RAID-1](https://mail-index.NetBSD.org/port-xen/2006/03/01/0010.html) |
explains how to set up booting a dom0 with Xen using grub with |
explains how to set up booting a dom0 with Xen using grub with |
NetBSD's RAIDframe. (This is obsolete with the use of NetBSD's native |
NetBSD's RAIDframe. (This is obsolete with the use of NetBSD's native |
boot.) |
boot. Now, just create a system with RAID-1, and alter /boot.cfg as |
|
described above.) |
|
|
Configuring Xen |
Configuring Xen |
--------------- |
--------------- |
Line 397 There will be no domUs, and none can be
|
Line 405 There will be no domUs, and none can be
|
to configure the dom0 daemons. |
to configure the dom0 daemons. |
|
|
The daemons which should be run vary with Xen version and with whether |
The daemons which should be run vary with Xen version and with whether |
one is using xm or xl. The Xen 3.1 and 3.3 packages use xm. Xen 4.1 |
one is using xm or xl. The Xen 3.1, 3.3 and 4.1 packages use xm. Xen |
and higher packages use xl. While is is possible to use xm with some |
4.2 and up packages use xl. To use xm with 4.2, edit xendomains to |
4.x versions (TODO: 4.1 and 4.2?), the pkgsrc-provided rc.d scripts do |
use xm instead. |
not support this as of 2014-12-26, and thus the HOWTO does not support |
|
it either. (Make sure your packages are reasonably recent.) |
|
|
|
For "xm" (3.1 and 3.3), you should enable xend and xenbackendd (but |
For 3.1 and 3.3, you should enable xend and xenbackendd: |
note that you should be using 4.x): |
|
|
|
xend=YES |
xend=YES |
xenbackendd=YES |
xenbackendd=YES |
|
|
For "xl" (4.x), you should enabled xend and xencommons (xenstored). |
For 4.1 and up, you should enable xencommons. Not enabling xencommons |
Trying to boot 4.x without xencommons=YES will result in a hang; it is |
will result in a hang; it is necessary to hit ^C on the console to let |
necessary to hit ^C on the console to let the machine finish booting. |
the machine finish booting. If you are using xm (default in 4.1, or |
TODO: explain why xend is installed by the package. |
if you changed xendomains in 4.2), you should also enable xend: |
|
|
|
xend=YES # only if using xm, and only installed <= 4.2 |
xencommons=YES |
xencommons=YES |
|
|
The installation of NetBSD should already have created devices for xen |
|
(xencons, xenevt), but if they are not present, create them: |
|
|
|
cd /dev && sh MAKEDEV xen |
|
|
|
TODO: Recommend for/against xen-watchdog. |
TODO: Recommend for/against xen-watchdog. |
|
|
After you have configured the daemons and either started them (in the |
After you have configured the daemons and either started them (in the |
Line 447 make this work, one should not expect to
|
Line 448 make this work, one should not expect to
|
(and thus xencommons). There is currently no reason to expect that |
(and thus xencommons). There is currently no reason to expect that |
this will get fixed any time soon. |
this will get fixed any time soon. |
|
|
|
### No-longer needed advice about devices |
|
|
|
The installation of NetBSD should already have created devices for xen |
|
(xencons, xenevt, xsd_kva), but if they are not present, create them: |
|
|
|
cd /dev && sh MAKEDEV xen |
|
|
anita (for testing NetBSD) |
anita (for testing NetBSD) |
-------------------------- |
-------------------------- |
|
|
Line 506 section.
|
Line 514 section.
|
# Install secondary boot loader |
# Install secondary boot loader |
cp -p /usr/mdec/boot / |
cp -p /usr/mdec/boot / |
# Create boot.cfg following earlier guidance: |
# Create boot.cfg following earlier guidance: |
menu=Xen:load /netbsd-XEN3PAE_DOM0.gz console=pc;multiboot /xen.gz dom0_mem=256M |
menu=Xen:load /netbsd-XEN3PAE_DOM0.gz console=pc;multiboot /xen.gz dom0_mem=512M |
menu=Xen.ok:load /netbsd-XEN3PAE_DOM0.ok.gz console=pc;multiboot /xen.ok.gz dom0_mem=256M |
menu=Xen.ok:load /netbsd-XEN3PAE_DOM0.ok.gz console=pc;multiboot /xen.ok.gz dom0_mem=512M |
menu=GENERIC:boot |
menu=GENERIC:boot |
menu=GENERIC single-user:boot -s |
menu=GENERIC single-user:boot -s |
menu=GENERIC.ok:boot netbsd.ok |
menu=GENERIC.ok:boot netbsd.ok |
Line 549 Hardware known to work
|
Line 557 Hardware known to work
|
Arguably, this section is misplaced, and there should be a page of |
Arguably, this section is misplaced, and there should be a page of |
hardware that runs NetBSD/amd64 well, with the mostly-well-founded |
hardware that runs NetBSD/amd64 well, with the mostly-well-founded |
assumption that NetBSD/xen runs fine on any modern hardware that |
assumption that NetBSD/xen runs fine on any modern hardware that |
NetBSD/amd64 runs well on. Until then, we give motherboard/CPU/RAM |
NetBSD/amd64 runs well on. Until then, we give motherboard/CPU (and |
triples to aid those choosing a motherboard. Note that Xen systems |
sometimes RAM) pairs/triples to aid those choosing a motherboard. |
usually do not run X, so a listing here does not imply that X works at |
Note that Xen systems usually do not run X, so a listing here does not |
all. |
imply that X works at all. |
|
|
Supermicro X9SRL-F, Xeon E5-1650 v2, 96 GiB ECC |
Supermicro X9SRL-F, Xeon E5-1650 v2, 96 GiB ECC |
Supermicro ??, Atom C2758 (8 core), 32 GiB ECC |
Supermicro ??, Atom C2758 (8 core), 32 GiB ECC |
Line 561 all.
|
Line 569 all.
|
Older hardware: |
Older hardware: |
|
|
Intel D915GEV, Pentium4 CPU 3.40GHz, 4GB 533MHz Synchronous DDR2 |
Intel D915GEV, Pentium4 CPU 3.40GHz, 4GB 533MHz Synchronous DDR2 |
|
INTEL DG33FB, "Intel(R) Core(TM)2 Duo CPU E6850 @ 3.00GHz" |
|
INTEL DG33FB, "Intel(R) Core(TM)2 Duo CPU E8400 @ 3.00GHz" |
|
|
Running Xen under qemu |
Running Xen under qemu |
---------------------- |
---------------------- |
Line 578 In 2015-01, the following combination wa
|
Line 588 In 2015-01, the following combination wa
|
dom0 kernel: NetBSD/amd64 6.1.5 |
dom0 kernel: NetBSD/amd64 6.1.5 |
Xen tools: xentools42-4.2.5 from pkgsrc |
Xen tools: xentools42-4.2.5 from pkgsrc |
|
|
See [PR 47720](http://gnats.netbsd.org/47720) for a problem with dom0 |
See [PR 47720](https://gnats.netbsd.org/47720) for a problem with dom0 |
shutdown. |
shutdown. |
|
|
Unprivileged domains (domU) |
Unprivileged domains (domU) |
Line 694 for the first virtual disk for the domU
|
Line 704 for the first virtual disk for the domU
|
the file serves two purposes. One is that preallocating the contents |
the file serves two purposes. One is that preallocating the contents |
improves performance. The other is that vnd on sparse files has |
improves performance. The other is that vnd on sparse files has |
failed to work. TODO: give working/notworking NetBSD versions for |
failed to work. TODO: give working/notworking NetBSD versions for |
sparse vnd. Note that the use of file/vnd for Xen is not really |
sparse vnd and gnats reference. Note that the use of file/vnd for Xen |
different than creating a file-backed virtual disk for some other |
is not really different than creating a file-backed virtual disk for |
purpose, except that xentools handles the vnconfig commands. To |
some other purpose, except that xentools handles the vnconfig |
create an empty 4G virtual disk, simply do |
commands. To create an empty 4G virtual disk, simply do |
|
|
dd if=/dev/zero of=foo-xbd0 bs=1m count=4096 |
dd if=/dev/zero of=foo-xbd0 bs=1m count=4096 |
|
|
Line 731 guest, one can create /dev/hda1 in /dev,
|
Line 741 guest, one can create /dev/hda1 in /dev,
|
The third element is "w" for writable disks, and "r" for read-only |
The third element is "w" for writable disks, and "r" for read-only |
disks. |
disks. |
|
|
|
Note that NetBSD by default creates only vnd[0123]. If you need more |
|
than 4 total virtual disks at a time, run e.g. "./MAKEDEV vnd4" in the |
|
dom0. |
|
|
|
Note that NetBSD by default creates only xbd[0123]. If you need more |
|
virtual disks in a domU, run e.g. "./MAKEDEV xbd4" in the domU. |
|
|
Virtual Networking |
Virtual Networking |
------------------ |
------------------ |
|
|
Line 767 Sizing domains
|
Line 784 Sizing domains
|
|
|
Modern x86 hardware has vast amounts of resources. However, many |
Modern x86 hardware has vast amounts of resources. However, many |
virtual servers can function just fine on far less. A system with |
virtual servers can function just fine on far less. A system with |
256M of RAM and a 4G disk can be a reasonable choice. Note that it is |
512M of RAM and a 4G disk can be a reasonable choice. Note that it is |
far easier to adjust virtual resources than physical ones. For |
far easier to adjust virtual resources than physical ones. For |
memory, it's just a config file edit and a reboot. For disk, one can |
memory, it's just a config file edit and a reboot. For disk, one can |
create a new file and vnconfig it (or lvm), and then dump/restore, |
create a new file and vnconfig it (or lvm), and then dump/restore, |
Line 1055 to update the special boot partition.
|
Line 1072 to update the special boot partition.
|
Amazon |
Amazon |
------ |
------ |
|
|
See the [Amazon EC2 page](../amazon_ec2/). |
See the [Amazon EC2 page](/amazon_ec2/). |
|
|
Using npf |
|
--------- |
|
|
|
In standard kernels, npf is a module, and thus cannot be loaded in a |
|
DOMU kernel. |
|
|
|
TODO: Explain how to compile npf into a custom kernel, answering (but |
|
note that the problem was caused by not booting the right kernel) |
|
[this email to |
|
netbsd-users](http://mail-index.netbsd.org/netbsd-users/2014/12/26/msg015576.html). |
|
|
|
TODO items for improving NetBSD/xen |
TODO items for improving NetBSD/xen |
=================================== |
=================================== |
Line 1081 TODO items for improving NetBSD/xen
|
Line 1087 TODO items for improving NetBSD/xen
|
fragsize/blocksize (UFS2 support may be present; the point is to |
fragsize/blocksize (UFS2 support may be present; the point is to |
make it so that with any UFS1/UFS2 file system setup that works |
make it so that with any UFS1/UFS2 file system setup that works |
with NetBSD grub will also work). |
with NetBSD grub will also work). |
See [pkg/40258](http://gnats.netbsd.org/40258). |
See [pkg/40258](https://gnats.netbsd.org/40258). |
* Push patches upstream. |
* Push patches upstream. |
* Get UFS2 patches into pvgrub. |
* Get UFS2 patches into pvgrub. |
* Add support for PV ops to a version of /boot, and make it usable as |
* Add support for PV ops to a version of /boot, and make it usable as |
a kernel in Xen, similar to pvgrub. |
a kernel in Xen, similar to pvgrub. |
* Solve somehow the issue with modules for GENERIC not being loadable |
|
in a Xen dom0 or domU kernel. |
|
|
|
Random pointers |
Random pointers |
=============== |
=============== |
|
|
TODO: This section contains links from elsewhere not yet integrated |
This section contains links from elsewhere not yet integrated into the |
into the HOWTO. |
HOWTO, and other guides. |
|
|
* http://www.lumbercartel.ca/library/xen/ |
* http://www.lumbercartel.ca/library/xen/ |
* http://pbraun.nethence.com/doc/sysutils/xen_netbsd_dom0.html |
* http://pbraun.nethence.com/doc/sysutils/xen_netbsd_dom0.html |
|
* https://gmplib.org/~tege/xen.html |