wikisrc/ports/xen/howto.mdwn - diff

Return to howto.mdwn CVS log

Up to [NetBSD Developer Wiki] / wikisrc / ports / xen

Diff for /wikisrc/ports/xen/howto.mdwn between versions 1.112 and 1.147

version 1.112, 2016/12/20 12:53:06	version 1.147, 2018/07/26 10:47:25
Line 1	Line 1
	[[!meta title="Xen HowTo"]]

Introduction	Introduction
============	============

[![[Xen	[![[Xen
screenshot]](http://www.netbsd.org/gallery/in-Action/hubertf-xens.png)](http://www.netbsd.org/gallery/in-Action/hubertf-xen.png)	screenshot]](https://www.netbsd.org/gallery/in-Action/hubertf-xens.png)](https://www.netbsd.org/gallery/in-Action/hubertf-xen.png)

Xen is a hypervisor (or virtual machine monitor) for x86 hardware	Xen is a hypervisor (or virtual machine monitor) for x86 hardware
(i686-class or higher), which supports running multiple guest	(i686-class or higher), which supports running multiple guest
Line 27 code for Xen and need not be aware that	Line 29 code for Xen and need not be aware that
Attempts to access hardware registers are trapped and emulated. This	Attempts to access hardware registers are trapped and emulated. This
style is less efficient but can run unmodified guests.	style is less efficient but can run unmodified guests.

Generally any amd64 machine will work with Xen and PV guests. In	Generally any machine that runs NetBSD/amd64 will work with Xen and PV
theory i386 computers without amd64 support can be used for Xen <=	guests. In theory i386 computers (without x86_64/amd64 support) can
4.2, but we have no recent reports of this working (this is a hint).	be used for Xen <= 4.2, but we have no recent reports of this working
For HVM guests, the VT or VMX CPU feature (Intel) or SVM/HVM/VT	(this is a hint). For HVM guests, hardware support is needed, but it
(amd64) is needed; "cpuctl identify 0" will show this. TODO: Clean up	is common on recent machines. For Intel CPUs, one needs the VT-x
and check the above features.	extension, shown in "cpuctl identify 0" as VMX. For AMD CPUs, one
	needs the AMD-V extensions, shown in "cpuctl identify 0" as SVM.
	There are further features for IOMMU virtualization, Intel's VT-d and
	AMD's AMD-Vi. TODO: Explain whether Xen on NetBSD makes use of these
	features. TODO: Review by someone who really understands this.

	Note that a FreeBSD dom0 requires VT-x and VT-d (or equivalent); this
	is because the FreeBSD dom0 does not run in PV mode.

At boot, the dom0 kernel is loaded as a module with Xen as the kernel.	At boot, the dom0 kernel is loaded as a module with Xen as the kernel.
The dom0 can start one or more domUs. (Booting is explained in detail	The dom0 can start one or more domUs. (Booting is explained in detail
Line 44 attempts to address both the case of run	Line 53 attempts to address both the case of run
and running domUs under it (NetBSD and other), and also running NetBSD	and running domUs under it (NetBSD and other), and also running NetBSD
as a domU in a VPS.	as a domU in a VPS.

Some versions of Xen support "PCI passthrough", which means that	Xen 3.1 in pkgsrc used to support "PCI passthrough", which means that
specific PCI devices can be made available to a specific domU instead	specific PCI devices can be made available to a specific domU instead
of the dom0. This can be useful to let a domU run X11, or access some	of the dom0. This can be useful to let a domU run X11, or access some
network interface or other peripheral.	network interface or other peripheral.
Line 62 things must be done, guiding the reader	Line 71 things must be done, guiding the reader
path when there are no known good reasons to stray.	path when there are no known good reasons to stray.

This HOWTO presumes a basic familiarity with the Xen system	This HOWTO presumes a basic familiarity with the Xen system
architecture. This HOWTO presumes familiarity with installing NetBSD	architecture, with installing NetBSD on i386/amd64 hardware, and with
on i386/amd64 hardware and installing software from pkgsrc.	installing software from pkgsrc. See also the [Xen
See also the [Xen website](http://www.xenproject.org/).	website](http://www.xenproject.org/).

Versions of Xen and NetBSD	Versions of Xen and NetBSD
==========================	==========================
Line 74 of Xen version and NetBSD version. This	Line 83 of Xen version and NetBSD version. This
which version to choose. Versions not in pkgsrc and older unsupported	which version to choose. Versions not in pkgsrc and older unsupported
versions of NetBSD are intentionally ignored.	versions of NetBSD are intentionally ignored.

Xen	The term "amd64" is used to refer to both the NetBSD port and to the
---	hardware architecture on which it runs. (Such hardware is made by
	both Intel and AMD, and in 2016 a normal PC has this CPU
	architecture.)

	Xen versions
	------------

In NetBSD, Xen is provided in pkgsrc, via matching pairs of packages	In NetBSD, Xen is provided in pkgsrc, via matching pairs of packages
xenkernel and xentools. We will refer only to the kernel versions,	xenkernel and xentools. We will refer only to the kernel versions,
but note that both packages must be installed together and must have	but note that both packages must be installed together and must have
matching versions.	matching versions.

xenkernel3 provides Xen 3.1. It is no longer maintained by Xen, and the last applied security patch was in	Versions available in pkgsrc:
2011. Thus, it should not be used. It supports PCI passthrough,
which is why people use it anyway. Xen 3.1 supports i386, both PAE and
non-PAE.

xenkernel33 provides Xen 3.3. It is no longer maintained by Xen, and
the last applied security patch was in 2012. Thus, it should not be
used. Xen 3.3 supports i386, but only in PAE mode. There are no good
reasons to run this version.

xenkernel41 provides Xen 4.1. It is no longer maintained by Xen, but
as of 2016-12 received backported security patches. Xen 4.1 supports
i386, but only in PAE mode. There are no good reasons to run this
version.

xenkernel42 provides Xen 4.2. It is no longer maintained by Xen, but
as of 2016-12 received backported security patches. Xen 4.2 supports
i386, but only in PAE mode. The only reason to run this is if you
need to use xm instead of xl, or if you need to run an i386 dom0
(because your hardware is i386 only).

xenkernel45 provides Xen 4.5. It is no longer maintained by Xen, but
as of 2016-12 it received security patches. Xen 4.5 requires an amd64
dom0, but domUs can be amd64 or i386 PAE. TODO: It is either a
conservative choice or somewhat old.

xenkernel45 provides Xen 4.6. It is new to pkgsrc in 2016-05. It is
no longer maintained by Xen, but as of 2016-12 it received security
patches. Xen 4.6 requires an amd64 dom0, but domUs can be amd64 or
i386 PAE. TODO: It is either a somewhat aggressive choice or the
standard choice

See also the [Xen Security Advisory page](http://xenbits.xen.org/xsa/).	[[!table data="""
	Xen Version \|Package Name \|Xen CPU Support \|EOL'ed By Upstream
	4.2 \|xenkernel42 \|32bit, 64bit \|Yes
	4.5 \|xenkernel45 \|64bit \|Yes
	4.6 \|xenkernel46 \|64bit \|Partially
	4.8 \|xenkernel48 \|64bit \|No
	4.11 \|xenkernel411 \|64bit \|No
	"""]]

Ideally newer versions of Xen will be added to pkgsrc.	See also the [Xen Security Advisory page](http://xenbits.xen.org/xsa/).

Note that NetBSD support is called XEN3. It works with Xen 3 and Xen	Note: Xen 4.2 was the last version to support 32bit CPUs.
4 because the hypercall interface has been stable.

Xen command program	Xen command program
-------------------	-------------------
Line 127 Xen command program	Line 117 Xen command program
Early Xen used a program called xm to manipulate the system from the	Early Xen used a program called xm to manipulate the system from the
dom0. Starting in 4.1, a replacement program with similar behavior	dom0. Starting in 4.1, a replacement program with similar behavior
called xl is provided, but it does not work well in 4.1. In 4.2, both	called xl is provided, but it does not work well in 4.1. In 4.2, both
xm and xl work fine. 4.4 is the last version that has xm. You must	xm and xl work fine. 4.4 is the last version that has xm.
choose one or the other, because it affects which daemons you run.
However, the rc.d scripts provided by xentools packages expect a
particular version, and you should use the version used by the
scripts.

NetBSD	You must make a global choice to use xm or xl, because it affects not
------	only which command you use, but the command used by rc.d scripts
	(specifically xendomains) and which daemons should be run. The
	xentools packages provide xl for 4.2 and up.

	In 4.2, you can choose to use xm by simply changing the ctl_command
	variable and setting xend=YES in rc.conf.

	With xl, virtual devices are configured in parallel, which can cause
	problems if they are written assuming serial operation (e.g., updating
	firewall rules without explicit locking). There is now locking for
	the provided scripts, which works for normal casses (e.g, file-backed
	xbd, where a vnd must be allocated). But, as of 201612, it has not
	been adequately tested for a complex custom setup with a large number
	of interfaces.

The netbsd-6, netbsd-7, and -current branches are all reasonable	NetBSD versions
	---------------

	The netbsd-7, netbsd-8, and -current branches are all reasonable
choices, with more or less the same considerations for non-Xen use.	choices, with more or less the same considerations for non-Xen use.
Therefore, netbsd-7 is recommended as the stable version of the most	NetBSD 8 is recommended as the stable version of the most recent
recent release for production use. For those wanting to learn Xen or	release for production use.
without production stability concerns, netbsd-7 is still likely most
appropriate, but -current is also a reasonable choice. Xen runs fine	For developing Xen, netbsd-current may be appropriate.
on netbsd-5, but the xentools packages are likely difficult to build.

As of NetBSD 6, a NetBSD domU will support multiple vcpus. There is	As of NetBSD 6, a NetBSD domU will support multiple vcpus. There is
no SMP support for NetBSD as dom0. (The dom0 itself doesn't really	no SMP support for NetBSD as dom0. (The dom0 itself doesn't really
need SMP for dom0 functions; the lack of support is really a problem	need SMP for dom0 functions; the lack of support is really a problem
when using a dom0 as a normal computer.)	when using a dom0 as a normal computer.)

	Note: NetBSD support is called XEN3. However, it does support Xen 4,
	because the hypercall interface has remained identical.

Architecture	Architecture
------------	------------

Xen itself can run on i386 (Xen < 3.1) or amd64 machines (all Xen	Xen itself can run on i386 (Xen < 4.2) or amd64 hardware (all Xen
versions). (Practically, almost any computer where one would want to	versions). (Practically, almost any computer where one would want to
run Xen today supports amd64.)	run Xen today supports amd64.)

Xen, the dom0 kernel, and each domU kernel can be either i386 or	Xen, the dom0 system, and each domU system can be either i386 or
amd64. When building a xenkernel package, one obtains i386 on an i386	amd64. When building a xenkernel package, one obtains an i386 Xen
host, and amd64 on an amd64 host. If the Xen kernel is i386, then the	kernel on an i386 host, and an amd64 Xen kernel on an amd64 host. If
dom0 kernel and all domU kernels must be i386. With an amd64 Xen	the Xen kernel is i386, then the dom0 kernel and all domU kernels must
kernel, an amd64 dom0 kernel is known to work, and an i386PAE dom0	be i386. With an amd64 Xen kernel, an amd64 dom0 kernel is known to
kernel should in theory work. An amd64 Xen/dom0 is known to support	work, and an i386 dom0 kernel should in theory work. An amd64
both i386PAE and amd64 domUs.	Xen/dom0 is known to support both i386 and amd64 domUs.

i386 dom0 and domU kernels must be PAE (except for Xen 3.1); these are	i386 dom0 and domU kernels must be PAE (except for an i386 Xen 3.1
built by default. (Note that emacs (at least) fails if run on i386	kernel, where one can use non-PAE for dom0 and all domUs); PAE kernels
with PAE when built without, and vice versa, presumably due to bugs in	are included in the NetBSD default build. (Note that emacs (at least)
the undump code.)	fails if run on i386 with PAE when built without, and vice versa,
	presumably due to bugs in the undump code.)
Because of the above, the standard approach is to use amd64 for the
dom0.	Because of the above, the standard approach is to use an amd64 Xen
	kernel and NetBSD/amd64 for the dom0. For domUs, NetBSD/i386 (with
Xen 4.2 is the last version to support i386 as a host. TODO: Clarify	the PAE kernel) and NetBSD/amd64 are in widespread use, and there is
if this is about the CPU, the Xen kernel, or the dom0 kernel having to	little to no Xen-specific reason to prefer one over the other.
be amd64.
	Note that to use an i386 dom0 with Xen 4.5 or higher, one must build
	(or obtain from pre-built packages) an amd64 Xen kernel and install
	that on the system. (One must also use a PAE i386 kernel, but this is
	also required with an i386 Xen kernel.). Almost no one in the
	NetBSD/Xen community does this, and the standard, well-tested,
	approach is to use an amd64 dom0.

	A [posting on
	xen-devel](https://lists.xen.org/archives/html/xen-devel/2012-07/msg00085.html)
	explained that PV system call overhead was higher on amd64, and thus
	there is some notion that i386 guests are faster. It goes on to
	caution that the total situation is complex and not entirely
	understood. On top of that caution, the post is about Linux, not
	NetBSD. TODO: Include link to benchmarks, if someone posts them.

Stability	Stability
---------	---------

Mostly, NetBSD as a dom0 or domU is quite stable.	Mostly, NetBSD as a dom0 or domU is quite stable. However, just like every
However, there are some open PRs indicating problems.	other architecture, there are some open PRs indicating problems.

- [PR 48125](http://gnats.netbsd.org/48125)
- [PR 47720](http://gnats.netbsd.org/47720)

Note also that there are issues with sparse vnd(4) instances, but	Note also that there are issues with sparse vnd(4) instances, but
these are not about Xen -- they just are noticed with sparse vnd(4)	these are not about Xen -- they just are noticed with sparse vnd(4)
Line 193 instances in support of virtual disks in	Line 207 instances in support of virtual disks in
Recommendation	Recommendation
--------------	--------------

Therefore, this HOWTO recommends running xenkernel45 or xenkernel46,	Therefore, this HOWTO recommends running xenkernel46, xl, the NetBSD 7
xl, the NetBSD 7 stable branch, and to use an amd64 kernel as the	stable branch, and therefore to use an amd64 kernel as the dom0.
dom0. Either the i386PAE or amd64 version of NetBSD may be used as	Either the i386PAE or amd64 version of NetBSD may be used as domUs.
domUs.
	A tentative replacement recommendation is xenkernel48, xl, and NetBSD
Build problems	8.
--------------
	Because bugs are fixed quite often, and because of Xen security
Ideally, all versions of Xen in pkgsrc would build on all versions of	advisories, it is good to stay up to date with NetBSD (tracking a
NetBSD on both i386 and amd64. However, that isn't the case. Besides	stable branch), with the Xen kernel (tracking a Xen version via
aging code and aging compilers, qemu (included in xentools for HVM	pkgsrc), and with the Xen tools. Specifically, NetBSD (-7 and
support) is difficult to build. The following are known to work or FAIL:	-current) got an important fix affecting dom0/domU timesharing in
	November, 2015, and xentools46 got a fix to enable Ubuntu guests to
xenkernel3 netbsd-5 amd64	boot in December, 2016.
xentools3 netbsd-5 amd64
xentools3=hvm netbsd-5 amd64 ????
xenkernel33 netbsd-5 amd64
xentools33 netbsd-5 amd64
xenkernel41 netbsd-5 amd64
xentools41 netbsd-5 amd64
xenkernel42 netbsd-5 amd64
xentools42 netbsd-5 amd64

xenkernel3 netbsd-6 i386 FAIL
xentools3 netbsd-6 i386
xentools3-hvm netbsd-6 i386 FAIL (dependencies fail)
xenkernel33 netbsd-6 i386
xentools33 netbsd-6 i386
xenkernel41 netbsd-6 i386
xentools41 netbsd-6 i386
xenkernel42 netbsd-6 i386
xentools42 netbsd-6 i386 *MIXED

(all 3 and 33 seem to FAIL)
xenkernel41 netbsd-7 i386
xentools41 netbsd-7 i386
xenkernel42 netbsd-7 i386
xentools42 netbsd-7 i386 ??FAIL

(*On netbsd-6 i386, there is a xentools42 in the 2014Q3 official builds,
but it does not build for gdt.)

NetBSD as a dom0	NetBSD as a dom0
================	================
Line 248 configuration.	Line 235 configuration.

For experimenting with Xen, a machine with as little as 1G of RAM and	For experimenting with Xen, a machine with as little as 1G of RAM and
100G of disk can work. For running many domUs in productions, far	100G of disk can work. For running many domUs in productions, far
more will be needed.	more will be needed; e.g. 4-8G and 1T of disk is reasonable for a
	half-dozen domUs of 512M and 32G each. Basically, the RAM and disk
	have to be bigger than the sum of the RAM/disk needs of the dom0 and
	all the domUs.

	In 2018-05, trouble booting a dom0 was reported with 256M of RAM: with
	512M it worked reliably. This does not make sense, but if you see
	"not ELF" after Xen boots, try increasing dom0 RAM.

Styles of dom0 operation	Styles of dom0 operation
------------------------	------------------------
Line 271 Xen daemons when not running Xen.	Line 265 Xen daemons when not running Xen.
Note that NetBSD as dom0 does not support multiple CPUs. This will	Note that NetBSD as dom0 does not support multiple CPUs. This will
limit the performance of the Xen/dom0 workstation approach. In theory	limit the performance of the Xen/dom0 workstation approach. In theory
the only issue is that the "backend drivers" are not yet MPSAFE:	the only issue is that the "backend drivers" are not yet MPSAFE:
http://mail-index.netbsd.org/netbsd-users/2014/08/29/msg015195.html	https://mail-index.netbsd.org/netbsd-users/2014/08/29/msg015195.html

Installation of NetBSD	Installation of NetBSD
----------------------	----------------------
Line 313 Installation of Xen	Line 307 Installation of Xen
-------------------	-------------------

In the dom0, install sysutils/xenkernel42 and sysutils/xentools42 from	In the dom0, install sysutils/xenkernel42 and sysutils/xentools42 from
pkgsrc (or another matching pair).	pkgsrc (or another matching pair). See [the pkgsrc
See [the pkgsrc	documentation](https://www.NetBSD.org/docs/pkgsrc/) for help with
documentation](http://www.NetBSD.org/docs/pkgsrc/) for help with pkgsrc.	pkgsrc. Ensure that your packages are recent; the HOWTO does not
	contemplate old builds.


For Xen 3.1, support for HVM guests is in sysutils/xentool3-hvm. More	For Xen 3.1, support for HVM guests is in sysutils/xentool3-hvm. More
recent versions have HVM support integrated in the main xentools	recent versions have HVM support integrated in the main xentools
Line 333 releasedir/i386/binary/kernel/netbsd-XEN	Line 329 releasedir/i386/binary/kernel/netbsd-XEN
should not use Xen 3.1.) Both xen and the NetBSD kernel may be (and	should not use Xen 3.1.) Both xen and the NetBSD kernel may be (and
typically are) left compressed.	typically are) left compressed.

In a dom0 kernel, kernfs is mandatory for xend to communicate with the	In a dom0, kernfs is mandatory for xend to communicate with the
kernel, so ensure that /kern is in fstab. TODO: Say this is default,	kernel, so ensure that /kern is in fstab. (A standard NetBSD install
or file a PR and give a reference.	should already mount /kern.)

Because you already installed NetBSD, you have a working boot setup	Because you already installed NetBSD, you have a working boot setup
with an MBR bootblock, either bootxx_ffsv1 or bootxx_ffsv2 at the	with an MBR bootblock, either bootxx_ffsv1 or bootxx_ffsv2 at the
beginning of your root file system, /boot present, and likely	beginning of your root file system, have /boot, and likely also
/boot.cfg. (If not, fix before continuing!)	/boot.cfg. (If not, fix before continuing!)

Add a line to to /boot.cfg to boot Xen. See boot.cfg(5) for an	Add a line to to /boot.cfg to boot Xen. See boot.cfg(5) for an
example. The basic line is	example. The basic line is

menu=Xen:load /netbsd-XEN3_DOM0.gz console=pc;multiboot /xen.gz dom0_mem=256M	menu=Xen:load /netbsd-XEN3_DOM0.gz console=pc;multiboot /xen.gz dom0_mem=512M

which specifies that the dom0 should have 256M, leaving the rest to be	which specifies that the dom0 should have 512M, leaving the rest to be
allocated for domUs. To use a serial console, use	allocated for domUs. To use a serial console, use

menu=Xen:load /netbsd-XEN3_DOM0.gz console=com0;multiboot /xen.gz dom0_mem=256M console=com1 com1=9600,8n1	menu=Xen:load /netbsd-XEN3_DOM0.gz;multiboot /xen.gz dom0_mem=512M console=com1 com1=9600,8n1

which will use the first serial port for Xen (which counts starting	which will use the first serial port for Xen (which counts starting
from 1), forcing speed/parity, and also for NetBSD (which counts	from 1, unlike NetBSD which counts starting from 0), forcing
starting at 0). In an attempt to add performance, one can also add	speed/parity. Because the NetBSD command line lacks a
	"console=pc" argument, it will use the default "xencons" console device,
	which directs the console I/O through Xen to the same console device Xen
	itself uses (in this case, the serial port).

	In an attempt to add performance, one can also add

dom0_max_vcpus=1 dom0_vcpus_pin	dom0_max_vcpus=1 dom0_vcpus_pin

Line 367 and other than dom0 memory and max_vcpus	Line 368 and other than dom0 memory and max_vcpus
necessary.	necessary.

As with non-Xen systems, you should have a line to boot /netbsd (a	As with non-Xen systems, you should have a line to boot /netbsd (a
kernel that works without Xen) and fallback versions of the non-Xen	kernel that works without Xen). Consider a line to boot /netbsd.ok (a
kernel, Xen, and the dom0 kernel.	fallback version of the non-Xen kernel, updated manually when you are
	sure /netbsd is ok). Consider also a line to boot fallback versions
	of Xen and the dom0 kernel, but note that non-Xen NetBSD can be used
	to resolve Xen booting issues.

	Probably you want a default=N line to choose Xen in the absence of
	intervention.

Now, reboot so that you are running a DOM0 kernel under Xen, rather	Now, reboot so that you are running a DOM0 kernel under Xen, rather
than GENERIC without Xen.	than GENERIC without Xen.
Line 378 Using grub (historic)	Line 385 Using grub (historic)

Before NetBSD's native bootloader could support Xen, the use of	Before NetBSD's native bootloader could support Xen, the use of
grub was recommended. If necessary, see the	grub was recommended. If necessary, see the
[old grub information](/ports/xen/howto-grub/).	[old grub information](/ports/xen/howto-grub).

The [HowTo on Installing into	The [HowTo on Installing into
RAID-1](http://mail-index.NetBSD.org/port-xen/2006/03/01/0010.html)	RAID-1](https://mail-index.NetBSD.org/port-xen/2006/03/01/0010.html)
explains how to set up booting a dom0 with Xen using grub with	explains how to set up booting a dom0 with Xen using grub with
NetBSD's RAIDframe. (This is obsolete with the use of NetBSD's native	NetBSD's RAIDframe. (This is obsolete with the use of NetBSD's native
boot.)	boot. Now, just create a system with RAID-1, and alter /boot.cfg as
	described above.)

Configuring Xen	Configuring Xen
---------------	---------------
Line 397 There will be no domUs, and none can be	Line 405 There will be no domUs, and none can be
to configure the dom0 daemons.	to configure the dom0 daemons.

The daemons which should be run vary with Xen version and with whether	The daemons which should be run vary with Xen version and with whether
one is using xm or xl. The Xen 3.1 and 3.3 packages use xm. Xen 4.1	one is using xm or xl. The Xen 3.1, 3.3 and 4.1 packages use xm. Xen
and higher packages use xl. While is is possible to use xm with some	4.2 and up packages use xl. To use xm with 4.2, edit xendomains to
4.x versions (TODO: 4.1 and 4.2?), the pkgsrc-provided rc.d scripts do	use xm instead.
not support this as of 2014-12-26, and thus the HOWTO does not support
it either. (Make sure your packages are reasonably recent.)

For "xm" (3.1 and 3.3), you should enable xend and xenbackendd (but	For 3.1 and 3.3, you should enable xend and xenbackendd:
note that you should be using 4.x):

xend=YES	xend=YES
xenbackendd=YES	xenbackendd=YES

For "xl" (4.x), you should enabled xend and xencommons (xenstored).	For 4.1 and up, you should enable xencommons. Not enabling xencommons
Trying to boot 4.x without xencommons=YES will result in a hang; it is	will result in a hang; it is necessary to hit ^C on the console to let
necessary to hit ^C on the console to let the machine finish booting.	the machine finish booting. If you are using xm (default in 4.1, or
TODO: explain why xend is installed by the package.	if you changed xendomains in 4.2), you should also enable xend:

	xend=YES # only if using xm, and only installed <= 4.2
xencommons=YES	xencommons=YES

The installation of NetBSD should already have created devices for xen
(xencons, xenevt), but if they are not present, create them:

cd /dev && sh MAKEDEV xen

TODO: Recommend for/against xen-watchdog.	TODO: Recommend for/against xen-watchdog.

After you have configured the daemons and either started them (in the	After you have configured the daemons and either started them (in the
Line 447 make this work, one should not expect to	Line 448 make this work, one should not expect to
(and thus xencommons). There is currently no reason to expect that	(and thus xencommons). There is currently no reason to expect that
this will get fixed any time soon.	this will get fixed any time soon.

	### No-longer needed advice about devices

	The installation of NetBSD should already have created devices for xen
	(xencons, xenevt, xsd_kva), but if they are not present, create them:

	cd /dev && sh MAKEDEV xen

anita (for testing NetBSD)	anita (for testing NetBSD)
--------------------------	--------------------------

Line 506 section.	Line 514 section.
# Install secondary boot loader	# Install secondary boot loader
cp -p /usr/mdec/boot /	cp -p /usr/mdec/boot /
# Create boot.cfg following earlier guidance:	# Create boot.cfg following earlier guidance:
menu=Xen:load /netbsd-XEN3PAE_DOM0.gz console=pc;multiboot /xen.gz dom0_mem=256M	menu=Xen:load /netbsd-XEN3PAE_DOM0.gz console=pc;multiboot /xen.gz dom0_mem=512M
menu=Xen.ok:load /netbsd-XEN3PAE_DOM0.ok.gz console=pc;multiboot /xen.ok.gz dom0_mem=256M	menu=Xen.ok:load /netbsd-XEN3PAE_DOM0.ok.gz console=pc;multiboot /xen.ok.gz dom0_mem=512M
menu=GENERIC:boot	menu=GENERIC:boot
menu=GENERIC single-user:boot -s	menu=GENERIC single-user:boot -s
menu=GENERIC.ok:boot netbsd.ok	menu=GENERIC.ok:boot netbsd.ok
Line 549 Hardware known to work	Line 557 Hardware known to work
Arguably, this section is misplaced, and there should be a page of	Arguably, this section is misplaced, and there should be a page of
hardware that runs NetBSD/amd64 well, with the mostly-well-founded	hardware that runs NetBSD/amd64 well, with the mostly-well-founded
assumption that NetBSD/xen runs fine on any modern hardware that	assumption that NetBSD/xen runs fine on any modern hardware that
NetBSD/amd64 runs well on. Until then, we give motherboard/CPU/RAM	NetBSD/amd64 runs well on. Until then, we give motherboard/CPU (and
triples to aid those choosing a motherboard. Note that Xen systems	sometimes RAM) pairs/triples to aid those choosing a motherboard.
usually do not run X, so a listing here does not imply that X works at	Note that Xen systems usually do not run X, so a listing here does not
all.	imply that X works at all.

Supermicro X9SRL-F, Xeon E5-1650 v2, 96 GiB ECC	Supermicro X9SRL-F, Xeon E5-1650 v2, 96 GiB ECC
Supermicro ??, Atom C2758 (8 core), 32 GiB ECC	Supermicro ??, Atom C2758 (8 core), 32 GiB ECC
Line 561 all.	Line 569 all.
Older hardware:	Older hardware:

Intel D915GEV, Pentium4 CPU 3.40GHz, 4GB 533MHz Synchronous DDR2	Intel D915GEV, Pentium4 CPU 3.40GHz, 4GB 533MHz Synchronous DDR2
	INTEL DG33FB, "Intel(R) Core(TM)2 Duo CPU E6850 @ 3.00GHz"
	INTEL DG33FB, "Intel(R) Core(TM)2 Duo CPU E8400 @ 3.00GHz"

Running Xen under qemu	Running Xen under qemu
----------------------	----------------------
Line 578 In 2015-01, the following combination wa	Line 588 In 2015-01, the following combination wa
dom0 kernel: NetBSD/amd64 6.1.5	dom0 kernel: NetBSD/amd64 6.1.5
Xen tools: xentools42-4.2.5 from pkgsrc	Xen tools: xentools42-4.2.5 from pkgsrc

See [PR 47720](http://gnats.netbsd.org/47720) for a problem with dom0	See [PR 47720](https://gnats.netbsd.org/47720) for a problem with dom0
shutdown.	shutdown.

Unprivileged domains (domU)	Unprivileged domains (domU)
Line 694 for the first virtual disk for the domU	Line 704 for the first virtual disk for the domU
the file serves two purposes. One is that preallocating the contents	the file serves two purposes. One is that preallocating the contents
improves performance. The other is that vnd on sparse files has	improves performance. The other is that vnd on sparse files has
failed to work. TODO: give working/notworking NetBSD versions for	failed to work. TODO: give working/notworking NetBSD versions for
sparse vnd. Note that the use of file/vnd for Xen is not really	sparse vnd and gnats reference. Note that the use of file/vnd for Xen
different than creating a file-backed virtual disk for some other	is not really different than creating a file-backed virtual disk for
purpose, except that xentools handles the vnconfig commands. To	some other purpose, except that xentools handles the vnconfig
create an empty 4G virtual disk, simply do	commands. To create an empty 4G virtual disk, simply do

dd if=/dev/zero of=foo-xbd0 bs=1m count=4096	dd if=/dev/zero of=foo-xbd0 bs=1m count=4096

Line 731 guest, one can create /dev/hda1 in /dev,	Line 741 guest, one can create /dev/hda1 in /dev,
The third element is "w" for writable disks, and "r" for read-only	The third element is "w" for writable disks, and "r" for read-only
disks.	disks.

	Note that NetBSD by default creates only vnd[0123]. If you need more
	than 4 total virtual disks at a time, run e.g. "./MAKEDEV vnd4" in the
	dom0.

	Note that NetBSD by default creates only xbd[0123]. If you need more
	virtual disks in a domU, run e.g. "./MAKEDEV xbd4" in the domU.

Virtual Networking	Virtual Networking
------------------	------------------

Line 767 Sizing domains	Line 784 Sizing domains

Modern x86 hardware has vast amounts of resources. However, many	Modern x86 hardware has vast amounts of resources. However, many
virtual servers can function just fine on far less. A system with	virtual servers can function just fine on far less. A system with
256M of RAM and a 4G disk can be a reasonable choice. Note that it is	512M of RAM and a 4G disk can be a reasonable choice. Note that it is
far easier to adjust virtual resources than physical ones. For	far easier to adjust virtual resources than physical ones. For
memory, it's just a config file edit and a reboot. For disk, one can	memory, it's just a config file edit and a reboot. For disk, one can
create a new file and vnconfig it (or lvm), and then dump/restore,	create a new file and vnconfig it (or lvm), and then dump/restore,
Line 1055 to update the special boot partition.	Line 1072 to update the special boot partition.
Amazon	Amazon
------	------

See the [Amazon EC2 page](../amazon_ec2/).	See the [Amazon EC2 page](/amazon_ec2/).

Using npf
---------

In standard kernels, npf is a module, and thus cannot be loaded in a
DOMU kernel.

TODO: Explain how to compile npf into a custom kernel, answering (but
note that the problem was caused by not booting the right kernel)
[this email to
netbsd-users](http://mail-index.netbsd.org/netbsd-users/2014/12/26/msg015576.html).

TODO items for improving NetBSD/xen	TODO items for improving NetBSD/xen
===================================	===================================
Line 1081 TODO items for improving NetBSD/xen	Line 1087 TODO items for improving NetBSD/xen
fragsize/blocksize (UFS2 support may be present; the point is to	fragsize/blocksize (UFS2 support may be present; the point is to
make it so that with any UFS1/UFS2 file system setup that works	make it so that with any UFS1/UFS2 file system setup that works
with NetBSD grub will also work).	with NetBSD grub will also work).
See [pkg/40258](http://gnats.netbsd.org/40258).	See [pkg/40258](https://gnats.netbsd.org/40258).
* Push patches upstream.	* Push patches upstream.
* Get UFS2 patches into pvgrub.	* Get UFS2 patches into pvgrub.
* Add support for PV ops to a version of /boot, and make it usable as	* Add support for PV ops to a version of /boot, and make it usable as
a kernel in Xen, similar to pvgrub.	a kernel in Xen, similar to pvgrub.
* Solve somehow the issue with modules for GENERIC not being loadable
in a Xen dom0 or domU kernel.

Random pointers	Random pointers
===============	===============

TODO: This section contains links from elsewhere not yet integrated	This section contains links from elsewhere not yet integrated into the
into the HOWTO.	HOWTO, and other guides.

* http://www.lumbercartel.ca/library/xen/	* http://www.lumbercartel.ca/library/xen/
* http://pbraun.nethence.com/doc/sysutils/xen_netbsd_dom0.html	* http://pbraun.nethence.com/doc/sysutils/xen_netbsd_dom0.html
	* https://gmplib.org/~tege/xen.html

CVSweb for NetBSD wikisrc <wikimaster@NetBSD.org> software: FreeBSD-CVSweb

Removed from v.1.112
changed lines
	Added in v.1.147