|
version 1.180, 2020/11/15 14:31:58
|
version 1.185, 2020/11/15 14:59:54
|
|
Line 8 systems which operate in an unprivileged
|
Line 8 systems which operate in an unprivileged
|
| from the domU systems are forwarded by the Xen hypervisor to the dom0 to be |
from the domU systems are forwarded by the Xen hypervisor to the dom0 to be |
| fulfilled. |
fulfilled. |
| |
|
| Xen supports different styles of guests; see [PV on HVM](https://wiki.xen.org/wiki/PV_on_HVM) and [PVH(v2)](https://wiki.xenproject.org/wiki/PVH_(v2\)_Domu) for upstream documentation. |
This HOWTO presumes a basic familiarity with the Xen system |
| |
architecture, with installing NetBSD on amd64 hardware, and with |
| |
installing software from pkgsrc. See also the [Xen |
| |
website](http://www.xenproject.org/). |
| |
|
| |
[[!toc]] |
| |
|
| |
# Overview |
| |
|
| |
The basic concept of Xen is that the hypervisor (xenkernel) runs on |
| |
the hardware, and runs a privileged domain ("dom0") that can access |
| |
disks/networking/etc. One then runs additonal unprivileged domains |
| |
(each a "domU"), presumably to do something useful. |
| |
|
| |
This HOWTO addresses how to run a NetBSD dom0 (and hence also build |
| |
xen itself). It also addresses how to run domUs in that environment, |
| |
and how to deal with having a domU in a Xen environment run by someone |
| |
else and/or not running NetBSD. |
| |
|
| |
## Guest Styles |
| |
|
| |
Xen supports different styles of guests. |
| |
|
| [[!table data=""" |
[[!table data=""" |
| Style of guest |Supported by NetBSD |
Style of guest |Supported by NetBSD |
|
Line 21 PVH |current-only (domU, dom0 not yet)
|
Line 42 PVH |current-only (domU, dom0 not yet)
|
| In Para-Virtualized (PV) mode, the guest OS does not attempt to access |
In Para-Virtualized (PV) mode, the guest OS does not attempt to access |
| hardware directly, but instead makes hypercalls to the hypervisor; PV |
hardware directly, but instead makes hypercalls to the hypervisor; PV |
| guests must be specifically coded for Xen. |
guests must be specifically coded for Xen. |
| |
See [PV](https://wiki.xen.org/wiki/Paravirtualization_(PV\)). |
| |
|
| In HVM mode, no guest modification is required; however, hardware |
In HVM mode, no guest modification is required; however, hardware |
| support is required, such as VT-x on Intel CPUs and SVM on AMD CPUs. |
support is required, such as VT-x on Intel CPUs and SVM on AMD CPUs. |
|
Line 28 The dom0 runs qemu to emulate hardware.
|
Line 50 The dom0 runs qemu to emulate hardware.
|
| |
|
| In PVHVM mode, the guest runs as HVM, but additionally can use PV |
In PVHVM mode, the guest runs as HVM, but additionally can use PV |
| drivers for efficiency. |
drivers for efficiency. |
| |
See [PV on HVM](https://wiki.xen.org/wiki/PV_on_HVM). |
| |
|
| There have been two PVH modes: original PVH and PVHv2. Original PVH |
There have been two PVH modes: original PVH and PVHv2. Original PVH |
| was based on PV mode and is no longer relevant at all. PVHv2 is |
was based on PV mode and is no longer relevant at all. PVHv2 is |
| basically lightweight HVM with PV drivers. A critical feature of it |
basically lightweight HVM with PV drivers. A critical feature of it |
| is that qemu is not needed; the hypervisor can do the emulation that |
is that qemu is not needed; the hypervisor can do the emulation that |
| is required. Thus, a dom0 can be PVHv2. |
is required. Thus, a dom0 can be PVHv2. |
| |
|
| The source code uses PVH and config files use pvh; this refers to PVHv2. |
The source code uses PVH and config files use pvh; this refers to PVHv2. |
| |
See [PVH(v2)](https://wiki.xenproject.org/wiki/PVH_(v2\)_Domu). |
| |
|
| At boot, the dom0 kernel is loaded as a module with Xen as the kernel. |
At system boot, the dom0 kernel is loaded as a module with Xen as the kernel. |
| The dom0 can start one or more domUs. (Booting is explained in detail |
The dom0 can start one or more domUs. (Booting is explained in detail |
| in the dom0 section.) |
in the dom0 section.) |
| |
|
| This HOWTO presumes a basic familiarity with the Xen system |
## CPU Architecture |
| architecture, with installing NetBSD on i386/amd64 hardware, and with |
|
| installing software from pkgsrc. See also the [Xen |
|
| website](http://www.xenproject.org/). |
|
| |
|
| [[!toc]] |
Xen runs on x86_64 hardware (the NetBSD amd64 port). |
| |
|
| |
There is a concept of Xen running on ARM, but there are no reports of this working with NetBSD. |
| |
|
| |
The dom0 system should be amd64. (Instructions for i386PAE dom0 have been removed from the HOWTO.) |
| |
|
| # Versions and Support |
The domU can be i386PAE or amd64. |
| |
i386PAE at one point was considered as [faster](https://lists.xen.org/archives/html/xen-devel/2012-07/msg00085.html) than amd64. |
| |
|
| |
## Xen Versions |
| |
|
| In NetBSD, Xen is provided in pkgsrc, via matching pairs of packages |
In NetBSD, Xen is provided in pkgsrc, via matching pairs of packages |
| xenkernel and xentools. We will refer only to the kernel versions, |
xenkernel and xentools. We will refer only to the kernel versions, |
|
Line 58 matching versions.
|
Line 85 matching versions.
|
| Versions available in pkgsrc: |
Versions available in pkgsrc: |
| |
|
| [[!table data=""" |
[[!table data=""" |
| Xen Version |Package Name |Xen CPU Support |xm? |EOL'ed By Upstream |
Xen Version |Package Name |Xen CPU Support |EOL'ed By Upstream |
| 4.11 |xenkernel411 |x86_64 | |No |
4.11 |xenkernel411 |x86_64 |No |
| 4.13 |xenkernel413 |x86_64 | |No |
4.13 |xenkernel413 |x86_64 |No |
| """]] |
"""]] |
| |
|
| See also the [Xen Security Advisory page](http://xenbits.xen.org/xsa/). |
See also the [Xen Security Advisory page](http://xenbits.xen.org/xsa/). |
| |
|
| Multiprocessor (SMP) support in NetBSD differs depending on the domain: |
Older Xen had a python-based management tool called xm, now replaced |
| |
by xl. |
| |
|
| [[!table data=""" |
## NetBSD versions |
| Domain |Supports SMP |
|
| dom0 |No |
|
| domU |Yes |
|
| """]] |
|
| |
|
| Note: NetBSD support is called XEN3. However, it does support Xen 4, |
Xen has been supported in NetBSD for a long time, at least since 2005. |
| because the hypercall interface has remained identical. |
Initially Xen was PV only. |
| |
|
| Older Xen had a python-based management tool called xm, now replaced |
NetBSD 8 and up support PV and HVM modes. |
| by xl. |
|
| |
|
| Architecture |
Support for PVHVM and PVH is available only in NetBSD-current. |
| ------------ |
|
| |
|
| Xen 4.5 and later runs on x86_64 hardware (the NetBSD amd64 port). |
NetBSD as a dom0 does not run SMP, because some drivers are not yet |
| There is a concept of Xen running on ARM, but there are no reports of this working with NetBSD. |
safe for this. \todo Link to more information about what needs work. |
| |
|
| The dom0 system should be amd64. (Instructions for i386PAE dom0 have been removed from the HOWTO.) |
NetBSD, when run as a domU, can and does typically run SMP. |
| |
|
| The domU can be i386PAE or amd64. |
Note: NetBSD support is called XEN3. However, it does support Xen 4, |
| i386PAE at one point was considered as [faster](https://lists.xen.org/archives/html/xen-devel/2012-07/msg00085.html) than amd64. |
because the hypercall interface has remained identical. |
| |
|
| # Creating a dom0 |
# Creating a dom0 |
| |
|
|
Line 100 In 2018-05, trouble booting a dom0 was r
|
Line 122 In 2018-05, trouble booting a dom0 was r
|
| 512M it worked reliably. This does not make sense, but if you see |
512M it worked reliably. This does not make sense, but if you see |
| "not ELF" after Xen boots, try increasing dom0 RAM. |
"not ELF" after Xen boots, try increasing dom0 RAM. |
| |
|
| Installation of NetBSD |
## Installation of NetBSD |
| ---------------------- |
|
| |
|
| [Install NetBSD/amd64](/guide/inst/) |
[Install NetBSD/amd64](/guide/inst/) |
| just as you would if you were not using Xen. |
just as you would if you were not using Xen. |
| |
|
| Installation of Xen |
## Installation of Xen |
| ------------------- |
|
| |
|
| We will consider that you chose to use Xen 4.13, with NetBSD/amd64 as |
We will consider that you chose to use Xen 4.13, with NetBSD/amd64 as |
| dom0. In the dom0, install xenkernel48 and xentools48 from pkgsrc. |
dom0. In the dom0, install xenkernel48 and xentools48 from pkgsrc. |
|
Line 197 make this work, one should not expect to
|
Line 217 make this work, one should not expect to
|
| (and thus xencommons). There is currently no reason to expect that |
(and thus xencommons). There is currently no reason to expect that |
| this will get fixed any time soon. |
this will get fixed any time soon. |
| |
|
| anita (for testing NetBSD) |
## anita (for testing NetBSD) |
| -------------------------- |
|
| |
|
| With the setup so far, one should be able to run |
With the setup so far, one should be able to run |
| anita (see pkgsrc/misc/py-anita) to test NetBSD releases, by doing (as |
anita (see pkgsrc/misc/py-anita) to test NetBSD releases, by doing (as |
|
Line 208 root, because anita must create a domU):
|
Line 227 root, because anita must create a domU):
|
| anita --vmm=xl test file:///usr/obj/i386/ |
anita --vmm=xl test file:///usr/obj/i386/ |
| """]] |
"""]] |
| |
|
| Xen-specific NetBSD issues |
## Xen-specific NetBSD issues |
| -------------------------- |
|
| |
|
| There are (at least) two additional things different about NetBSD as a |
There are (at least) two additional things different about NetBSD as a |
| dom0 kernel compared to hardware. |
dom0 kernel compared to hardware. |
|
Line 223 options as GENERIC. While it is debatab
|
Line 241 options as GENERIC. While it is debatab
|
| bug, users should be aware of this and can simply add missing config |
bug, users should be aware of this and can simply add missing config |
| items if desired. |
items if desired. |
| |
|
| Updating NetBSD in a dom0 |
## Updating NetBSD in a dom0 |
| ------------------------- |
|
| |
|
| This is just like updating NetBSD on bare hardware, assuming the new |
This is just like updating NetBSD on bare hardware, assuming the new |
| version supports the version of Xen you are running. Generally, one |
version supports the version of Xen you are running. Generally, one |
|
Line 234 and adjusts `/etc`.
|
Line 251 and adjusts `/etc`.
|
| Note that one must update both the non-Xen kernel typically used for |
Note that one must update both the non-Xen kernel typically used for |
| rescue purposes and the DOM0 kernel used with Xen. |
rescue purposes and the DOM0 kernel used with Xen. |
| |
|
| Converting from grub to /boot |
## Converting from grub to /boot |
| ----------------------------- |
|
| |
|
| These instructions were used to convert a system from |
These instructions were used to convert a system from |
| grub to /boot. The system was originally installed in February of |
grub to /boot. The system was originally installed in February of |
|
Line 265 default=1
|
Line 281 default=1
|
| timeout=30 |
timeout=30 |
| """]] |
"""]] |
| |
|
| Upgrading Xen versions |
## Upgrading Xen versions |
| --------------------- |
|
| |
|
| Minor version upgrades are trivial. Just rebuild/replace the |
Minor version upgrades are trivial. Just rebuild/replace the |
| xenkernel version and copy the new xen.gz to `/` (where `/boot.cfg` |
xenkernel version and copy the new xen.gz to `/` (where `/boot.cfg` |
|
Line 287 mediated by Xen, and configured in the d
|
Line 302 mediated by Xen, and configured in the d
|
| Entropy in domUs can be an issue; physical disks and network are on |
Entropy in domUs can be an issue; physical disks and network are on |
| the dom0. NetBSD's /dev/random system works, but is often challenged. |
the dom0. NetBSD's /dev/random system works, but is often challenged. |
| |
|
| Config files |
## Config files |
| ------------ |
|
| |
|
| See /usr/pkg/share/examples/xen/xlexample* |
See /usr/pkg/share/examples/xen/xlexample* |
| for a small number of well-commented examples, mostly for running |
for a small number of well-commented examples, mostly for running |
|
Line 334 equivalent to pushing the power button;
|
Line 348 equivalent to pushing the power button;
|
| power-press event and do a clean shutdown. Shutting down the dom0 |
power-press event and do a clean shutdown. Shutting down the dom0 |
| will trigger controlled shutdowns of all configured domUs. |
will trigger controlled shutdowns of all configured domUs. |
| |
|
| CPU and memory |
## CPU and memory |
| -------------- |
|
| |
|
| A domain is provided with some number of vcpus, up to the number |
A domain is provided with some number of vcpus, up to the number |
| of CPUs seen by the hypervisor. For a domU, it is controlled |
of CPUs seen by the hypervisor. For a domU, it is controlled |
|
Line 349 than the available memory.
|
Line 362 than the available memory.
|
| Xen also provides a "balloon" driver, which can be used to let domains |
Xen also provides a "balloon" driver, which can be used to let domains |
| use more memory temporarily. |
use more memory temporarily. |
| |
|
| Virtual disks |
## Virtual disks |
| ------------- |
|
| |
|
| In domU config files, the disks are defined as a sequence of 3-tuples: |
In domU config files, the disks are defined as a sequence of 3-tuples: |
| |
|
|
Line 499 It is also desirable to add
|
Line 511 It is also desirable to add
|
| powerd=YES |
powerd=YES |
| |
|
| in rc.conf. This way, the domain will be properly shut down if |
in rc.conf. This way, the domain will be properly shut down if |
| `xm shutdown -R` or `xm shutdown -H` is used on the dom0. |
`xl shutdown -R` or `xl shutdown -H` is used on the dom0. |
| |
\todo Check the translation to xl. |
| |
|
| It is not strictly necessary to have a kernel (as /netbsd) in the domU |
It is not strictly necessary to have a kernel (as /netbsd) in the domU |
| file system. However, various programs (e.g. netstat) will use that |
file system. However, various programs (e.g. netstat) will use that |
|
Line 702 NetBSD apparently only use specific vers
|
Line 715 NetBSD apparently only use specific vers
|
| |
|
| \todo Explain better. |
\todo Explain better. |
| |
|
| pvgrub |
## Boot methods |
| ------ |
|
| |
### pvgrub |
| |
|
| pvgrub is a version of grub that uses PV operations instead of BIOS |
pvgrub is a version of grub that uses PV operations instead of BIOS |
| calls. It is booted from the dom0 as the domU kernel, and then reads |
calls. It is booted from the dom0 as the domU kernel, and then reads |
|
Line 728 partition for the kernel with the intent
|
Line 742 partition for the kernel with the intent
|
| which leads to /netbsd not being the actual kernel. One must remember |
which leads to /netbsd not being the actual kernel. One must remember |
| to update the special boot partition. |
to update the special boot partition. |
| |
|
| pygrub |
### pygrub |
| ------- |
|
| |
|
| pygrub runs in the dom0 and looks into the domU file system. This |
pygrub runs in the dom0 and looks into the domU file system. This |
| implies that the domU must have a kernel in a file system in a format |
implies that the domU must have a kernel in a file system in a format |
|
Line 743 historical DomU images using it still wo
|
Line 756 historical DomU images using it still wo
|
| As of 2014, pygrub seems to be of mostly historical |
As of 2014, pygrub seems to be of mostly historical |
| interest. New DomUs should use pvgrub. |
interest. New DomUs should use pvgrub. |
| |
|
| Amazon |
## Specific Providers |
| ------ |
|
| |
### Amazon |
| |
|
| See the [Amazon EC2 page](/amazon_ec2/). |
See the [Amazon EC2 page](/amazon_ec2/). |
![[BACK]](/icons/back.gif){kind=icon}
Return to howto.mdwn CVS log ![[TXT]](/icons/text.gif){kind=icon}
![[DIR]](/icons/dir.gif){kind=icon}
Up to [NetBSD Developer Wiki] / wikisrc / ports / xen