version 1.157, 2018/08/27 16:54:42
|
version 1.158, 2018/09/05 09:16:10
|
Line 89 than amd64.
|
Line 89 than amd64.
|
NetBSD as a dom0 |
NetBSD as a dom0 |
================ |
================ |
|
|
NetBSD can be used as a dom0 and works very well. The following |
In order to install a NetBSD as a dom0, one must first install a normal |
sections address installation, updating NetBSD, and updating Xen. |
NetBSD system, and then pivot the install to a dom0 install by changing |
|
the kernel and boot configuration. |
Note that it doesn't make sense to talk about installing a dom0 OS |
|
without also installing Xen itself. We first address installing |
|
NetBSD, which is not yet a dom0, and then adding Xen, pivoting the |
|
NetBSD install to a dom0 install by just changing the kernel and boot |
|
configuration. |
|
|
|
In 2018-05, trouble booting a dom0 was reported with 256M of RAM: with |
In 2018-05, trouble booting a dom0 was reported with 256M of RAM: with |
512M it worked reliably. This does not make sense, but if you see |
512M it worked reliably. This does not make sense, but if you see |
"not ELF" after Xen boots, try increasing dom0 RAM. |
"not ELF" after Xen boots, try increasing dom0 RAM. |
|
|
Styles of dom0 operation |
|
------------------------ |
|
|
|
There are two basic ways to use Xen. The traditional method is for |
|
the dom0 to do absolutely nothing other than providing support to some |
|
number of domUs. Such a system was probably installed for the sole |
|
purpose of hosting domUs, and sits in a server room on a UPS. |
|
|
|
The other way is to put Xen under a normal-usage computer, so that the |
|
dom0 is what the computer would have been without Xen, perhaps a |
|
desktop or laptop. Then, one can run domUs at will. |
|
|
|
Installation of NetBSD |
Installation of NetBSD |
---------------------- |
---------------------- |
|
|
Line 125 Installation of Xen
|
Line 108 Installation of Xen
|
|
|
We will consider that you chose to use Xen 4.8, with NetBSD/amd64 as |
We will consider that you chose to use Xen 4.8, with NetBSD/amd64 as |
dom0. In the dom0, install xenkernel48 and xentools48 from pkgsrc. |
dom0. In the dom0, install xenkernel48 and xentools48 from pkgsrc. |
Ensure that your packages are recent. |
|
|
|
Once this is done, install the Xen kernel itself: |
Once this is done, install the Xen kernel itself: |
|
|
Line 158 itself uses (in this case, the serial po
|
Line 140 itself uses (in this case, the serial po
|
|
|
In an attempt to add performance, one can also add `dom0_max_vcpus=1 dom0_vcpus_pin`, |
In an attempt to add performance, one can also add `dom0_max_vcpus=1 dom0_vcpus_pin`, |
to force only one vcpu to be provided (since NetBSD dom0 can't use |
to force only one vcpu to be provided (since NetBSD dom0 can't use |
more) and to pin that vcpu to a physical CPU. |
more) and to pin that vcpu to a physical CPU. Xen has |
|
[many boot options](http://xenbits.xenproject.org/docs/4.8-testing/misc/xen-command-line.html), |
Xen has [many boot |
|
options](http://xenbits.xenproject.org/docs/4.5-testing/misc/xen-command-line.html), |
|
and other than dom0 memory and max_vcpus, they are generally not |
and other than dom0 memory and max_vcpus, they are generally not |
necessary. |
necessary. |
|
|
Now, reboot so that you are running a DOM0 kernel under Xen, rather |
Enable `xencommons`: |
than GENERIC without Xen. |
|
|
|
Configuring Xen |
|
--------------- |
|
|
|
Now, you have a system that will boot Xen and the dom0 kernel, but not |
|
do anything else special. Make sure that you have rebooted into Xen. |
|
There will be no domUs, and none can be started because you still have |
|
to configure the dom0 daemons. |
|
|
|
Since Xen 4.2, the tool which should be used is `xl`. |
|
|
|
For 4.1 and up, you should enable `xencommons`: |
|
|
|
[[!template id=filecontent name="/etc/rc.conf" text=""" |
[[!template id=filecontent name="/etc/rc.conf" text=""" |
xencommons=YES |
xencommons=YES |
"""]] |
"""]] |
|
|
Not enabling xencommons will result in a hang; it is necessary to hit `^C` on |
Now, reboot so that you are running a DOM0 kernel under Xen, rather |
the console to let the machine finish booting: |
than GENERIC without Xen. |
|
|
TODO: Recommend for/against xen-watchdog. |
TODO: Recommend for/against xen-watchdog. |
|
|
After you have configured the daemons and either started them (in the |
Once the reboot is done, use `xl` to inspect Xen's boot messages, |
order given) or rebooted, use `xl` to inspect Xen's boot messages, |
|
available resources, and running domains. For example: |
available resources, and running domains. For example: |
|
|
[[!template id=programlisting text=""" |
[[!template id=programlisting text=""" |
Line 311 the dom0. NetBSD's /dev/random system w
|
Line 277 the dom0. NetBSD's /dev/random system w
|
Config files |
Config files |
------------ |
------------ |
|
|
There is no good order to present config files and the concepts |
See /usr/pkg/share/examples/xen/xlexample*, |
surrounding what is being configured. We first show an example config |
|
file, and then in the various sections give details. |
|
|
|
See /usr/pkg/share/examples/xen/xmexample*, |
|
for a large number of well-commented examples, mostly for running |
for a large number of well-commented examples, mostly for running |
GNU/Linux. |
GNU/Linux. |
|
|
Line 359 equivalent to pushing the power button;
|
Line 321 equivalent to pushing the power button;
|
power-press event and do a clean shutdown. Shutting down the dom0 |
power-press event and do a clean shutdown. Shutting down the dom0 |
will trigger controlled shutdowns of all configured domUs. |
will trigger controlled shutdowns of all configured domUs. |
|
|
domU kernels |
|
------------ |
|
|
|
On a physical computer, the BIOS reads sector 0, and a chain of boot |
|
loaders finds and loads a kernel. Normally this comes from the root |
|
file system. With Xen domUs, the process is totally different. The |
|
normal path is for the domU kernel to be a file in the dom0's |
|
file system. At the request of the dom0, Xen loads that kernel into a |
|
new domU instance and starts execution. domU kernels can be anyplace. |
|
|
|
Note that loading the domU kernel from the dom0 implies that boot |
|
blocks, /boot, /boot.cfg, and so on are all ignored in the domU. |
|
See the VPS section near the end for discussion of alternate ways to |
|
obtain domU kernels. |
|
|
|
CPU and memory |
CPU and memory |
-------------- |
-------------- |
|
|
Line 392 use more memory temporarily.
|
Line 339 use more memory temporarily.
|
Virtual disks |
Virtual disks |
------------- |
------------- |
|
|
With the file/vnd style, typically one creates a directory, |
In domU config files, the disks are defined as a sequence of 3-tuples: |
e.g. /u0/xen, on a disk large enough to hold virtual disks for all |
|
domUs. Then, for each domU disk, one writes zeros to a file that then |
* The first element is "method:/path/to/disk". Common methods are |
serves to hold the virtual disk's bits; a suggested name is foo-xbd0 |
"file:" for file-backed vnd, and "phy:" for something that is already |
for the first virtual disk for the domU called foo. Writing zeros to |
a device. |
the file serves two purposes. One is that preallocating the contents |
|
improves performance. The other is that vnd on sparse files has |
* The second element is an artifact of how virtual disks are passed to |
failed to work. TODO: give working/notworking NetBSD versions for |
Linux, and a source of confusion with NetBSD Xen usage. Linux domUs |
sparse vnd and gnats reference. Note that the use of file/vnd for Xen |
are given a device name to associate with the disk, and values like |
is not really different than creating a file-backed virtual disk for |
"hda1" or "sda1" are common. In a NetBSD domU, the first disk appears |
some other purpose, except that xentools handles the vnconfig |
as xbd0, the second as xbd1, and so on. However, xl demands a |
commands. To create an empty 4G virtual disk, simply do |
second argument. The name given is converted to a major/minor by |
|
calling stat(2) on the name in /dev and this is passed to the domU. |
dd if=/dev/zero of=foo-xbd0 bs=1m count=4096 |
In the general case, the dom0 and domU can be different operating |
|
systems, and it is an unwarranted assumption that they have consistent |
Do not use qemu-img-xen, because this will create sparse file. There |
numbering in /dev, or even that the dom0 OS has a /dev. With NetBSD |
have been recent (2015) reports of sparse vnd(4) devices causing |
as both dom0 and domU, using values of 0x0 for the first disk and 0x1 |
lockups, but there is apparently no PR. |
for the second works fine and avoids this issue. For a GNU/Linux |
|
guest, one can create /dev/hda1 in /dev, or to pass 0x301 for |
With the lvm style, one creates logical devices. They are then used |
/dev/hda1. |
similarly to vnds. TODO: Add an example with lvm. |
|
|
|
In domU config files, the disks are defined as a sequence of 3-tuples. |
|
The first element is "method:/path/to/disk". Common methods are |
|
"file:" for file-backed vnd. and "phy:" for something that is already |
|
a (TODO: character or block) device. |
|
|
|
The second element is an artifact of how virtual disks are passed to |
|
Linux, and a source of confusion with NetBSD Xen usage. Linux domUs |
|
are given a device name to associate with the disk, and values like |
|
"hda1" or "sda1" are common. In a NetBSD domU, the first disk appears |
|
as xbd0, the second as xbd1, and so on. However, xm/xl demand a |
|
second argument. The name given is converted to a major/minor by |
|
calling stat(2) on the name in /dev and this is passed to the domU. |
|
In the general case, the dom0 and domU can be different operating |
|
systems, and it is an unwarranted assumption that they have consistent |
|
numbering in /dev, or even that the dom0 OS has a /dev. With NetBSD |
|
as both dom0 and domU, using values of 0x0 for the first disk and 0x1 |
|
for the second works fine and avoids this issue. For a GNU/Linux |
|
guest, one can create /dev/hda1 in /dev, or to pass 0x301 for |
|
/dev/hda1. |
|
|
|
The third element is "w" for writable disks, and "r" for read-only |
* The third element is "w" for writable disks, and "r" for read-only |
disks. |
disks. |
|
|
|
Example: |
|
[[!template id=filecontent name="/usr/pkg/etc/xen/foo" text=""" |
|
disk = [ 'file:/n0/xen/foo-wd0,0x0,w' ] |
|
"""]] |
|
|
Note that NetBSD by default creates only vnd[0123]. If you need more |
Note that NetBSD by default creates only vnd[0123]. If you need more |
than 4 total virtual disks at a time, run e.g. "./MAKEDEV vnd4" in the |
than 4 total virtual disks at a time, run e.g. "./MAKEDEV vnd4" in the |
Line 449 Virtual Networking
|
Line 380 Virtual Networking
|
|
|
Xen provides virtual Ethernets, each of which connects the dom0 and a |
Xen provides virtual Ethernets, each of which connects the dom0 and a |
domU. For each virtual network, there is an interface "xvifN.M" in |
domU. For each virtual network, there is an interface "xvifN.M" in |
the dom0, and in domU index N, a matching interface xennetM (NetBSD |
the dom0, and a matching interface xennetM (NetBSD name) in domU index N. |
name). The interfaces behave as if there is an Ethernet with two |
The interfaces behave as if there is an Ethernet with two |
adapters connected. From this primitive, one can construct various |
adapters connected. From this primitive, one can construct various |
configurations. We focus on two common and useful cases for which |
configurations. We focus on two common and useful cases for which |
there are existing scripts: bridging and NAT. |
there are existing scripts: bridging and NAT. |
Line 477 The MAC address specified is the one use
|
Line 408 The MAC address specified is the one use
|
domain. The interface in dom0 will use this address XOR'd with |
domain. The interface in dom0 will use this address XOR'd with |
00:00:00:01:00:00. Random MAC addresses are assigned if not given. |
00:00:00:01:00:00. Random MAC addresses are assigned if not given. |
|
|
Sizing domains |
|
-------------- |
|
|
|
Modern x86 hardware has vast amounts of resources. However, many |
|
virtual servers can function just fine on far less. A system with |
|
512M of RAM and a 4G disk can be a reasonable choice. Note that it is |
|
far easier to adjust virtual resources than physical ones. For |
|
memory, it's just a config file edit and a reboot. For disk, one can |
|
create a new file and vnconfig it (or lvm), and then dump/restore, |
|
just like updating physical disks, but without having to be there and |
|
without those pesky connectors. |
|
|
|
Starting domains automatically |
Starting domains automatically |
------------------------------ |
------------------------------ |
|
|
To start domains foo at bar at boot and shut them down cleanly on dom0 |
To start domains `domU-netbsd` and `domU-linux` at boot and shut them |
shutdown, in rc.conf add: |
down cleanly on dom0 shutdown, add the following in rc.conf: |
|
|
xendomains="foo bar" |
|
|
|
Note that earlier versions of the xentools41 xendomains rc.d script |
[[!template id=filecontent name="/etc/rc.conf" text=""" |
used xl, when one should use xm with 4.1. |
xendomains="domU-netbsd domU-linux" |
|
"""]] |
|
|
Creating specific unprivileged domains (domU) |
Creating a domU |
============================================= |
=============== |
|
|
Creating domUs is almost entirely independent of operating system. We |
Creating domUs is almost entirely independent of operating system. We |
have already presented the basics of config files. Note that you must |
have already presented the basics of config files. Note that you must |
have already completed the dom0 setup so that "xl list" (or "xm list") |
have already completed the dom0 setup so that "xl list" works. |
works. |
|
|
|
Creating an unprivileged NetBSD domain (domU) |
Creating a NetBSD domU |
--------------------------------------------- |
---------------------- |
|
|
See the earlier config file, and adjust memory. Decide on how much |
See the earlier config file, and adjust memory. Decide on how much |
storage you will provide, and prepare it (file or lvm). |
storage you will provide, and prepare it (file or lvm). |
Line 580 not really a Xen-specific issue, but bec
|
Line 497 not really a Xen-specific issue, but bec
|
obtained from the dom0, it is far more likely to be out of sync or |
obtained from the dom0, it is far more likely to be out of sync or |
missing with Xen.) |
missing with Xen.) |
|
|
Creating an unprivileged Linux domain (domU) |
Creating a Linux domU |
-------------------------------------------- |
--------------------- |
|
|
Creating unprivileged Linux domains isn't much different from |
Creating unprivileged Linux domains isn't much different from |
unprivileged NetBSD domains, but there are some details to know. |
unprivileged NetBSD domains, but there are some details to know. |
Line 624 To get the Linux console right, you need
|
Line 541 To get the Linux console right, you need
|
to your configuration since not all Linux distributions auto-attach a |
to your configuration since not all Linux distributions auto-attach a |
tty to the xen console. |
tty to the xen console. |
|
|
Creating an unprivileged Solaris domain (domU) |
Creating a Solaris domU |
---------------------------------------------- |
----------------------- |
|
|
See possibly outdated |
See possibly outdated |
[Solaris domU instructions](/ports/xen/howto-solaris/). |
[Solaris domU instructions](/ports/xen/howto-solaris/). |