version 1.156, 2018/08/25 09:42:48
|
version 1.157, 2018/08/27 16:54:42
|
Line 1
|
Line 1
|
[[!meta title="Xen HowTo"]] |
[[!meta title="Xen HowTo"]] |
|
|
Introduction |
Xen is a Type 1 hypervisor which supports running multiple guest operating |
============ |
systems on a single physical machine. One uses the Xen kernel to control the |
|
CPU, memory and console, a dom0 operating system which mediates access to |
Xen is a hypervisor for x86 hardware, which supports running multiple guest |
other hardware (e.g., disks, network, USB), and one or more domU operating |
operating systems on a single physical machine. Xen is a Type 1 or |
systems which operate in an unprivileged virtualized environment. IO requests |
bare-metal hypervisor; one uses the Xen kernel to control the CPU, |
from the domU systems are forwarded by the Xen hypervisor to the dom0 to be |
memory and console, a dom0 operating system which mediates access to |
fulfilled. |
other hardware (e.g., disks, network, USB), and one or more domU |
|
operating systems which operate in an unprivileged virtualized |
|
environment. IO requests from the domU systems are forwarded by the |
|
Xen hypervisor to the dom0 to be fulfilled. |
|
|
|
Xen supports different styles of guest: |
Xen supports different styles of guest: |
|
|
Line 41 This HOWTO attempts to address both the
|
Line 37 This HOWTO attempts to address both the
|
on hardware and running domUs under it (NetBSD and other), and also |
on hardware and running domUs under it (NetBSD and other), and also |
running NetBSD as a domU in a VPS. |
running NetBSD as a domU in a VPS. |
|
|
Versions of Xen and NetBSD |
Versions and Support |
========================== |
==================== |
|
|
Most of the installation concepts and instructions are independent |
|
of Xen version and NetBSD version. This section gives advice on |
|
which version to choose. Versions not in pkgsrc and older unsupported |
|
versions of NetBSD are intentionally ignored. |
|
|
|
The term "amd64" is used to refer to both the NetBSD port and to the |
|
hardware architecture on which it runs. Such hardware is generally |
|
made by both Intel and AMD, and common on PC computers. |
|
|
|
Xen versions |
|
------------ |
|
|
|
In NetBSD, Xen is provided in pkgsrc, via matching pairs of packages |
In NetBSD, Xen is provided in pkgsrc, via matching pairs of packages |
xenkernel and xentools. We will refer only to the kernel versions, |
xenkernel and xentools. We will refer only to the kernel versions, |
Line 74 Xen Version |Package Name |Xen CPU Suppo
|
Line 58 Xen Version |Package Name |Xen CPU Suppo
|
|
|
See also the [Xen Security Advisory page](http://xenbits.xen.org/xsa/). |
See also the [Xen Security Advisory page](http://xenbits.xen.org/xsa/). |
|
|
Note: Xen 4.2 was the last version to support 32bit CPUs. |
Multiprocessor (SMP) support in NetBSD differs depending on the domain: |
|
|
NetBSD versions |
|
--------------- |
|
|
|
NetBSD 8 is recommended as the stable version of the most recent |
|
release for production use. |
|
|
|
For developing Xen, netbsd-current may be appropriate. |
|
|
|
As of NetBSD 6, a NetBSD domU will support multiple vcpus. There is |
[[!table data=""" |
no SMP support for NetBSD as dom0. (The dom0 itself doesn't really |
Domain |Supports SMP |
need SMP for dom0 functions; the lack of support is really a problem |
dom0 |No |
when using a dom0 as a normal computer.) |
domU |Yes |
|
"""]] |
|
|
Note: NetBSD support is called XEN3. However, it does support Xen 4, |
Note: NetBSD support is called XEN3. However, it does support Xen 4, |
because the hypercall interface has remained identical. |
because the hypercall interface has remained identical. |
Line 95 because the hypercall interface has rema
|
Line 72 because the hypercall interface has rema
|
Architecture |
Architecture |
------------ |
------------ |
|
|
Xen itself runs on amd64 hardware. Practically, almost any computer |
Xen itself runs on x86_64 hardware. |
where one would want to run Xen today supports amd64. |
|
|
|
The dom0 system, plus each domU, can be either i386PAE or amd64. |
The dom0 system, plus each domU, can be either i386PAE or amd64. |
i386 without PAE is not supported. |
i386 without PAE is not supported. |
|
|
The standard approach is to use NetBSD/amd64 for the dom0. For domUs, |
The standard approach is to use NetBSD/amd64 for the dom0. |
NetBSD/i386 (PAE) and NetBSD/amd64 are in widespread use, and there is |
|
little to no Xen-specific reason to prefer one over the other. |
To use an i386PAE dom0, one must build or obtain a 64bit Xen kernel and |
|
install it on the system. |
Note that to use an i386 dom0 with Xen 4.5 or higher, one must build |
|
(or obtain from pre-built packages) an amd64 Xen kernel and install |
For domUs, i386PAE is considered as |
that on the system. Almost no one in the NetBSD/Xen community does |
[faster](https://lists.xen.org/archives/html/xen-devel/2012-07/msg00085.html) |
this, and the standard, well-tested, approach is to use an amd64 dom0. |
than amd64. |
|
|
A [posting on |
|
xen-devel](https://lists.xen.org/archives/html/xen-devel/2012-07/msg00085.html) |
|
explained that PV system call overhead was higher on amd64, and thus |
|
there is some notion that i386 guests are faster. It goes on to |
|
caution that the total situation is complex and not entirely |
|
understood. On top of that caution, the post is about Linux, not |
|
NetBSD. |
|
|
|
NetBSD as a dom0 |
NetBSD as a dom0 |
================ |
================ |
|
|
NetBSD can be used as a dom0 and works very well. The following |
NetBSD can be used as a dom0 and works very well. The following |
sections address installation, updating NetBSD, and updating Xen. |
sections address installation, updating NetBSD, and updating Xen. |
|
|
Note that it doesn't make sense to talk about installing a dom0 OS |
Note that it doesn't make sense to talk about installing a dom0 OS |
without also installing Xen itself. We first address installing |
without also installing Xen itself. We first address installing |
NetBSD, which is not yet a dom0, and then adding Xen, pivoting the |
NetBSD, which is not yet a dom0, and then adding Xen, pivoting the |
NetBSD install to a dom0 install by just changing the kernel and boot |
NetBSD install to a dom0 install by just changing the kernel and boot |
configuration. |
configuration. |
|
|
For experimenting with Xen, a machine with as little as 1G of RAM and |
|
100G of disk can work. For running many domUs in productions, far |
|
more will be needed; e.g. 4-8G and 1T of disk is reasonable for a |
|
half-dozen domUs of 512M and 32G each. Basically, the RAM and disk |
|
have to be bigger than the sum of the RAM/disk needs of the dom0 and |
|
all the domUs. |
|
|
|
In 2018-05, trouble booting a dom0 was reported with 256M of RAM: with |
In 2018-05, trouble booting a dom0 was reported with 256M of RAM: with |
512M it worked reliably. This does not make sense, but if you see |
512M it worked reliably. This does not make sense, but if you see |
"not ELF" after Xen boots, try increasing dom0 RAM. |
"not ELF" after Xen boots, try increasing dom0 RAM. |
Line 150 purpose of hosting domUs, and sits in a
|
Line 112 purpose of hosting domUs, and sits in a
|
|
|
The other way is to put Xen under a normal-usage computer, so that the |
The other way is to put Xen under a normal-usage computer, so that the |
dom0 is what the computer would have been without Xen, perhaps a |
dom0 is what the computer would have been without Xen, perhaps a |
desktop or laptop. Then, one can run domUs at will. Purists will |
desktop or laptop. Then, one can run domUs at will. |
deride this as less secure than the previous approach, and for a |
|
computer whose purpose is to run domUs, they are right. But Xen and a |
|
dom0 (without domUs) is not meaningfully less secure than the same |
|
things running without Xen. One can boot Xen or boot regular NetBSD |
|
alternately with little problems, simply refraining from starting the |
|
Xen daemons when not running Xen. |
|
|
|
Note that NetBSD as dom0 does not support multiple CPUs. This will |
|
limit the performance of the Xen/dom0 workstation approach. In theory |
|
the only issue is that the "backend drivers" are not yet MPSAFE: |
|
https://mail-index.netbsd.org/netbsd-users/2014/08/29/msg015195.html |
|
|
|
Installation of NetBSD |
Installation of NetBSD |
---------------------- |
---------------------- |
|
|
First, |
[Install NetBSD/amd64](/guide/inst/) |
[install NetBSD/amd64](/guide/inst/) |
|
just as you would if you were not using Xen. |
just as you would if you were not using Xen. |
However, the partitioning approach is very important. |
|
|
|
If you want to use RAIDframe for the dom0, there are no special issues |
|
for Xen. Typically one provides RAID storage for the dom0, and the |
|
domU systems are unaware of RAID. The 2nd-stage loader bootxx_* skips |
|
over a RAID1 header to find /boot from a file system within a RAID |
|
partition; this is no different when booting Xen. |
|
|
|
There are 4 styles of providing backing storage for the virtual disks |
|
used by domUs: raw partitions, LVM, file-backed vnd(4), and SAN. |
|
|
|
With raw partitions, one has a disklabel (or gpt) partition sized for |
|
each virtual disk to be used by the domU. (If you are able to predict |
|
how domU usage will evolve, please add an explanation to the HOWTO. |
|
Seriously, needs tend to change over time.) |
|
|
|
One can use [lvm(8)](/guide/lvm/) to create logical devices to use |
|
for domU disks. This is almost as efficient as raw disk partitions |
|
and more flexible. Hence raw disk partitions should typically not |
|
be used. |
|
|
|
One can use files in the dom0 file system, typically created by dd'ing |
|
/dev/zero to create a specific size. This is somewhat less efficient, |
|
but very convenient, as one can cp the files for backup, or move them |
|
between dom0 hosts. |
|
|
|
Finally, in theory one can place the files backing the domU disks in a |
|
SAN. (This is an invitation for someone who has done this to add a |
|
HOWTO page.) |
|
|
|
Installation of Xen |
Installation of Xen |
------------------- |
------------------- |
Line 212 Once this is done, install the Xen kerne
|
Line 133 Once this is done, install the Xen kerne
|
# cp /usr/pkg/xen48-kernel/xen.gz / |
# cp /usr/pkg/xen48-kernel/xen.gz / |
"""]] |
"""]] |
|
|
Then, place a NetBSD XEN3_DOM0 kernel in /, copied from |
Then, place a NetBSD XEN3_DOM0 kernel in the `/` directory, copied from |
releasedir/amd64/binary/kernel/netbsd-XEN3_DOM0.gz of a NetBSD build. |
`releasedir/amd64/binary/kernel/netbsd-XEN3_DOM0.gz` of a NetBSD build. |
|
|
Add a line to /boot.cfg to boot Xen: |
Add a line to /boot.cfg to boot Xen: |
|
|
[[!template id=programlisting text=""" |
[[!template id=filecontent name="/boot.cfg" text=""" |
menu=Xen:load /netbsd-XEN3_DOM0.gz console=pc;multiboot /xen.gz dom0_mem=512M |
menu=Xen:load /netbsd-XEN3_DOM0.gz console=pc;multiboot /xen.gz dom0_mem=512M |
"""]] |
"""]] |
|
|
which specifies that the dom0 should have 512MB of ram, leaving the rest |
which specifies that the dom0 should have 512MB of ram, leaving the rest |
to be allocated for domUs. To use a serial console, use |
to be allocated for domUs. To use a serial console, use |
|
|
[[!template id=programlisting text=""" |
[[!template id=filecontent name="/boot.cfg" text=""" |
menu=Xen:load /netbsd-XEN3_DOM0.gz;multiboot /xen.gz dom0_mem=512M console=com1 com1=9600,8n1 |
menu=Xen:load /netbsd-XEN3_DOM0.gz;multiboot /xen.gz dom0_mem=512M console=com1 com1=9600,8n1 |
"""]] |
"""]] |
|
|
Line 235 speed/parity. Because the NetBSD comman
|
Line 156 speed/parity. Because the NetBSD comman
|
which directs the console I/O through Xen to the same console device Xen |
which directs the console I/O through Xen to the same console device Xen |
itself uses (in this case, the serial port). |
itself uses (in this case, the serial port). |
|
|
In an attempt to add performance, one can also add: |
In an attempt to add performance, one can also add `dom0_max_vcpus=1 dom0_vcpus_pin`, |
|
|
[[!template id=programlisting text=""" |
|
dom0_max_vcpus=1 dom0_vcpus_pin |
|
"""]] |
|
|
|
to force only one vcpu to be provided (since NetBSD dom0 can't use |
to force only one vcpu to be provided (since NetBSD dom0 can't use |
more) and to pin that vcpu to a physical CPU. |
more) and to pin that vcpu to a physical CPU. |
|
|
Line 249 options](http://xenbits.xenproject.org/d
|
Line 165 options](http://xenbits.xenproject.org/d
|
and other than dom0 memory and max_vcpus, they are generally not |
and other than dom0 memory and max_vcpus, they are generally not |
necessary. |
necessary. |
|
|
As with non-Xen systems, you should have a line to boot /netbsd (a |
|
kernel that works without Xen). |
|
|
|
Now, reboot so that you are running a DOM0 kernel under Xen, rather |
Now, reboot so that you are running a DOM0 kernel under Xen, rather |
than GENERIC without Xen. |
than GENERIC without Xen. |
|
|
Using grub (historic) |
|
--------------------- |
|
|
|
Before NetBSD's native bootloader could support Xen, the use of |
|
grub was recommended. If necessary, see the |
|
[old grub information](/ports/xen/howto-grub). |
|
|
|
The [HowTo on Installing into |
|
RAID-1](https://mail-index.NetBSD.org/port-xen/2006/03/01/0010.html) |
|
explains how to set up booting a dom0 with Xen using grub with |
|
NetBSD's RAIDframe. (This is obsolete with the use of NetBSD's native |
|
boot. Now, just create a system with RAID-1, and alter /boot.cfg as |
|
described above.) |
|
|
|
Configuring Xen |
Configuring Xen |
--------------- |
--------------- |
|
|
Line 277 do anything else special. Make sure tha
|
Line 176 do anything else special. Make sure tha
|
There will be no domUs, and none can be started because you still have |
There will be no domUs, and none can be started because you still have |
to configure the dom0 daemons. |
to configure the dom0 daemons. |
|
|
The daemons which should be run vary with Xen version and with whether |
Since Xen 4.2, the tool which should be used is `xl`. |
one is using xm or xl. Xen 4.2 and up packages use xl. To use xm with 4.2, |
|
edit xendomains to use xm instead. |
|
|
|
For 4.1 and up, you should enable xencommons. Not enabling xencommons |
|
will result in a hang; it is necessary to hit ^C on the console to let |
|
the machine finish booting. If you are using xm (default in 4.1, or |
|
if you changed xendomains in 4.2), you should also enable xend: |
|
|
|
[[!template id=programlisting text=""" |
For 4.1 and up, you should enable `xencommons`: |
xend=YES # only if using xm, and only installed <= 4.2 |
|
|
[[!template id=filecontent name="/etc/rc.conf" text=""" |
xencommons=YES |
xencommons=YES |
"""]] |
"""]] |
|
|
|
Not enabling xencommons will result in a hang; it is necessary to hit `^C` on |
|
the console to let the machine finish booting: |
|
|
TODO: Recommend for/against xen-watchdog. |
TODO: Recommend for/against xen-watchdog. |
|
|
After you have configured the daemons and either started them (in the |
After you have configured the daemons and either started them (in the |
order given) or rebooted, use xm or xl to inspect Xen's boot messages, |
order given) or rebooted, use `xl` to inspect Xen's boot messages, |
available resources, and running domains. An example with xl follows: |
available resources, and running domains. For example: |
|
|
[[!template id=programlisting text=""" |
[[!template id=programlisting text=""" |
# xl dmesg |
# xl dmesg |
Line 311 Xen logs will be in /var/log/xen.
|
Line 207 Xen logs will be in /var/log/xen.
|
|
|
### Issues with xencommons |
### Issues with xencommons |
|
|
xencommons starts xenstored, which stores data on behalf of dom0 and |
`xencommons` starts `xenstored`, which stores data on behalf of dom0 and |
domUs. It does not currently work to stop and start xenstored. |
domUs. It does not currently work to stop and start xenstored. |
Certainly all domUs should be shutdown first, following the sort order |
Certainly all domUs should be shutdown first, following the sort order |
of the rc.d scripts. However, the dom0 sets up state with xenstored, |
of the rc.d scripts. However, the dom0 sets up state with xenstored, |
Line 321 make this work, one should not expect to
|
Line 217 make this work, one should not expect to
|
(and thus xencommons). There is currently no reason to expect that |
(and thus xencommons). There is currently no reason to expect that |
this will get fixed any time soon. |
this will get fixed any time soon. |
|
|
### No-longer needed advice about devices |
|
|
|
The installation of NetBSD should already have created devices for xen |
|
(xencons, xenevt, xsd_kva), but if they are not present, create them: |
|
|
|
[[!template id=programlisting text=""" |
|
cd /dev && sh MAKEDEV xen |
|
"""]] |
|
|
|
anita (for testing NetBSD) |
anita (for testing NetBSD) |
-------------------------- |
-------------------------- |
|
|
With the setup so far (assuming 4.2/xl), one should be able to run |
With the setup so far (assuming 4.8/xl), one should be able to run |
anita (see pkgsrc/misc/py-anita) to test NetBSD releases, by doing (as |
anita (see pkgsrc/misc/py-anita) to test NetBSD releases, by doing (as |
root, because anita must create a domU): |
root, because anita must create a domU): |
|
|
Line 341 root, because anita must create a domU):
|
Line 228 root, because anita must create a domU):
|
anita --vmm=xl test file:///usr/obj/i386/ |
anita --vmm=xl test file:///usr/obj/i386/ |
"""]] |
"""]] |
|
|
Alternatively, one can use --vmm=xm to use xm-based domU creation |
|
instead (and must, on Xen <= 4.1). TODO: confirm that anita xl really works. |
|
|
|
Xen-specific NetBSD issues |
Xen-specific NetBSD issues |
-------------------------- |
-------------------------- |
|
|
Line 352 dom0 kernel compared to hardware.
|
Line 236 dom0 kernel compared to hardware.
|
|
|
One is that the module ABI is different because some of the #defines |
One is that the module ABI is different because some of the #defines |
change, so one must build modules for Xen. As of netbsd-7, the build |
change, so one must build modules for Xen. As of netbsd-7, the build |
system does this automatically. TODO: check this. (Before building |
system does this automatically. |
Xen modules was added, it was awkward to use modules to the point |
|
where it was considered that it did not work.) |
|
|
|
The other difference is that XEN3_DOM0 does not have exactly the same |
The other difference is that XEN3_DOM0 does not have exactly the same |
options as GENERIC. While it is debatable whether or not this is a |
options as GENERIC. While it is debatable whether or not this is a |
Line 367 Updating NetBSD in a dom0
|
Line 249 Updating NetBSD in a dom0
|
This is just like updating NetBSD on bare hardware, assuming the new |
This is just like updating NetBSD on bare hardware, assuming the new |
version supports the version of Xen you are running. Generally, one |
version supports the version of Xen you are running. Generally, one |
replaces the kernel and reboots, and then overlays userland binaries |
replaces the kernel and reboots, and then overlays userland binaries |
and adjusts /etc. |
and adjusts `/etc`. |
|
|
Note that one must update both the non-Xen kernel typically used for |
Note that one must update both the non-Xen kernel typically used for |
rescue purposes and the DOM0 kernel used with Xen. |
rescue purposes and the DOM0 kernel used with Xen. |
Line 375 rescue purposes and the DOM0 kernel used
|
Line 257 rescue purposes and the DOM0 kernel used
|
Converting from grub to /boot |
Converting from grub to /boot |
----------------------------- |
----------------------------- |
|
|
These instructions were [TODO: will be] used to convert a system from |
These instructions were used to convert a system from |
grub to /boot. The system was originally installed in February of |
grub to /boot. The system was originally installed in February of |
2006 with a RAID1 setup and grub to boot Xen 2, and has been updated |
2006 with a RAID1 setup and grub to boot Xen 2, and has been updated |
over time. Before these commands, it was running NetBSD 6 i386, Xen |
over time. Before these commands, it was running NetBSD 6 i386, Xen |
4.1 and grub, much like the message linked earlier in the grub |
4.1 and grub, much like the message linked earlier in the grub |
section. |
section. |
|
|
# Install MBR bootblocks on both disks. |
[[!template id=programlisting text=""" |
fdisk -i /dev/rwd0d |
# Install MBR bootblocks on both disks. |
fdisk -i /dev/rwd1d |
fdisk -i /dev/rwd0d |
# Install NetBSD primary boot loader (/ is FFSv1) into RAID1 components. |
fdisk -i /dev/rwd1d |
installboot -v /dev/rwd0d /usr/mdec/bootxx_ffsv1 |
# Install NetBSD primary boot loader (/ is FFSv1) into RAID1 components. |
installboot -v /dev/rwd1d /usr/mdec/bootxx_ffsv1 |
installboot -v /dev/rwd0d /usr/mdec/bootxx_ffsv1 |
# Install secondary boot loader |
installboot -v /dev/rwd1d /usr/mdec/bootxx_ffsv1 |
cp -p /usr/mdec/boot / |
# Install secondary boot loader |
# Create boot.cfg following earlier guidance: |
cp -p /usr/mdec/boot / |
menu=Xen:load /netbsd-XEN3PAE_DOM0.gz console=pc;multiboot /xen.gz dom0_mem=512M |
# Create boot.cfg following earlier guidance: |
menu=Xen.ok:load /netbsd-XEN3PAE_DOM0.ok.gz console=pc;multiboot /xen.ok.gz dom0_mem=512M |
menu=Xen:load /netbsd-XEN3PAE_DOM0.gz console=pc;multiboot /xen.gz dom0_mem=512M |
menu=GENERIC:boot |
menu=Xen.ok:load /netbsd-XEN3PAE_DOM0.ok.gz console=pc;multiboot /xen.ok.gz dom0_mem=512M |
menu=GENERIC single-user:boot -s |
menu=GENERIC:boot |
menu=GENERIC.ok:boot netbsd.ok |
menu=GENERIC single-user:boot -s |
menu=GENERIC.ok single-user:boot netbsd.ok -s |
menu=GENERIC.ok:boot netbsd.ok |
menu=Drop to boot prompt:prompt |
menu=GENERIC.ok single-user:boot netbsd.ok -s |
default=1 |
menu=Drop to boot prompt:prompt |
timeout=30 |
default=1 |
|
timeout=30 |
TODO: actually do this and fix it if necessary. |
"""]] |
|
|
Upgrading Xen versions |
Upgrading Xen versions |
--------------------- |
--------------------- |
|
|
Minor version upgrades are trivial. Just rebuild/replace the |
Minor version upgrades are trivial. Just rebuild/replace the |
xenkernel version and copy the new xen.gz to / (where /boot.cfg |
xenkernel version and copy the new xen.gz to `/` (where `/boot.cfg` |
references it), and reboot. |
references it), and reboot. |
|
|
Major version upgrades are conceptually not difficult, but can run |
|
into all the issues found when installing Xen. Assuming migration |
|
from 4.1 to 4.2, remove the xenkernel41 and xentools41 packages and |
|
install the xenkernel42 and xentools42 packages. Copy the 4.2 xen.gz |
|
to /. |
|
|
|
Ensure that the contents of /etc/rc.d/xen* are correct. Specifically, |
|
they must match the package you just installed and not be left over |
|
from some previous installation. |
|
|
|
Enable the correct set of daemons; see the configuring section above. |
|
(Upgrading from 3.x to 4.x without doing this will result in a hang.) |
|
|
|
Ensure that the domU config files are valid for the new version. |
|
Specifically, for 4.x remove autorestart=True, and ensure that disks |
|
are specified with numbers as the second argument, as the examples |
|
above show, and not NetBSD device names. |
|
|
|
Unprivileged domains (domU) |
Unprivileged domains (domU) |
=========================== |
=========================== |
|
|
This section describes general concepts about domUs. It does not |
This section describes general concepts about domUs. It does not |
address specific domU operating systems or how to install them. The |
address specific domU operating systems or how to install them. The |
config files for domUs are typically in /usr/pkg/etc/xen, and are |
config files for domUs are typically in `/usr/pkg/etc/xen`, and are |
typically named so that the file name, domU name and the domU's host |
typically named so that the file name, domU name and the domU's host |
name match. |
name match. |
|
|
Line 451 There is no good order to present config
|
Line 315 There is no good order to present config
|
surrounding what is being configured. We first show an example config |
surrounding what is being configured. We first show an example config |
file, and then in the various sections give details. |
file, and then in the various sections give details. |
|
|
See (at least in xentools41) /usr/pkg/share/examples/xen/xmexample*, |
See /usr/pkg/share/examples/xen/xmexample*, |
for a large number of well-commented examples, mostly for running |
for a large number of well-commented examples, mostly for running |
GNU/Linux. |
GNU/Linux. |
|
|
The following is an example minimal domain configuration file |
The following is an example minimal domain configuration file. The domU |
"/usr/pkg/etc/xen/foo". It is (with only a name change) an actual |
serves as a network file server. |
known working config file on Xen 4.1 (NetBSD 5 amd64 dom0 and NetBSD 5 |
|
i386 domU). The domU serves as a network file server. |
[[!template id=filecontent name="/usr/pkg/etc/xen/foo" text=""" |
|
name = "domU-id" |
# -*- mode: python; -*- |
kernel = "/netbsd-XEN3PAE_DOMU-i386-foo.gz" |
|
memory = 1024 |
name = "domU-id" |
vif = [ 'mac=aa:00:00:d1:00:09,bridge=bridge0' ] |
kernel = "/netbsd-XEN3PAE_DOMU-i386-foo.gz" |
disk = [ 'file:/n0/xen/foo-wd0,0x0,w', |
memory = 1024 |
'file:/n0/xen/foo-wd1,0x1,w' ] |
vif = [ 'mac=aa:00:00:d1:00:09,bridge=bridge0' ] |
"""]] |
disk = [ 'file:/n0/xen/foo-wd0,0x0,w', |
|
'file:/n0/xen/foo-wd1,0x1,w' ] |
|
|
|
The domain will have name given in the name setting. The kernel has the |
The domain will have name given in the `name` setting. The kernel has the |
host/domU name in it, so that on the dom0 one can update the various |
host/domU name in it, so that on the dom0 one can update the various |
domUs independently. The vif line causes an interface to be provided, |
domUs independently. The `vif` line causes an interface to be provided, |
with a specific mac address (do not reuse MAC addresses!), in bridge |
with a specific mac address (do not reuse MAC addresses!), in bridge |
mode. Two disks are provided, and they are both writable; the bits |
mode. Two disks are provided, and they are both writable; the bits |
are stored in files and Xen attaches them to a vnd(4) device in the |
are stored in files and Xen attaches them to a vnd(4) device in the |
dom0 on domain creation. The system treats xbd0 as the boot device |
dom0 on domain creation. The system treats xbd0 as the boot device |
without needing explicit configuration. |
without needing explicit configuration. |
|
|
By default xm looks for domain config files in /usr/pkg/etc/xen. Note |
By default, `xl` looks for domain config files in `/usr/pkg/etc/xen`. Note |
that "xm create" takes the name of a config file, while other commands |
that "xl create" takes the name of a config file, while other commands |
take the name of a domain. To create the domain, connect to the |
take the name of a domain. |
console, create the domain while attaching the console, shutdown the |
|
domain, and see if it has finished stopping, do (or xl with Xen >= |
Examples of commands: |
4.2): |
|
|
|
xm create foo |
|
xm console foo |
|
xm create -c foo |
|
xm shutdown foo |
|
xm list |
|
|
|
Typing ^] will exit the console session. Shutting down a domain is |
[[!template id=programlisting text=""" |
|
xl create foo |
|
xl console foo |
|
xl create -c foo |
|
xl shutdown foo |
|
xl list |
|
"""]] |
|
|
|
Typing `^]` will exit the console session. Shutting down a domain is |
equivalent to pushing the power button; a NetBSD domU will receive a |
equivalent to pushing the power button; a NetBSD domU will receive a |
power-press event and do a clean shutdown. Shutting down the dom0 |
power-press event and do a clean shutdown. Shutting down the dom0 |
will trigger controlled shutdowns of all configured domUs. |
will trigger controlled shutdowns of all configured domUs. |
Line 504 loaders finds and loads a kernel. Norma
|
Line 367 loaders finds and loads a kernel. Norma
|
file system. With Xen domUs, the process is totally different. The |
file system. With Xen domUs, the process is totally different. The |
normal path is for the domU kernel to be a file in the dom0's |
normal path is for the domU kernel to be a file in the dom0's |
file system. At the request of the dom0, Xen loads that kernel into a |
file system. At the request of the dom0, Xen loads that kernel into a |
new domU instance and starts execution. While domU kernels can be |
new domU instance and starts execution. domU kernels can be anyplace. |
anyplace, reasonable places to store domU kernels on the dom0 are in / |
|
(so they are near the dom0 kernel), in /usr/pkg/etc/xen (near the |
|
config files), or in /u0/xen (where the vdisks are). |
|
|
|
Note that loading the domU kernel from the dom0 implies that boot |
Note that loading the domU kernel from the dom0 implies that boot |
blocks, /boot, /boot.cfg, and so on are all ignored in the domU. |
blocks, /boot, /boot.cfg, and so on are all ignored in the domU. |
Line 518 CPU and memory
|
Line 378 CPU and memory
|
-------------- |
-------------- |
|
|
A domain is provided with some number of vcpus, less than the number |
A domain is provided with some number of vcpus, less than the number |
of CPUs seen by the hypervisor. (For a dom0, this is controlled by |
of CPUs seen by the hypervisor. For a domU, it is controlled |
the boot argument "dom0_max_vcpus=1".) For a domU, it is controlled |
|
from the config file by the "vcpus = N" directive. |
from the config file by the "vcpus = N" directive. |
|
|
A domain is provided with memory; this is controlled in the config |
A domain is provided with memory; this is controlled in the config |
Line 528 sum of the the memory allocated to the d
|
Line 387 sum of the the memory allocated to the d
|
than the available memory. |
than the available memory. |
|
|
Xen also provides a "balloon" driver, which can be used to let domains |
Xen also provides a "balloon" driver, which can be used to let domains |
use more memory temporarily. TODO: Explain better, and explain how |
use more memory temporarily. |
well it works with NetBSD. |
|
|
|
Virtual disks |
Virtual disks |
------------- |
------------- |
Line 605 interfaces to the bridge. One specifies
|
Line 463 interfaces to the bridge. One specifies
|
config file. The bridge must be set up already in the dom0; an |
config file. The bridge must be set up already in the dom0; an |
example /etc/ifconfig.bridge0 is: |
example /etc/ifconfig.bridge0 is: |
|
|
create |
[[!template id=filecontent name="/etc/ifconfig.bridge0" text=""" |
up |
create |
!brconfig bridge0 add wm0 |
up |
|
!brconfig bridge0 add wm0 |
|
"""]] |
|
|
With NAT, the domU perceives itself to be behind a NAT running on the |
With NAT, the domU perceives itself to be behind a NAT running on the |
dom0. This is often appropriate when running Xen on a workstation. |
dom0. This is often appropriate when running Xen on a workstation. |
Line 908 Amazon
|
Line 768 Amazon
|
|
|
See the [Amazon EC2 page](/amazon_ec2/). |
See the [Amazon EC2 page](/amazon_ec2/). |
|
|
TODO items for improving NetBSD/xen |
|
=================================== |
|
|
|
* Make the NetBSD dom0 kernel work with SMP. |
|
* Test the Xen 4.5 packages adequately to be able to recommend them as |
|
the standard approach. |
|
* Get PCI passthrough working on Xen 4.5 |
|
* Get pvgrub into pkgsrc, either via xentools or separately. |
|
* grub |
|
* Check/add support to pkgsrc grub2 for UFS2 and arbitrary |
|
fragsize/blocksize (UFS2 support may be present; the point is to |
|
make it so that with any UFS1/UFS2 file system setup that works |
|
with NetBSD grub will also work). |
|
See [pkg/40258](https://gnats.netbsd.org/40258). |
|
* Push patches upstream. |
|
* Get UFS2 patches into pvgrub. |
|
* Add support for PV ops to a version of /boot, and make it usable as |
|
a kernel in Xen, similar to pvgrub. |
|
|
|
Random pointers |
Random pointers |
=============== |
=============== |
|
|