--- wikisrc/ports/xen/howto.mdwn 2014/12/26 13:15:32 1.43 +++ wikisrc/ports/xen/howto.mdwn 2014/12/26 20:00:44 1.48 @@ -181,6 +181,10 @@ NetBSD, which is not yet a dom0, and the NetBSD install to a dom0 install by just changing the kernel and boot configuration. +For experimenting with Xen, a machine with as little as 1G of RAM and +100G of disk can work. For running many domUs in productions, far +more will be needed. + Styles of dom0 operation ------------------------ @@ -421,16 +425,84 @@ mediated by Xen, and configured in the d Entropy in domUs can be an issue; physical disks and network are on the dom0. NetBSD's /dev/random system works, but is often challenged. +Config files +------------ + +There is no good order to present config files and the concepts +surrounding what is being configured. We first show an example config +file, and then in the various sections give details. + +See (at least in xentools41) /usr/pkg/share/examples/xen/xmexample*, +for a large number of well-commented examples, mostly for running +GNU/Linux. + +The following is an example minimal domain configuration file +"/usr/pkg/etc/xen/foo". It is (with only a name change) an actual +known working config file on Xen 4.1 (NetBSD 5 amd64 dom0 and NetBSD 5 +i386 domU). The domU serves as a network file server. + + # -*- mode: python; -*- + + kernel = "/netbsd-XEN3PAE_DOMU-i386-foo.gz" + memory = 1024 + vif = [ 'mac=aa:00:00:d1:00:09,bridge=bridge0' ] + disk = [ 'file:/n0/xen/foo-wd0,0x0,w', + 'file:/n0/xen/foo-wd1,0x1,w' ] + +The domain will have the same name as the file. The kernel has the +host/domU name in it, so that on the dom0 one can update the various +domUs independently. The vif line causes an interface to be provided, +with a specific mac address (do not reuse MAC addresses!), in bridge +mode. Two disks are provided, and they are both writable; the bits +are stored in files and Xen attaches them to a vnd(4) device in the +dom0 on domain creation. The system treates xbd0 as the boot device +without needing explicit configuration. + +By default xm looks for domain config files in /usr/pkg/etc/xen. Note +that "xm create" takes the name of a config file, while other commands +take the name of a domain. To create the domain, connect to the +console, create the domain while attaching the console, shutdown the +domain, and see if it has finished stopping, do (or xl with Xen >= +4.2): + + xm create foo + xm console foo + xm create -c foo + xm shutdown foo + xm list + +Typing ^] will exit the console session. Shutting down a domain is +equivalent to pushing the power button; a NetBSD domU will receive a +power-press event and do a clean shutdown. Shutting down the dom0 +will trigger controlled shutdowns of all configured domUs. + +domU kernels +------------ + +On a physical computer, the BIOS reads sector 0, and a chain of boot +loaders finds and loads a kernel. Normally this comes from the root +filesystem. With Xen domUs, the process is totally different. The +normal path is for the domU kernel to be a file in the dom0's +filesystem. At the request of the dom0, Xen loads that kernel into a +new domU instance and starts execution. While domU kernels can be +anyplace, reasonable places to store domU kernels on the dom0 are in / +(so they are near the dom0 kernel), in /usr/pkg/etc/xen (near the +config files), or in /u0/xen (where the vdisks are). + +See the VPS section near the end for discussion of alternate ways to +obtain domU kernels. + CPU and memory -------------- -A domain is provided with some number of vcpus, less than the -number of cpus seen by the hypervisor. For a dom0, this is controlled -by the boot argument "dom0_max_vcpus=1". For a domU, it is controlled -from the config file. - -A domain is provided with memory, In the straightforward case, the sum -of the the memory allocated to the dom0 and all domUs must be less +A domain is provided with some number of vcpus, less than the number +of cpus seen by the hypervisor. (For a dom0, this is controlled by +the boot argument "dom0_max_vcpus=1".) For a domU, it is controlled +from the config file by the "vcpus = N" directive. + +A domain is provided with memory; this is controlled in the config +file by "memory = N" (in megabytes). In the straightforward case, the +sum of the the memory allocated to the dom0 and all domUs must be less than the available memory. Xen also provides a "balloon" driver, which can be used to let domains @@ -456,23 +528,55 @@ create an empty 4G virtual disk, simply dd if=/dev/zero of=foo-xbd0 bs=1m count=4096 With the lvm style, one creates logical devices. They are then used -similarly to vnds. +similarly to vnds. TODO: Add an example with lvm. -Virtual Networking ------------------- +In domU config files, the disks are defined as a sequence of 3-tuples. +The first element is "method:/path/to/disk". Common methods are +"file:" for file-backed vnd. and "phy:" for something that is already +a (TODO: character or block) device. + +The second element is an artifact of how virtual disks are passed to +Linux, and a source of confusion with NetBSD Xen usage. Linux domUs +are given a device name to associate with the disk, and values like +"hda1" or "sda1" are common. In a NetBSD domU, the first disk appears +as xbd0, the second as xbd1, and so on. However, xm/xl demand a +second argument. The name given is converted to a major/minor by +consulting /dev and this is passed to the domU (TODO: check this). In +the general case, the dom0 and domU can be different operating +systems, and it is an unwarranted assumption that they have consistent +numbering in /dev, or even that the dom0 OS has a /dev. With NetBSD +as both dom0 and domU, using values of 0x0 for the first disk and 0x1 +for the second works fine and avoids this issue. -TODO: explain xvif concept, and that it's general. +The third element is "w" for writable disks, and "r" for read-only +disks. -There are two normal styles: bridging and NAT. +Virtual Networking +------------------ -With bridging, the domU perceives itself to be on the same network as -the dom0. For server virtualization, this is usually best. +Xen provides virtual ethernets, each of which connects the dom0 and a +domU. For each virtual network, there is an interface "xvifN.M" in +the dom0, and in domU index N, a matching interface xennetM (NetBSD +name). The interfaces behave as if there is an Ethernet with two +adaptors connected. From this primitive, one can construct various +configurations. We focus on two common and useful cases for which +there are existing scripts: bridging and NAT. + +With bridging (in the example above), the domU perceives itself to be +on the same network as the dom0. For server virtualization, this is +usually best. Bridging is accomplished by creating a bridge(4) device +and adding the dom0's physical interface and the various xvifN.0 +interfaces to the bridge. One specifies "bridge=bridge0" in the domU +config file. The bridge must be set up already in the dom0; an +example /etc/ifconfig.bridge0 is: + + create + up + !brconfig bridge0 add wm0 With NAT, the domU perceives itself to be behind a NAT running on the dom0. This is often appropriate when running Xen on a workstation. - -One can construct arbitrary other configurations, but there is no -script support. +TODO: NAT appears to be configured by "vif = [ '' ]". Sizing domains -------------- @@ -486,36 +590,16 @@ create a new file and vnconfig it (or lv just like updating physical disks, but without having to be there and without those pesky connectors. -domU kernels ------------- +Starting domains automatically +------------------------------ -On a physical computer, the BIOS reads sector 0, and a chain of boot -loaders finds and loads a kernel. Normally this comes from the root -filesystem. With Xen domUs, the process is totally different. The -normal path is for the domU kernel to be a file in the dom0's -filesystem. At the request of the dom0, Xen loads that kernel into a -new domU instance and starts execution. While domU kernels can be -anyplace, reasonable places to store domU kernels on the dom0 are in / -(so they are near the dom0 kernel), in /usr/pkg/etc/xen (near the -config files), or in /u0/xen (where the vdisks are). +To start domains foo at bar at boot and shut them down cleanly on dom0 +shutdown, in rc.conf add: -See the VPS section near the end for discussion of alternate ways to -obtain domU kernels. - -Config files ------------- - -TODO: give example config files. Use both lvm and vnd. - -TODO: explain the mess with 3 arguments for disks and how to cope (0x1). - -Starting domains ----------------- - -TODO: Explain "xm start" and "xl start". Explain rc.d/xendomains. + xendomains="foo bar" TODO: Explain why 4.1 rc.d/xendomains has xl, when one should use xm -on 4.1. +on 4.1. Or fix the xentools41 package to have xm Creating specific unprivileged domains (domU) ============================================= @@ -651,7 +735,7 @@ working vif-bridge is also provided with #!/bin/sh #============================================================================ - # $NetBSD: howto.mdwn,v 1.42 2014/12/26 13:10:00 gdt Exp $ + # $NetBSD: howto.mdwn,v 1.47 2014/12/26 18:35:45 gdt Exp $ # # /usr/pkg/etc/xen/vif-bridge # @@ -1041,3 +1125,12 @@ TODO: Perhaps reference panix, prmgr, am TODO: Somewhere, discuss pvgrub and py-grub to load the domU kernel from the domU filesystem. + +Using npf +--------- + +In standard kernels, npf is a module, and thus cannot be loadeed in a +DOMU kernel. + +TODO: explain how to compile npf into a custom kernel, answering: +http://mail-index.netbsd.org/netbsd-users/2014/12/26/msg015576.html