[BACK]Return to howto.mdwn CVS log [TXT] [DIR] Up to [NetBSD Developer Wiki] / wikisrc / ports / xen

Diff for /wikisrc/ports/xen/howto.mdwn between versions 1.110 and 1.111

version 1.110, 2016/12/20 04:09:54 version 1.111, 2016/12/20 08:18:01
Line 30  style is less efficient but can run unmo Line 30  style is less efficient but can run unmo
 Generally any amd64 machine will work with Xen and PV guests.  In  Generally any amd64 machine will work with Xen and PV guests.  In
 theory i386 computers without amd64 support can be used for Xen <=  theory i386 computers without amd64 support can be used for Xen <=
 4.2, but we have no recent reports of this working (this is a hint).  4.2, but we have no recent reports of this working (this is a hint).
 For HVM guests, the VT or VMX cpu feature (Intel) or SVM/HVM/VT  For HVM guests, the VT or VMX CPU feature (Intel) or SVM/HVM/VT
 (amd64) is needed; "cpuctl identify 0" will show this.  TODO: Clean up  (amd64) is needed; "cpuctl identify 0" will show this.  TODO: Clean up
 and check the above features.  and check the above features.
   
Line 49  specific PCI devices can be made availab Line 49  specific PCI devices can be made availab
 of the dom0.  This can be useful to let a domU run X11, or access some  of the dom0.  This can be useful to let a domU run X11, or access some
 network interface or other peripheral.  network interface or other peripheral.
   
 NetBSD 6 and earlier supported Xen 2; support was removed fro NetBSD  NetBSD 6 and earlier supported Xen 2; support was removed from NetBSD
 7.  Xen 2 has been removed from pkgsrc.  7.  Xen 2 has been removed from pkgsrc.
   
 Prerequisites  Prerequisites
Line 77  versions of NetBSD are intentionally ign Line 77  versions of NetBSD are intentionally ign
 Xen  Xen
 ---  ---
   
 In NetBSD, xen is provided in pkgsrc, via matching pairs of packages  In NetBSD, Xen is provided in pkgsrc, via matching pairs of packages
 xenkernel and xentools.  We will refer only to the kernel versions,  xenkernel and xentools.  We will refer only to the kernel versions,
 but note that both packages must be installed together and must have  but note that both packages must be installed together and must have
 matching versions.  matching versions.
Line 158  run Xen today supports amd64.) Line 158  run Xen today supports amd64.)
   
 Xen, the dom0 kernel, and each domU kernel can be either i386 or  Xen, the dom0 kernel, and each domU kernel can be either i386 or
 amd64.  When building a xenkernel package, one obtains i386 on an i386  amd64.  When building a xenkernel package, one obtains i386 on an i386
 host, and amd64 on an amd64 host.  If the xen kernel is i386, then the  host, and amd64 on an amd64 host.  If the Xen kernel is i386, then the
 dom0 kernel and all domU kernels must be i386.  With an amd64 xen  dom0 kernel and all domU kernels must be i386.  With an amd64 Xen
 kernel, an amd64 dom0 kernel is known to work, and an i386PAE dom0  kernel, an amd64 dom0 kernel is known to work, and an i386PAE dom0
 kernel should in theory work.  An amd64 xen/dom0 is known to support  kernel should in theory work.  An amd64 Xen/dom0 is known to support
 both i386PAE and amd64 domUs.  both i386PAE and amd64 domUs.
   
 i386 dom0 and domU kernels must be PAE (except for Xen 3.1); these are  i386 dom0 and domU kernels must be PAE (except for Xen 3.1); these are
Line 173  Because of the above, the standard appro Line 173  Because of the above, the standard appro
 dom0.  dom0.
   
 Xen 4.2 is the last version to support i386 as a host.  TODO: Clarify  Xen 4.2 is the last version to support i386 as a host.  TODO: Clarify
 if this is about the CPU, the xen kernel, or the dom0 kernel having to  if this is about the CPU, the Xen kernel, or the dom0 kernel having to
 be amd64.  be amd64.
   
   
Line 284  However, the partitioning approach is ve Line 284  However, the partitioning approach is ve
 If you want to use RAIDframe for the dom0, there are no special issues  If you want to use RAIDframe for the dom0, there are no special issues
 for Xen.  Typically one provides RAID storage for the dom0, and the  for Xen.  Typically one provides RAID storage for the dom0, and the
 domU systems are unaware of RAID.  The 2nd-stage loader bootxx_* skips  domU systems are unaware of RAID.  The 2nd-stage loader bootxx_* skips
 over a RAID1 header to find /boot from a filesystem within a RAID  over a RAID1 header to find /boot from a file system within a RAID
 partition; this is no different when booting Xen.  partition; this is no different when booting Xen.
   
 There are 4 styles of providing backing storage for the virtual disks  There are 4 styles of providing backing storage for the virtual disks
Line 300  for domU disks.  This is almost as effic Line 300  for domU disks.  This is almost as effic
 and more flexible.  Hence raw disk partitions should typically not  and more flexible.  Hence raw disk partitions should typically not
 be used.  be used.
   
 One can use files in the dom0 filesystem, typically created by dd'ing  One can use files in the dom0 file system, typically created by dd'ing
 /dev/zero to create a specific size.  This is somewhat less efficient,  /dev/zero to create a specific size.  This is somewhat less efficient,
 but very convenient, as one can cp the files for backup, or move them  but very convenient, as one can cp the files for backup, or move them
 between dom0 hosts.  between dom0 hosts.
Line 333  releasedir/i386/binary/kernel/netbsd-XEN Line 333  releasedir/i386/binary/kernel/netbsd-XEN
 should not use Xen 3.1.)  Both xen and the NetBSD kernel may be (and  should not use Xen 3.1.)  Both xen and the NetBSD kernel may be (and
 typically are) left compressed.  typically are) left compressed.
   
 In a dom0 kernel, kernfs is mandatory for xend to comunicate with the  In a dom0 kernel, kernfs is mandatory for xend to communicate with the
 kernel, so ensure that /kern is in fstab.  TODO: Say this is default,  kernel, so ensure that /kern is in fstab.  TODO: Say this is default,
 or file a PR and give a reference.  or file a PR and give a reference.
   
 Because you already installed NetBSD, you have a working boot setup  Because you already installed NetBSD, you have a working boot setup
 with an MBR bootblock, either bootxx_ffsv1 or bootxx_ffsv2 at the  with an MBR bootblock, either bootxx_ffsv1 or bootxx_ffsv2 at the
 beginning of your root filesystem, /boot present, and likely  beginning of your root file system, /boot present, and likely
 /boot.cfg.  (If not, fix before continuing!)  /boot.cfg.  (If not, fix before continuing!)
   
 Add a line to to /boot.cfg to boot Xen.  See boot.cfg(5) for an  Add a line to to /boot.cfg to boot Xen.  See boot.cfg(5) for an
Line 359  starting at 0).  In an attempt to add pe Line 359  starting at 0).  In an attempt to add pe
         dom0_max_vcpus=1 dom0_vcpus_pin          dom0_max_vcpus=1 dom0_vcpus_pin
   
 to force only one vcpu to be provided (since NetBSD dom0 can't use  to force only one vcpu to be provided (since NetBSD dom0 can't use
 more) and to pin that vcpu to a physical cpu.  TODO: benchmark this.  more) and to pin that vcpu to a physical CPU.  TODO: benchmark this.
   
 Xen has [many boot  Xen has [many boot
 options](http://xenbits.xenproject.org/docs/4.5-testing/misc/xen-command-line.html),  options](http://xenbits.xenproject.org/docs/4.5-testing/misc/xen-command-line.html),
 and other tham dom0 memory and max_vcpus, they are generally not  and other than dom0 memory and max_vcpus, they are generally not
 necessary.  necessary.
   
 As with non-Xen systems, you should have a line to boot /netbsd (a  As with non-Xen systems, you should have a line to boot /netbsd (a
Line 411  note that you should be using 4.x): Line 411  note that you should be using 4.x):
   
 For "xl" (4.x), you should enabled xend and xencommons (xenstored).  For "xl" (4.x), you should enabled xend and xencommons (xenstored).
 Trying to boot 4.x without xencommons=YES will result in a hang; it is  Trying to boot 4.x without xencommons=YES will result in a hang; it is
 necessary to hig ^C on the console to let the machine finish booting.  necessary to hit ^C on the console to let the machine finish booting.
 TODO: explain why xend is installed by the package.  TODO: explain why xend is installed by the package.
   
         xencommons=YES          xencommons=YES
Line 465  Xen-specific NetBSD issues Line 465  Xen-specific NetBSD issues
 There are (at least) two additional things different about NetBSD as a  There are (at least) two additional things different about NetBSD as a
 dom0 kernel compared to hardware.  dom0 kernel compared to hardware.
   
 One is that the module ABI is different because some of the #ddefines  One is that the module ABI is different because some of the #defines
 change, so one must build modules for Xen.  As of netbsd-7, the build  change, so one must build modules for Xen.  As of netbsd-7, the build
 system does this automatically.  TODO: check this.  (Before building  system does this automatically.  TODO: check this.  (Before building
 Xen modules was added, it was awkward to use modules to the point  Xen modules was added, it was awkward to use modules to the point
Line 497  over time.  Before these commands, it wa Line 497  over time.  Before these commands, it wa
 4.1 and grub, much like the message linked earlier in the grub  4.1 and grub, much like the message linked earlier in the grub
 section.  section.
   
         # Install mbr bootblocks on both disks.           # Install MBR bootblocks on both disks. 
         fdisk -i /dev/rwd0d          fdisk -i /dev/rwd0d
         fdisk -i /dev/rwd1d          fdisk -i /dev/rwd1d
         # Install NetBSD primary boot loader (/ is FFSv1) into RAID1 components.          # Install NetBSD primary boot loader (/ is FFSv1) into RAID1 components.
Line 505  section. Line 505  section.
         installboot -v /dev/rwd1d /usr/mdec/bootxx_ffsv1          installboot -v /dev/rwd1d /usr/mdec/bootxx_ffsv1
         # Install secondary boot loader          # Install secondary boot loader
         cp -p /usr/mdec/boot /          cp -p /usr/mdec/boot /
         # Create boog.cfg following earlier guidance:          # Create boot.cfg following earlier guidance:
         menu=Xen:load /netbsd-XEN3PAE_DOM0.gz console=pc;multiboot /xen.gz dom0_mem=256M          menu=Xen:load /netbsd-XEN3PAE_DOM0.gz console=pc;multiboot /xen.gz dom0_mem=256M
         menu=Xen.ok:load /netbsd-XEN3PAE_DOM0.ok.gz console=pc;multiboot /xen.ok.gz dom0_mem=256M          menu=Xen.ok:load /netbsd-XEN3PAE_DOM0.ok.gz console=pc;multiboot /xen.ok.gz dom0_mem=256M
         menu=GENERIC:boot          menu=GENERIC:boot
Line 590  config files for domUs are typically in  Line 590  config files for domUs are typically in 
 typically named so that the file name, domU name and the domU's host  typically named so that the file name, domU name and the domU's host
 name match.  name match.
   
 The domU is provided with cpu and memory by Xen, configured by the  The domU is provided with CPU and memory by Xen, configured by the
 dom0.  The domU is provided with disk and network by the dom0,  dom0.  The domU is provided with disk and network by the dom0,
 mediated by Xen, and configured in the dom0.  mediated by Xen, and configured in the dom0.
   
Line 627  domUs independently.  The vif line cause Line 627  domUs independently.  The vif line cause
 with a specific mac address (do not reuse MAC addresses!), in bridge  with a specific mac address (do not reuse MAC addresses!), in bridge
 mode.  Two disks are provided, and they are both writable; the bits  mode.  Two disks are provided, and they are both writable; the bits
 are stored in files and Xen attaches them to a vnd(4) device in the  are stored in files and Xen attaches them to a vnd(4) device in the
 dom0 on domain creation.  The system treates xbd0 as the boot device  dom0 on domain creation.  The system treats xbd0 as the boot device
 without needing explicit configuration.  without needing explicit configuration.
   
 By default xm looks for domain config files in /usr/pkg/etc/xen.  Note  By default xm looks for domain config files in /usr/pkg/etc/xen.  Note
Line 653  domU kernels Line 653  domU kernels
   
 On a physical computer, the BIOS reads sector 0, and a chain of boot  On a physical computer, the BIOS reads sector 0, and a chain of boot
 loaders finds and loads a kernel.  Normally this comes from the root  loaders finds and loads a kernel.  Normally this comes from the root
 filesystem.  With Xen domUs, the process is totally different.  The  file system.  With Xen domUs, the process is totally different.  The
 normal path is for the domU kernel to be a file in the dom0's  normal path is for the domU kernel to be a file in the dom0's
 filesystem.  At the request of the dom0, Xen loads that kernel into a  file system.  At the request of the dom0, Xen loads that kernel into a
 new domU instance and starts execution.  While domU kernels can be  new domU instance and starts execution.  While domU kernels can be
 anyplace, reasonable places to store domU kernels on the dom0 are in /  anyplace, reasonable places to store domU kernels on the dom0 are in /
 (so they are near the dom0 kernel), in /usr/pkg/etc/xen (near the  (so they are near the dom0 kernel), in /usr/pkg/etc/xen (near the
Line 670  CPU and memory Line 670  CPU and memory
 --------------  --------------
   
 A domain is provided with some number of vcpus, less than the number  A domain is provided with some number of vcpus, less than the number
 of cpus seen by the hypervisor.  (For a dom0, this is controlled by  of CPUs seen by the hypervisor.  (For a dom0, this is controlled by
 the boot argument "dom0_max_vcpus=1".)  For a domU, it is controlled  the boot argument "dom0_max_vcpus=1".)  For a domU, it is controlled
 from the config file by the "vcpus = N" directive.  from the config file by the "vcpus = N" directive.
   
Line 734  disks. Line 734  disks.
 Virtual Networking  Virtual Networking
 ------------------  ------------------
   
 Xen provides virtual ethernets, each of which connects the dom0 and a  Xen provides virtual Ethernets, each of which connects the dom0 and a
 domU.  For each virtual network, there is an interface "xvifN.M" in  domU.  For each virtual network, there is an interface "xvifN.M" in
 the dom0, and in domU index N, a matching interface xennetM (NetBSD  the dom0, and in domU index N, a matching interface xennetM (NetBSD
 name).  The interfaces behave as if there is an Ethernet with two  name).  The interfaces behave as if there is an Ethernet with two
 adaptors connected.  From this primitive, one can construct various  adapters connected.  From this primitive, one can construct various
 configurations.  We focus on two common and useful cases for which  configurations.  We focus on two common and useful cases for which
 there are existing scripts: bridging and NAT.  there are existing scripts: bridging and NAT.
   
Line 782  shutdown, in rc.conf add: Line 782  shutdown, in rc.conf add:
   
         xendomains="foo bar"          xendomains="foo bar"
   
 Note that earlier versions of the xentools41 xendomains rc.d scripth  Note that earlier versions of the xentools41 xendomains rc.d script
 usd xl, when one should use xm with 4.1.  used xl, when one should use xm with 4.1.
   
 Creating specific unprivileged domains (domU)  Creating specific unprivileged domains (domU)
 =============================================  =============================================
Line 799  Creating an unprivileged NetBSD domain ( Line 799  Creating an unprivileged NetBSD domain (
 See the earlier config file, and adjust memory.  Decide on how much  See the earlier config file, and adjust memory.  Decide on how much
 storage you will provide, and prepare it (file or lvm).  storage you will provide, and prepare it (file or lvm).
   
 While the kernel will be obtained from the dom0 filesystem, the same  While the kernel will be obtained from the dom0 file system, the same
 file should be present in the domU as /netbsd so that tools like  file should be present in the domU as /netbsd so that tools like
 savecore(8) can work.   (This is helpful but not necessary.)  savecore(8) can work.   (This is helpful but not necessary.)
   
Line 862  in rc.conf. This way, the domain will be Line 862  in rc.conf. This way, the domain will be
 `xm shutdown -R` or `xm shutdown -H` is used on the dom0.  `xm shutdown -R` or `xm shutdown -H` is used on the dom0.
   
 It is not strictly necessary to have a kernel (as /netbsd) in the domU  It is not strictly necessary to have a kernel (as /netbsd) in the domU
 filesystem.  However, various programs (e.g. netstat) will use that  file system.  However, various programs (e.g. netstat) will use that
 kernel to look up symbols to read from kernel virtual memory.  If  kernel to look up symbols to read from kernel virtual memory.  If
 /netbsd is not the running kernel, those lookups will fail.  (This is  /netbsd is not the running kernel, those lookups will fail.  (This is
 not really a Xen-specific issue, but because the domU kernel is  not really a Xen-specific issue, but because the domU kernel is
Line 901  Then copy the files from a working Linux Line 901  Then copy the files from a working Linux
 `/etc` (fstab, network config).  It should also be possible to extract  `/etc` (fstab, network config).  It should also be possible to extract
 binary packages such as .rpm or .deb directly to the mounted partition  binary packages such as .rpm or .deb directly to the mounted partition
 using the appropriate tool, possibly running under NetBSD's Linux  using the appropriate tool, possibly running under NetBSD's Linux
 emulation.  Once the filesystem has been populated, umount it.  If  emulation.  Once the file system has been populated, umount it.  If
 desirable, the filesystem can be converted to ext3 using tune2fs -j.  desirable, the file system can be converted to ext3 using tune2fs -j.
 It should now be possible to boot the Linux guest domain, using one of  It should now be possible to boot the Linux guest domain, using one of
 the vmlinuz-\*-xenU kernels available in the Xen binary distribution.  the vmlinuz-\*-xenU kernels available in the Xen binary distribution.
   
 To get the linux console right, you need to add:  To get the Linux console right, you need to add:
   
     extra = "xencons=tty1"      extra = "xencons=tty1"
   
 to your configuration since not all linux distributions auto-attach a  to your configuration since not all Linux distributions auto-attach a
 tty to the xen console.  tty to the xen console.
   
 Creating an unprivileged Solaris domain (domU)  Creating an unprivileged Solaris domain (domU)
Line 954  the domU's config file, with the format  Line 954  the domU's config file, with the format 
         pci = [ '0000:00:06.0', '0000:00:0a.0' ]          pci = [ '0000:00:06.0', '0000:00:0a.0' ]
   
 In the domU an "xpci" device will show up, to which one or more pci  In the domU an "xpci" device will show up, to which one or more pci
 busses will attach.  Then the PCI drivers will attach to PCI busses as  buses will attach.  Then the PCI drivers will attach to PCI buses as
 usual.  Note that the default NetBSD DOMU kernels do not have "xpci"  usual.  Note that the default NetBSD DOMU kernels do not have "xpci"
 or any PCI drivers built in by default; you have to build your own  or any PCI drivers built in by default; you have to build your own
 kernel to use PCI devices in a domU.  Here's a kernel config example;  kernel to use PCI devices in a domU.  Here's a kernel config example;
Line 962  note that only the "xpci" lines are unus Line 962  note that only the "xpci" lines are unus
   
         include         "arch/i386/conf/XEN3_DOMU"          include         "arch/i386/conf/XEN3_DOMU"
   
         # Add support for PCI busses to the XEN3_DOMU kernel          # Add support for PCI buses to the XEN3_DOMU kernel
         xpci* at xenbus ?          xpci* at xenbus ?
         pci* at xpci ?          pci* at xpci ?
   
Line 1001  only a few are mentioned that specifical Line 1001  only a few are mentioned that specifical
   
 VPS operators provide varying degrees of access and mechanisms for  VPS operators provide varying degrees of access and mechanisms for
 configuration.  The big issue is usually how one controls which kernel  configuration.  The big issue is usually how one controls which kernel
 is booted, because the kernel is nominally in the dom0 filesystem (to  is booted, because the kernel is nominally in the dom0 file system (to
 which VPS users do not normally have acesss).  A second issue is how  which VPS users do not normally have access).  A second issue is how
 to install NetBSD.  to install NetBSD.
 A VPS user may want to compile a kernel for security updates, to run  A VPS user may want to compile a kernel for security updates, to run
 npf, run IPsec, or any other reason why someone would want to change  npf, run IPsec, or any other reason why someone would want to change
 their kernel.  their kernel.
   
 One approach is to have an adminstrative interface to upload a kernel,  One approach is to have an administrative interface to upload a kernel,
 or to select from a prepopulated list.  Other approaches are pygrub  or to select from a prepopulated list.  Other approaches are pygrub
 (deprecated) and pvgrub, which are ways to have a bootloader obtain a  (deprecated) and pvgrub, which are ways to have a bootloader obtain a
 kernel from the domU filesystem.  This is closer to a regular physical  kernel from the domU file system.  This is closer to a regular physical
 computer, where someone who controls a machine can replace the kernel.  computer, where someone who controls a machine can replace the kernel.
   
 A second issue is multiple CPUs.  With NetBSD 6, domUs support  A second issue is multiple CPUs.  With NetBSD 6, domUs support
Line 1021  CPUs for NetBSD domUs. Line 1021  CPUs for NetBSD domUs.
 pygrub  pygrub
 -------  -------
   
 pygrub runs in the dom0 and looks into the domU filesystem.  This  pygrub runs in the dom0 and looks into the domU file system.  This
 implies that the domU must have a kernel in a filesystem in a format  implies that the domU must have a kernel in a file system in a format
 known to pygrub.  As of 2014, pygrub seems to be of mostly historical  known to pygrub.  As of 2014, pygrub seems to be of mostly historical
 interest.  interest.
   
Line 1031  pvgrub Line 1031  pvgrub
   
 pvgrub is a version of grub that uses PV operations instead of BIOS  pvgrub is a version of grub that uses PV operations instead of BIOS
 calls.  It is booted from the dom0 as the domU kernel, and then reads  calls.  It is booted from the dom0 as the domU kernel, and then reads
 /grub/menu.lst and loads a kernel from the domU filesystem.  /grub/menu.lst and loads a kernel from the domU file system.
   
 [Panix](http://www.panix.com/) lets users use pvgrub.  Panix reports  [Panix](http://www.panix.com/) lets users use pvgrub.  Panix reports
 that pvgrub works with FFsv2 with 16K/2K and 32K/4K block/frag sizes  that pvgrub works with FFsv2 with 16K/2K and 32K/4K block/frag sizes
Line 1050  does not support all aspects of modern F Line 1050  does not support all aspects of modern F
 that FFSv2 works fine.  At prgmr, typically one has an ext2 or FAT  that FFSv2 works fine.  At prgmr, typically one has an ext2 or FAT
 partition for the kernel with the intent that grub can understand it,  partition for the kernel with the intent that grub can understand it,
 which leads to /netbsd not being the actual kernel.  One must remember  which leads to /netbsd not being the actual kernel.  One must remember
 to update the special boot partiion.  to update the special boot partition.
   
 Amazon  Amazon
 ------  ------
Line 1079  TODO items for improving NetBSD/xen Line 1079  TODO items for improving NetBSD/xen
 * grub  * grub
   * Check/add support to pkgsrc grub2 for UFS2 and arbitrary    * Check/add support to pkgsrc grub2 for UFS2 and arbitrary
     fragsize/blocksize (UFS2 support may be present; the point is to      fragsize/blocksize (UFS2 support may be present; the point is to
     make it so that with any UFS1/UFS2 filesystem setup that works      make it so that with any UFS1/UFS2 file system setup that works
     with NetBSD grub will also work).      with NetBSD grub will also work).
     See [pkg/40258](http://gnats.netbsd.org/40258).      See [pkg/40258](http://gnats.netbsd.org/40258).
   * Push patches upstream.    * Push patches upstream.
Removed from v.1.110  
changed lines
  Added in v.1.111

CVSweb for NetBSD wikisrc <wikimaster@NetBSD.org> software: FreeBSD-CVSweb