Diff for /wikisrc/ports/xen/howto.mdwn between versions 1.113 and 1.132

version 1.113, 2016/12/20 13:09:58 version 1.132, 2016/12/20 19:59:38
Line 27  code for Xen and need not be aware that  Line 27  code for Xen and need not be aware that 
 Attempts to access hardware registers are trapped and emulated.  This  Attempts to access hardware registers are trapped and emulated.  This
 style is less efficient but can run unmodified guests.  style is less efficient but can run unmodified guests.
   
 Generally any amd64 machine will work with Xen and PV guests.  In  Generally any machine that runs NetBSD/amd64 will work with Xen and PV
 theory i386 computers without amd64 support can be used for Xen <=  guests.  In theory i386 computers (without x86_64/amd64 support) can
 4.2, but we have no recent reports of this working (this is a hint).  be used for Xen <= 4.2, but we have no recent reports of this working
 For HVM guests, the VT or VMX CPU feature (Intel) or SVM/HVM/VT  (this is a hint).  For HVM guests, hardware support is needed, but it
 (amd64) is needed; "cpuctl identify 0" will show this.  TODO: Clean up  is common on recent machines.  For Intel CPUs, one needs the VT-x
 and check the above features.  extension, shown in "cpuctl identify 0" as VMX.  For AMD CPUs, one
   needs the AMD-V extensions, shown in "cpuctl identify 0" as SVM.
   There are further features for IOMMU virtualization, Intel's VT-d and
   AMD's AMD-Vi.  TODO: Explain whether Xen on NetBSD makes use of these
   features.  TODO: Review by someone who really understands this.
   
   Note that a FreeBSD dom0 requires VT-x and VT-d (or equivalent); this
   is because the FreeBSD dom0 does not run in PV mode.
   
 At boot, the dom0 kernel is loaded as a module with Xen as the kernel.  At boot, the dom0 kernel is loaded as a module with Xen as the kernel.
 The dom0 can start one or more domUs.  (Booting is explained in detail  The dom0 can start one or more domUs.  (Booting is explained in detail
Line 44  attempts to address both the case of run Line 51  attempts to address both the case of run
 and running domUs under it (NetBSD and other), and also running NetBSD  and running domUs under it (NetBSD and other), and also running NetBSD
 as a domU in a VPS.  as a domU in a VPS.
   
 Some versions of Xen support "PCI passthrough", which means that  Xen 3.1 in pkgsrc supports "PCI passthrough", which means that
 specific PCI devices can be made available to a specific domU instead  specific PCI devices can be made available to a specific domU instead
 of the dom0.  This can be useful to let a domU run X11, or access some  of the dom0.  This can be useful to let a domU run X11, or access some
 network interface or other peripheral.  network interface or other peripheral.
Line 62  things must be done, guiding the reader  Line 69  things must be done, guiding the reader 
 path when there are no known good reasons to stray.  path when there are no known good reasons to stray.
   
 This HOWTO presumes a basic familiarity with the Xen system  This HOWTO presumes a basic familiarity with the Xen system
 architecture.  This HOWTO presumes familiarity with installing NetBSD  architecture, with installing NetBSD on i386/amd64 hardware, and with
 on i386/amd64 hardware and installing software from pkgsrc.  installing software from pkgsrc.  See also the [Xen
 See also the [Xen website](http://www.xenproject.org/).  website](http://www.xenproject.org/).
   
 Versions of Xen and NetBSD  Versions of Xen and NetBSD
 ==========================  ==========================
Line 103  i386, but only in PAE mode.  The only re Line 110  i386, but only in PAE mode.  The only re
 need to use xm instead of xl, or if you need to run an i386 dom0  need to use xm instead of xl, or if you need to run an i386 dom0
 (because your hardware is i386 only).  (because your hardware is i386 only).
   
 xenkernel45 provides Xen 4.5.  It is no longer maintained by Xen, but  xenkernel45 provides Xen 4.5.  As of 2016-12, security patches were
 as of 2016-12 it received security patches.  Xen 4.5 requires an amd64  released by Xen and applied to pkgsrc.  Xen 4.5 requires using a dom0
 dom0, but domUs can be amd64 or i386 PAE.  TODO: It is either a  running NetBSD's amd64 port (Intel or AMD hardware is fine), but domUs
 conservative choice or somewhat old.  can be amd64 or i386 PAE.  While slightly old, n4.5 has been tested and
   run by others, so it is the conservative choice.
 xenkernel45 provides Xen 4.6.  It is new to pkgsrc in 2016-05.  It is  
 no longer maintained by Xen, but as of 2016-12 it received security  xenkernel46 provides Xen 4.6.  It is new to pkgsrc as of 2016-05.  As
 patches.  Xen 4.6 requires an amd64 dom0, but domUs can be amd64 or  of 2016-12, security patches were released by Xen and applied to
 i386 PAE.  TODO: It is either a somewhat aggressive choice or the  pkgsrc.  Xen 4.6 similarly requires a NetBSD/amd64 dom0, but domUs can
 standard choice  be amd64 or i386 PAE.  For new installations, 4.6 is probably the
   appropriate choice and it will likely soon be the standard approach.
   
 Xen 4.7 and 4.8 are not in pkgsrc.  Xen 4.7 (released 2016-06) and 4.8 (released 2016-12) are not yet in
   pkgsrc.
   
 See also the [Xen Security Advisory page](http://xenbits.xen.org/xsa/).  See also the [Xen Security Advisory page](http://xenbits.xen.org/xsa/).
   
Line 127  Xen command program Line 136  Xen command program
 Early Xen used a program called xm to manipulate the system from the  Early Xen used a program called xm to manipulate the system from the
 dom0.  Starting in 4.1, a replacement program with similar behavior  dom0.  Starting in 4.1, a replacement program with similar behavior
 called xl is provided, but it does not work well in 4.1.  In 4.2, both  called xl is provided, but it does not work well in 4.1.  In 4.2, both
 xm and xl work fine.  4.4 is the last version that has xm.  You must  xm and xl work fine.  4.4 is the last version that has xm.
 choose one or the other, because it affects which daemons you run.  
 However, the rc.d scripts provided by xentools packages expect a  You must make a global choice to use xm or xl, because it affects not
 particular version, and you should use the version used by the  only which command you use, but the command used by rc.d scripts
 scripts.  (specifically xendomains) and which daemons should be run.  The
   xentools packages provide xm for 3.1, 3.3 and 4.1 and xl for 4.2 and up.
   
   In 4.2, you can choose to use xm by simply changing the ctl_command
   variable.
   
   With xl, virtual devices are configured in parallel, which can cause
   problems if they are written assuming serial operation (e.g., updating
   firewall rules without explicit locking).  There is now locking for
   the provided scripts, which works for normal casses (e.g, file-backed
   xbd, where a vnd must be allocated).  But, as of 201612, it has not
   been adequately tested for a complex custom setup with a large number
   of interfaces.
   
 NetBSD  NetBSD
 ------  ------
Line 139  NetBSD Line 160  NetBSD
 The netbsd-6, netbsd-7, and -current branches are all reasonable  The netbsd-6, netbsd-7, and -current branches are all reasonable
 choices, with more or less the same considerations for non-Xen use.  choices, with more or less the same considerations for non-Xen use.
 Therefore, netbsd-7 is recommended as the stable version of the most  Therefore, netbsd-7 is recommended as the stable version of the most
 recent release for production use.  For those wanting to learn Xen or  recent release for production use.  In addition, netbsd-7 and -current
 without production stability concerns, netbsd-7 is still likely most  have a important scheduler fix (in November of 2015) affecting
 appropriate, but -current is also a reasonable choice.  Xen runs fine  contention between dom0 and domUs; see
 on netbsd-5, but the xentools packages are likely difficult to build.  https://releng.netbsd.org/cgi-bin/req-7.cgi?show=1040 for a
   description.  For those wanting to learn Xen or without production
   stability concerns, netbsd-7 is still likely most appropriate, but
   -current is also a reasonable choice.  (Xen runs ok on netbsd-5, but
   the xentools packages are likely difficult to build, and netbsd-5 is
   not supported.)
   
 As of NetBSD 6, a NetBSD domU will support multiple vcpus.  There is  As of NetBSD 6, a NetBSD domU will support multiple vcpus.  There is
 no SMP support for NetBSD as dom0.  (The dom0 itself doesn't really  no SMP support for NetBSD as dom0.  (The dom0 itself doesn't really
Line 198  xl, the NetBSD 7 stable branch, and to u Line 224  xl, the NetBSD 7 stable branch, and to u
 dom0.  Either the i386PAE or amd64 version of NetBSD may be used as  dom0.  Either the i386PAE or amd64 version of NetBSD may be used as
 domUs.  domUs.
   
 Build problems  Status
 --------------  ------
   
 Ideally, all versions of Xen in pkgsrc would build on all versions of  
 NetBSD on both i386 and amd64.  However, that isn't the case.  Besides  
 aging code and aging compilers, qemu (included in xentools for HVM  
 support) is difficult to build.  The following are known to work or FAIL:  
   
         xenkernel3 netbsd-5 amd64  
         xentools3 netbsd-5 amd64  
         xentools3=hvm netbsd-5 amd64 ????  
         xenkernel33 netbsd-5 amd64  
         xentools33 netbsd-5 amd64  
         xenkernel41 netbsd-5 amd64  
         xentools41 netbsd-5 amd64  
         xenkernel42 netbsd-5 amd64  
         xentools42 netbsd-5 amd64  
   
         xenkernel3 netbsd-6 i386 FAIL  
         xentools3 netbsd-6 i386  
         xentools3-hvm netbsd-6 i386 FAIL (dependencies fail)  
         xenkernel33 netbsd-6 i386  
         xentools33 netbsd-6 i386  
         xenkernel41 netbsd-6 i386  
         xentools41 netbsd-6 i386  
         xenkernel42 netbsd-6 i386  
         xentools42 netbsd-6 i386 *MIXED  
   
         (all 3 and 33 seem to FAIL)  
         xenkernel41 netbsd-7 i386  
         xentools41 netbsd-7 i386  
         xenkernel42 netbsd-7 i386  
         xentools42 netbsd-7 i386 ??FAIL  
   
 (*On netbsd-6 i386, there is a xentools42 in the 2014Q3 official builds,  Ideally, all versions of Xen in pkgsrc would build on all supported
 but it does not build for gdt.)  versions of NetBSD/amd64, to the point where this section would be
   silly.  However, that has not always been the case.  Besides aging
   code and aging compilers, qemu (included in xentools for HVM support)
   is difficult to build.  Note that there is intentionally no data for
   4.5+ up for i386, and often omits xentools info if the corresponding
   kernel fails.
   
   The following table gives status, with the date last checked
   (generally on the most recent quarterly branch).  The first code is
   "builds" if it builds ok, and "FAIL" for a failure to build.  The
   second code/date only appears for xenkernel* and is "works" if it runs
   ok as a dom0, and "FAIL" if it won't boot or run a domU.
   
           xenkernel3 netbsd-6 i386 FAIL 201612
           xenkernel33 netbsd-6 i386 FAIL 201612
           xenkernel41 netbsd-6 i386 builds 201612
           xenkernel42 netbsd-6 i386 builds 201612
           xentools3 netbsd-6 i386 FAIL 201612
           xentools33 netbsd-6 i386 FAIL 201612
           xentools41 netbsd-6 i386 builds 201612
           xentools42 netbsd-6 i386 FAIL 201612
   
           xenkernel3 netbsd-7 i386 FAIL 201412
           xenkernel33 netbsd-7 i386 FAIL 201412
           xenkernel41 netbsd-7 i386 builds 201412
           xenkernel42 netbsd-7 i386 builds 201412
           xentools41 netbsd-7 i386 builds 201412
           xentools42 netbsd-7 i386 ??FAIL 201412
   
           xenkernel3 netbsd-6 amd64 FAIL 201612
           xenkernel33 netbsd-6 amd64 FAIL 201612
           xenkernel41 netbsd-6 amd64 builds 201612 works 201612
           xenkernel42 netbsd-6 amd64 builds 201612 works 201612
           xenkernel45 netbsd-6 amd64 builds 201612
           xenkernel46 netbsd-6 amd64 builds 201612
           xentools41 netbsd-6 amd64 builds 201612
           xentools42 netbsd-6 amd64 builds 201612
           xentools45 netbsd-6 amd64 builds 201612
           xentools46 netbsd-6 amd64 FAIL 201612
   
           xenkernel3 netbsd-7 amd64 builds 201612
           xenkernel33 netbsd-7 amd64 builds 201612
           xenkernel41 netbsd-7 amd64 builds 201612
           xenkernel42 netbsd-7 amd64 builds 201612
           xenkernel45 netbsd-7 amd64 builds 201612
           xenkernel46 netbsd-7 amd64 builds 201612
           xentools3 netbsd-7 amd64 builds 201612
           xentools3-hvm netbsd-7 amd64 builds 201612
           xentools33 netbsd-7 amd64 FAIL 201612
           xentools41 netbsd-7 amd64 builds 201612
           xentools42 netbsd-7 amd64 builds 201612
           xentools45 netbsd-7 amd64 builds 201612
           xentools46 netbsd-7 amd64 builds 201612
   
 NetBSD as a dom0  NetBSD as a dom0
 ================  ================
Line 313  Installation of Xen Line 360  Installation of Xen
 -------------------  -------------------
   
 In the dom0, install sysutils/xenkernel42 and sysutils/xentools42 from  In the dom0, install sysutils/xenkernel42 and sysutils/xentools42 from
 pkgsrc (or another matching pair).  pkgsrc (or another matching pair).  See [the pkgsrc
 See [the pkgsrc  documentation](http://www.NetBSD.org/docs/pkgsrc/) for help with
 documentation](http://www.NetBSD.org/docs/pkgsrc/) for help with pkgsrc.  pkgsrc.  Ensure that your packages are recent; the HOWTO does not
   contemplate old builds.
   
   
 For Xen 3.1, support for HVM guests is in sysutils/xentool3-hvm.  More  For Xen 3.1, support for HVM guests is in sysutils/xentool3-hvm.  More
 recent versions have HVM support integrated in the main xentools  recent versions have HVM support integrated in the main xentools
Line 367  and other than dom0 memory and max_vcpus Line 416  and other than dom0 memory and max_vcpus
 necessary.  necessary.
   
 As with non-Xen systems, you should have a line to boot /netbsd (a  As with non-Xen systems, you should have a line to boot /netbsd (a
 kernel that works without Xen) and fallback versions of the non-Xen  kernel that works without Xen).  Consider a line to boot /netbsd.ok (a
 kernel, Xen, and the dom0 kernel.  fallback version of the non-Xen kernel, updated manually when you are
   sure /netbsd is ok).  Consider also a line to boot fallback versions
   of Xen and the dom0 kernel, but note that non-Xen NetBSD can be used
   to resolve Xen booting issues.
   
   Probably you want a default=N line to choose Xen in the absence of
   intervention.
   
 Now, reboot so that you are running a DOM0 kernel under Xen, rather  Now, reboot so that you are running a DOM0 kernel under Xen, rather
 than GENERIC without Xen.  than GENERIC without Xen.
Line 397  There will be no domUs, and none can be  Line 452  There will be no domUs, and none can be 
 to configure the dom0 daemons.  to configure the dom0 daemons.
   
 The daemons which should be run vary with Xen version and with whether  The daemons which should be run vary with Xen version and with whether
 one is using xm or xl.  The Xen 3.1 and 3.3 packages use xm.  Xen 4.1  one is using xm or xl.  The Xen 3.1, 3.3 and 4.1 packages use xm.  Xen
 and higher packages use xl.  While is is possible to use xm with some  4.2 and up packages use xl.  To use xm with 4.2, edit xendomains to
 4.x versions (TODO: 4.1 and 4.2?), the pkgsrc-provided rc.d scripts do  use xm instead.
 not support this as of 2014-12-26, and thus the HOWTO does not support  
 it either.  (Make sure your packages are reasonably recent.)  
   
 For "xm" (3.1 and 3.3), you should enable xend and xenbackendd (but  For 3.1 and 3.3, you should enable xend and xenbackendd:
 note that you should be using 4.x):  
   
         xend=YES          xend=YES
         xenbackendd=YES          xenbackendd=YES
   
 For "xl" (4.x), you should enabled xend and xencommons (xenstored).  For 4.1 and up, you should enable xencommons.  Not enabling xencommons
 Trying to boot 4.x without xencommons=YES will result in a hang; it is  will result in a hang; it is necessary to hit ^C on the console to let
 necessary to hit ^C on the console to let the machine finish booting.  the machine finish booting.  If you are using xm (default in 4.1, or
 TODO: explain why xend is installed by the package.  if you changed xendomains in 4.2), you should also enable xend:
   
           xend=YES # only if using xm, and only installed <= 4.2
         xencommons=YES          xencommons=YES
   
 The installation of NetBSD should already have created devices for xen  
 (xencons, xenevt), but if they are not present, create them:  
   
         cd /dev && sh MAKEDEV xen  
   
 TODO: Recommend for/against xen-watchdog.  TODO: Recommend for/against xen-watchdog.
   
 After you have configured the daemons and either started them (in the  After you have configured the daemons and either started them (in the
Line 447  make this work, one should not expect to Line 495  make this work, one should not expect to
 (and thus xencommons).  There is currently no reason to expect that  (and thus xencommons).  There is currently no reason to expect that
 this will get fixed any time soon.  this will get fixed any time soon.
   
   ### No-longer needed advice about devices
   
   The installation of NetBSD should already have created devices for xen
   (xencons, xenevt, xsd_kva), but if they are not present, create them:
   
           cd /dev && sh MAKEDEV xen
   
 anita (for testing NetBSD)  anita (for testing NetBSD)
 --------------------------  --------------------------
   
Line 549  Hardware known to work Line 604  Hardware known to work
 Arguably, this section is misplaced, and there should be a page of  Arguably, this section is misplaced, and there should be a page of
 hardware that runs NetBSD/amd64 well, with the mostly-well-founded  hardware that runs NetBSD/amd64 well, with the mostly-well-founded
 assumption that NetBSD/xen runs fine on any modern hardware that  assumption that NetBSD/xen runs fine on any modern hardware that
 NetBSD/amd64 runs well on.  Until then, we give motherboard/CPU/RAM  NetBSD/amd64 runs well on.  Until then, we give motherboard/CPU (and
 triples to aid those choosing a motherboard.  Note that Xen systems  sometimes RAM) pairs/triples to aid those choosing a motherboard.
 usually do not run X, so a listing here does not imply that X works at  Note that Xen systems usually do not run X, so a listing here does not
 all.  imply that X works at all.
   
         Supermicro X9SRL-F, Xeon E5-1650 v2, 96 GiB ECC          Supermicro X9SRL-F, Xeon E5-1650 v2, 96 GiB ECC
         Supermicro ??, Atom C2758 (8 core), 32 GiB ECC          Supermicro ??, Atom C2758 (8 core), 32 GiB ECC
Line 561  all. Line 616  all.
 Older hardware:  Older hardware:
   
         Intel D915GEV, Pentium4 CPU 3.40GHz, 4GB 533MHz Synchronous DDR2          Intel D915GEV, Pentium4 CPU 3.40GHz, 4GB 533MHz Synchronous DDR2
           INTEL DG33FB, "Intel(R) Core(TM)2 Duo CPU     E6850  @ 3.00GHz"
           INTEL DG33FB, "Intel(R) Core(TM)2 Duo CPU     E8400  @ 3.00GHz"
   
 Running Xen under qemu  Running Xen under qemu
 ----------------------  ----------------------
Line 694  for the first virtual disk for the domU  Line 751  for the first virtual disk for the domU 
 the file serves two purposes.  One is that preallocating the contents  the file serves two purposes.  One is that preallocating the contents
 improves performance.  The other is that vnd on sparse files has  improves performance.  The other is that vnd on sparse files has
 failed to work.  TODO: give working/notworking NetBSD versions for  failed to work.  TODO: give working/notworking NetBSD versions for
 sparse vnd.  Note that the use of file/vnd for Xen is not really  sparse vnd and gnats reference.  Note that the use of file/vnd for Xen
 different than creating a file-backed virtual disk for some other  is not really different than creating a file-backed virtual disk for
 purpose, except that xentools handles the vnconfig commands.  To  some other purpose, except that xentools handles the vnconfig
 create an empty 4G virtual disk, simply do  commands.  To create an empty 4G virtual disk, simply do
   
         dd if=/dev/zero of=foo-xbd0 bs=1m count=4096          dd if=/dev/zero of=foo-xbd0 bs=1m count=4096
   
Line 731  guest, one can create /dev/hda1 in /dev, Line 788  guest, one can create /dev/hda1 in /dev,
 The third element is "w" for writable disks, and "r" for read-only  The third element is "w" for writable disks, and "r" for read-only
 disks.  disks.
   
   Note that NetBSD by default creates only vnd[0123].  If you need more
   than 4 total virtual disks at a time, run e.g. "./MAKEDEV vnd4" in the
   dom0.
   
   Note that NetBSD by default creates only xbd[0123].  If you need more
   virtual disks in a domU, run e.g. "./MAKEDEV xbd4" in the domU.
   
 Virtual Networking  Virtual Networking
 ------------------  ------------------
   
Line 1092  TODO items for improving NetBSD/xen Line 1156  TODO items for improving NetBSD/xen
 Random pointers  Random pointers
 ===============  ===============
   
 TODO: This section contains links from elsewhere not yet integrated  This section contains links from elsewhere not yet integrated into the
 into the HOWTO.  HOWTO, and other guides.
   
 * http://www.lumbercartel.ca/library/xen/  * http://www.lumbercartel.ca/library/xen/
 * http://pbraun.nethence.com/doc/sysutils/xen_netbsd_dom0.html  * http://pbraun.nethence.com/doc/sysutils/xen_netbsd_dom0.html
   * https://gmplib.org/~tege/xen.html

Removed from v.1.113  
changed lines
  Added in v.1.132


CVSweb for NetBSD wikisrc <wikimaster@NetBSD.org> software: FreeBSD-CVSweb