Diff for /wikisrc/ports/xen/howto.mdwn between versions 1.93 and 1.110

version 1.93, 2015/03/05 13:27:37 version 1.110, 2016/12/20 04:09:54
Line 49  specific PCI devices can be made availab Line 49  specific PCI devices can be made availab
 of the dom0.  This can be useful to let a domU run X11, or access some  of the dom0.  This can be useful to let a domU run X11, or access some
 network interface or other peripheral.  network interface or other peripheral.
   
 NetBSD used to support Xen2; this has been removed.  NetBSD 6 and earlier supported Xen 2; support was removed fro NetBSD
   7.  Xen 2 has been removed from pkgsrc.
   
 Prerequisites  Prerequisites
 -------------  -------------
Line 81  xenkernel and xentools.  We will refer o Line 82  xenkernel and xentools.  We will refer o
 but note that both packages must be installed together and must have  but note that both packages must be installed together and must have
 matching versions.  matching versions.
   
 xenkernel3 and xenkernel33 provide Xen 3.1 and 3.3.  These no longer  xenkernel3 provides Xen 3.1.  It is no longer maintained by Xen, and the last applied security patch was in
 receive security patches and should not be used.  Xen 3.1 supports PCI  2011. Thus, it should not be used.  It supports PCI passthrough,
 passthrough.  Xen 3.1 supports non-PAE on i386.  which is why people use it anyway. Xen 3.1 supports i386, both PAE and
   non-PAE.
 xenkernel41 provides Xen 4.1.  This is no longer maintained by Xen,  
 but as of 2014-12 receives backported security patches.  It is a  xenkernel33 provides Xen 3.3.  It is no longer maintained by Xen, and
 reasonable although trailing-edge choice.  the last applied security patch was in 2012.  Thus, it should not be
   used.  Xen 3.3 supports i386, but only in PAE mode.  There are no good
   reasons to run this version.
   
   xenkernel41 provides Xen 4.1.  It is no longer maintained by Xen, but
   as of 2016-11 received backported security patches.  Xen 4.1 supports
   i386, but only in PAE mode.  There are no good reasons to run this
   version.
   
   xenkernel42 provides Xen 4.2.  It is no longer maintained by Xen, but
   as of 2016-11 received backported security patches.  Xen 4.2 supports
   i386, but only in PAE mode.  The only reason to run this is if you
   need to use xm instead of xl, or if you need to run an i386 dom0
   (because your hardware is i386 only).
   
   xenkernel45 provides Xen 4.5.  It is no longer maintained by Xen, but
   as of 2016-11 it received security patches.  Xen 4.5 requires an amd64
   dom0, but domUs can be amd64 or i386 PAE.  TODO: It is either a
   conservative choice or somewhat old.
   
   xenkernel45 provides Xen 4.6.  It is new to pkgsrc in 2016-05.  It is
   no longer maintained by Xen, but as of 2016-11 it received security
   patches.  Xen 4.6 requires an amd64 dom0, but domUs can be amd64 or
   i386 PAE.  TODO: It is either a somewhat aggressive choice or the
   standard choice
   
 xenkernel42 provides Xen 4.2.  This is maintained by Xen, but old as  See also the [Xen Security Advisory page](http://xenbits.xen.org/xsa/).
 of 2014-12.  
   
 xenkernel45 provides Xen 4.5.  This is new to pkgsrc as of 2015-01 and  
 not yet recommended for other than experimental/testing use.  
   
 Ideally newer versions of Xen will be added to pkgsrc.  Ideally newer versions of Xen will be added to pkgsrc.
   
Line 108  dom0.  Starting in 4.1, a replacement pr Line 129  dom0.  Starting in 4.1, a replacement pr
 called xl is provided, but it does not work well in 4.1.  In 4.2, both  called xl is provided, but it does not work well in 4.1.  In 4.2, both
 xm and xl work fine.  4.4 is the last version that has xm.  You must  xm and xl work fine.  4.4 is the last version that has xm.  You must
 choose one or the other, because it affects which daemons you run.  choose one or the other, because it affects which daemons you run.
   However, the rc.d scripts provided by xentools packages expect a
   particular version, and you should use the version used by the
   scripts.
   
 NetBSD  NetBSD
 ------  ------
   
 The netbsd-5, netbsd-6, netbsd-7, and -current branches are all  The netbsd-6, netbsd-7, and -current branches are all reasonable
 reasonable choices, with more or less the same considerations for  choices, with more or less the same considerations for non-Xen use.
 non-Xen use.  Therefore, netbsd-6 is recommended as the stable version  Therefore, netbsd-7 is recommended as the stable version of the most
 of the most recent release for production use.  For those wanting to  recent release for production use.  For those wanting to learn Xen or
 learn Xen or without production stability concerns, netbsd-7 is likely  without production stability concerns, netbsd-7 is still likely most
 most appropriate.  appropriate, but -current is also a reasonable choice.  Xen runs fine
   on netbsd-5, but the xentools packages are likely difficult to build.
   
 As of NetBSD 6, a NetBSD domU will support multiple vcpus.  There is  As of NetBSD 6, a NetBSD domU will support multiple vcpus.  There is
 no SMP support for NetBSD as dom0.  (The dom0 itself doesn't really  no SMP support for NetBSD as dom0.  (The dom0 itself doesn't really
 need SMP; the lack of support is really a problem when using a dom0 as  need SMP for dom0 functions; the lack of support is really a problem
 a normal computer.)  when using a dom0 as a normal computer.)
   
 Architecture  Architecture
 ------------  ------------
   
 Xen itself can run on i386 or amd64 machines.  (Practically, almost  Xen itself can run on i386 (Xen < 3.1) or amd64 machines (all Xen
 any computer where one would want to run Xen supports amd64.)  If  versions).  (Practically, almost any computer where one would want to
 using an i386 NetBSD kernel for the dom0, PAE is required (PAE  run Xen today supports amd64.)
 versions are built by default).  While i386 dom0 works fine, amd64 is  
 recommended as more normal.  Xen, the dom0 kernel, and each domU kernel can be either i386 or
   amd64.  When building a xenkernel package, one obtains i386 on an i386
   host, and amd64 on an amd64 host.  If the xen kernel is i386, then the
   dom0 kernel and all domU kernels must be i386.  With an amd64 xen
   kernel, an amd64 dom0 kernel is known to work, and an i386PAE dom0
   kernel should in theory work.  An amd64 xen/dom0 is known to support
   both i386PAE and amd64 domUs.
   
   i386 dom0 and domU kernels must be PAE (except for Xen 3.1); these are
   built by default.  (Note that emacs (at least) fails if run on i386
   with PAE when built without, and vice versa, presumably due to bugs in
   the undump code.)
   
   Because of the above, the standard approach is to use amd64 for the
   dom0.
   
 Xen 4.2 is the last version to support i386 as a host.  TODO: Clarify  Xen 4.2 is the last version to support i386 as a host.  TODO: Clarify
 if this is about the CPU having to be amd64, or about the dom0 kernel  if this is about the CPU, the xen kernel, or the dom0 kernel having to
 having to be amd64.  be amd64.
   
 One can then run i386 domUs and amd64 domUs, in any combination.  If  
 running an i386 NetBSD kernel as a domU, the PAE version is required.  
 (Note that emacs (at least) fails if run on i386 with PAE when built  
 without, and vice versa, presumably due to bugs in the undump code.)  
   
 Stability  Stability
 ---------  ---------
Line 152  However, there are some open PRs indicat Line 187  However, there are some open PRs indicat
  - [PR 47720](http://gnats.netbsd.org/47720)   - [PR 47720](http://gnats.netbsd.org/47720)
   
 Note also that there are issues with sparse vnd(4) instances, but  Note also that there are issues with sparse vnd(4) instances, but
 these are not about Xen.  these are not about Xen -- they just are noticed with sparse vnd(4)
   instances in support of virtual disks in a dom0.
   
 Recommendation  Recommendation
 --------------  --------------
   
 Therefore, this HOWTO recommends running xenkernel42 (and xentools42),  Therefore, this HOWTO recommends running xenkernel45 or xenkernel46,
 xl, the NetBSD 6 stable branch, and to use an amd64 kernel as the  xl, the NetBSD 7 stable branch, and to use an amd64 kernel as the
 dom0.  Either the i386 or amd64 of NetBSD may be used as domUs.  dom0.  Either the i386PAE or amd64 version of NetBSD may be used as
   domUs.
   
 Build problems  Build problems
 --------------  --------------
Line 357  Xen logs will be in /var/log/xen. Line 394  Xen logs will be in /var/log/xen.
 Now, you have a system that will boot Xen and the dom0 kernel, but not  Now, you have a system that will boot Xen and the dom0 kernel, but not
 do anything else special.  Make sure that you have rebooted into Xen.  do anything else special.  Make sure that you have rebooted into Xen.
 There will be no domUs, and none can be started because you still have  There will be no domUs, and none can be started because you still have
 to configure the dom0 tools.  The daemons which should be run vary  to configure the dom0 daemons.
 with Xen version and with whether one is using xm or xl.  Note that  
 xend is for supporting "xm", and should only be used if you plan on  
 using "xm".  Do NOT enable xend if you plan on using "xl" as it will  
 cause problems.  Running xl without xencommons=YES (and starting it)  
 will result in a hang (so don't do that; follow the HOWTO!).  
   
 The installation of NetBSD should already have created devices for xen  
 (xencons, xenevt), but if they are not present, create them:  
   
         cd /dev && sh MAKEDEV xen  
   
 TODO: Give 3.1 advice (or remove it from pkgsrc).  The daemons which should be run vary with Xen version and with whether
   one is using xm or xl.  The Xen 3.1 and 3.3 packages use xm.  Xen 4.1
   and higher packages use xl.  While is is possible to use xm with some
   4.x versions (TODO: 4.1 and 4.2?), the pkgsrc-provided rc.d scripts do
   not support this as of 2014-12-26, and thus the HOWTO does not support
   it either.  (Make sure your packages are reasonably recent.)
   
 For 3.3 (and thus xm), add to rc.conf (but note that you should have  For "xm" (3.1 and 3.3), you should enable xend and xenbackendd (but
 installed 4.1 or 4.2):  note that you should be using 4.x):
   
         xend=YES          xend=YES
         xenbackendd=YES          xenbackendd=YES
   
 For 4.1 (and thus xm; xl is believed not to work well), add to rc.conf:  For "xl" (4.x), you should enabled xend and xencommons (xenstored).
   Trying to boot 4.x without xencommons=YES will result in a hang; it is
         xencommons=YES  necessary to hig ^C on the console to let the machine finish booting.
         xend=YES  TODO: explain why xend is installed by the package.
   
 (If you are using xentools41 from before 2014-12-26, change  
 rc.d/xendomains to use xm rather than xl.)  
   
 For 4.2 with xm, add to rc.conf  
   
         xencommons=YES  
         xend=YES  
   
 For 4.2 with xl, add to rc.conf:  
   
         xencommons=YES          xencommons=YES
         TODO: explain if there is a xend replacement  
   
 For 4.5 (and thus with xl), add to rc.conf:  The installation of NetBSD should already have created devices for xen
   (xencons, xenevt), but if they are not present, create them:
   
         xencommons=YES          cd /dev && sh MAKEDEV xen
         TODO: explain if there is a xend replacement  
   
 TODO: Recommend for/against xen-watchdog.  TODO: Recommend for/against xen-watchdog.
   
 After you have configured the daemons and either started them (in the  After you have configured the daemons and either started them (in the
 order given) or rebooted, use xm or xl to inspect Xen's boot messages,  order given) or rebooted, use xm or xl to inspect Xen's boot messages,
 available resources, and running domains.  An example with xm follows:  available resources, and running domains.  An example with xl follows:
   
         # xm dmesg          # xl dmesg
         [xen's boot info]          [xen's boot info]
         # xm info          # xl info
         [available memory, etc.]          [available memory, etc.]
         # xm list          # xl list
         Name              Id  Mem(MB)  CPU  State  Time(s)  Console          Name              Id  Mem(MB)  CPU  State  Time(s)  Console
         Domain-0           0       64    0  r----     58.1          Domain-0           0       64    0  r----     58.1
   
 With xl, the commands are the same, and the output may be slightly  
 different.  TODO: add example output for xl before the xm example,  
 after confirming on 4.2 and resolving the TODO about rc.conf.  
   
 ### Issues with xencommons  ### Issues with xencommons
   
 xencommons starts xenstored, which stores data on behalf of dom0 and  xencommons starts xenstored, which stores data on behalf of dom0 and
Line 448  Xen-specific NetBSD issues Line 465  Xen-specific NetBSD issues
 There are (at least) two additional things different about NetBSD as a  There are (at least) two additional things different about NetBSD as a
 dom0 kernel compared to hardware.  dom0 kernel compared to hardware.
   
 One is that modules are not usable in DOM0 kernels, so one must  One is that the module ABI is different because some of the #ddefines
 compile in what's needed.  It's not really that modules cannot work,  change, so one must build modules for Xen.  As of netbsd-7, the build
 but that modules must be built for XEN3_DOM0 because some of the  system does this automatically.  TODO: check this.  (Before building
 defines change and the normal module builds don't do this.  Basically,  Xen modules was added, it was awkward to use modules to the point
 enabling Xen changes the kernel ABI, and the module build system  where it was considered that it did not work.)
 doesn't cope with this.  
   
 The other difference is that XEN3_DOM0 does not have exactly the same  The other difference is that XEN3_DOM0 does not have exactly the same
 options as GENERIC.  While it is debatable whether or not this is a  options as GENERIC.  While it is debatable whether or not this is a
Line 502  section. Line 518  section.
   
 TODO: actually do this and fix it if necessary.  TODO: actually do this and fix it if necessary.
   
 Updating Xen versions  Upgrading Xen versions
 ---------------------  ---------------------
   
 Updating Xen is conceptually not difficult, but can run into all the  Minor version upgrades are trivial.  Just rebuild/replace the
 issues found when installing Xen.  Assuming migration from 4.1 to 4.2,  xenkernel version and copy the new xen.gz to / (where /boot.cfg
 remove the xenkernel41 and xentools41 packages and install the  references it), and reboot.
 xenkernel42 and xentools42 packages.  Copy the 4.2 xen.gz to /.  
   Major version upgrades are conceptually not difficult, but can run
 Ensure that the contents of /etc/rc.d/xen* are correct.  Enable the  into all the issues found when installing Xen.  Assuming migration
 correct set of daemons.  Ensure that the domU config files are valid  from 4.1 to 4.2, remove the xenkernel41 and xentools41 packages and
 for the new version.  install the xenkernel42 and xentools42 packages.  Copy the 4.2 xen.gz
   to /.
   
   Ensure that the contents of /etc/rc.d/xen* are correct.  Specifically,
   they must match the package you just installed and not be left over
   from some previous installation.
   
   Enable the correct set of daemons; see the configuring section above.
   (Upgrading from 3.x to 4.x without doing this will result in a hang.)
   
   Ensure that the domU config files are valid for the new version.
   Specifically, for 4.x remove autorestart=True, and ensure that disks
   are specified with numbers as the second argument, as the examples
   above show, and not NetBSD device names.
   
   Hardware known to work
   ----------------------
   
   Arguably, this section is misplaced, and there should be a page of
   hardware that runs NetBSD/amd64 well, with the mostly-well-founded
   assumption that NetBSD/xen runs fine on any modern hardware that
   NetBSD/amd64 runs well on.  Until then, we give motherboard/CPU/RAM
   triples to aid those choosing a motherboard.  Note that Xen systems
   usually do not run X, so a listing here does not imply that X works at
   all.
   
           Supermicro X9SRL-F, Xeon E5-1650 v2, 96 GiB ECC
           Supermicro ??, Atom C2758 (8 core), 32 GiB ECC
           ASUS M5A78L-M/USB3 AM3+ microATX, AMD Piledriver X8 4000MHz, 16 GiB ECC
   
   Older hardware:
   
           Intel D915GEV, Pentium4 CPU 3.40GHz, 4GB 533MHz Synchronous DDR2
   
 Running Xen under qemu  Running Xen under qemu
 ----------------------  ----------------------
Line 761  i386 and amd64 provide the following ker Line 808  i386 and amd64 provide the following ker
   
         i386 XEN3_DOMU          i386 XEN3_DOMU
         i386 XEN3PAE_DOMU          i386 XEN3PAE_DOMU
         amd64 XEN3_DOMU          amd64 XEN3_DOMU
   
 Unless using Xen 3.1 (and you shouldn't) with i386-mode Xen, you must  Unless using Xen 3.1 (and you shouldn't) with i386-mode Xen, you must
 use the PAE version of the i386 kernel.  use the PAE version of the i386 kernel.
Line 822  not really a Xen-specific issue, but bec Line 869  not really a Xen-specific issue, but bec
 obtained from the dom0, it is far more likely to be out of sync or  obtained from the dom0, it is far more likely to be out of sync or
 missing with Xen.)  missing with Xen.)
   
 Your domain should be now ready to work, enjoy.  
   
 Creating an unprivileged Linux domain (domU)  Creating an unprivileged Linux domain (domU)
 --------------------------------------------  --------------------------------------------
   
Line 1010  to update the special boot partiion. Line 1055  to update the special boot partiion.
 Amazon  Amazon
 ------  ------
   
 TODO: add link to NetBSD amazon howto.  See the [Amazon EC2 page](../amazon_ec2/).
   
 Using npf  Using npf
 ---------  ---------
Line 1018  Using npf Line 1063  Using npf
 In standard kernels, npf is a module, and thus cannot be loaded in a  In standard kernels, npf is a module, and thus cannot be loaded in a
 DOMU kernel.  DOMU kernel.
   
 TODO: explain how to compile npf into a custom kernel, answering (but  TODO: Explain how to compile npf into a custom kernel, answering (but
 note that the problem was caused by not booting the right kernel):  note that the problem was caused by not booting the right kernel)
 http://mail-index.netbsd.org/netbsd-users/2014/12/26/msg015576.html  [this email to
   netbsd-users](http://mail-index.netbsd.org/netbsd-users/2014/12/26/msg015576.html).
   
 TODO items for improving NetBSD/xen  TODO items for improving NetBSD/xen
 ===================================  ===================================

Removed from v.1.93  
changed lines
  Added in v.1.110


CVSweb for NetBSD wikisrc <wikimaster@NetBSD.org> software: FreeBSD-CVSweb