File:  [NetBSD Developer Wiki] / wikisrc / rumpkernel.mdwn
Revision 1.1: download - view: text, annotated - select for diffs
Sun Jan 5 14:53:58 2014 UTC (6 years, 9 months ago) by pooka
Branches: MAIN
CVS tags: HEAD
Convert htdocs/docs/rump/ to wiki
(or least attempt to. i have no idea what i'm doing)

    1: **Contents**
    2: 
    3: [[!toc levels=2]]
    4: 
    5: # The Anykernel and Rump Kernels
    6: 
    7: ## About
    8: 
    9: A driver abstracts an underlying entity. For example, a TCP/IP driver
   10: abstracts the details required to perform networking, the Fast File
   11: System (FFS) driver abstracts how the file system blocks are laid out on
   12: the storage medium, and a PCI network card driver abstracts how to
   13: access device registers to send and receive packets. The kernel
   14: architecture controls how kernel drivers run with respect to other
   15: system components. Some examples of kernel architectures are the
   16: monolithic kernel, microkernel and exokernel. In contrast to the above
   17: architectures, NetBSD is an *anykernel*. This means that some kernel
   18: drivers can be run according to any kernel architecture instead of being
   19: limited to a single one.
   20: 
   21: <!--
   22:     TODO
   23:     ![](rumparch.png)
   24: -->
   25: 
   26: When a driver is not run as part of the monolithic kernel, e.g. when it
   27: is run as a microkernel style server, the driver is hosted in a *rump
   28: kernel*. A rump kernel is an ultralightweight virtualized kernel running
   29: on top of high-level hypercall interface. Instead of a low-level
   30: hypercall API typically seen with operating systems with operations such
   31: as "modify page table", the rump kernel hypercall API provides
   32: high-level operations such as "run this code in a thread".
   33: 
   34: Currently, three implementations of the rump kernel hypercall interface
   35: exist.
   36: 
   37: -   The POSIX implementation is included in the NetBSD tree and makes
   38:     rump kernels to run as userspace processes on most operating systems
   39:     such as NetBSD, Linux and Solaris.
   40: -   The Xen implementation allows running rump kernels directly as Xen
   41:     DomU's without an intermediate operating system.
   42: -   The Linux kernel hypervisor allows rump kernels to run inside the
   43:     Linux kernel.
   44: 
   45: Rump kernels are radically different from OS virtualization technologies
   46: such as KVM, containers and usermode operating systems. A rump kernel
   47: does not support hosting application processes because a rump kernel is
   48: aimed at virtualizing kernel drivers and application virtualization
   49: would be pure overhead. Instead, existing entities such as processes
   50: from a hosting OS are used as clients for the rump kernel ("application"
   51: in the figure).
   52: 
   53: As a result of the above design choices, rump kernels are extremely
   54: lightweight. The bootstrap time for rump kernels on POSIX hosts is
   55: measured in milliseconds and memory footprint in 100kB's. This means
   56: that a rump kernel can be bootstrapped for example as part of a command
   57: line tool for virtually no cost or user impact. Rump kernels also
   58: mandate very little from the hypercall implementation meaning that rump
   59: kernels, and by extension NetBSD kernel drivers, can be hosted in
   60: virtually any environment.
   61: 
   62: Use cases for rump kernels include:
   63: 
   64: -   **Code reuse**: kernel drivers can be reused without having to run a
   65:     whole OS. For example, a full-featured TCP/IP stack (IPv6, IPSec,
   66:     etc.) can be included in an embedded appliance without having to
   67:     write the stack from scratch or waste resources on running an entire
   68:     OS.
   69: -   **Kernel driver virtualization**: every rump kernel has its own
   70:     state. Furthermore, the functionality offered by multiple rump
   71:     kernels running on the same host does not need to be equal. For
   72:     example, multiple different networking stacks optimized for
   73:     different purposes are possible.
   74: -   **Security**: when hosted on a POSIX system, a rump kernel runs in
   75:     its own instance of a userspace process. For example, it is widely
   76:     published that file system drivers are vulnerable to untrusted file
   77:     system images. Unlike on other general purpose operating systems, on
   78:     NetBSD it is possible to mount untrusted file systems, such as those
   79:     on a USB stick, in an isolated server with the kernel file system
   80:     driver. This isolates attacks and prevents kernel compromises while
   81:     not requiring to maintain separate userspace implementations of the
   82:     file system drivers or use other resource-intensive approaches such
   83:     as virtual machines.
   84: -   **Easy prototyping and development**: kernel code can be developed
   85:     as a normal userspace application. Once development is finished, the
   86:     code can simply be complied into the kernel. This is a much more
   87:     convenient and straightforward approach to kernel development than
   88:     the use of virtual machines.
   89: -   **Safe testing**: kernel code can be tested in userspace on any host
   90:     without risk of the test host being affected. Again, virtual
   91:     machines are not required.
   92: 
   93: ## Further Reading
   94: 
   95: ### Dissertation
   96: 
   97: The following is the definitive guide to the anykernel and rump kernels
   98: and supercedes all earlier publications and terminology on the subject.
   99: 
  100: -   [Flexible Operating System Internals: The Design and Implementation
  101:     of the Anykernel and Rump
  102:     Kernels](http://lib.tkk.fi/Diss/2012/isbn9789526049175/isbn9789526049175.pdf)
  103: 
  104: ### Software using rump kernels
  105: 
  106: These links are interesting for people who want to use rump kernels in
  107: addition to reading about them.
  108: 
  109: -   [Scripts for building rump kernels for POSIX
  110:     systems](https://github.com/anttikantee/buildrump.sh)
  111: -   [Rump kernel hypercall implementation for Xen; rump kernels as Xen
  112:     DomU's](https://github.com/anttikantee/rumpuser-xen)
  113: -   [fs-utils: File system image access
  114:     utilities](https://github.com/stacktic/fs-utils)
  115: -   Fast userspace packet processing: TCP/IP stack for use with
  116:     [DPDK](https://github.com/anttikantee/dpdk-rumptcpip) or
  117:     [netmap](https://github.com/anttikantee/netmap-rumptcpip)
  118: 
  119: ### Articles, Tutorials & Howtos
  120: 
  121: -   [Running rump kernels and applications on Xen without a full
  122:     OS](http://blog.NetBSD.org/tnf/entry/running_applications_on_the_xen)
  123: -   [PCI device driver support in rump kernels on
  124:     Xen](http://blog.NetBSD.org/tnf/entry/pci_driver_support_for_rump)
  125: -   [Experiment with a rump kernel hypervisor for the Linux
  126:     kernel](http://blog.NetBSD.org/tnf/entry/a_rump_kernel_hypervisor_for)
  127:     (allows rump kernels to run *in* the Linux kernel)
  128: -   [Experiment on compiling rump kernels to javascript and running them
  129:     in the
  130:     browser](http://blog.NetBSD.org/tnf/entry/kernel_drivers_compiled_to_javascript)
  131: -   [Kernel Servers using
  132:     Rump](http://www.NetBSD.org/docs/rump/sysproxy.html)
  133: -   [Tutorial On Rump Kernel Servers and
  134:     Clients](http://www.NetBSD.org/docs/rump/sptut.html)
  135: -   [Revolutionizing Kernel Development: Testing With
  136:     Rump](http://blog.NetBSD.org/tnf/entry/revolutionizing_kernel_development_testing_with)
  137: 
  138: ### Conference publications and talks
  139: 
  140: -   "The Anykernel and Rump Kernels" gives a general overview and
  141:     demonstrates rump kernels on Windows and in Firefox. The
  142:     [video](http://video.fosdem.org/2013/maintracks/K.1.105/The_Anykernel_and_Rump_Kernels.webm),
  143:     [slides](https://fosdem.org/2013/schedule/event/operating_systems_anykernel/attachments/slides/244/export/events/attachments/operating_systems_anykernel/slides/244/fosdem2013_rumpkern.pdf)
  144:     and an
  145:     [interview](https://archive.fosdem.org/2013/interviews/2013-antii-kantee/)
  146:     are available. Presented at FOSDEM 2013 (Operating Systems track).
  147: -   "Rump Device Drivers: Shine On You Kernel Diamond" describes device
  148:     driver and USB. The
  149:     [paper](http://ftp.NetBSD.org/pub/NetBSD/misc/pooka/tmp/rumpdev.pdf)
  150:     and [video presentation](http://www.youtube.com/watch?v=3AJNxa33pzk)
  151:     are available. Presented at AsiaBSDCon 2010.
  152: -   "Fs-utils: File Systems Access Tools for Userland" describes
  153:     fs-utils, an mtools-like utility kit which uses rump kernel file
  154:     systems as a backend. The
  155:     [paper](http://www.ukuug.org/events/eurobsdcon2009/papers/ebc09_fs-utils.pdf)
  156:     is available. Presented at EuroBSDCon 2009.
  157: -   "Rump File Systems: Kernel Code Reborn" describes kernel file system
  158:     code and its uses in userspace. The
  159:     [paper](http://usenix.org/events/usenix09/tech/full_papers/kantee/kantee.pdf)
  160:     and
  161:     [slides](http://usenix.org/events/usenix09/tech/slides/kantee.pdf)
  162:     are available. Presented at the 2009 USENIX Annual Technical
  163:     Conference.
  164: -   "Kernel Development in Userspace - The Rump Approach" describes
  165:     doing kernel development with rump kernels. The
  166:     [paper](http://www.bsdcan.org/2009/schedule/attachments/104_rumpdevel.pdf)
  167:     and
  168:     [slides](http://www.bsdcan.org/2009/schedule/attachments/105_bsdcan09-kantee.pdf)
  169:     are available. Presented at BSDCan 2009.
  170: -   "Environmental Independence: BSD Kernel TCP/IP in Userspace"
  171:     describes networking in rump kernels. The
  172:     [paper](http://2009.asiabsdcon.org/papers/abc2009-P5A-paper.pdf) and
  173:     [video presentation](http://www.youtube.com/watch?v=RxFctq8A0WI) are
  174:     available. Presented at AsiaBSDCon 2009.
  175: 
  176: ### Manual pages
  177: 
  178: The manpages provide the usual type of information. Start from
  179: [rump.3](http://man.NetBSD.org/cgi-bin/man-cgi?rump++NetBSD-current) and
  180: follow the cross-references in "SEE ALSO".
  181: 
  182: ## Discuss
  183: 
  184: Any topic related to rump kernels can be discussed on the
  185: [rumpkernel-users mailing
  186: list](https://lists.sourceforge.net/lists/listinfo/rumpkernel-users).
  187: Alternatively, you can use a NetBSD mailing which is related to a
  188: specific subtopic.
  189: 
  190: The IRC channel for rump kernels is **\#rumpkernel** on
  191: **irc.freenode.net**.
  192: 
  193: ## Availability
  194: 
  195: The anykernel and rump kernels were first introduced as a prototype in
  196: NetBSD 5.0. A stable version with numerous new features and improvements
  197: was shipped with NetBSD 6.0.
  198: 
  199: ## Source Code
  200: 
  201: All of the source code is available from the NetBSD source tree and can
  202: be obtained with the usual methods.
  203: 
  204: You can also [browse](http://cvsweb.NetBSD.org/bsdweb.cgi/src/) the
  205: source code history online. Code is found from all areas of the source
  206: tree. Some examples of where to look include
  207: [src/lib](http://cvsweb.NetBSD.org/bsdweb.cgi/src/lib/),
  208: [src/usr.bin](http://cvsweb.NetBSD.org/bsdweb.cgi/src/usr.bin/) and
  209: [src/sys/rump](http://cvsweb.NetBSD.org/bsdweb.cgi/src/sys/rump/).

CVSweb for NetBSD wikisrc <wikimaster@NetBSD.org> software: FreeBSD-CVSweb