Annotation of wikisrc/rumpkernel.mdwn, revision 1.3

1.1       pooka       1: **Contents**
                      3: [[!toc levels=2]]
                      5: # The Anykernel and Rump Kernels
                      7: ## About
                      9: A driver abstracts an underlying entity. For example, a TCP/IP driver
                     10: abstracts the details required to perform networking, the Fast File
                     11: System (FFS) driver abstracts how the file system blocks are laid out on
                     12: the storage medium, and a PCI network card driver abstracts how to
                     13: access device registers to send and receive packets. The kernel
                     14: architecture controls how kernel drivers run with respect to other
                     15: system components. Some examples of kernel architectures are the
                     16: monolithic kernel, microkernel and exokernel. In contrast to the above
                     17: architectures, NetBSD is an *anykernel*. This means that some kernel
                     18: drivers can be run according to any kernel architecture instead of being
                     19: limited to a single one.
                     21: <!--
                     22:     TODO
                     23:     ![](rumparch.png)
                     24: -->
                     26: When a driver is not run as part of the monolithic kernel, e.g. when it
                     27: is run as a microkernel style server, the driver is hosted in a *rump
                     28: kernel*. A rump kernel is an ultralightweight virtualized kernel running
                     29: on top of high-level hypercall interface. Instead of a low-level
                     30: hypercall API typically seen with operating systems with operations such
                     31: as "modify page table", the rump kernel hypercall API provides
                     32: high-level operations such as "run this code in a thread".
                     34: Currently, three implementations of the rump kernel hypercall interface
                     35: exist.
1.2       pooka      37: -   The POSIX implementation is included in the NetBSD tree and allows
1.1       pooka      38:     rump kernels to run as userspace processes on most operating systems
                     39:     such as NetBSD, Linux and Solaris.
                     40: -   The Xen implementation allows running rump kernels directly as Xen
                     41:     DomU's without an intermediate operating system.
                     42: -   The Linux kernel hypervisor allows rump kernels to run inside the
                     43:     Linux kernel.
                     45: Rump kernels are radically different from OS virtualization technologies
                     46: such as KVM, containers and usermode operating systems. A rump kernel
                     47: does not support hosting application processes because a rump kernel is
                     48: aimed at virtualizing kernel drivers and application virtualization
                     49: would be pure overhead. Instead, existing entities such as processes
                     50: from a hosting OS are used as clients for the rump kernel ("application"
                     51: in the figure).
                     53: As a result of the above design choices, rump kernels are extremely
                     54: lightweight. The bootstrap time for rump kernels on POSIX hosts is
                     55: measured in milliseconds and memory footprint in 100kB's. This means
                     56: that a rump kernel can be bootstrapped for example as part of a command
                     57: line tool for virtually no cost or user impact. Rump kernels also
1.2       pooka      58: mandate very little from the hypercall implementation meaning that
                     59: rump kernels, and by extension NetBSD kernel drivers, can be hosted in
                     60: virtually any environment -- for example, rump kernels do not require
                     61: a platform with an MMU.
1.1       pooka      62: 
                     63: Use cases for rump kernels include:
                     65: -   **Code reuse**: kernel drivers can be reused without having to run a
                     66:     whole OS. For example, a full-featured TCP/IP stack (IPv6, IPSec,
                     67:     etc.) can be included in an embedded appliance without having to
                     68:     write the stack from scratch or waste resources on running an entire
                     69:     OS.
                     70: -   **Kernel driver virtualization**: every rump kernel has its own
                     71:     state. Furthermore, the functionality offered by multiple rump
                     72:     kernels running on the same host does not need to be equal. For
                     73:     example, multiple different networking stacks optimized for
                     74:     different purposes are possible.
                     75: -   **Security**: when hosted on a POSIX system, a rump kernel runs in
                     76:     its own instance of a userspace process. For example, it is widely
                     77:     published that file system drivers are vulnerable to untrusted file
                     78:     system images. Unlike on other general purpose operating systems, on
                     79:     NetBSD it is possible to mount untrusted file systems, such as those
                     80:     on a USB stick, in an isolated server with the kernel file system
                     81:     driver. This isolates attacks and prevents kernel compromises while
                     82:     not requiring to maintain separate userspace implementations of the
                     83:     file system drivers or use other resource-intensive approaches such
                     84:     as virtual machines.
                     85: -   **Easy prototyping and development**: kernel code can be developed
                     86:     as a normal userspace application. Once development is finished, the
                     87:     code can simply be complied into the kernel. This is a much more
                     88:     convenient and straightforward approach to kernel development than
                     89:     the use of virtual machines.
                     90: -   **Safe testing**: kernel code can be tested in userspace on any host
                     91:     without risk of the test host being affected. Again, virtual
                     92:     machines are not required.
                     94: ## Further Reading
1.2       pooka      96: ### Book
1.1       pooka      97: 
1.2       pooka      98: The following book is the definitive guide to the anykernel and rump
                     99: kernels and supercedes all earlier publications and terminology on
                    100: the subject.
1.1       pooka     101: 
                    102: -   [Flexible Operating System Internals: The Design and Implementation
                    103:     of the Anykernel and Rump
                    104:     Kernels](
1.2       pooka     106: Note that the book was finalized in summer 2012, so while the fundamentals
                    107: are still accurate, some of the problems described in "Future Work"
                    108: have already been solved.  Check out the links below.
1.1       pooka     111: ### Software using rump kernels
1.2       pooka     113: While the NetBSD source tree hosts the base kernel drivers and hypercall
                    114: implementation, more I/O drivers, infrastructure scripts and hypercall
                    115: implementations are hosted elsewhere.  Most of the code is hosted
                    116: under the [rump kernels]( organization
                    117: on github.  Some highlights include:
1.1       pooka     118: 
                    119: -   [Scripts for building rump kernels for POSIX
1.2       pooka     120:     systems](
                    121: -   The [rumprun]( package
                    122:     allows portable building and running of unmodified NetBSD userspace
                    123:     applications -- extremely useful for configuring rump kernels (e.g.
                    124:     network interfaces and routing tables)
1.1       pooka     125: -   [Rump kernel hypercall implementation for Xen; rump kernels as Xen
1.2       pooka     126:     DomU's](
1.1       pooka     127: -   [fs-utils: File system image access
                    128:     utilities](
                    129: -   Fast userspace packet processing: TCP/IP stack for use with
1.3     ! pooka     130:     [DPDK](, 
        !           131:     [netmap]( or
        !           132:     [Snabb Switch](
1.1       pooka     133: 
                    134: ### Articles, Tutorials & Howtos
                    136: -   [Running rump kernels and applications on Xen without a full
                    137:     OS](
                    138: -   [PCI device driver support in rump kernels on
                    139:     Xen](
                    140: -   [Experiment with a rump kernel hypervisor for the Linux
                    141:     kernel](
                    142:     (allows rump kernels to run *in* the Linux kernel)
                    143: -   [Experiment on compiling rump kernels to javascript and running them
                    144:     in the
                    145:     browser](
                    146: -   [Kernel Servers using
                    147:     Rump](
                    148: -   [Tutorial On Rump Kernel Servers and
                    149:     Clients](
                    150: -   [Revolutionizing Kernel Development: Testing With
                    151:     Rump](
                    153: ### Conference publications and talks
                    155: -   "The Anykernel and Rump Kernels" gives a general overview and
                    156:     demonstrates rump kernels on Windows and in Firefox. The
                    157:     [video](,
                    158:     [slides](
                    159:     and an
                    160:     [interview](
                    161:     are available. Presented at FOSDEM 2013 (Operating Systems track).
                    162: -   "Rump Device Drivers: Shine On You Kernel Diamond" describes device
                    163:     driver and USB. The
                    164:     [paper](
                    165:     and [video presentation](
                    166:     are available. Presented at AsiaBSDCon 2010.
                    167: -   "Fs-utils: File Systems Access Tools for Userland" describes
                    168:     fs-utils, an mtools-like utility kit which uses rump kernel file
                    169:     systems as a backend. The
                    170:     [paper](
                    171:     is available. Presented at EuroBSDCon 2009.
                    172: -   "Rump File Systems: Kernel Code Reborn" describes kernel file system
                    173:     code and its uses in userspace. The
                    174:     [paper](
                    175:     and
                    176:     [slides](
                    177:     are available. Presented at the 2009 USENIX Annual Technical
                    178:     Conference.
                    179: -   "Kernel Development in Userspace - The Rump Approach" describes
                    180:     doing kernel development with rump kernels. The
                    181:     [paper](
                    182:     and
                    183:     [slides](
                    184:     are available. Presented at BSDCan 2009.
                    185: -   "Environmental Independence: BSD Kernel TCP/IP in Userspace"
                    186:     describes networking in rump kernels. The
                    187:     [paper]( and
                    188:     [video presentation]( are
                    189:     available. Presented at AsiaBSDCon 2009.
                    191: ### Manual pages
                    193: The manpages provide the usual type of information. Start from
                    194: [rump.3]( and
                    195: follow the cross-references in "SEE ALSO".
                    197: ## Discuss
                    199: Any topic related to rump kernels can be discussed on the
                    200: [rumpkernel-users mailing
                    201: list](
                    202: Alternatively, you can use a NetBSD mailing which is related to a
                    203: specific subtopic.
                    205: The IRC channel for rump kernels is **\#rumpkernel** on
                    206: ****.
                    208: ## Availability
                    210: The anykernel and rump kernels were first introduced as a prototype in
                    211: NetBSD 5.0. A stable version with numerous new features and improvements
                    212: was shipped with NetBSD 6.0.
                    214: ## Source Code
                    216: All of the source code is available from the NetBSD source tree and can
                    217: be obtained with the usual methods.
                    219: You can also [browse]( the
                    220: source code history online. Code is found from all areas of the source
                    221: tree. Some examples of where to look include
                    222: [src/lib](,
                    223: [src/usr.bin]( and
                    224: [src/sys/rump](

CVSweb for NetBSD wikisrc <> software: FreeBSD-CVSweb