Annotation of wikisrc/rumpkernel.mdwn, revision 1.3
1.1 pooka 1: **Contents**
2:
3: [[!toc levels=2]]
4:
5: # The Anykernel and Rump Kernels
6:
7: ## About
8:
9: A driver abstracts an underlying entity. For example, a TCP/IP driver
10: abstracts the details required to perform networking, the Fast File
11: System (FFS) driver abstracts how the file system blocks are laid out on
12: the storage medium, and a PCI network card driver abstracts how to
13: access device registers to send and receive packets. The kernel
14: architecture controls how kernel drivers run with respect to other
15: system components. Some examples of kernel architectures are the
16: monolithic kernel, microkernel and exokernel. In contrast to the above
17: architectures, NetBSD is an *anykernel*. This means that some kernel
18: drivers can be run according to any kernel architecture instead of being
19: limited to a single one.
20:
21: <!--
22: TODO
23: 
24: -->
25:
26: When a driver is not run as part of the monolithic kernel, e.g. when it
27: is run as a microkernel style server, the driver is hosted in a *rump
28: kernel*. A rump kernel is an ultralightweight virtualized kernel running
29: on top of high-level hypercall interface. Instead of a low-level
30: hypercall API typically seen with operating systems with operations such
31: as "modify page table", the rump kernel hypercall API provides
32: high-level operations such as "run this code in a thread".
33:
34: Currently, three implementations of the rump kernel hypercall interface
35: exist.
36:
1.2 pooka 37: - The POSIX implementation is included in the NetBSD tree and allows
1.1 pooka 38: rump kernels to run as userspace processes on most operating systems
39: such as NetBSD, Linux and Solaris.
40: - The Xen implementation allows running rump kernels directly as Xen
41: DomU's without an intermediate operating system.
42: - The Linux kernel hypervisor allows rump kernels to run inside the
43: Linux kernel.
44:
45: Rump kernels are radically different from OS virtualization technologies
46: such as KVM, containers and usermode operating systems. A rump kernel
47: does not support hosting application processes because a rump kernel is
48: aimed at virtualizing kernel drivers and application virtualization
49: would be pure overhead. Instead, existing entities such as processes
50: from a hosting OS are used as clients for the rump kernel ("application"
51: in the figure).
52:
53: As a result of the above design choices, rump kernels are extremely
54: lightweight. The bootstrap time for rump kernels on POSIX hosts is
55: measured in milliseconds and memory footprint in 100kB's. This means
56: that a rump kernel can be bootstrapped for example as part of a command
57: line tool for virtually no cost or user impact. Rump kernels also
1.2 pooka 58: mandate very little from the hypercall implementation meaning that
59: rump kernels, and by extension NetBSD kernel drivers, can be hosted in
60: virtually any environment -- for example, rump kernels do not require
61: a platform with an MMU.
1.1 pooka 62:
63: Use cases for rump kernels include:
64:
65: - **Code reuse**: kernel drivers can be reused without having to run a
66: whole OS. For example, a full-featured TCP/IP stack (IPv6, IPSec,
67: etc.) can be included in an embedded appliance without having to
68: write the stack from scratch or waste resources on running an entire
69: OS.
70: - **Kernel driver virtualization**: every rump kernel has its own
71: state. Furthermore, the functionality offered by multiple rump
72: kernels running on the same host does not need to be equal. For
73: example, multiple different networking stacks optimized for
74: different purposes are possible.
75: - **Security**: when hosted on a POSIX system, a rump kernel runs in
76: its own instance of a userspace process. For example, it is widely
77: published that file system drivers are vulnerable to untrusted file
78: system images. Unlike on other general purpose operating systems, on
79: NetBSD it is possible to mount untrusted file systems, such as those
80: on a USB stick, in an isolated server with the kernel file system
81: driver. This isolates attacks and prevents kernel compromises while
82: not requiring to maintain separate userspace implementations of the
83: file system drivers or use other resource-intensive approaches such
84: as virtual machines.
85: - **Easy prototyping and development**: kernel code can be developed
86: as a normal userspace application. Once development is finished, the
87: code can simply be complied into the kernel. This is a much more
88: convenient and straightforward approach to kernel development than
89: the use of virtual machines.
90: - **Safe testing**: kernel code can be tested in userspace on any host
91: without risk of the test host being affected. Again, virtual
92: machines are not required.
93:
94: ## Further Reading
95:
1.2 pooka 96: ### Book
1.1 pooka 97:
1.2 pooka 98: The following book is the definitive guide to the anykernel and rump
99: kernels and supercedes all earlier publications and terminology on
100: the subject.
1.1 pooka 101:
102: - [Flexible Operating System Internals: The Design and Implementation
103: of the Anykernel and Rump
104: Kernels](http://lib.tkk.fi/Diss/2012/isbn9789526049175/isbn9789526049175.pdf)
105:
1.2 pooka 106: Note that the book was finalized in summer 2012, so while the fundamentals
107: are still accurate, some of the problems described in "Future Work"
108: have already been solved. Check out the links below.
109:
110:
1.1 pooka 111: ### Software using rump kernels
112:
1.2 pooka 113: While the NetBSD source tree hosts the base kernel drivers and hypercall
114: implementation, more I/O drivers, infrastructure scripts and hypercall
115: implementations are hosted elsewhere. Most of the code is hosted
116: under the [rump kernels](https://github.com/rumpkernel/) organization
117: on github. Some highlights include:
1.1 pooka 118:
119: - [Scripts for building rump kernels for POSIX
1.2 pooka 120: systems](https://github.com/rumpkernel/buildrump.sh)
121: - The [rumprun](https://github.com/rumpkernel/rumprun/) package
122: allows portable building and running of unmodified NetBSD userspace
123: applications -- extremely useful for configuring rump kernels (e.g.
124: network interfaces and routing tables)
1.1 pooka 125: - [Rump kernel hypercall implementation for Xen; rump kernels as Xen
1.2 pooka 126: DomU's](https://github.com/rumpkernel/rumpuser-xen)
1.1 pooka 127: - [fs-utils: File system image access
128: utilities](https://github.com/stacktic/fs-utils)
129: - Fast userspace packet processing: TCP/IP stack for use with
1.3 ! pooka 130: [DPDK](https://github.com/rumpkernel/dpdk-rumptcpip),
! 131: [netmap](https://github.com/rumpkernel/netmap-rumptcpip) or
! 132: [Snabb Switch](https://github.com/anttikantee/snabbswitch/tree/rumpkernel/)
1.1 pooka 133:
134: ### Articles, Tutorials & Howtos
135:
136: - [Running rump kernels and applications on Xen without a full
137: OS](http://blog.NetBSD.org/tnf/entry/running_applications_on_the_xen)
138: - [PCI device driver support in rump kernels on
139: Xen](http://blog.NetBSD.org/tnf/entry/pci_driver_support_for_rump)
140: - [Experiment with a rump kernel hypervisor for the Linux
141: kernel](http://blog.NetBSD.org/tnf/entry/a_rump_kernel_hypervisor_for)
142: (allows rump kernels to run *in* the Linux kernel)
143: - [Experiment on compiling rump kernels to javascript and running them
144: in the
145: browser](http://blog.NetBSD.org/tnf/entry/kernel_drivers_compiled_to_javascript)
146: - [Kernel Servers using
147: Rump](http://www.NetBSD.org/docs/rump/sysproxy.html)
148: - [Tutorial On Rump Kernel Servers and
149: Clients](http://www.NetBSD.org/docs/rump/sptut.html)
150: - [Revolutionizing Kernel Development: Testing With
151: Rump](http://blog.NetBSD.org/tnf/entry/revolutionizing_kernel_development_testing_with)
152:
153: ### Conference publications and talks
154:
155: - "The Anykernel and Rump Kernels" gives a general overview and
156: demonstrates rump kernels on Windows and in Firefox. The
157: [video](http://video.fosdem.org/2013/maintracks/K.1.105/The_Anykernel_and_Rump_Kernels.webm),
158: [slides](https://fosdem.org/2013/schedule/event/operating_systems_anykernel/attachments/slides/244/export/events/attachments/operating_systems_anykernel/slides/244/fosdem2013_rumpkern.pdf)
159: and an
160: [interview](https://archive.fosdem.org/2013/interviews/2013-antii-kantee/)
161: are available. Presented at FOSDEM 2013 (Operating Systems track).
162: - "Rump Device Drivers: Shine On You Kernel Diamond" describes device
163: driver and USB. The
164: [paper](http://ftp.NetBSD.org/pub/NetBSD/misc/pooka/tmp/rumpdev.pdf)
165: and [video presentation](http://www.youtube.com/watch?v=3AJNxa33pzk)
166: are available. Presented at AsiaBSDCon 2010.
167: - "Fs-utils: File Systems Access Tools for Userland" describes
168: fs-utils, an mtools-like utility kit which uses rump kernel file
169: systems as a backend. The
170: [paper](http://www.ukuug.org/events/eurobsdcon2009/papers/ebc09_fs-utils.pdf)
171: is available. Presented at EuroBSDCon 2009.
172: - "Rump File Systems: Kernel Code Reborn" describes kernel file system
173: code and its uses in userspace. The
174: [paper](http://usenix.org/events/usenix09/tech/full_papers/kantee/kantee.pdf)
175: and
176: [slides](http://usenix.org/events/usenix09/tech/slides/kantee.pdf)
177: are available. Presented at the 2009 USENIX Annual Technical
178: Conference.
179: - "Kernel Development in Userspace - The Rump Approach" describes
180: doing kernel development with rump kernels. The
181: [paper](http://www.bsdcan.org/2009/schedule/attachments/104_rumpdevel.pdf)
182: and
183: [slides](http://www.bsdcan.org/2009/schedule/attachments/105_bsdcan09-kantee.pdf)
184: are available. Presented at BSDCan 2009.
185: - "Environmental Independence: BSD Kernel TCP/IP in Userspace"
186: describes networking in rump kernels. The
187: [paper](http://2009.asiabsdcon.org/papers/abc2009-P5A-paper.pdf) and
188: [video presentation](http://www.youtube.com/watch?v=RxFctq8A0WI) are
189: available. Presented at AsiaBSDCon 2009.
190:
191: ### Manual pages
192:
193: The manpages provide the usual type of information. Start from
194: [rump.3](http://man.NetBSD.org/cgi-bin/man-cgi?rump++NetBSD-current) and
195: follow the cross-references in "SEE ALSO".
196:
197: ## Discuss
198:
199: Any topic related to rump kernels can be discussed on the
200: [rumpkernel-users mailing
201: list](https://lists.sourceforge.net/lists/listinfo/rumpkernel-users).
202: Alternatively, you can use a NetBSD mailing which is related to a
203: specific subtopic.
204:
205: The IRC channel for rump kernels is **\#rumpkernel** on
206: **irc.freenode.net**.
207:
208: ## Availability
209:
210: The anykernel and rump kernels were first introduced as a prototype in
211: NetBSD 5.0. A stable version with numerous new features and improvements
212: was shipped with NetBSD 6.0.
213:
214: ## Source Code
215:
216: All of the source code is available from the NetBSD source tree and can
217: be obtained with the usual methods.
218:
219: You can also [browse](http://cvsweb.NetBSD.org/bsdweb.cgi/src/) the
220: source code history online. Code is found from all areas of the source
221: tree. Some examples of where to look include
222: [src/lib](http://cvsweb.NetBSD.org/bsdweb.cgi/src/lib/),
223: [src/usr.bin](http://cvsweb.NetBSD.org/bsdweb.cgi/src/usr.bin/) and
224: [src/sys/rump](http://cvsweb.NetBSD.org/bsdweb.cgi/src/sys/rump/).
CVSweb for NetBSD wikisrc <wikimaster@NetBSD.org> software: FreeBSD-CVSweb
![[BACK]](/icons/back.gif){kind=icon}
Return to rumpkernel.mdwn CVS log ![[TXT]](/icons/text.gif){kind=icon}
![[DIR]](/icons/dir.gif){kind=icon}
Up to [NetBSD Developer Wiki] / wikisrc