QEMU for NetBSD
# Current status
Summary of the current status of QEMU hosted on a NetBSD host.
## What works?
Quick summary:
- The NetBSD target builds and works out of the box with elementary features.
- There is a pkgsrc package that ships a recent version.
- QEMU should work with all currently supported NetBSD versions starting from 6.x.
- QEMU as of today is not PaX MPROTECT safe.
- All tests in the check target pass.
- NetBSD is a maintained platform.
## Standard features matrix
Elementary features for NetBSD are probably well supported.
Name | Description | NetBSD status
----------------------|----------------------------------------------------|--------------
python | Python programming language | should work
smbd | Samba | net/samba, untested
git | GIT VCS | should work
debug | | should work
sanitizers | ASan, UBSan, ASan headers | should work
sanitizers Fibers | Asan headers with fibers | Futue GCC/Clang should work
stack-protector | Stack protector | works
audio oss | OSS audio backend | should work
audio sdl | SDL audio backend | should work
coroutine ucontext | | should work
coroutine sigaltstack | | should work
coroutine windows | | N/A, Windows specific
slirp | User networking | works
tcg-interpreter | Tiny Code Generator | should work
malloc-trim | GNU malloc(3) optimization | N/A, Linux specific
gcov | Test Coverage Program | should work
gprof | GNU profiling | should work
profiler | | should work
## Optional feature matrix
Additional features support various extensions,
most of them are a matter of using the emulator with a 3rd party and are OS independent.
Part of features require OS specific extensions.
Name | Description | NetBSD status
---------------|----------------------------------------------------|--------------
system | all system emulation targets | works
user | supported user emulation targets | broken
linux-user | all linux usermode emulation targets | N/A
bsd-user | all BSD usermode emulation targets | broken (FreeBSD ships local patches)
docs | build documentation | should work
guest-agent | build the QEMU Guest Agent | unknown, probably broken
guest-agent-msi| build guest agent Windows MSI installation package | unknown, probably broken
pie | Position Independent Executables | should work
modules | QEMU modules support (not kernel modules) | unknown, should work
debug-tcg | TCG debugging (default is disabled) | should work
debug-info | debugging information | should work
sparse | sparse checker | should work
gnutls | GNUTLS cryptography support | should work
nettle | nettle cryptography support | should work
gcrypt | libgcrypt cryptography support | should work
sdl | SDL UI | should work
--with-sdlabi | select preferred SDL ABI 1.2 or 2.0 | should work, not imporant
gtk | gtk UI | should work
--with-gtkabi | select preferred GTK ABI 2.0 or 3.0 | should work, not important
vte | vte support for the gtk UI | should work
curses | curses UI | works with native NetBSD 8.0 curses(3)
vnc | VNC UI support | should work
vnc-sasl | SASL encryption for VNC server | should work
vnc-jpeg | JPEG lossy compression for VNC server | should work
vnc-png | PNG compression for VNC server | should work
cocoa | Cocoa UI (Mac OS X only) | N/A, Darwin specific
virtfs | VirtFS | probably broken
mpath | Multipath persistent reservation passthrough | unknown, problably not supported
xen | xen backend driver support | unknown
xen-pci-passthrough | PCI passthrough support for Xen | unknown, probably not supported
brlapi | BrlAPI (Braile) | unknown, should work
curl | curl connectivity | should work
membarrier | membarrier system call (for Linux 4.14+ or Windows)| not supported
fdt | fdt device tree | unknown, should work
bluez | bluez stack connectivity | unknown
kvm | KVM acceleration support | N/A , Linux specific kernel APIs required in userland
hax | HAX acceleration support | mostly works
hvf | Hypervisor.framework acceleration support | not ported, Darwin specific
whpx | Windows Hypervisor Platform acceleration support | N/A
rdma | Enable RDMA-based migration and PVRDMA support | unknown, probably not supported
vde | support for vde network | probably not supported and Linux specific as of today
netmap | support for netmap network | not supported (FreeBSD specific?)
linux-aio | Linux AIO support | not supported, Linux specific
cap-ng | libcap-ng support | not supported, Linux specific
attr | attr and xattr support | N/A, Linux specific ?
vhost-net | vhost-net acceleration support | not supported
vhost-crypto | vhost-crypto acceleration support | not supported
spice | spice | unknown, probably not supported
rbd | rados block device (rbd) | unknown
libiscsi | iscsi support | unknown
libnfs | nfs support | unknown
smartcard | smartcard support (libcacard) | unknown
libusb | libusb (for usb passthrough) | unknown
live-block-migration | Block migration in the main migration stream| unknown
usb-redir | usb network redirection support | unknown
lzo | support of lzo compression library | should work
snappy | support of snappy compression library | should work
bzip2 | support of bzip2 compression library | should work
seccomp | seccomp support | Linux specific (?)
coroutine-pool | coroutine freelist (better performance) | unknown
glusterfs | GlusterFS backend | unknown
tpm | TPM support | unknown
libssh2 | ssh block device support | should work
numa | libnuma support | not ported
libxml2 | for Parallels image format | should work
tcmalloc | tcmalloc support | should work
jemalloc | jemalloc support | works
replication | replication support | not ported, N/A ?, Linux specific
vhost-vsock | virtio sockets device support | not ported
opengl | opengl support | unknown
virglrenderer | virgl rendering support | unknown
xfsctl | xfsctl support | N/A / not ported
qom-cast-debug | cast debugging support | unknown
tools | build qemu-io, qemu-nbd and qemu-image tools | works (userland nbd only)
vxhs | Veritas HyperScale vDisk backend support | N/A ?
crypto-afalg | Linux AF_ALG crypto backend driver | N/A / not ported
vhost-user | vhost-user support | not ported
capstone | capstone disassembler support | should work
## External features
1. HQEMU
> HQEMU is a retargetable and multi-threaded dynamic binary translator on multicores. It integrates QEMU and LLVM as its building blocks. The translator in the enhanced QEMU acts as a fast translator with low translation overhead. The optimization-intensive LLVM optimizer running on separate threads dynamically improves code for higher performance. With the hybrid QEMU+LLVM approach, HQEMU can achieve low translation overhead and good translated code quality.
> HQEMU supports process-level emulation and full-system virtualization. It provides translation modes of running the QEMU translator and LLVM optimizer in one process, or running the LLVM optimizer as a stand-alone optimization server (version 0.13.0).
2. DPDK
> DPDK is a set of libraries and drivers for fast packet processing.
3. The ultimate CPU emulator
> Unicorn is a lightweight multi-platform, multi-architecture CPU emulator framework.
## Prioritized tasks
1. Upstream remaining local pkgsrc patches, mostly (old?) NetBSD and SmartOS related ones.
2. Make QEMU PaX MPROTECT safe.
3. Develop NetBSD USER emulation, attempt to either share the code with FreeBSD (bsd-user, old broken, downstream patches) or with Linux (linux-user, recent, actively maintained). Preferred approach is to share as much code with linux-user as possible, regardless of the state of bsd-user.
## HAXM
> HAXM is a cross-platform hardware-assisted virtualization engine (hypervisor), widely used as an accelerator for Android Emulator and QEMU. It has always supported running on Windows and macOS, and has been ported to other host operating systems as well, such as Linux and NetBSD.
> HAXM runs as a kernel-mode driver on the host operating system, and provides a KVM-like interface to user space, thereby enabling applications like QEMU to utilize the hardware virtualization capabilities built into modern Intel CPUs, namely Intel Virtualization Technology.
More information on the porting process is available on The NetBSD blog:
A tutorial (outdated) by an early user of the support:
NetBSD/amd64 is the only supported host of the package, but it shouldn't be difficult to expand it to Darwin, Windows and Linux if there would be users and maintainers for these OSes.
NetBSD as host version 8.0 and HEAD (8.99.3x) tested. The minimal version that could work in theory is 6.x, but everything prior 8.0 is untested.
There is no longer need to keep a patched host kernel.
There is need to keep at least a copy of syssrc in /usr/src(/sys).
Guests known to work:
* NetBSD/amd64 8.99.3x (other variations untested)
* Windows 7 32-bit
* FreeBSD 12 32-bit
* Linux (noapic boot)
* Minix3 i386
* Plan9 (i386?)
* FREEDOS
Other OSes are either broken or untested.
Usage:
1. Install emulators/haxm from pkgsrc
2. Install emulators/qemu 3.1.0nb5 or newer from pkgsrc
3. Use auxiliary scripts for HAXM (superuser privileges needed)
4. Append HAXM option to qemu (qemu --accel hax)
Auxiliary scripts in the HAXM package:
* haxm-mknod # creates /dev entries
* haxm-modload # insert the HAXM kernel module
* haxm-modunload # remove the HAXM module from kernel
Optionally grant access to the HAXM device nodes to user(s):
* /dev/HAX
* /dev/hax_vm/vm{00-07}
* /dev/hax_vm{00-07}/vcpu{00-16}