Return to qemu.mdwn CVS log

Up to [NetBSD Developer Wiki] / wikisrc / users / kamil

Change the syntax attempting to improve the presentation of wiki page

QEMU for NetBSD

# Current status

Summary of the current status of QEMU hosted on a NetBSD host.

## What works?

Quick summary:

 - The NetBSD target builds and works out of the box with elementary features.

 - There is a pkgsrc package that ships a recent version.

 - QEMU should work with all currently supported NetBSD versions starting from 6.x.

 - QEMU as of today is not PaX MPROTECT safe.

 - All tests in the check target pass.

 - NetBSD is a maintained platform.

## Standard features matrix

Elementary features for NetBSD are probably well supported.

Name                  | Description                                        | NetBSD status
----------------------|----------------------------------------------------|--------------
python                | Python programming language                        | should work
smbd                  | Samba                                              | net/samba, untested
git                   | GIT VCS                                            | should work
debug                 |                                                    | should work
sanitizers            | ASan, UBSan, ASan headers                          | should work
sanitizers Fibers     | Asan headers with fibers                           | Futue GCC/Clang should work
stack-protector       | Stack protector                                    | works
audio oss             | OSS audio backend                                  | should work
audio sdl             | SDL audio backend                                  | should work
coroutine ucontext    |                                                    | should work
coroutine sigaltstack |                                                    | should work
coroutine windows     |                                                    | N/A, Windows specific
slirp                 | User networking                                    | works                     
tcg-interpreter       | Tiny Code Generator                                | should work
malloc-trim           | GNU malloc(3) optimization                         | N/A, Linux specific
gcov                  | Test Coverage Program                              | should work
gprof                 | GNU profiling                                      | should work
profiler              |                                                    | should work


## Optional feature matrix

Additional features support various extensions,
most of them are a matter of using the emulator with a 3rd party and are OS independent.

Part of features require OS specific extensions.

Name           | Description                                        | NetBSD status
---------------|----------------------------------------------------|--------------
system         | all system emulation targets                       | works
user           | supported user emulation targets                   | broken
linux-user     | all linux usermode emulation targets               | N/A
bsd-user       | all BSD usermode emulation targets                 | broken (FreeBSD ships local patches)
docs           | build documentation                                | should work
guest-agent    | build the QEMU Guest Agent                         | unknown, probably broken
guest-agent-msi| build guest agent Windows MSI installation package | unknown, probably broken
pie            | Position Independent Executables                   | should work
modules        | modules support                                    | unknown, should work
debug-tcg      | TCG debugging (default is disabled)                | should work
debug-info     | debugging information                              | should work
sparse         | sparse checker                                     | should work
gnutls         | GNUTLS cryptography support                        | should work
nettle         | nettle cryptography support                        | should work
gcrypt         | libgcrypt cryptography support                     | should work
sdl            | SDL UI                                             | should work
--with-sdlabi  |   select preferred SDL ABI 1.2 or 2.0              | should work, not imporant
gtk            | gtk UI                                             | should work
--with-gtkabi  |   select preferred GTK ABI 2.0 or 3.0              | should work, not important
vte            | vte support for the gtk UI                         | should work
curses         | curses UI                                          | works with native NetBSD 8.0 curses(3) 
vnc            | VNC UI support                                     | should work
vnc-sasl       | SASL encryption for VNC server                     | should work
vnc-jpeg       | JPEG lossy compression for VNC server              | should work
vnc-png        | PNG compression for VNC server                     | should work
cocoa          | Cocoa UI (Mac OS X only)                           | N/A, Darwin specific
virtfs         | VirtFS                                             | probably broken
mpath          | Multipath persistent reservation passthrough       | unknown, problably not supported
xen            | xen backend driver support                         | unknown
xen-pci-passthrough |   PCI passthrough support for Xen             | unknown, probably not supported
brlapi         | BrlAPI (Braile)                                    | unknown, should work
curl           | curl connectivity                                  | should work
membarrier     | membarrier system call (for Linux 4.14+ or Windows)| not supported
fdt            | fdt device tree                                    | unknown, should work
bluez          | bluez stack connectivity                           | unknown
kvm            | KVM acceleration support                           | N/A , Linux specific kernel APIs required in userland
hax            | HAX acceleration support                           | not ported, Windows/Darwin specific as of today
hvf            | Hypervisor.framework acceleration support          | not ported, Darwin specific
whpx           | Windows Hypervisor Platform acceleration support   | N/A
rdma           | Enable RDMA-based migration and PVRDMA support     | unknown, probably not supported
vde            | support for vde network                            | probably not supported and Linux specific as of today
netmap         | support for netmap network                         | not supported (FreeBSD specific?)
linux-aio      | Linux AIO support                                  | not supported, Linux specific
cap-ng         | libcap-ng support                                  | not supported, Linux specific
attr           | attr and xattr support                             | N/A, Linux specific ?
vhost-net      | vhost-net acceleration support                     | not supported
vhost-crypto   | vhost-crypto acceleration support                  | not supported
spice          | spice                                              | unknown, probably not supported
rbd            | rados block device (rbd)                           | unknown
libiscsi       | iscsi support                                      | unknown
libnfs         | nfs support                                        | unknown
smartcard      | smartcard support (libcacard)                      | unknown
libusb         | libusb (for usb passthrough)                       | unknown
live-block-migration |  Block migration in the main migration stream| unknown
usb-redir      | usb network redirection support                    | unknown
lzo            | support of lzo compression library                 | should work
snappy         | support of snappy compression library              | should work
bzip2          | support of bzip2 compression library               | should work
seccomp        | seccomp support                                    | Linux specific (?)
coroutine-pool | coroutine freelist (better performance)            | unknown
glusterfs      | GlusterFS backend                                  | unknown
tpm            | TPM support                                        | unknown
libssh2        | ssh block device support                           | should work
numa           | libnuma support                                    | not ported
libxml2        | for Parallels image format                         | should work
tcmalloc       | tcmalloc support                                   | should work
jemalloc       | jemalloc support                                   | works
replication    | replication support                                | not ported, N/A ?, Linux specific
vhost-vsock    | virtio sockets device support                      | not ported
opengl         | opengl support                                     | unknown
virglrenderer  | virgl rendering support                            | unknown
xfsctl         | xfsctl support                                     | N/A / not ported
qom-cast-debug | cast debugging support                             | unknown
tools          | build qemu-io, qemu-nbd and qemu-image tools       | works (userland nbd only)
vxhs           | Veritas HyperScale vDisk backend support           | N/A ?
crypto-afalg   | Linux AF_ALG crypto backend driver                 | N/A / not ported
vhost-user     | vhost-user support                                 | not ported
capstone       | capstone disassembler support                      | should work

## External features

1. HQEMU

> HQEMU is a retargetable and multi-threaded dynamic binary translator on multicores. It integrates QEMU and LLVM as its building blocks. The translator in the enhanced QEMU acts as a fast translator with low translation overhead. The optimization-intensive LLVM optimizer running on separate threads dynamically improves code for higher performance. With the hybrid QEMU+LLVM approach, HQEMU can achieve low translation overhead and good translated code quality.
> HQEMU supports process-level emulation and full-system virtualization. It provides translation modes of running the QEMU translator and LLVM optimizer in one process, or running the LLVM optimizer as a stand-alone optimization server (version 0.13.0).

[http://itanium.iis.sinica.edu.tw/hqemu/](http://itanium.iis.sinica.edu.tw/hqemu/)

2. DPDK

> DPDK is a set of libraries and drivers for fast packet processing.

[https://dpdk.org/](https://dpdk.org/)

3. The ultimate CPU emulator

> Unicorn is a lightweight multi-platform, multi-architecture CPU emulator framework. 

[https://www.unicorn-engine.org/](https://www.unicorn-engine.org/)

## Prioritized tasks

1. Upstream remaining local pkgsrc patches, mostly (old?) NetBSD and SmartOS related ones.

2. Make QEMU PaX MPROTECT safe.

3. Develop NetBSD USER emulation, attempt to either share the code with FreeBSD (bsd-user, old broken, downstream patches) or with Linux (linux-user, recent, actively maintained). Preferred approach is to share as much code with linux-user as possible, regardless of the state of bsd-user.

4. Develop HAXM backend support for the NetBSD kernel.

## HAXM hardware assisted virtualization

Features:

 - Open Source
 - HAXM is an Intel hardware assisted virtualization.
 - HAXM supports up to 8 active VMs.
 - HAXM supports up to 16 active VCPUs per VM.
 - ioctl(2) based API with statically created devices (/dev/HAX, /dev/haxm_vm/vmXX, /dev/haxm_vm/haxm_vmXX_vcpuYY)

HAXM porting for NetBSD rationale:

 - Relatively low cost (compared to alternatives) to get it aboard to the NetBSD kernel.
 - Relatively non-intrusive (compared to alternatives like bhyve and KVM)
 - Scratch but non-functional port for NetBSD is ready.
 - Active (paid developers) upstream to maintain this
 - Clean separation between kernel specific and independent parts (there are at least 2 backends: Darwin and Windows)
 - QEMU frontend already exists, API is ioctl(2) based
 - Good license (BSD-3-clause) for the kernel driver
 - Tailored down for desktop users, initially prepared for Android developers

HAXM shortcomings:

 - No AMD support
 - Required relatively recent Intel CPU
 - Not as flexible as KVM-like solutions for embedded or server usage
 - Not necessarily ready (not tested) to run every OS in existence, there are known failures to execute some versions of the Linux kernel

Current HAXM/NetBSD status:

 - The module builds and loads into the kernel (triggering a panic)
 - Revisit the UVM APIs to ship all the needed features of sharing memory between user and kernel that is used by HAXM

> https://github.com/krytarowski/haxm/tree/netbsd-1
> 
> $ cd netbsd && make

External links:

[https://www.qemu.org/2017/11/22/haxm-usage-windows/](https://www.qemu.org/2017/11/22/haxm-usage-windows/)

[https://software.intel.com/en-us/articles/intel-hardware-accelerated-execution-manager-intel-haxm](https://software.intel.com/en-us/articles/intel-hardware-accelerated-execution-manager-intel-haxm)

[https://github.com/intel/haxm](https://github.com/intel/haxm)