Google Summer of Code logo We are very happy to announce The NetBSD Foundation Google Summer of Code 2018 projects:

We have started the community bonding period where students get in touch with mentors and community and get more confident with documentation and the code base. The coding period will start from May 14 until August 6.

Please welcome all our students and a big good luck to students and mentors!

A big thank you also to Google and The NetBSD Foundation Organization Administrators!

Looking forward to a nice Google Summer of Code!

Posted mid-morning Tuesday, May 1st, 2018 Tags:
For the past month I've been mostly working on improving the kernel code in the ptrace(2) API. Additionally, I've prepared support for reading NetBSD/aarch64 core(5) files.


I've rebased my old patches for handling NetBSD core(5) files to recent LLDB from HEAD (7.0svn). I had to implement the support for aarch64, as requested by Joerg Sonnenberger for debugging NetBSD/arm64 bugs. The NetBSD case is special here, as we set general purpose registers in 128bit containers, while other OSes prefer 64-bit ones. I had to add a local kludge to cast the interesting 64-bit part of the value.

I've generated a local prebuilt toolchain with lldb prepared for a NetbBSD/amd64 host and shared with developers.

Debug Registers

I've improved the security of Debug Registers, with the following changes:

  • x86_dbregs_setup_initdbstate() stores the initial status of Debug Registers, I've explicitly disabled some of their bits for further operation. This register context is used when initializing Debug Registers for a userland thread. This is a paranoid measure used just in case a bootloader uses them and leaves the bits lit up.
  • I've added a new sysctl(3) switch security.models.extensions.user_set_dbregs: "Whether unprivileged users may set CPU Debug Registers". The Debug Registers use privileged operations and we decided to add another check, disabling them by default. There is no regress in the elementary functionality and a user can still read the Debug Register context.
  • This new sysctl(3) functionality has been covered by ATF checks in existing tests (i386 and amd64 for now).

fork(2) and vfork(2)

I've pushed various changes in the kernel subsystems and ATF userland tests for the ptrace(2) functionality:

  • I've integrated all fork(2) and vfork(2) tests into the shared function body. They tests PTRACE_FORK, PTRACE_VFORK and PTRACE_VFORK_DONE. There are now eight fork1..fork8 and eight vfork1..vfork8 tests (total 16). I'm planning to cover several remaining corner case scenarios in the forking code.
  • I've removed an unused variable from linux_sys_clone(), which calls fork1(9). This provides an opportunity to refactor the generic code.
  • I've refactored start_init() setting inside it the initproc pointer. This eliminates the remaining user of rnewprocp in the fork1(9) API.
  • I've removed the rnewprocp argument from fork1(9). This argument in the current implementation could cause use-after-free in corner cases.
  • PTRACE_VFORK has been implemented and made functional.

Security hardening

I've prohibited calling PT_TRACE_ME for a process that is either initproc (PID1) or a direct child of PID1.

I've prohibited calling PT_ATTACH from initproc. This shouldn't happen in normal circumstances, but just in case this would lead to invalid branches in the kernel.

With the above alternations, I've removed a bug causing a creation of a process that is not debuggable. It's worth to note that this bug still exists in other popular kernels. A simple reproducer for pre-8.0 and other OSes using ptrace(2) (Linux, most BSDs ...):

    $ cat antidebug.c
    #include <sys/types.h>
    #include <sys/ptrace.h>
    #include <stdlib.h>
    #include <unistd.h>
    #include <stdio.h>
    #include <errno.h>
    main(int argc, char **argv)
            pid_t child;
            int rv;
            int n = 0;
            child = fork();
            if (child == 0) {
                    while (getppid() != 1)
                    rv = ptrace(PT_TRACE_ME, 0, 0, 0);
                    if (rv != 0)
                    printf("Try to detach to me with a debugger!! ");
                    printf("haha My PID is %d\n", getpid());
                    while (1) {
                            printf("%d\n", n++);

Additionally it's no longer valid calling and returning success PT_TRACE_ME when a process is already traced.

These security changes are covered by new ATF ptrace(2) tests:

  1. traceme_pid1_parent - Assert that a process cannot mark its parent a debugger twice.
  2. traceme_twice - Verify that PT_TRACE_ME is not allowed when our parent is PID1.

Other ptrace(2) enhancements

A list of other changes:

  • I've refactored part of the existing ATF ptrace(2) tests, focusing on code deduplication and covering more corner cases. I'm now especially testing PT_TRACE_ME existing scenarios with signals from various categories: SIGSTOP, SIGCONT, SIGKILL, SIGABRT, SIGHUP. Some signals are non-maskable ones, some generate core(5) files etc. They might use different kernel paths and I'm now covering them in a more uniform and hopefully more maintainable form.
  • In the kernel, I've removed unused branch in the proc_stop_done() and sigswitch() functions. This functionality was used in deprecated filesystem tracing feature (/proc). It prevented emitting child signals to a debugging program, namely with the SIGCHLD signal.

    The modern solution to perform tracing without signals in a debugger is to spawn a debugging server and outsource the tracing functionality to it. This is done in software like gdb-server, lldb-server etc.

  • I've removed a stray XXX comment from I and D read/write operations, and added a proper explanation:

    +        /*
    +        * The I and D separate address space has been inherited from PDP-11.
    +        * The 16-bit UNIX started with a single address space per program,
    +        * but was extended to two 16-bit (2 x 64kb) address spaces.
    +        *
    +        * We no longer maintain this feature in maintained architectures, but
    +        * we keep the API for backward compatiblity. Currently the I and D
    +        * operations are exactly the same and not distinguished in debuggers.
    +        */
  • I've improved the WCOREDUMP() checker in the ATF tests, casting it to either 1 or 0. This helps to handler correctly the tests generating core(5) files.
  • Improve of the proc_stoptrace() function. proc_stoptrace() is dedicated for emitting a syscall trap for a debugger, either on entry or exit of the system function routine.


    • Change an if() branch of an invalid condition of being traced by initproc (PID1) to KASSERT(9).
    • Assert that the current process has set appropriate flags (PSL_TRACED and PSL_SYSCALL).
    • Use ktrpoint(KTR_PSIG) and ktrpsig()/e_ktrpsig() in order to register the emitted signal for the ktrace(1) event debugging.


A critical Problem Report kern/51630 regarding lack of PTRACE_VFORK implementation has been fixed. This means that there are no other unimplemented API calls, but there are still bugs in the existing ones.

With fixes and addition of new test cases, as of today we are passing 961 ptrace(2) tests and skipping 1 (out of 1018 total).

Plan for the next milestone

Cover the remaining forking corner-cases in the context of debuggers with new ATF tests and fix the remaining bugs.

The first step is to implement proper support for handling PT_TRACE_ME-traced scenarios from a vfork(2)ed child. Next I plan to keep covering the corner cases of the forking code and finish this by removal of subtle bugs that are still left in the code since the SMP'ification.

This work was sponsored by The NetBSD Foundation.

The NetBSD Foundation is a non-profit organization and welcomes any donations to help us continue funding projects and services to the open-source community. Please consider visiting the following URL, and chip in what you can:

Posted late Tuesday night, May 2nd, 2018 Tags:

I've recently been gifted a fancy laptop - a Dell XPS 15 9550.
I want to run NetBSD on it and have it run well, and I've set aside time to achieve this.
Things work surprisingly well out of the box - touchscreen and touchpad work, but need to be calibrated better. 4K video is displayed, but without graphical acceleration, which is somewhat slow. I could adjust the resolution to be lower, but haven't tried yet.

These are some of the lessons I learned from doing this, being new to drivers and networking.

rtsx - SD card reader

This one was simple and straightforward.
The PCI ID of my SD is 10ec:525a, a realtek PCI device with ID 0x525a.
Normal SD card readers are supported by a generic driver, but mine wasn't.
I've been told that when SD cards don't present themselves as the generic device, it is for good reason.

Adding the PCI device ID to sys/dev/pci/pcidevs would have revealed the adjacent entries:

product REALTEK RTS5229		0x5229	RTS5229 PCI-E Card Reader
product REALTEK RTS5249		0x5249	RTS5249 PCI-E Card Reader
product REALTEK RTS525A		0x525A	RTS525A PCI-E Card Reader <- my addition
product REALTEK RTL8402		0x5286	RTL8402 PCI-E Card Reader

These are all supported by rtsx(4), which is a driver originally from OpenBSD.
Looking at the driver in OpenBSD, it has diverged since, and indeed someone had added RTS525A support.

I've initially tried to synchronize the driver, but it seemed beyond my capabilities, and just went to steal the support.
Found the individual commit and it was kinda ugly, there are conditionals for variants in some functions.
I've added the RTS525A in the matching conditionals.

It would have been nice to make it less ugly, but I'm only adding a bit of fuel to the fire.
I've received hints that drivers for hardware from some manufacturers is going to be ugly either way.

I saw that "read-only detect" is not supported. Curious if I could added, I wondered what happens if you try to write when the SD card's read-only switch is held. Apparently this is a suggestion for software, I could read back the contents I wrote after detach. perhaps not having the support isn't too bad.

bwfm - wireless networking

Previously, Broadcom wifi in a laptop would have resulted in sadness, followed by disassembly and hopeful replacement, unless the manufacturer has chosen to whitelist wireless cards, in which case I am doomed to a USB wifi card.
Thanks to bluerise@openbsd, jmcneill and Broadcom for open sourcing their drivers, we now have a port of the linux brcmfmac driver, for Broadcom FullMAC wireless devices. Mine is one.

These devices do a lot of 802.11 weight lifting on their own. Unfortunately that means that if they hadn't had their firmware updated, vulnerabilities are likely, such as KRACK. Previously jmcneill had made it work for the USB variant, and even made it use some of the kernel's 802.11 code, allowing to avoid some vulnerabilities.

He did not port the PCI variant, added later in OpenBSD. I've set out to steal this code.

First and foremost, I need the ability to express 'please build the pci variant code', or to add it to the pci config.
The files I have for the driver are:

sys/dev/ic/{bwfm.c, bwfmvar.h, bwfmreg.h}
The USB variant already exists, it uses:
sys/dev/ic/{bwfm.c, bwfmvar.h, bwfmreg.h}
ic is something shared among pci, usb, sdio code. the code for it is already built due to being added to sys/conf/files.
I only need to add the PCI code.

The file for this is sys/dev/pci/files.pci. Fortunately it has plenty of examples, and here is what I added:

# Broadcom FullMAC USB wireless adapter
attach	bwfm at pci with bwfm_pci: firmload
file	dev/pci/if_bwfm_pci.c		bwfm_pci

I've had to adapt APIs - some OpenBSD ones not existing in NetBSD. Fortunately there are lots of drivers to look around. I've made a very simple if_rxr_* and made it static. MCLGETI I copied from another driver, again static.
My first attempt at if_rxr_* (however simple) didn't seem to work. I rewrote it and made sure to KASSERT assumptions I assume are correct.
I was hoping to find an existing "NetBSD-ish" API for this, but I'm not sure there is something quite like it, and it is pretty simple. But it was a good opportunity to learn more about NetBSD APIs.

Once I got it to compile and not panic on attach, I had a problem. sending a message to the device failed to get the appropriate response, and attach failed.
I've spent a long time reading the initialization sequence, in OpenBSD and in linux brcmfmac, but couldn't find anything. I was ready to give up, but then I decided to diff my file to OpenBSD, and saw that in an attempt to calm type warnings (fatal by default, as NetBSD builds with -Werror), I made some unnecessary and incorrect changes.
I've switched to building without -Werror (that means passing -V NOGCCERROR=1 to, or adding to mk.conf) and reverted this changes, and then it attached cleanly.
In retrospect, I shouldn't have made unnecessary changes as a beginner. -V NOGCCERROR=1 is there. Device work first, cosmetic warning fixes second.

After it had attached, I tried to connect. I hoped wpa_supplicant -d would tell why it failed, but it only said that the connection times out.
Like many drivers in the *BSD world, bwfm has a BWFM_DEBUG and a debug printf that is controllable. I turned it on to get these messages from bwfm_rx_event:

bwfm0: newstate 0 -> 1
bwfm0: newstate 1 -> 1
bwfm_rx_event: buf 0xffffe404b97f9010 len 354 datalen 280 code 69 status 8 reason 0
bwfm_rx_event: buf 0xffffe404b97f9010 len 354 datalen 280 code 69 status 8 reason 0
bwfm_rx_event: buf 0xffffe404b97f9010 len 362 datalen 288 code 69 status 8 reason 0
bwfm_rx_event: buf 0xffffe404b97f9010 len 354 datalen 280 code 69 status 8 reason 0
bwfm_rx_event: buf 0xffffe404b97f9010 len 354 datalen 280 code 69 status 8 reason 0
bwfm_rx_event: buf 0xffffe404b97f9010 len 354 datalen 280 code 69 status 8 reason 0
bwfm_rx_event: buf 0xffffe404b97f9010 len 362 datalen 288 code 69 status 8 reason 0
bwfm_rx_event: buf 0xffffe404b97f9010 len 354 datalen 280 code 69 status 8 reason 0
bwfm_rx_event: buf 0xffffe404b97f9010 len 86 datalen 12 code 69 status 0 reason 0
bwfm_rx_event: buf 0xffffe404b9837010 len 74 datalen 0 code 26 status 0 reason 0
bwfm0: newstate 1 -> 2
bwfm_rx_event: buf 0xffffe404b9837010 len 82 datalen 8 code 0 status 3 reason 0
bwfm0: newstate 2 -> 1

To decipher the messages, I picked up what the various numbers means. Deciphered, it says:

802.11 state INIT -> SCAN
802.11 state SCAN -> SCAN
received messages from the driver:
code 69 (scan result)
code 26 (scan end)
802.11 state SCAN -> AUTH
code 0 (SET_SSID) status 3 (NO NETWORKS)
802.11 state AUTH -> SCAN

Authenticating is failing to find a matching network. But why?
Now experienced with the previous experience ("I've found my problem! it was me!"), I went back to some more attempts to sync the driver, which changed the SSID-setting code.

With this corrected, the driver worked as expected.

In total, it would have been easier to start with a working driver and adapting it to make some changes, like having the USB variant or testing on OpenBSD, but I managed anyway, it just took longer.
In two weeks I will receive the USB variant, but the driver is already working for me. I had trouble installing OpenBSD on the same machine.

I'll clean up my changes and post it for review soon.

Thanks to jak, pgoyette, rkujawa, phone for various fixes.
Thanks to maxv for fixing a bug affecting my use of a network stack API.
Thanks to bluerise@openbsd and jmcneill for the prior driver work.
Thanks to Broadcom for providing open source drivers under a friendly license.

remaining work - graphical acceleration.

NetBSD's x86 graphical acceleration is mostly from Linux, from linux 3.15.
This predates support for my Skylake GPU and nVidia graphics on my machine, and also the ability to dual-GPU effectively.
I'll try to sync the drivers and made previous attempts to learn about graphics, but I can't promise anything.

Posted late Thursday evening, May 3rd, 2018 Tags:

Pinebook is an affordable 64-bit ARM notebook. Today we're going to take a look at the kernel output at startup and talk about what hardware support is available on NetBSD.

Loaded initial symtab at 0xffffffc0007342c0, strtab at 0xffffffc0007e3208, # entries 29859
Copyright (c) 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005,
2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014, 2015, 2016, 2017,
2018 The NetBSD Foundation, Inc.  All rights reserved.
Copyright (c) 1982, 1986, 1989, 1991, 1993
The Regents of the University of California.  All rights reserved.

NetBSD 8.99.17 (GENERIC64) #272: Mon May 21 07:25:46 ADT 2018
total memory = 2007 MB
avail memory = 1942 MB

Pinebook comes with 2GB RAM standard. A small amount of this is reserved by the kernel and framebuffer.

timecounter: Timecounters tick every 10.000 msec
armfdt0 (root)
fdt0 at armfdt0: Pine64 Pinebook
fdt1 at fdt0
fdt2 at fdt0

NetBSD uses flattened device-tree (FDT) to enumerate devices on all Allwinner based SoCs. On a running system, you can inspect the device tree using the ofctl(8) utility:

$ ofctl -p /sound  
[Caching 111 nodes and 697 properties]
compatible              73696d70 6c652d61 7564696f 2d636172   "simple-audio-car
            0010:       6400.... ........ ........ ........   d"
name                    736f756e 6400.... ........ ........   "sound"
                        69327300 ........ ........ ........   i2s.
                        0000001c ........ ........ ........   ....
simple-audio-card,name  53554e35 30492041 7564696f 20436172   SUN50I Audio Car
            0010:       6400.... ........ ........ ........   d.
cpus0 at fdt0
cpu0 at cpus0: Cortex-A53 r0p4 (Cortex V8-A core)
cpu0: CPU Affinity 0-0-0-0
cpu0: IC enabled, DC enabled, EL0/EL1 stack Alignment check enabled
cpu0: Cache Writeback Granule 16B, Exclusives Reservation Granule 16B
cpu0: Dcache line 64, Icache line 64
cpu0: L1 32KB/64B 2-way read-allocate VIPT Instruction cache
cpu0: L1 32KB/64B 4-way write-back read-allocate write-allocate PIPT Data cache
cpu0: L2 512KB/64B 16-way write-back read-allocate write-allocate PIPT Unified cache
cpu0: revID=0x80, PMCv3, 4k table, 64k table, 16bit ASID
cpu0: auxID=0x11120, FP, CRC32, SHA1, SHA256, AES+PMULL, NEON, rounding, NaN propagation, denormals, 32x64bitRegs, Fused Multiply-Add

Pinebook's Allwinner A64 processor is based on the ARM Cortex-A53. It is designed to run at frequencies up to 1.2GHz.

cpufreqdt0 at cpu0

Dynamic voltage and frequency scaling (DVFS) is supported for the CPU. More on this in a bit...

cpu1 at cpus0: disabled (uniprocessor kernel)
cpu2 at cpus0: disabled (uniprocessor kernel)
cpu3 at cpus0: disabled (uniprocessor kernel)

The A64 is a quad core design. NetBSD's aarch64 pmap does not yet support SMP, so three cores are disabled for now.

gic0 at fdt1: GIC
armgic0 at gic0: Generic Interrupt Controller, 224 sources (215 valid)
armgic0: 16 Priorities, 192 SPIs, 7 PPIs, 16 SGIs

The interrupt controller is a standard ARM GIC-400 design.

fclock0 at fdt0: 24000000 Hz fixed clock
fclock1 at fdt0: 32768 Hz fixed clock
fclock2 at fdt0: 16000000 Hz fixed clock

Fixed clock input signals.

gtmr0 at fdt0: Generic Timer
gtmr0: interrupting on GIC irq 27
armgtmr0 at gtmr0: ARMv7 Generic 64-bit Timer (24000 kHz)
timecounter: Timecounter "armgtmr0" frequency 24000000 Hz quality 500

Standard ARMv7/ARMv8 architected timer.

sun50ia64ccu0 at fdt1: A64 CCU
sun50ia64rccu0 at fdt1: A64 PRCM CCU

Clock drivers for managing PLLs, module clock dividers, clock gating, software resets, etc. Information about the clock tree is exported in the hw.clk sysctl namespace (root access required to read these values).

# sysctl hw.clk.sun50ia64ccu0.mmc2
hw.clk.sun50ia64ccu0.mmc2.rate = 200000000
hw.clk.sun50ia64ccu0.mmc2.parent = pll_periph0_2x
hw.clk.sun50ia64ccu0.mmc2.parent_domain = sun50ia64ccu0
sunxinmi0 at fdt1: R_INTC

NMI interrupt controller. This is wired to the POK (power on key) pin on the power management IC. In NetBSD, a long (>1.5s) press of the power button on Pinebook triggers a power down event.

sunxigpio0 at fdt1: PIO
gpio0 at sunxigpio0: 103 pins
sunxigpio0: interrupting on GIC irq 43
sunxigpio1 at fdt1: PIO
gpio1 at sunxigpio1: 13 pins
sunxigpio1: interrupting on GIC irq 77

GPIO and multi-function I/O pin support.

sunxipwm0 at fdt1: PWM

Pulse-width modulation driver. This is used on Pinebook to control the brightness of the display backlight.

sunxiusbphy0 at fdt1: USB PHY

USB 2.0 PHY, responsible for powering up and initialization of USB ports.

sunxirsb0 at fdt1: RSB
sunxirsb0: interrupting on GIC irq 71
iic0 at sunxirsb0: I2C bus

Allwinner "Reduced Serial Bus" is an I2C-like interface used to communicate with power management ICs.

axppmic0 at iic0 addr 0x3a3: AXP803
axpreg0 at axppmic0: dldo1
axpreg1 at axppmic0: dldo2
axpreg2 at axppmic0: dldo3
axpreg3 at axppmic0: dldo4
axpreg4 at axppmic0: eldo1
axpreg5 at axppmic0: eldo2
axpreg6 at axppmic0: eldo3
axpreg7 at axppmic0: fldo1
axpreg8 at axppmic0: fldo2
axpreg9 at axppmic0: dcdc1
axpreg10 at axppmic0: dcdc2
axpreg11 at axppmic0: dcdc3
axpreg12 at axppmic0: dcdc4
axpreg13 at axppmic0: dcdc5
axpreg14 at axppmic0: dcdc6
axpreg15 at axppmic0: aldo1
axpreg16 at axppmic0: aldo2
axpreg17 at axppmic0: aldo3

The X-Powers AXP803 power management IC (PMIC) controls power outputs and battery charging. Power, charging, and battery status is exported to userland with the envsys(4) API:

$ envstat -d axppmic0
                   Current  CritMax  WarnMax  WarnMin  CritMin  Unit
   ACIN present:      TRUE
   VBUS present:     FALSE
battery present:      TRUE
       charging:     FALSE
   charge state:    NORMAL
battery percent:       100                                      none
sun6idma0 at fdt1: DMA controller (8 channels)
sun6idma0: interrupting on GIC irq 82

8-channel programmable DMA controller. Used by the audio driver.

/soc/syscon@1c00000 at fdt1 not configured

A miscellaneous register set of registers. The Allwinner A64 ethernet driver uses these resources to configure the PHY interface. Not used on Pinebook.

ausoc0 at fdt0: SUN50I Audio Card
a64acodec0 at fdt1: A64 Audio Codec (analog part)
sunxii2s0 at fdt1: Audio Codec (digital part)
sun8icodec0 at fdt1: Audio Codec
sun8icodec0: interrupting on GIC irq 60

Analog audio codec. On Allwinner A64 this functionality is spread across multiple modules. The ausoc driver is responsible for talking to these drivers and exposing a single audio device. Initialization is deferred until all drivers are ready, so the audio device is attached below.

psci0 at fdt0: PSCI 0.2

ARM "Power State Coordination Interface". NetBSD uses this to implement reset, power off, and (for multiprocessor kernels) starting secondary CPUs.

gpiokeys0 at fdt0: Lid Switch

Pinebook has a Hall effect sensor wired to a GPIO to signal whether the lid is open or closed. On NetBSD state change events are sent to powerd, as well as broadcast to other drivers in the kernel. For example, the backlight driver will automatically turn off the display when the lid is closed.

pwmbacklight0 at fdt0: PWM Backlight <0 30 40 50 60 70 80 90 100>

The backlight brightness on Pinebook is controlled by the PWM controller. These controls are exported via sysctl:

$ sysctl hw.pwmbacklight0
hw.pwmbacklight0.levels = 0 30 40 50 60 70 80 90 100
hw.pwmbacklight0.level = 80

In addition, the framebuffer driver will pass WSDISPLAYIO_SVIDEO ioctl requests through to the backlight driver, so screen blanking in Xorg will work.

sunximmc0 at fdt1: SD/MMC controller
sunximmc0: interrupting on GIC irq 92
sunximmc1 at fdt1: SD/MMC controller
sunximmc1: interrupting on GIC irq 93
sunximmc2 at fdt1: SD/MMC controller
sunximmc2: interrupting on GIC irq 94

SD/MMC controllers. On Pinebook, they are used for the following functions:

  • #0: SD card slot
  • #1: SDIO Wi-Fi
  • #2: Onboard eMMC

motg0 at fdt1: USB OTG
motg0: interrupting on GIC irq 103
usb0 at motg0: USB revision 2.0

USB OTG controller. NetBSD supports this in host mode only.

ehci0 at fdt1: EHCI
ehci0: interrupting on GIC irq 104
ehci0: EHCI version 1.0
ehci0: 1 companion controller, 1 port
usb1 at ehci0: USB revision 2.0
ohci0 at fdt1: OHCI
ohci0: interrupting on GIC irq 105
ohci0: OHCI version 1.0
usb2 at ohci0: USB revision 1.0
ehci1 at fdt1: EHCI
ehci1: interrupting on GIC irq 106
ehci1: EHCI version 1.0
ehci1: 1 companion controller, 1 port
usb3 at ehci1: USB revision 2.0
ohci1 at fdt1: OHCI
ohci1: interrupting on GIC irq 107
ohci1: OHCI version 1.0
usb4 at ohci1: USB revision 1.0

USB 2.0 host controllers.

com0 at fdt1: ns16550a, working fifo
com0: interrupting on GIC irq 32

Serial console.

sunxirtc0 at fdt1: RTC

Real-time clock. Note that the Pinebook does not include a battery backed RTC.

sunxiwdt0 at fdt1: Watchdog
sunxiwdt0: default watchdog period is 16 seconds

The watchdog timer can be enabled with the wdogctl(8) utility:

# wdogctl
Available watchdog timers:
        sunxiwdt0, 16 second period
sunxithermal0 at fdt1: Thermal sensor controller
sunxithermal0: interrupting on GIC irq 63
sunxithermal0: cpu: alarm 85C hyst 58C shut 106C
sunxithermal0: gpu1: alarm 85C hyst 58C shut 106C
sunxithermal0: gpu2: alarm 85C hyst 58C shut 106C

CPU and GPU temperatures are exported to userland with the envsys(4) API:

$ envstat -d sunxithermal0
                     Current  CritMax  WarnMax  WarnMin  CritMin  Unit
  CPU temperature:    54.000                                      degC
GPU temperature 1:    52.000                                      degC
GPU temperature 2:    52.000                                      degC

In addition, the thermal sensor driver coordinates with the DVFS driver to automatically throttle CPU frequency and voltage when temperatures exceed alarm thresholds.

genfb0 at fdt2: Simple Framebuffer (1366x768 32-bpp @ 0xbe000000)
genfb0: switching to framebuffer console
genfb0: framebuffer at 0xbe000000, size 1366x768, depth 32, stride 5464
wsdisplay0 at genfb0 kbdmux 1: console (default, vt100 emulation)
wsmux1: connecting to wsdisplay0
wsdisplay0: screen 1-3 added (default, vt100 emulation)

NetBSD will automatically use a framebuffer configured by U-Boot using the simplefb protocol. Since the simple frame buffer driver uses genfb(4), it is supported by Xorg out of the box.

ausoc0: codec: sun8icodec0, cpu: sunxii2s0, aux: a64acodec0
audio0 at ausoc0: full duplex, playback, capture, mmap, independent
ausoc0: Virtual format configured - Format SLINEAR, precision 16, channels 2, frequency 48000
ausoc0: Latency: 256 milliseconds

Analog playback and capture are supported on Pinebook. In addition, headphone jack sensing is supported to automatically mute the internal speakers when headphones are plugged in. The following mixer controls are available:

$ mixerctl -a
spkr0 at audio0: PC Speaker (synthesized)
wsbell at spkr0 not configured

Pseudo PC speaker driver (not currently enabled in GENERIC64 kernel).

timecounter: Timecounter "clockinterrupt" frequency 100 Hz quality 0
Lid Switch: lid opened.
cpufreqdt0: 1152.000 MHz, 1300000 uV
cpufreqdt0: 1104.000 MHz, 1260000 uV
cpufreqdt0: 1008.000 MHz, 1200000 uV
cpufreqdt0: 816.000 MHz, 1080000 uV
cpufreqdt0: 648.000 MHz, 1040000 uV
cpufreqdt0: 408.000 MHz, 1000000 uV

Dynamic voltage and frequency scaling is supported. The desired frequency can be set manually with sysctl, or automatically by installing sysutils/estd from pkgsrc.

# sysctl machdep.cpu = 1152
machdep.cpu.frequency.current = 1152
machdep.cpu.frequency.available = 1152 1104 1008 816 648 408
sdmmc0 at sunximmc0
sdmmc1 at sunximmc1
sdmmc2 at sunximmc2
uhub0 at usb0: NetBSD (0000) MOTG root hub (0000), class 9/0, rev 2.00/1.00, addr 1
uhub0: 1 port with 1 removable, self powered
uhub1 at usb1: NetBSD (0000) EHCI root hub (0000), class 9/0, rev 2.00/1.00, addr 1
uhub1: 1 port with 1 removable, self powered
uhub2 at usb2: NetBSD (0000) OHCI root hub (0000), class 9/0, rev 1.00/1.00, addr 1
uhub2: 1 port with 1 removable, self powered
uhub3 at usb3: NetBSD (0000) EHCI root hub (0000), class 9/0, rev 2.00/1.00, addr 1
uhub3: 1 port with 1 removable, self powered
uhub4 at usb4: NetBSD (0000) OHCI root hub (0000), class 9/0, rev 1.00/1.00, addr 1
uhub4: 1 port with 1 removable, self powered
IPsec: Initialized Security Association Processing.
(manufacturer 0x24c, product 0xb703, standard function interface code 0x7) at sdmmc1 function 1 not configured

SDIO Wi-Fi is not yet supported.

ld2 at sdmmc2: <0x88:0x0103:NCard :0x00:0x30601836:0x000>
ld2: 14800 MB, 7517 cyl, 64 head, 63 sec, 512 bytes/sect x 30310400 sectors
ld2: 8-bit width, HS200, 64 MB cache, 200.000 MHz

Pinebook comes with 16GB of built-in eMMC storage. NetBSD drives this in High Speed HS200 mode, which can support up to 200MB/s at 200MHz.

uhub5 at uhub3 port 1: vendor 05e3 (0x5e3) USB2.0 Hub (0x608), class 9/0, rev 2.00/88.32, addr 2
uhub5: single transaction translator
uhub5: 4 ports with 1 removable, self powered
uhidev0 at uhub5 port 1 configuration 1 interface 0
uhidev0: HAILUCK CO.,LTD (0x258a) USB KEYBOARD (0x0c), rev 1.10/1.00, addr 3, iclass 3/1
ukbd0 at uhidev0: 8 Variable keys, 6 Array codes
wskbd0 at ukbd0: console keyboard, using wsdisplay0
uhidev1 at uhub5 port 1 configuration 1 interface 1
uhidev1: HAILUCK CO.,LTD (0x258a) USB KEYBOARD (0x0c), rev 1.10/1.00, addr 3, iclass 3/0
uhidev1: 9 report ids
ums0 at uhidev1 reportid 1: 5 buttons, W and Z dirs
wsmouse0 at ums0 mux 0
uhid0 at uhidev1 reportid 2: input=1, output=0, feature=0
uhid1 at uhidev1 reportid 3: input=3, output=0, feature=0
uhid2 at uhidev1 reportid 5: input=0, output=0, feature=5
uhid3 at uhidev1 reportid 6: input=0, output=0, feature=1039
uhid4 at uhidev1 reportid 9: input=1, output=0, feature=0

The Pinebook's keyboard and touchpad are connected internally to a USB controller.

The sleep (Fn+Esc), home (Fn+F1), volume down (Fn+F3), volume up (Fn+F4), and mute (Fn+F5) keys on the keyboard are mapped to uhid(4) devices. These can be used with usbhidaction(1):

# cat /etc/usbhidaction.conf 
Consumer:Consumer_Control.Consumer:Volume_Up                    1
        mixerctl -n -w outputs.master++
Consumer:Consumer_Control.Consumer:Volume_Down                  1
        mixerctl -n -w outputs.master--
Consumer:Consumer_Control.Consumer:Mute                         1
        mixerctl -n -w outputs.mute++
Consumer:Consumer_Control.Consumer:AC_Home                      1
        /etc/powerd/scripts/hotkey_button AC_Home pressed
Generic_Desktop:System_Control.Generic_Desktop:System_Sleep     1
        /etc/powerd/scripts/sleep_button System_Sleep pressed
# /usr/bin/usbhidaction -c /etc/usbhidaction.conf -f /dev/uhid0 -i -p /var/run/ 
# /usr/bin/usbhidaction -c /etc/usbhidaction.conf -f /dev/uhid1 -i -p /var/run/ 
urtwn0 at uhub5 port 2
urtwn0: Realtek (0x7392) 802.11n WLAN Adapter (0x7811), rev 2.00/2.00, addr 4
urtwn0: MAC/BB RTL8188CUS, RF 6052 1T1R, address 80:1f:02:94:40:12
urtwn0: 1 rx pipe, 2 tx pipes
urtwn0: 11b rates: 1Mbps 2Mbps 5.5Mbps 11Mbps
urtwn0: 11g rates: 1Mbps 2Mbps 5.5Mbps 11Mbps 6Mbps 9Mbps 12Mbps 18Mbps 24Mbps 36Mbps 48Mbps 54Mbps

Externally connected USB Wi-Fi adapter (not included).

uvideo0 at uhub5 port 3 configuration 1 interface 0: Image Processor (0x90c) USB 2.0 PC Cam (0x37c), rev 2.00/0.05, addr 5
video0 at uvideo0: Image Processor (0x90c) USB 2.0 PC Cam (0x37c), rev 2.00/0.05, addr 5

The built-in video camera is supported using standard V4L2 APIs.

$ videoctl -a
info.cap.card=Image Processor (0x90c) USB 2.0
boot device: ld2
root on ld2a dumps on ld2b

NetBSD supports a wide variety of Allwinner boards. The wiki contains more information about supported Allwinner SoCs and peripherals.

Posted at lunch time on Monday, May 21st, 2018 Tags:
As a part of a funded project, I am conducting a security audit of NetBSD’s network stack. The work will end soon, and I would like to briefly present some results.

Fixing, Strengthening, Simplifying

Over the last five months, hundreds of patches were committed to the source tree as a result of this work. Dozens of bugs were fixed, among which a good number of actual, remotely-triggerable vulnerabilities.

Changes were made to strengthen the networking subsystems and improve code quality: reinforce the mbuf API, add many KASSERTs to enforce assumptions, simplify packet handling, and verify compliance with RFCs. This was done in several layers of the NetBSD kernel, from device drivers to L4 handlers.

A lot of cleanup took place, too. For example I managed to remove more than one thousand lines of code in IPsec, while at the same time improving robustness and performance. This kind of cleanup results in a networking code that is much easier to understand and maintain.

The fixes for critical bugs were quickly propagated to the stable branches (NetBSD-6, NetBSD-7) and the NetBSD-8_BETA branch. Along the way, several changes too were discreetly propagated, when they were considered as good mitigations against possible attack vectors.

Fixes in Other Operating Systems

In the course of investigating several bugs discovered in NetBSD, I happened to look at the network stacks of other operating systems, to see whether they had already fixed the issues, and if so how. Needless to say, I found bugs there too.

So far the trophies are:

SA2018-003 (1) 6.2-Errata-#005 (2) SA-18:01.ipsec (2)
SA2018-004 (1) 6.2-Errata-#006 (1) SA-18:05.ipsec (2)
SA2018-006 (5+) 6.2-Errata-#007 (1)
SA2018-007 (10) 6.2-Errata-#010 (1)
SA2018-008 (2) 6.3-Errata-#006 (2)
6.3-Errata-#008 (1)

Fig. A: advisory_name (number_of_bugs).

Of course, I am focusing on NetBSD, so it is no surprise the number of bugs found there is higher than in the other OSes.

Also, it is to be noted that FreeBSD hasn’t yet published advisories for several bugs that I reported to them (which they nonetheless fixed pretty quickly).

Some Examples

The IPv6 Buffer Overflow

In January I discovered, in NetBSD’s IPv6 stack, a subtle buffer overflow, which turned out to affect the other BSDs as well.

The overflow allowed an attacker to write one byte of packet-controlled data into ‘packet_storage+off’, where ‘off’ could be approximately controlled too.

This allowed at least a pretty bad remote DoS: by sending specially-crafted packets in a loop, an attacker could overwrite several areas of memory with wrong values, which would eventually lead to undefined behavior and crash.

One way of exploiting this bug was to use a special combination of nested fragments.

Fig. B: A simplified view.

In short, when receiving the last fragment of a packet, the kernel would iterate over the previous IPv6 options of the packet, assuming that everything was located in the first mbuf. It was possible to break this assumption, by sending a fragment nested into another.

Given the nature of the bug there were probably other (and perhaps more direct) ways to trigger it.

This overflow was of course fixed pretty quickly, but in addition the NetBSD kernel was changed to automatically drop nested fragments. This is an example of the many miscellaneous changes made in order to strengthen the network stack.

The IPsec Infinite Loop

When receiving an IPv6-AH packet, the IPsec entry point was not correctly computing the length of the IPv6 suboptions, and this, before authentication. As a result, a specially-crafted IPv6 packet could trigger an infinite loop in the kernel (making it unresponsive). In addition this flaw allowed a limited buffer overflow - where the data being written was however not controllable by the attacker.

The other BSDs too were affected by this vulnerability. In addition they were subject to another buffer overflow, in IPv4-AH this time, which happened to have been already fixed in NetBSD several years earlier.


While looking at the IPv6 Multicast code, I stumbled across a pretty simple yet pretty bad mistake: at one point the Pim6 entry point would return IPPROTO_NONE instead of IPPROTO_DONE. Returning IPPROTO_NONE was entirely wrong: it caused the kernel to keep iterating on the IPv6 packet chain, while the packet storage was already freed.

Therefore it was possible to remotely cause a use-after-free if the packet was forged in such a way that the kernel would take the IPPROTO_NONE branch. Generally the kernel would panic shortly afterwards, because it figured out it was double-freeing the same packet storage. (A use-after-free detector is also one of the things that were added in NetBSD to prevent the exploitation of such bugs.)

This typo was found in the Multicast entry code, which is enabled only with a particular, non-default configuration. Therefore it didn’t affect a default install of NetBSD.

While looking for other occurrences of this typo, I found the exact same bug in the exact same place in FreeBSD. Curiously enough, OpenBSD had the same bug too, but in a totally different place: the typo existed in their EtherIP entry point, but there, it was more dangerous, because it was in a branch taken by default, and therefore a default install of OpenBSD was vulnerable.

The PF Signedness Bug

A bug was found in NetBSD’s implementation of the PF firewall, that did not affect the other BSDs. In the initial PF code a particular macro was used as an alias to a number. This macro formed a signed integer.

In NetBSD, however, the macro was defined differently: it contained the sizeof statement. This was a terrible mistake, because it resulted in an unsigned integer.

This was not the intended signedness. Given that the macro in question was used to perform length validation checks in PF’s TCP-SYN entry point when a modulate rule was active, it was easy to cause a remote DoS by sending a malformed packet.

But PF was not a component I was supposed to look at as part of my work. So how did I still manage to find this bug? Well, while closing dozens of reports in NetBSD’s Problem Report database, I stumbled across a PR from 2010, which was briefly saying that PF’s TCP-SYN entry point could crash the kernel if a special packet was received. Looking at the PF code, it was clear, after two minutes, where the problem was.

The NPF Integer Overflow

An integer overflow could be triggered in NPF, when parsing an IPv6 packet with large options. This could cause NPF to look for the L4 payload at the wrong offset within the packet, and it allowed an attacker to bypass any L4 filtering rule on IPv6.

Fig. C: Simplified example of an exploit.

In the example above, NPF allows the packet to enter, based on validation performed on the wrong TCP header (orange, dashed lines). The kernel reads the correct TCP header (red), and delivers it to a socket. NPF was supposed to reject it.

More generally, several problems existed in NPF’s handling of IPv6 packets.

The IPsec Fragment Attack

(A more detailed example)

I noticed some time ago that when reassembling fragments (in either IPv4 or IPv6), the kernel was not removing the M_PKTHDR flag on the secondary mbufs in mbuf chains. This flag is supposed to indicate that a given mbuf is the head of the chain it forms; having the flag on secondary mbufs was suspicious.

Later, deep in the IPsec-ESP handler, I found a function that was assuming that M_PKTHDR was set only in the first mbuf of the chain – assumption that evidently didn’t hold, since reassembled fragments had several M_PKTHDRs. Later in the handler, it resulted in a wrong length being stored in the mbuf header (the packet had become shorter than the actual length registered). The wrong length would in turn trigger a kernel panic shortly afterwards.

This remote DoS was triggerable if the ESP payload was located in a secondary mbuf of the chain, which never is the case in practice. The function in question was called after ESP decryption. Therefore, in order for an attacker to reach this place, it was necessary to send a correct ESP payload – which meant having the ESP key. So at a first glance, this looked like a non-critical bug.

But there was still a theoretical way to exploit the bug. In the case of IPv6 at least, the IP options can be huge, and more importantly, can be located in the unencrypted part of the packet. Let’s say you are MITMing a NetBSD host that is having an ESP conversation with another host. You intercept a packet directed to the NetBSD host. You take the ESP payload as-is (which you can’t decrypt since you don’t have the key), you craft a new two-fragment packet: you put the other IPsec host’s IPv6 address as source, insert a dummy IP option in the first fragment, insert the ESP payload as-is in the second fragment, and send the two fragments to the NetBSD host.

The NetBSD host reassembles the fragments, decrypts the ESP payload correctly, reaches the aforementioned handler, miscomputes the packet length, and crashes.

The other BSDs were affected.

And the rest...

Many security fixes and improvements in different places that I didn’t list here.

What Now

Not all protocols and layers of the network stack were verified, because of time constraints, and also because of unexpected events: the recent x86 CPU bugs, which I was the only one able to fix promptly. A todo list will be left when the project end date is reached, for someone else to pick up. Me perhaps, later this year? We’ll see.

This security audit of NetBSD’s network stack is sponsored by The NetBSD Foundation, and serves all users of BSD-derived operating systems. The NetBSD Foundation is a non-profit organization, and welcomes any donations that help continue funding projects of this kind.

Posted early Monday morning, May 28th, 2018 Tags: