NetBSD Wiki/ports/evbarm/ NetBSD/evbarm on Raspberry Pi

This page describes the NetBSD/evbarm and NetBSD/evbarm64 (sometimes written /aarch64) ports on Raspberry Pi hardware. Most board variants are believed supported, and specific boards known to work are listed on this page. We use e.g. "RPI2" to refer to "Raspberry Pi 2" to save precious bytes on this page.

As of early 2024, NetBSD does not support the Raspberry Pi 5.

Initial Raspberry Pi support was introduced in NetBSD 6.0. NetBSD 7.0 added complete support for the board, along with introducing support for the quad-core Raspberry Pi 2 board. Raspberry Pi 3 support was added for NetBSD 8. NetBSD 9 supports aarch64, meaning using the newer processors in 64-bit mode. NetBSD 10 supports the RPI 4.

raspberrypi.jpg

  1. What works
    1. NetBSD 8
    2. NetBSD 9
    3. NetBSD 10
    4. NetBSD current
    5. What needs documenting if it works
    6. What needs work
  2. CPU types
  3. Installation
    1. Using standard images
    2. Building yourself
    3. SD card structure and booting process
    4. Console approaches
    5. Installation via ebijun's image
    6. Links
  4. Issues and Workarounds
    1. RPI4 xhci
  5. Maintaining a system
    1. Booting single user
    2. vcgencmd
    3. Updating the kernel
    4. RPI4 UEFI 3 GB
    5. Updating dtb files
    6. Updating the firmware
    7. Booting
    8. Split-mode aarch32/aarch64
  6. X11 and GPU
    1. Video playback
    2. OpenGL ES
    3. Quake 3
    4. RetroArch / Libretro
  7. Developer notes
    1. Updating the firmware version in the NetBSD sources
    2. Testing with anita and qemu
  8. Misc notes
    1. Power supply needed (or: why there is a little rainbow square in the top-right corner?)
    2. Xenon death flash (Raspberry Pi 2 is camera-shy)

(Raspberry Pi image by Christopher Lee used under CC-By-2.0 license)

What works

Note that NetBSD 8 is very out of date; asking about it will surely result in advice to upgrade. The HOWTO only address NetBSD 9, 10, and current.

In general, many aspects of NetBSD on RPI are not different from NetBSD on any other computer and the HOWTO tends to not discuss them.

NetBSD 8

While NetBSD 8 is of historic interest only, we document what was working in 8.

NetBSD 9

NetBSD 10

NetBSD current

What needs documenting if it works

What needs work

CPU types

Note that one can run a build of earmv6hf on the 2 and 3. There will still be a kernel7, built to use the 2/3 hardware, but with the armv6 instruction set. In theory the code compiled for earmv7hf will be faster, but anecdotal experience is that it doesn't matter that much. Builds of NetBSD for earlier revisions of ARM are unsupported.

\todo Explain if one can run earmv7hf on RPI4 (not that there is any reason to do so).

Installation

Using standard images

The simplest way is to download the appropriate SD card image from the NetBSD mirrors:

Decompress it and write it to the SD card:

$ gunzip armv7.img.gz
$ dd if=armv7.img of=/dev/rld0d conv=sync bs=1m progress=1

If you're not using NetBSD, your operating system's dd command's arguments may vary. On Windows, try Rawrite32.

Building yourself

Getting sources and building a release with build.sh is not special for evbarm. However, the evbarm port has a very large number of CPU types, compared to i386 and amd64 which have one each. The standard approach is to use -m to define MACHINE and -a to define MACHINE_ARCH. build.sh supports aliases that can be passed as a MACHINE value, but denote both MACHINE and a MACHINE_ARCH. The third line uses an alias and is equal to the second, for RPI2/3. Note that the aliases start with "evb" while the MACHINE_ARCH values do not, and that aliases have "-el" or "-eb", while the MACHINE_ARCH values have no suffix or "eb".

Consider setting RELEASEMACHINEDIR if you wish to build multiple MACHINE_ARCH values for a MACHINE; see build.sh. Use something like "evbarm-earmv7hf", so that 1) earvm6 and earmv7 don't collide and 2) anita will recognize it as a type of evbarm.

SD card structure and booting process

The RPI1 looks for firmware and kernel.img on the first FAT32 MBR partition of the uSD card. A separate kernel (kernel7.img) is used on RPI2 and RPI3. The NetBSD kernel will then find a NetBSD MBR partition and within that the root disklabel partition, and use that FFS partition as the root filesystem.

The RPI4 is similar, but can (only?) look in the first FAT32 partition within the GPT. \todo Explain more.

A 2 GB card is the smallest workable size that the installation image will fit on. After the first boot, the system resizes the NetBSD root partition to fill the card. Note that swap is after /boot and before /, and not contained in the NetBSD fdisk partition. However, if you don't try to change the partition structure, this should not cause you any trouble.

Note that SD cards generally have limited write tolerance, so you may wish to disable atime updates via the noatime option, as is done by the default installation.

https://www.raspberrypi.org/documentation/configuration/config-txt/boot.md

DTBs

Note that generally, a single dtb is loaded. On NetBSD 9, the dtb file for the system is loaded by the bootloader (in flash).

The RPI bootloader looks for a magic string in a trailer after the kernel to determine if it should use DTB support (the new normal) or something called ATAG (apparently the old way). See upstream commit introducing DTB trailer for more information.

Kernel format variants

Multiple kernels are provided in releasedir/binary/kernels.

In netbsd-8, there are only GENERIC and GENERIC.bin. RPI uses the GENERIC.bin version as kernel.img/kernel7.img, and does not use GENERIC (regular ELF).

In netbsd-9, 4 versions are provided and GENERIC.img is used as kernel.img/kernel7.img.

In netbsd-9, there are only GENERIC and GENERIC.img.

netbsd-GENERIC.gz

This is regular ELF and not used on RPI. However it is installed as /netbsd.

netbsd-GENERIC.img.gz

On NetBSD >=9, the kernel with the .img suffix has the trailer to cause the bootloader to load DTB files. Without DTBs, very little useful will happen! On NetBSD 10 aarch64, this is /boot/netbsd.img.

netbsd-GENERIC.bin.gz

This is like .img, but without the trailer for DTB. This file should be used on -8. Do not install it as kernel7.img on -9: it will not boot.

netbsd-GENERIC.ub.gz

This is for u-boot, not used on RPI and present only for NetBSD 9 (not 8, not 10).

boot methods

There are (at least) two boot methods: the tradtional RPI method and UEFI.

The RPI expects an MSDOS filesystem on the uSD. Through RPI3, MBR is ok. With RPI4, GPT is ok.

Traditional booting

This is the method used if one uses arm.img or arm64.img from the release build, without trying to convert to UEFI.

Boot code in some kind of processor EEPROM loads some of the files from the uSD, probably start.elf or start4.elf. The file config.txt then controls the next steps.

This is the method that has traditionally been used for NetBSD on RPI. One can configure a serial console by editing files in the uSD, and never having a monitor.

UEFI booting

With UEFI, one first boots "UEFI firmware" that then loads the NetBSD kernel and provides a UEFI interface. This makes the RPI seem more like a PC, and of course whether that's a good thing depends on your perspective. There is richer support for configuring where to boot from.

The netbsd-10 image boot partition contains EFI/BOOT/bootaa64.efi.

This is called "UEFI firmware" even though the bits are on uSD rather than EEPROM/flash. However, it functions like a UEFI BIOS in a PC.

See the UEFI firmware page. Write the UEFI firmware to the SD card. Then either insert a USB drive with the standard NetBSD arm64.img written to it, or have the FFS partition from that image on the rest of the uSD card. The RPI will then boot from that filesystem.

See also this page about UEFI on RPI which is from an unknown time.

Console approaches

The standard approach is to use a USB keyboard and an HDMI monitor for installation.

This section was written for the traditional boot process.

Serial Console

By default NetBSD images are set to use HDMI. If you wish to use a serial console, mount the FAT32 partition on another system and edit cmdline.txt and remove '"console=fb"'.

Enabling ssh for installation without any console

(Note that this section is more broadly applicable than RPI.)

If you want to enable ssh with the standard image, so that you can log in over the net without either a serial or HDMI console, you can edit the configuration of a uSD card before booting. On another computer, mount the ffs partition, place /root/.ssh/authorized_keys, uncomment PermitRootLogin in /etc/ssh/sshd_config, and comment out the rc_configure=NO in /etc/rc.conf. Besides having to find the IP address (e.g. from DHCP server logs), you will have to wait for the partition resizing and reboot.

As an alternative, see creds_msdos(8) which describes a way to install a key via /boot/creds.txt. Note that creds_msdos is not part of a normal system installation and is only present on install media. As of 10.0_RC3, the sshkey option is broken, and sshkeyfile will not work with a file containing spaces (but this is Unix and files should not have spaces :-). This is probably fixed in 10.0_RC4.

Installation with sshramdisk image

\todo Verify if this is still accurate. If only for earmv6, explain why or file bug to have the rest support it. If rpi_inst is historical, gc this section.

build.sh (and hence the FTP site) also creates an image 'rpi_inst.img.gz' specifically for installation without HDMI or a serial console, when built for earmv6hf. Note that this image is much smaller and that you will need to fetch the sets over the network. To use this method, write that image to a uSD card as above, and then:

The rpi_inst.img.gz image will only work for systems that use earmv6hf kernels (so not RPI2/3). See this port-arm message for details.

Installation via ebijun's image

As an alternative to the standard installation images, Jun Ebihara provides an install image for Raspberry Pi that includes packages. It is based on NetBSD-current and is built for earmv6hf, and thus will work on Raspberry Pi 1, 2 and 3. This image is typically updated every few weeks.

Links

The following pages have been published by NetBSD community members. (Note that some of them are old.)

Issues and Workarounds

RPI4 xhci

With the netbsd-10 arm64.img on a RPI4 (most of them), the pci driver is missing and therefore xhci will not attach, so the USB ports will not work. One workaround is to switch to UEFI, but that leads to a 3GB memory limit and needing a monitor. Another is to add kernel config. One can also add the hardware rng. Adding the following to GENERIC64.local results in both working; you likely also need a dtb that includes the RNG. \todo Explain why this isn't in GENERIC64 or link to a PR.

GENERIC64

bcm2838pcie* at fdt?                    # STB PCIe host controller
bcm2838rng* at fdt?                     # RPI4 RNG

There is some need to load firmware for the xhci driver, but apparently that works, once the above is added.

Maintaining a system

Booting single user

\todo Describe how to boot single user via the serial console and via the fb console.

vcgencmd

The program vcgencmd, referenced in the boot section, can be found in pkgsrc/misc/raspberrypi-userland.

Updating the kernel

\todo Explain if updating firmware is necessary when e.g. moving from 8 to 9, or 9 to current.

RPI4 UEFI 3 GB

To work around bugs in hardware (that may or may not be fixed in recent RPI4) and because not all OSes have workarounds, the UEFI firmware's default is to limit RAM to 3GB. NetBSD 10 can be used with more, so this needs to be configured in UEFI.

Updating dtb files

NetBSD 8

On NetBSD 8, dtb files are not used.

NetBSD 9

Build a release. gunzip the armv7.img, vnconfig it, and mount the MSDOS partition (e) e.g. on /mnt. Copy the dtb files from /mnt/foo.dtb to /boot, and from /mnt/dtb/foo.dtb to /boot/dtb.

It seems that some systems, including RPI, require dtb files in /boot, and some expect them in /boot/dtb.

NetBSD 10

When updating, ensure that /boot is mounted and that you unpack the dtb set.

Updating the firmware

It is somewhat likely that running NetBSD from a given branch X with firmware from a branch Y < X will not go well. It is unclear if firmware from a branch Y > X will work. It is standard practice to use firmware from the right branch. An alternative view is that newer firmware is usually better, and the the firmware needs to be new enough for the hardware.

A section below describes the process of updating NetBSD's copy of the firmware from upstream, with testing, by NetBSD developers. This section is about updating a system's firmware from the firmware in a version of NetBSD.

(Updating the firmware is harder than it should be.) Build a release. gunzip the armv7.img, vnconfig it, and mount the MSDOS partition (e) e.g. on /mnt. Copy files from that to /boot that have changes, carefully.

Relevant files include bootcode.bin, start.elf and start_cd.elf.

Compare cmdline.txt, but beware that just overwriting it will lose customizations like using the serial console instead of the framebuffer.

\todo Explain where the firmware is in the source tree, and note that it is not in the installed system image (such as /usr/mdec). Explain how to update a system (presumably /boot) from either an installed system's new firmware files, or the source tree. Explain any particular cautions.

\todo Explain if using updated firmware from one branch (e.g. netbsd-current) on a system using a different branch (e.g. netbsd-8) is safe. Explain if pullups are done to release branches with new firmware.

Booting

The device boots by finding a file "bootcode.bin". The primary location is a FAT32 partition on the uSD card, and an additional location is on a USB drive. See the upstream documentation on booting and read all the subpages.

The standard approach is to use a uSD card, with a fdisk partition table containing a FAT32 partition marked active, and a NetBSD partition. The NetBSD partition will then contain a disklabel, pointing to an FFS partition (a), a swap partition (b) and the FAT32 boot partition mounted as /boot (e). The file /boot/cmdline.txt has a line to set the root partition.

One wrinkle in the standard approach is that the disk layout is "boot swap /", but the NetBSD fdisk partition starts at the location of /, so the swap partition is not within the NetBSD fdisk partition. The / partition can hold a disklabel, while swap cannot. It is normal to have swap after / (and thus within the fdisk partition), but the arrangement used permits growing / on first boot, for the typical case where a larger uSD is used, compared to the minimum image size.

An alternate approach is to have the boot FAT32 partition as above, but to have the entire system including root on an external disk. This is configured by changing root=ld0a to root=sd0a or root=dk0 (depending on disklabel/GPT). Besides greater space, part of the point is to avoid writing to the uSD card.

A third approach, workable on the Pi 3 only, is to configure USB host booting (already enabled on the 3+; see the upstream documentation) and have the boot partition also on the external device. In this case the external device must have an MBR because the hardware's first-stage boot does not have GPT support. In theory the procedure to program USB host boot mode will function on a NetBSD system because the programming is done by bootcode.bin. \todo Confirm that putting program_usb_boot_mode=1 in config.txt and booting works to program the OTP bit. Confirm that one can then boot NetBSD from external USB.

There is no well-defined USB enumeration order, so the preferred approach if one has multiple USB mass storage devices is to use named wedges in both fstab and cmdline.txt.

Split-mode aarch32/aarch64

\todo Verify this, and add any necessary cautions about boot code.

The aarch64 kernel can run aarch32 binaries, so one can boot an aarch64 kernel on a system with an aarch32 userland.

X11 and GPU

Video acceleration currently only works with 32-bit (ARMv7 and ARMv6) kernels due to the Broadcom code not being 64-bit clean.

Since applications require specialized support for the GPU, only a few applications are normally accelerated. NetBSD/aarch64 normally uses llvmpipe to provide fast parallel CPU-driven support for OpenGL, so should be faster when running normal applications.

The situation should be improved, ideally by writing a DRM/KMS driver.

Video playback

Accelerated video playback is supported with the OMXPlayer application and through GStreamer with the omx plugin.

OpenGL ES

Accelerated OpenGL ES is supported. The GL ES client libraries are included with the misc/raspberrypi-userland package.

Quake 3

A Raspberry Pi optimized build of ioquake3 is available in the games/ioquake3-raspberrypi package. To use it, the following additional resources are required:

Place the pak0.pk3 file in the /usr/pkg/lib/ioquake3/baseq3 directory.

RetroArch / Libretro

Using emulators/retroarch it is possible to run many emulators at full speed the Raspberry Pi. Emulator cores for various gaming consoles are available in the emulators/libretro-* packages. To begin using retroarch:

Developer notes

These notes are for people working on improvements to RPI support in NetBSD.

Updating the firmware version in the NetBSD sources

(Note that trying new firmware may result in a non-bootable system, so be prepared to recover the bootable media with another system.)

Upstream firmware releases are on GitHub. Copy all files except kernel*.img into /boot and reboot.

New firmware should pass all of the following tests before being committed to NetBSD.

Tests should be run on all of rpi[0123].

Testing with anita and qemu

See the anita section in the evbarm page.

It is not currently known how to emulate a RPI in qemu, and therefore anita does not yet have support for this. \todo Add a command-line example to run qemu emulating some RPI model.

Misc notes

Miscellaneous notes about Raspberry PI.

Power supply needed (or: why there is a little rainbow square in the top-right corner?)

Raspberry Pi devices are powered by 5V micro USB and a 2.5A (2500mA) power supply is recommended. For more information please read:

https://www.raspberrypi.org/documentation/faqs/#pi-power

Power glitches can also manifest in other ways, e.g. with an USB disk plugged:

sd0(umass0:0:0:0): generic HBA error
sd0: cache synchronization failed

Using a recommended power supply avoids most such issues.

Xenon death flash (Raspberry Pi 2 is camera-shy)

When using laser pointers or xenon flashes in cameras (or other flashes of high-intensity long-wave light) against a Raspberry Pi 2 the Pi can power itself off. For more information please read:

https://www.raspberrypi.org/blog/xenon-death-flash-a-free-physics-lesson/

Add a comment
Last edited late Thursday morning, October 10th, 2024
Preferences | Logout