NetBSD Wiki/tutorials/
how to reduce kernel size
Contents
- Introduction
- Example: OMAP 2420
- Default TISDP2420, 2251 kb
- Optimize for size with -Os, 2059 kb
- Remove debugging symbols, without -g, 2059 kb
- DIAGNOSTIC, 1867 kb
- VERBOSE_INIT_ARM, 1867 kb
- KTRACE, 1867 kb
- DDB, 1867 kb
- COMPAT_30, 1867 kb
- ksyms, 1510 kb
- PTRACE, 1510 kb
- FFS, 1510 kb
- MSDOSFS, 1509 kb
- PTYFS, 1509 kb
- bpfilter, 1445 kb
- INSECURE, 1445 kb
- INET6, 1316 kb
- MALLOC_NOINLINE, 1316 kb
- PIPE_SOCKETPAIR, 1316 kb
- VMSWAP, 1316 kb
- SYSVMSG, SYSVSEM, SYSVSHM, 1252 kb
- rnd, 1252 kb
- md, 1252 kb
- maxusers 2, 1252 kb
- MFS, 1124 kb
- COREDUMP, 1124 kb
- Kernel memory consumption
- References
Introduction
The NetBSD kernel is not big, but it can be made smaller. Generic size reduction steps are:
- compile with size optimizations
- removing debugging and logging features
- remove non-essentian functionality
Here is an example which shows how these steps were applied to a NetBSD kernel for an ARM development board. As a result the kernel size got reduced from 2251 to 1124 kilobytes.
Since kernel executable size is not the only measure of kernel memory usage, some tools and their example output are also shown.
Example: OMAP 2420
The board receives the kernel via TFTP and the root filesystem is on NFS. Most important components on the board are the serial ports and the Ethernet adapter.
NetBSD current has a default kernel configuration for the board: sys/arch/evbarm/conf/TISDP2420. This file includes a number of other configuration files, but after compilation the resulting configuration is available as sys/arch/evbarm/compile/obj/TISDP2420/config_file.h.
The configuration file structure and most options are explained in options(4) manual page. In this example options were changed by adding a no options OPTION_NAME directive or an options OPTION_NAME directive to the end of the default configuration file. By adding these to end of the file, previous declarations from included options files were easily replaced without changing the included files.
Default TISDP2420, 2251 kb
Default kernel is build with -O2 optimization and stripped with objdump. Many debugging options are still enabled.
$ ls -l sys/arch/evbarm/compile/obj/TISDP2420/netbsd*
-rwxr-xr-x 1 test test 2614515 2008-06-23 11:32 sys/arch/evbarm/compile/obj/TISDP2420/netbsd
-rwxr-xr-x 1 test test 2305536 2008-06-23 11:32 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.bin
-rwxr-xr-x 1 test test 13908834 2008-06-23 11:32 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.gdb
-rw-r--r-- 1 test test 3440335 2008-06-23 11:32 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.map
$ size sys/arch/evbarm/compile/obj/TISDP2420/netbsd
text data bss dec hex filename
1902956 339456 214740 2457152 257e40 sys/arch/evbarm/compile/obj/TISDP2420/netbsd
Dmesg shows how much memory is available after boot, when the kernel executable been loaded to RAM and most of the RAM based data structures have been initialized:
# dmesg|egrep "memory|gpm"
total memory = 62464 KB
avail memory = 58032 KB
DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x82536000, paddr=0x80710DISPC: omap2_lcd_alloc_fb(): memory
allocated at vaddr=0x8255c000, paddr=0x80740DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x82582000,
paddr=0x80760gpmc0 at mainbus0 base 0x6800a000: General Purpose Memory Controller, rev 2.0
gpmc0: CS#0 valid, addr 0x04000000, size 64MB
gpmc0: CS#1 valid, addr 0x08000000, size 16MB
sm0 at gpmc0 addr 0x08000300 intr 188
Optimize for size with -Os, 2059 kb
Compiler can optimize for size when build variable DEFCOPTS is set to -Os instead of the default -O2.
-rwxr-xr-x 1 test test 2458323 2008-06-25 09:48 sys/arch/evbarm/compile/obj/TISDP2420/netbsd
-rwxr-xr-x 1 test test 2109012 2008-06-25 09:48 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.bin
-rwxr-xr-x 1 test test 13390766 2008-06-25 09:48 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.gdb
-rw-r--r-- 1 test test 3446815 2008-06-25 09:48 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.map
text data bss dec hex filename
1764840 339540 214768 2319148 23632c sys/arch/evbarm/compile/obj/TISDP2420/netbsd
# dmesg | egrep "memory|gpm"
total memory = 62464 KB
avail memory = 58220 KB
DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x82537000, paddr=0x806e0DISPC: omap2_lcd_alloc_fb(): memory
allocated at vaddr=0x8255d000, paddr=0x80710DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x82583000,
paddr=0x80730gpmc0 at mainbus0 base 0x6800a000: General Purpose Memory Controller, rev 2.0
gpmc0: CS#0 valid, addr 0x04000000, size 64MB
gpmc0: CS#1 valid, addr 0x08000000, size 16MB
sm0 at gpmc0 addr 0x08000300 intr 188
Remove debugging symbols, without -g, 2059 kb
Build variable DEBUG contains the -g flag. Build without it. Results show that the kernel was already stripped of debug symbols by objdump, so this step is not usefull.
-rwxr-xr-x 1 test test 2502667 2008-06-25 09:58 sys/arch/evbarm/compile/obj/TISDP2420/netbsd
-rwxr-xr-x 1 test test 2109012 2008-06-25 09:58 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.bin
-rw-r--r-- 1 test test 3238409 2008-06-25 09:58 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.map
text data bss dec hex filename
1764840 339540 214768 2319148 23632c sys/arch/evbarm/compile/obj/TISDP2420/netbsd
# dmesg|egrep "memory|gpm"
total memory = 62464 KB
avail memory = 58220 KB
DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x82537000, paddr=0x806e0DISPC: omap2_lcd_alloc_fb(): memory
allocated at vaddr=0x8255d000, paddr=0x80710DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x82583000,
paddr=0x80730gpmc0 at mainbus0 base 0x6800a000: General Purpose Memory Controller, rev 2.0
gpmc0: CS#0 valid, addr 0x04000000, size 64MB
gpmc0: CS#1 valid, addr 0x08000000, size 16MB
sm0 at gpmc0 addr 0x08000300 intr 188
DIAGNOSTIC, 1867 kb
Kernel build without DIAGNOSTIC support.
-rwxr-xr-x 1 test test 2304678 2008-06-25 10:05 sys/arch/evbarm/compile/obj/TISDP2420/netbsd
-rwxr-xr-x 1 test test 1912404 2008-06-25 10:05 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.bin
-rw-r--r-- 1 test test 3208060 2008-06-25 10:05 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.map
text data bss dec hex filename
1567336 339540 214768 2121644 205fac sys/arch/evbarm/compile/obj/TISDP2420/netbsd
# dmesg|egrep "memory|gpm"
total memory = 62464 KB
avail memory = 58408 KB
DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x82538000, paddr=0x806b0DISPC: omap2_lcd_alloc_fb(): memory
allocated at vaddr=0x8255e000, paddr=0x806e0DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x82584000,
paddr=0x80700gpmc0 at mainbus0 base 0x6800a000: General Purpose Memory Controller, rev 2.0
gpmc0: CS#0 valid, addr 0x04000000, size 64MB
gpmc0: CS#1 valid, addr 0x08000000, size 16MB
sm0 at gpmc0 addr 0x08000300 intr 188
VERBOSE_INIT_ARM, 1867 kb
Kernel build without verbose ARM specific boot messages. VERBOSE_INIT_ARM seems to depend on DDB support.
-rwxr-xr-x 1 test test 2304616 2008-06-25 10:22 sys/arch/evbarm/compile/obj/TISDP2420/netbsd
-rwxr-xr-x 1 test test 1912404 2008-06-25 10:22 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.bin
-rw-r--r-- 1 test test 3207741 2008-06-25 10:22 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.map
text data bss dec hex filename
1565468 339540 214768 2119776 205860 sys/arch/evbarm/compile/obj/TISDP2420/netbsd
# dmesg|egrep "memory|gpm"
total memory = 62464 KB
avail memory = 58408 KB
DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x82538000, paddr=0x806b0DISPC: omap2_lcd_alloc_fb(): memory
allocated at vaddr=0x8255e000, paddr=0x806e0DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x82584000,
paddr=0x80700gpmc0 at mainbus0 base 0x6800a000: General Purpose Memory Controller, rev 2.0
gpmc0: CS#0 valid, addr 0x04000000, size 64MB
gpmc0: CS#1 valid, addr 0x08000000, size 16MB
sm0 at gpmc0 addr 0x08000300 intr 188
KTRACE, 1867 kb
Kernel without KTRACE system call tracing support. KTRACE seems to depend on DDB kernel debugger support, since boot hangs without it.
-rwxr-xr-x 1 test test 2303792 2008-06-25 10:39 sys/arch/evbarm/compile/obj/TISDP2420/netbsd
-rwxr-xr-x 1 test test 1912372 2008-06-25 10:39 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.bin
-rw-r--r-- 1 test test 3198575 2008-06-25 10:39 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.map
text data bss dec hex filename
1557776 339508 214704 2111988 2039f4 sys/arch/evbarm/compile/obj/TISDP2420/netbsd
# dmesg|egrep "memory|gpm"
total memory = 62464 KB
avail memory = 58408 KB
DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x82538000, paddr=0x806b0DISPC: omap2_lcd_alloc_fb(): memory
allocated at vaddr=0x8255e000, paddr=0x806e0DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x82584000,
paddr=0x80700gpmc0 at mainbus0 base 0x6800a000: General Purpose Memory Controller, rev 2.0
gpmc0: CS#0 valid, addr 0x04000000, size 64MB
gpmc0: CS#1 valid, addr 0x08000000, size 16MB
sm0 at gpmc0 addr 0x08000300 intr 188
DDB, 1867 kb
Build kernel without DDB in kernel debugger.
-rwxr-xr-x 1 test test 2260235 2008-06-25 10:42 sys/arch/evbarm/compile/obj/TISDP2420/netbsd
-rwxr-xr-x 1 test test 1911944 2008-06-25 10:42 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.bin
-rw-r--r-- 1 test test 3076696 2008-06-25 10:42 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.map
text data bss dec hex filename
1512444 339080 209440 2060964 1f72a4 sys/arch/evbarm/compile/obj/TISDP2420/netbsd
total memory = 62464 KB
avail memory = 58416 KB
DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x82538000, paddr=0x806b0DISPC: omap2_lcd_alloc_fb(): memory
allocated at vaddr=0x8255e000, paddr=0x806e0DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x82584000,
paddr=0x80700gpmc0 at mainbus0 base 0x6800a000: General Purpose Memory Controller, rev 2.0
gpmc0: CS#0 valid, addr 0x04000000, size 64MB
gpmc0: CS#1 valid, addr 0x08000000, size 16MB
sm0 at gpmc0 addr 0x08000300 intr 188
COMPAT_30, 1867 kb
Kernel without NetBSD 3.0 compatibility.
-rwxr-xr-x 1 test test 2259424 2008-06-25 10:55 sys/arch/evbarm/compile/obj/TISDP2420/netbsd
-rwxr-xr-x 1 test test 1911944 2008-06-25 10:55 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.bin
-rw-r--r-- 1 test test 3031468 2008-06-25 10:55 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.map
text data bss dec hex filename
1509828 339080 209440 2058348 1f686c sys/arch/evbarm/compile/obj/TISDP2420/netbsd
# dmesg|egrep "memory|gpm"
total memory = 62464 KB
avail memory = 58416 KB
DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x82538000, paddr=0x806b0DISPC: omap2_lcd_alloc_fb(): memory
allocated at vaddr=0x8255e000, paddr=0x806e0DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x82584000,
paddr=0x80700gpmc0 at mainbus0 base 0x6800a000: General Purpose Memory Controller, rev 2.0
gpmc0: CS#0 valid, addr 0x04000000, size 64MB
gpmc0: CS#1 valid, addr 0x08000000, size 16MB
sm0 at gpmc0 addr 0x08000300 intr 188
ksyms, 1510 kb
Kernel without /dev/ksyms support.
-rwxr-xr-x 1 test test 1925248 2008-06-25 10:59 sys/arch/evbarm/compile/obj/TISDP2420/netbsd
-rwxr-xr-x 1 test test 1546376 2008-06-25 10:59 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.bin
-rw-r--r-- 1 test test 3023265 2008-06-25 10:59 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.map
text data bss dec hex filename
1503148 39048 209056 1751252 1ab8d4 sys/arch/evbarm/compile/obj/TISDP2420/netbsd
# dmesg|egrep "memory|gpm"
total memory = 62464 KB
avail memory = 58764 KB
DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x8253a000, paddr=0x80660DISPC: omap2_lcd_alloc_fb(): memory
allocated at vaddr=0x82560000, paddr=0x80680DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x82586000,
paddr=0x806b0gpmc0 at mainbus0 base 0x6800a000: General Purpose Memory Controller, rev 2.0
gpmc0: CS#0 valid, addr 0x04000000, size 64MB
gpmc0: CS#1 valid, addr 0x08000000, size 16MB
sm0 at gpmc0 addr 0x08000300 intr 188
PTRACE, 1510 kb
Kernel without process tracing support.
-rwxr-xr-x 1 test test 1924948 2008-06-25 11:04 sys/arch/evbarm/compile/obj/TISDP2420/netbsd
-rwxr-xr-x 1 test test 1546376 2008-06-25 11:04 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.bin
-rw-r--r-- 1 test test 3015392 2008-06-25 11:04 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.map
text data bss dec hex filename
1499312 39048 209056 1747416 1aa9d8 sys/arch/evbarm/compile/obj/TISDP2420/netbsd
# dmesg|egrep "memory|gpm"
total memory = 62464 KB
avail memory = 58764 KB
DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x8253a000, paddr=0x80660DISPC: omap2_lcd_alloc_fb(): memory
allocated at vaddr=0x82560000, paddr=0x80680DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x82586000,
paddr=0x806b0gpmc0 at mainbus0 base 0x6800a000: General Purpose Memory Controller, rev 2.0
gpmc0: CS#0 valid, addr 0x04000000, size 64MB
gpmc0: CS#1 valid, addr 0x08000000, size 16MB
sm0 at gpmc0 addr 0x08000300 intr 188
FFS, 1510 kb
Kernel without fast filesystem support.
-rwxr-xr-x 1 test test 1924948 2008-06-25 11:09 sys/arch/evbarm/compile/obj/TISDP2420/netbsd
-rwxr-xr-x 1 test test 1546376 2008-06-25 11:09 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.bin
-rw-r--r-- 1 test test 3015329 2008-06-25 11:09 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.map
text data bss dec hex filename
1499248 39048 209056 1747352 1aa998 sys/arch/evbarm/compile/obj/TISDP2420/netbsd
# dmesg|egrep "memory|gpm"
total memory = 62464 KB
avail memory = 58764 KB
DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x8253a000, paddr=0x80660DISPC: omap2_lcd_alloc_fb(): memory
allocated at vaddr=0x82560000, paddr=0x80680DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x82586000,
paddr=0x806b0gpmc0 at mainbus0 base 0x6800a000: General Purpose Memory Controller, rev 2.0
gpmc0: CS#0 valid, addr 0x04000000, size 64MB
gpmc0: CS#1 valid, addr 0x08000000, size 16MB
sm0 at gpmc0 addr 0x08000300 intr 188
MSDOSFS, 1509 kb
Kernel without FAT filesystem support.
-rwxr-xr-x 1 test test 1887407 2008-06-25 11:21 sys/arch/evbarm/compile/obj/TISDP2420/netbsd
-rwxr-xr-x 1 test test 1546152 2008-06-25 11:21 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.bin
-rw-r--r-- 1 test test 2955657 2008-06-25 11:21 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.map
text data bss dec hex filename
1463132 38824 208672 1710628 1a1a24 sys/arch/evbarm/compile/obj/TISDP2420/netbsd
# dmesg|egrep "memory|gpm"
total memory = 62464 KB
avail memory = 58764 KB
DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x82538000, paddr=0x80660DISPC: omap2_lcd_alloc_fb(): memory
allocated at vaddr=0x8255e000, paddr=0x80680DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x82584000,
paddr=0x806b0gpmc0 at mainbus0 base 0x6800a000: General Purpose Memory Controller, rev 2.0
gpmc0: CS#0 valid, addr 0x04000000, size 64MB
gpmc0: CS#1 valid, addr 0x08000000, size 16MB
sm0 at gpmc0 addr 0x08000300 intr 188
PTYFS, 1509 kb
Kernel without pseudo TTY filesystem and pseudo-device pty.
-rwxr-xr-x 1 test test 1883462 2008-06-25 11:37 sys/arch/evbarm/compile/obj/TISDP2420/netbsd
-rwxr-xr-x 1 test test 1545916 2008-06-25 11:37 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.bin
-rw-r--r-- 1 test test 2911049 2008-06-25 11:37 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.map
text data bss dec hex filename
1446512 38588 208536 1693636 19d7c4 sys/arch/evbarm/compile/obj/TISDP2420/netbsd
# dmesg|egrep "memory|gpm"
total memory = 62464 KB
avail memory = 58764 KB
DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x82536000, paddr=0x80660DISPC: omap2_lcd_alloc_fb(): memory
allocated at vaddr=0x8255c000, paddr=0x80680DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x82582000,
paddr=0x806b0gpmc0 at mainbus0 base 0x6800a000: General Purpose Memory Controller, rev 2.0
gpmc0: CS#0 valid, addr 0x04000000, size 64MB
gpmc0: CS#1 valid, addr 0x08000000, size 16MB
sm0 at gpmc0 addr 0x08000300 intr 188
bpfilter, 1445 kb
Kernel without Berkeley packet filter pseudo-device support.
-rwxr-xr-x 1 test test 1848968 2008-06-25 11:46 sys/arch/evbarm/compile/obj/TISDP2420/netbsd
-rwxr-xr-x 1 test test 1480372 2008-06-25 11:46 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.bin
-rw-r--r-- 1 test test 2896335 2008-06-25 11:46 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.map
text data bss dec hex filename
1435984 38580 208272 1682836 19ad94 sys/arch/evbarm/compile/obj/TISDP2420/netbsd
# dmesg|egrep "memory|gpm"
total memory = 62464 KB
avail memory = 58828 KB
DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x82536000, paddr=0x80650DISPC: omap2_lcd_alloc_fb(): memory
allocated at vaddr=0x8255c000, paddr=0x80670DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x82582000,
paddr=0x806a0gpmc0 at mainbus0 base 0x6800a000: General Purpose Memory Controller, rev 2.0
gpmc0: CS#0 valid, addr 0x04000000, size 64MB
gpmc0: CS#1 valid, addr 0x08000000, size 16MB
sm0 at gpmc0 addr 0x08000300 intr 188
INSECURE, 1445 kb
Kernel without integrity protection.
-rwxr-xr-x 1 test test 1848968 2008-06-25 11:54 sys/arch/evbarm/compile/obj/TISDP2420/netbsd
-rwxr-xr-x 1 test test 1480372 2008-06-25 11:54 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.bin
-rw-r--r-- 1 test test 2896335 2008-06-25 11:54 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.map
text data bss dec hex filename
1435984 38580 208272 1682836 19ad94 sys/arch/evbarm/compile/obj/TISDP2420/netbsd
# dmesg|egrep "memory|gpm"
total memory = 62464 KB
avail memory = 58828 KB
DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x82536000, paddr=0x80650DISPC: omap2_lcd_alloc_fb(): memory
allocated at vaddr=0x8255c000, paddr=0x80670DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x82582000,
paddr=0x806a0gpmc0 at mainbus0 base 0x6800a000: General Purpose Memory Controller, rev 2.0
gpmc0: CS#0 valid, addr 0x04000000, size 64MB
gpmc0: CS#1 valid, addr 0x08000000, size 16MB
sm0 at gpmc0 addr 0x08000300 intr 188
INET6, 1316 kb
Kernel without IPv6 support.
-rwxr-xr-x 1 test test 1666318 2008-06-25 11:57 sys/arch/evbarm/compile/obj/TISDP2420/netbsd
-rwxr-xr-x 1 test test 1348532 2008-06-25 11:57 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.bin
-rw-r--r-- 1 test test 2666559 2008-06-25 11:57 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.map
text data bss dec hex filename
1271320 37812 199760 1508892 17061c sys/arch/evbarm/compile/obj/TISDP2420/netbsd
# dmesg|egrep "memory|gpm"
total memory = 62464 KB
avail memory = 58960 KB
DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x82537000, paddr=0x80630DISPC: omap2_lcd_alloc_fb(): memory
allocated at vaddr=0x8255d000, paddr=0x80650DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x82583000,
paddr=0x80680gpmc0 at mainbus0 base 0x6800a000: General Purpose Memory Controller, rev 2.0
gpmc0: CS#0 valid, addr 0x04000000, size 64MB
gpmc0: CS#1 valid, addr 0x08000000, size 16MB
sm0 at gpmc0 addr 0x08000300 intr 188
MALLOC_NOINLINE, 1316 kb
Kernel without inlined malloc functions.
-rwxr-xr-x 1 test test 1666318 2008-06-25 12:01 sys/arch/evbarm/compile/obj/TISDP2420/netbsd
-rwxr-xr-x 1 test test 1348532 2008-06-25 12:01 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.bin
-rw-r--r-- 1 test test 2666559 2008-06-25 12:01 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.map
text data bss dec hex filename
1271320 37812 199760 1508892 17061c sys/arch/evbarm/compile/obj/TISDP2420/netbsd
# dmesg|egrep "memory|gpm"
total memory = 62464 KB
avail memory = 58960 KB
DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x82537000, paddr=0x80630DISPC: omap2_lcd_alloc_fb(): memory
allocated at vaddr=0x8255d000, paddr=0x80650DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x82583000,
paddr=0x80680gpmc0 at mainbus0 base 0x6800a000: General Purpose Memory Controller, rev 2.0
gpmc0: CS#0 valid, addr 0x04000000, size 64MB
gpmc0: CS#1 valid, addr 0x08000000, size 16MB
sm0 at gpmc0 addr 0x08000300 intr 188
PIPE_SOCKETPAIR, 1316 kb
Kernel with smaller but slower pipe implementation.
-rwxr-xr-x 1 test test 1665096 2008-06-25 12:04 sys/arch/evbarm/compile/obj/TISDP2420/netbsd
-rwxr-xr-x 1 test test 1348468 2008-06-25 12:04 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.bin
-rw-r--r-- 1 test test 2658886 2008-06-25 12:04 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.map
text data bss dec hex filename
1264788 37748 199760 1502296 16ec58 sys/arch/evbarm/compile/obj/TISDP2420/netbsd
# dmesg|egrep "memory|gpm"
total memory = 62464 KB
avail memory = 58964 KB
DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x82537000, paddr=0x80630DISPC: omap2_lcd_alloc_fb(): memory
allocated at vaddr=0x8255d000, paddr=0x80650DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x82583000,
paddr=0x80670gpmc0 at mainbus0 base 0x6800a000: General Purpose Memory Controller, rev 2.0
gpmc0: CS#0 valid, addr 0x04000000, size 64MB
gpmc0: CS#1 valid, addr 0x08000000, size 16MB
sm0 at gpmc0 addr 0x08000300 intr 188
VMSWAP, 1316 kb
Kernel without swap support. Requires a build patch, but may still crash the kernel when uvm statistics are called from top, ps etc. programs.
--- a/sys/uvm/uvm_pdpolicy_clock.c
+++ b/sys/uvm/uvm_pdpolicy_clock.c
@@ -262,12 +262,13 @@ uvmpdpol_balancequeue(int swap_shortage)
/*
* if there's a shortage of swap slots, try to free it.
*/
-
+#if defined(VMSWAP)
if (swap_shortage > 0 && (p->pqflags & PQ_SWAPBACKED) != 0) {
if (uvmpd_trydropswap(p)) {
swap_shortage--;
}
}
+#endif /* defined(VMSWAP) */
/*
* if there's a shortage of inactive pages, deactivate.
-rwxr-xr-x 1 test test 1662968 2008-06-25 12:09 sys/arch/evbarm/compile/obj/TISDP2420/netbsd
-rwxr-xr-x 1 test test 1348392 2008-06-25 12:09 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.bin
-rw-r--r-- 1 test test 2642042 2008-06-25 12:09 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.map
text data bss dec hex filename
1248392 37672 199248 1485312 16aa00 sys/arch/evbarm/compile/obj/TISDP2420/netbsd
# dmesg|egrep "memory|gpm"
total memory = 62464 KB
avail memory = 58968 KB
DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x82537000, paddr=0x80630DISPC: omap2_lcd_alloc_fb(): memory
allocated at vaddr=0x8255d000, paddr=0x80650DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x82583000,
paddr=0x80670gpmc0 at mainbus0 base 0x6800a000: General Purpose Memory Controller, rev 2.0
gpmc0: CS#0 valid, addr 0x04000000, size 64MB
gpmc0: CS#1 valid, addr 0x08000000, size 16MB
sm0 at gpmc0 addr 0x08000300 intr 188
SYSVMSG, SYSVSEM, SYSVSHM, 1252 kb
Kernel without system V message queues, semaphores and shared memory.
-rwxr-xr-x 1 test test 1627140 2008-06-25 12:15 sys/arch/evbarm/compile/obj/TISDP2420/netbsd
-rwxr-xr-x 1 test test 1282748 2008-06-25 12:15 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.bin
-rw-r--r-- 1 test test 2612942 2008-06-25 12:15 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.map
text data bss dec hex filename
1225204 37564 198904 1461672 164da8 sys/arch/evbarm/compile/obj/TISDP2420/netbsd
# dmesg|egrep "memory|gpm"
total memory = 62464 KB
avail memory = 59032 KB
DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x82537000, paddr=0x80620DISPC: omap2_lcd_alloc_fb(): memory
allocated at vaddr=0x8255d000, paddr=0x80640DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x82583000,
paddr=0x80660gpmc0 at mainbus0 base 0x6800a000: General Purpose Memory Controller, rev 2.0
gpmc0: CS#0 valid, addr 0x04000000, size 64MB
gpmc0: CS#1 valid, addr 0x08000000, size 16MB
sm0 at gpmc0 addr 0x08000300 intr 188
rnd, 1252 kb
Kernel without /dev/random pseudo-device.
-rwxr-xr-x 1 test test 1625313 2008-06-25 12:25 sys/arch/evbarm/compile/obj/TISDP2420/netbsd
-rwxr-xr-x 1 test test 1282684 2008-06-25 12:25 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.bin
-rw-r--r-- 1 test test 2599464 2008-06-25 12:25 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.map
text data bss dec hex filename
1219556 37500 197944 1455000 163398 sys/arch/evbarm/compile/obj/TISDP2420/netbsd
# dmesg|egrep "memory|gpm"
total memory = 62464 KB
avail memory = 59032 KB
DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x82537000, paddr=0x80620DISPC: omap2_lcd_alloc_fb(): memory
allocated at vaddr=0x8255d000, paddr=0x80640DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x82583000,
paddr=0x80660gpmc0 at mainbus0 base 0x6800a000: General Purpose Memory Controller, rev 2.0
gpmc0: CS#0 valid, addr 0x04000000, size 64MB
gpmc0: CS#1 valid, addr 0x08000000, size 16MB
sm0 at gpmc0 addr 0x08000300 intr 188
md, 1252 kb
Kernel without support for memory devices like ramdisks.
-rwxr-xr-x 1 test test 1624021 2008-06-25 12:29 sys/arch/evbarm/compile/obj/TISDP2420/netbsd
-rwxr-xr-x 1 test test 1282588 2008-06-25 12:29 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.bin
-rw-r--r-- 1 test test 2581393 2008-06-25 12:29 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.map
text data bss dec hex filename
1214024 37404 197752 1449180 161cdc sys/arch/evbarm/compile/obj/TISDP2420/netbsd
# dmesg|egrep "memory|gpm"
total memory = 62464 KB
avail memory = 59032 KB
DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x82537000, paddr=0x80620DISPC: omap2_lcd_alloc_fb(): memory
allocated at vaddr=0x8255d000, paddr=0x80640DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x82583000,
paddr=0x80660gpmc0 at mainbus0 base 0x6800a000: General Purpose Memory Controller, rev 2.0
gpmc0: CS#0 valid, addr 0x04000000, size 64MB
gpmc0: CS#1 valid, addr 0x08000000, size 16MB
sm0 at gpmc0 addr 0x08000300 intr 188
maxusers 2, 1252 kb
Kernel with maxusers set to two instead of the default 32.
-rwxr-xr-x 1 test test 1624021 2008-06-25 12:34 sys/arch/evbarm/compile/obj/TISDP2420/netbsd
-rwxr-xr-x 1 test test 1282588 2008-06-25 12:34 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.bin
-rw-r--r-- 1 test test 2581393 2008-06-25 12:34 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.map
text data bss dec hex filename
1214024 37404 197752 1449180 161cdc sys/arch/evbarm/compile/obj/TISDP2420/netbsd
# dmesg|egrep "memory|gpm"
total memory = 62464 KB
avail memory = 59548 KB
DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x824b6000, paddr=0x805a0DISPC: omap2_lcd_alloc_fb(): memory
allocated at vaddr=0x824dc000, paddr=0x805c0DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x82502000,
paddr=0x805e0gpmc0 at mainbus0 base 0x6800a000: General Purpose Memory Controller, rev 2.0
gpmc0: CS#0 valid, addr 0x04000000, size 64MB
gpmc0: CS#1 valid, addr 0x08000000, size 16MB
sm0 at gpmc0 addr 0x08000300 intr 188
MFS, 1124 kb
Kernel without memory filesystem.
-rwxr-xr-x 1 test test 1483269 2008-06-25 12:37 sys/arch/evbarm/compile/obj/TISDP2420/netbsd
-rwxr-xr-x 1 test test 1151292 2008-06-25 12:37 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.bin
-rw-r--r-- 1 test test 2437216 2008-06-25 12:37 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.map
text data bss dec hex filename
1106936 37180 197688 1341804 14796c sys/arch/evbarm/compile/obj/TISDP2420/netbsd
# dmesg|egrep "memory|gpm"
total memory = 62464 KB
avail memory = 59672 KB
DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x824b3000, paddr=0x80580DISPC: omap2_lcd_alloc_fb(): memory
allocated at vaddr=0x824d9000, paddr=0x805a0DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x824ff000,
paddr=0x805d0gpmc0 at mainbus0 base 0x6800a000: General Purpose Memory Controller, rev 2.0
gpmc0: CS#0 valid, addr 0x04000000, size 64MB
gpmc0: CS#1 valid, addr 0x08000000, size 16MB
sm0 at gpmc0 addr 0x08000300 intr 188
COREDUMP, 1124 kb
Kernel without core dump support.
-rwxr-xr-x 1 test test 1482841 2008-06-25 13:34 sys/arch/evbarm/compile/obj/TISDP2420/netbsd
-rwxr-xr-x 1 test test 1151292 2008-06-25 13:34 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.bin
-rw-r--r-- 1 test test 2428587 2008-06-25 13:34 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.map
text data bss dec hex filename
1102708 37180 197688 1337576 1468e8 sys/arch/evbarm/compile/obj/TISDP2420/netbsd
# dmesg|egrep "memory|gpm"
total memory = 62464 KB
avail memory = 59672 KB
DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x824b3000, paddr=0x80580DISPC: omap2_lcd_alloc_fb(): memory
allocated at vaddr=0x824d9000, paddr=0x805a0DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x824ff000,
paddr=0x805d0gpmc0 at mainbus0 base 0x6800a000: General Purpose Memory Controller, rev 2.0
gpmc0: CS#0 valid, addr 0x04000000, size 64MB
gpmc0: CS#1 valid, addr 0x08000000, size 16MB
sm0 at gpmc0 addr 0x08000300 intr 188
Kernel memory consumption
The kernel executable size is not the only measure of kernel memory consumption. A number of userspace tools are able to show how the kernel and it's executable threads use memory while the system is running. Following chapters list some example tool output on the ARM OMAP 2420 board.
Using these tools is easy, but interpreting the numbers seems to require some indepth knowledge of the NetBSD kernel's memory management system uvm. More details on the meaning of these numbers as well as examples of found problems and countermeasures like kernel configurations and sysctl setting would be appreciated.
top
top shows overall memory usage in the system and an interesting process called system.
# top
load averages: 0.14, 0.03, 0.01 09:20:54
4 processes: 3 sleeping, 1 on CPU
CPU states: % user, % nice, % system, % interrupt, % idle
Memory: 1916K Act, 208K Wired, 996K Exec, 428K File, 52M Free
Swap:
PID USERNAME PRI NICE SIZE RES STATE TIME WCPU CPU COMMAND
51 root 43 0 976K 784K CPU 0:00 14.00% 0.68% top
0 root 125 0 0K 1008K schedule 0:00 0.00% 0.00% [system]
40 root 85 0 980K 956K wait 0:00 0.00% 0.00% sh
1 root 85 0 32K 500K wait 0:00 0.00% 0.00% init
# dmesg|grep -i mem
total memory = 62464 KB
avail memory = 58032 KB
DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x82536000, paddr=0x80710DISPC: omap2_lcd_alloc_fb(): memory
allocated at vaddr=0x8255c000, paddr=0x80740DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x82582000,
paddr=0x80760gpmc0 at mainbus0 base 0x6800a000: General Purpose Memory Controller, rev 2.0
# top -t
load averages: 0.00, 0.00, 0.00 up 0 days, 1:54 11:14:37
23 threads: 5 idle, 17 sleeping, 1 on CPU
CPU states: % user, % nice, % system, % interrupt, % idle
Memory: 3844K Act, 208K Wired, 1308K Exec, 2044K File, 50M Free
Swap:
PID LID USERNAME PRI STATE TIME WCPU CPU COMMAND NAME
0 6 root 223 IDLE 0:00 0.00% 0.00% [system] softser/0
0 3 root 222 IDLE 0:00 0.00% 0.00% [system] softnet/0
0 4 root 221 IDLE 0:00 0.00% 0.00% [system] softbio/0
0 5 root 220 IDLE 0:00 0.00% 0.00% [system] softclk/0
0 2 root 0 IDLE 0:00 0.00% 0.00% [system] idle/0
77 1 root 43 CPU 0:00 0.00% 0.00% top -
0 7 root 127 xcall 0:00 0.00% 0.00% [system] xcall/0
0 25 root 126 pgdaemon 0:00 0.00% 0.00% [system] pgdaemon
0 1 root 125 schedule 0:00 0.00% 0.00% [system] swapper
0 28 root 125 vmem_reh 0:00 0.00% 0.00% [system] vmem_rehash
0 27 root 125 aiodoned 0:00 0.00% 0.00% [system] aiodoned
0 9 root 125 cachegc 0:00 0.00% 0.00% [system] cachegc
0 8 root 125 vrele 0:00 0.00% 0.00% [system] vrele
0 26 root 124 syncer 0:00 0.00% 0.00% [system] ioflush
0 11 root 96 iicintr 0:00 0.00% 0.00% [system] iic0
ps
ps can show the system light weight processes/threads too with -s flag and lname displays a more sesible name:
# ps -awxs -o uid,pid,lid,nlwp,pri,ni,vsz,rss,command,lname
UID PID LID NLWP PRI NI VSZ RSS COMMAND LNAME
0 0 28 20 125 0 0 976 [system vmem_rehash
0 0 27 20 125 0 0 976 [system aiodoned
0 0 26 20 124 0 0 976 [system ioflush
0 0 25 20 126 0 0 976 [system pgdaemon
0 0 24 20 96 0 0 976 [system nfsio
0 0 23 20 96 0 0 976 [system nfsio
0 0 22 20 96 0 0 976 [system nfsio
0 0 21 20 96 0 0 976 [system nfsio
0 0 12 20 96 0 0 976 [system iic1
0 0 11 20 96 0 0 976 [system iic0
0 0 10 20 96 0 0 976 [system pmfevent
0 0 9 20 125 0 0 976 [system cachegc
0 0 8 20 125 0 0 976 [system vrele
0 0 7 20 127 0 0 976 [system xcall/0
0 0 6 20 223 0 0 976 [system softser/0
0 0 5 20 220 0 0 976 [system softclk/0
0 0 4 20 221 0 0 976 [system softbio/0
0 0 3 20 222 0 0 976 [system softnet/0
0 0 2 20 0 0 0 976 [system idle/0
0 0 1 20 125 0 0 976 [system swapper
0 1 1 1 85 0 32 500 init -
0 40 1 1 85 0 980 968 -sh -
0 62 1 1 42 0 976 676 ps -awx -
0 63 1 1 85 0 980 784 less -r -
pmap
Displaying process 0 memory map with pmap shows the kernel's memory map.
# pmap 0
80000000 17680K read/write/exec [ anon ]
81144000 4K read/write/exec [ anon ]
81145000 20K read/write/exec [ anon ]
8114A000 7328K read/write/exec [ kmem_map ]
81872000 4096K read/write/exec [ pager_map ]
81C72000 304K read/write/exec [ anon ]
81CBE000 4K read/write/exec [ anon ]
81CBF000 580K read/write/exec [ anon ]
81D50000 4096K read/write/exec [ exec_map ]
82150000 1200K read/write/exec [ phys_map ]
8227C000 2096K read/write/exec [ mb_map ]
82488000 20K read/write/exec [ anon ]
8248D000 8K read/write/exec [ uvm_aobj ]
8248F000 524K read/write/exec [ anon ]
82512000 48K read/write/exec [ uvm_aobj ]
8251E000 32K read/write/exec [ anon ]
82526000 8K read/write/exec [ uvm_aobj ]
82528000 48K read/write/exec [ anon ]
82534000 8K read/write/exec [ uvm_aobj ]
82536000 456K read/write/exec [ anon ]
825A8000 16K read/write/exec [ uvm_aobj ]
825AC000 12K read/write/exec [ anon ]
825AF000 48K read/write/exec [ uvm_aobj ]
825BB000 4K read/write/exec [ anon ]
825BC000 16K read/write/exec [ uvm_aobj ]
825C0000 56K read/write/exec [ anon ]
825CE000 8192K read/write/exec [ ubc_pager ]
82DCE000 52K read/write/exec [ anon ]
82DDB000 8K read/write/exec [ uvm_aobj ]
82DDD000 20K read/write/exec [ anon ]
82DE2000 8K read/write/exec [ uvm_aobj ]
82DE4000 4K read/write/exec [ anon ]
82DE6000 4K read/write/exec [ anon ]
82DE9000 8K read/write/exec [ uvm_aobj ]
82DEB000 8K read/write/exec [ anon ]
82DED000 8K read/write/exec [ uvm_aobj ]
82DEF000 16K read/write/exec [ anon ]
82DF7000 16K read/write/exec [ uvm_aobj ]
82DFB000 12K read/write/exec [ anon ]
82DFE000 8K read/write/exec [ uvm_aobj ]
82E00000 400K read/write/exec [ anon ]
82E68000 12K read/write/exec [ anon ]
82E70000 576K read/write/exec [ anon ]
total 48064K
vmstat
vmstat can display a lot of NetBSD memory management details and statistics, especially if the kernel is compiled with KMEMSTAT options. Output of vmstat -C -m on the ARM board.
References
- options(4)
- http://www.netbsd.org/docs/guide/en/chap-tuning.html#tuning-considerations-kernel
- Tuning NetBSD for performance
- KMEMSTAT http://mail-index.netbsd.org/tech-net/2006/03/27/0004.html
- VM tuning from swap perspective
- NetBSD Internals: Memory Management
- http://www.netbsd.org/docs/guide/en/chap-tuning.html#tuning-mtools
- How to reduce libc size
- http://mail-index.netbsd.org/port-arm/2008/06/27/msg000275.html
Add a comment