**Contents** [[!toc levels=3]] # Introduction The NetBSD kernel is not big, but it can be made smaller. Generic size reduction steps are: * compile with size optimizations * removing debugging and logging features * remove non-essentian functionality Here is an example which shows how these steps were applied to a NetBSD kernel for an ARM development board. As a result the kernel size got reduced from 2251 to 1124 kilobytes. Since kernel executable size is not the only measure of kernel memory usage, some tools and their example output are also shown. # Example: OMAP 2420 The board receives the kernel via TFTP and the root filesystem is on NFS. Most important components on the board are the serial ports and the Ethernet adapter. NetBSD current has a default kernel configuration for the board: **sys/arch/evbarm/conf/TISDP2420**. This file includes a number of other configuration files, but after compilation the resulting configuration is available as **sys/arch/evbarm/compile/obj/TISDP2420/config_file.h**. The configuration file structure and most options are explained in options(4) manual page. In this example options were changed by adding a **no options OPTION_NAME** directive or an **options OPTION_NAME** directive to the end of the default configuration file. By adding these to end of the file, previous declarations from included options files were easily replaced without changing the included files. ## Default TISDP2420, 2251 kb Default kernel is build with -O2 optimization and stripped with objdump. Many debugging options are still enabled. $ ls -l sys/arch/evbarm/compile/obj/TISDP2420/netbsd* -rwxr-xr-x 1 test test 2614515 2008-06-23 11:32 sys/arch/evbarm/compile/obj/TISDP2420/netbsd -rwxr-xr-x 1 test test 2305536 2008-06-23 11:32 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.bin -rwxr-xr-x 1 test test 13908834 2008-06-23 11:32 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.gdb -rw-r--r-- 1 test test 3440335 2008-06-23 11:32 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.map $ size sys/arch/evbarm/compile/obj/TISDP2420/netbsd text data bss dec hex filename 1902956 339456 214740 2457152 257e40 sys/arch/evbarm/compile/obj/TISDP2420/netbsd Dmesg shows how much memory is available after boot, when the kernel executable been loaded to RAM and most of the RAM based data structures have been initialized: # dmesg|egrep "memory|gpm" total memory = 62464 KB avail memory = 58032 KB DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x82536000, paddr=0x80710DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x8255c000, paddr=0x80740DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x82582000, paddr=0x80760gpmc0 at mainbus0 base 0x6800a000: General Purpose Memory Controller, rev 2.0 gpmc0: CS#0 valid, addr 0x04000000, size 64MB gpmc0: CS#1 valid, addr 0x08000000, size 16MB sm0 at gpmc0 addr 0x08000300 intr 188 ## Optimize for size with -Os, 2059 kb Compiler can optimize for size when build variable DEFCOPTS is set to **-Os** instead of the default **-O2**. -rwxr-xr-x 1 test test 2458323 2008-06-25 09:48 sys/arch/evbarm/compile/obj/TISDP2420/netbsd -rwxr-xr-x 1 test test 2109012 2008-06-25 09:48 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.bin -rwxr-xr-x 1 test test 13390766 2008-06-25 09:48 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.gdb -rw-r--r-- 1 test test 3446815 2008-06-25 09:48 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.map text data bss dec hex filename 1764840 339540 214768 2319148 23632c sys/arch/evbarm/compile/obj/TISDP2420/netbsd # dmesg | egrep "memory|gpm" total memory = 62464 KB avail memory = 58220 KB DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x82537000, paddr=0x806e0DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x8255d000, paddr=0x80710DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x82583000, paddr=0x80730gpmc0 at mainbus0 base 0x6800a000: General Purpose Memory Controller, rev 2.0 gpmc0: CS#0 valid, addr 0x04000000, size 64MB gpmc0: CS#1 valid, addr 0x08000000, size 16MB sm0 at gpmc0 addr 0x08000300 intr 188 ## Remove debugging symbols, without -g, 2059 kb Build variable DEBUG contains the **-g** flag. Build without it. Results show that the kernel was already stripped of debug symbols by objdump, so this step is not usefull. -rwxr-xr-x 1 test test 2502667 2008-06-25 09:58 sys/arch/evbarm/compile/obj/TISDP2420/netbsd -rwxr-xr-x 1 test test 2109012 2008-06-25 09:58 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.bin -rw-r--r-- 1 test test 3238409 2008-06-25 09:58 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.map text data bss dec hex filename 1764840 339540 214768 2319148 23632c sys/arch/evbarm/compile/obj/TISDP2420/netbsd # dmesg|egrep "memory|gpm" total memory = 62464 KB avail memory = 58220 KB DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x82537000, paddr=0x806e0DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x8255d000, paddr=0x80710DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x82583000, paddr=0x80730gpmc0 at mainbus0 base 0x6800a000: General Purpose Memory Controller, rev 2.0 gpmc0: CS#0 valid, addr 0x04000000, size 64MB gpmc0: CS#1 valid, addr 0x08000000, size 16MB sm0 at gpmc0 addr 0x08000300 intr 188 ## DIAGNOSTIC, 1867 kb Kernel build without DIAGNOSTIC support. -rwxr-xr-x 1 test test 2304678 2008-06-25 10:05 sys/arch/evbarm/compile/obj/TISDP2420/netbsd -rwxr-xr-x 1 test test 1912404 2008-06-25 10:05 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.bin -rw-r--r-- 1 test test 3208060 2008-06-25 10:05 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.map text data bss dec hex filename 1567336 339540 214768 2121644 205fac sys/arch/evbarm/compile/obj/TISDP2420/netbsd # dmesg|egrep "memory|gpm" total memory = 62464 KB avail memory = 58408 KB DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x82538000, paddr=0x806b0DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x8255e000, paddr=0x806e0DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x82584000, paddr=0x80700gpmc0 at mainbus0 base 0x6800a000: General Purpose Memory Controller, rev 2.0 gpmc0: CS#0 valid, addr 0x04000000, size 64MB gpmc0: CS#1 valid, addr 0x08000000, size 16MB sm0 at gpmc0 addr 0x08000300 intr 188 ## VERBOSE_INIT_ARM, 1867 kb Kernel build without verbose ARM specific boot messages. VERBOSE_INIT_ARM seems to depend on DDB support. -rwxr-xr-x 1 test test 2304616 2008-06-25 10:22 sys/arch/evbarm/compile/obj/TISDP2420/netbsd -rwxr-xr-x 1 test test 1912404 2008-06-25 10:22 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.bin -rw-r--r-- 1 test test 3207741 2008-06-25 10:22 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.map text data bss dec hex filename 1565468 339540 214768 2119776 205860 sys/arch/evbarm/compile/obj/TISDP2420/netbsd # dmesg|egrep "memory|gpm" total memory = 62464 KB avail memory = 58408 KB DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x82538000, paddr=0x806b0DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x8255e000, paddr=0x806e0DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x82584000, paddr=0x80700gpmc0 at mainbus0 base 0x6800a000: General Purpose Memory Controller, rev 2.0 gpmc0: CS#0 valid, addr 0x04000000, size 64MB gpmc0: CS#1 valid, addr 0x08000000, size 16MB sm0 at gpmc0 addr 0x08000300 intr 188 ## KTRACE, 1867 kb Kernel without KTRACE system call tracing support. KTRACE seems to depend on DDB kernel debugger support, since boot hangs without it. -rwxr-xr-x 1 test test 2303792 2008-06-25 10:39 sys/arch/evbarm/compile/obj/TISDP2420/netbsd -rwxr-xr-x 1 test test 1912372 2008-06-25 10:39 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.bin -rw-r--r-- 1 test test 3198575 2008-06-25 10:39 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.map text data bss dec hex filename 1557776 339508 214704 2111988 2039f4 sys/arch/evbarm/compile/obj/TISDP2420/netbsd # dmesg|egrep "memory|gpm" total memory = 62464 KB avail memory = 58408 KB DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x82538000, paddr=0x806b0DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x8255e000, paddr=0x806e0DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x82584000, paddr=0x80700gpmc0 at mainbus0 base 0x6800a000: General Purpose Memory Controller, rev 2.0 gpmc0: CS#0 valid, addr 0x04000000, size 64MB gpmc0: CS#1 valid, addr 0x08000000, size 16MB sm0 at gpmc0 addr 0x08000300 intr 188 ## DDB, 1867 kb Build kernel without DDB in kernel debugger. -rwxr-xr-x 1 test test 2260235 2008-06-25 10:42 sys/arch/evbarm/compile/obj/TISDP2420/netbsd -rwxr-xr-x 1 test test 1911944 2008-06-25 10:42 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.bin -rw-r--r-- 1 test test 3076696 2008-06-25 10:42 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.map text data bss dec hex filename 1512444 339080 209440 2060964 1f72a4 sys/arch/evbarm/compile/obj/TISDP2420/netbsd total memory = 62464 KB avail memory = 58416 KB DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x82538000, paddr=0x806b0DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x8255e000, paddr=0x806e0DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x82584000, paddr=0x80700gpmc0 at mainbus0 base 0x6800a000: General Purpose Memory Controller, rev 2.0 gpmc0: CS#0 valid, addr 0x04000000, size 64MB gpmc0: CS#1 valid, addr 0x08000000, size 16MB sm0 at gpmc0 addr 0x08000300 intr 188 ## COMPAT_30, 1867 kb Kernel without NetBSD 3.0 compatibility. -rwxr-xr-x 1 test test 2259424 2008-06-25 10:55 sys/arch/evbarm/compile/obj/TISDP2420/netbsd -rwxr-xr-x 1 test test 1911944 2008-06-25 10:55 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.bin -rw-r--r-- 1 test test 3031468 2008-06-25 10:55 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.map text data bss dec hex filename 1509828 339080 209440 2058348 1f686c sys/arch/evbarm/compile/obj/TISDP2420/netbsd # dmesg|egrep "memory|gpm" total memory = 62464 KB avail memory = 58416 KB DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x82538000, paddr=0x806b0DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x8255e000, paddr=0x806e0DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x82584000, paddr=0x80700gpmc0 at mainbus0 base 0x6800a000: General Purpose Memory Controller, rev 2.0 gpmc0: CS#0 valid, addr 0x04000000, size 64MB gpmc0: CS#1 valid, addr 0x08000000, size 16MB sm0 at gpmc0 addr 0x08000300 intr 188 ## ksyms, 1510 kb Kernel without /dev/ksyms support. -rwxr-xr-x 1 test test 1925248 2008-06-25 10:59 sys/arch/evbarm/compile/obj/TISDP2420/netbsd -rwxr-xr-x 1 test test 1546376 2008-06-25 10:59 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.bin -rw-r--r-- 1 test test 3023265 2008-06-25 10:59 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.map text data bss dec hex filename 1503148 39048 209056 1751252 1ab8d4 sys/arch/evbarm/compile/obj/TISDP2420/netbsd # dmesg|egrep "memory|gpm" total memory = 62464 KB avail memory = 58764 KB DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x8253a000, paddr=0x80660DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x82560000, paddr=0x80680DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x82586000, paddr=0x806b0gpmc0 at mainbus0 base 0x6800a000: General Purpose Memory Controller, rev 2.0 gpmc0: CS#0 valid, addr 0x04000000, size 64MB gpmc0: CS#1 valid, addr 0x08000000, size 16MB sm0 at gpmc0 addr 0x08000300 intr 188 ## PTRACE, 1510 kb Kernel without process tracing support. -rwxr-xr-x 1 test test 1924948 2008-06-25 11:04 sys/arch/evbarm/compile/obj/TISDP2420/netbsd -rwxr-xr-x 1 test test 1546376 2008-06-25 11:04 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.bin -rw-r--r-- 1 test test 3015392 2008-06-25 11:04 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.map text data bss dec hex filename 1499312 39048 209056 1747416 1aa9d8 sys/arch/evbarm/compile/obj/TISDP2420/netbsd # dmesg|egrep "memory|gpm" total memory = 62464 KB avail memory = 58764 KB DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x8253a000, paddr=0x80660DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x82560000, paddr=0x80680DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x82586000, paddr=0x806b0gpmc0 at mainbus0 base 0x6800a000: General Purpose Memory Controller, rev 2.0 gpmc0: CS#0 valid, addr 0x04000000, size 64MB gpmc0: CS#1 valid, addr 0x08000000, size 16MB sm0 at gpmc0 addr 0x08000300 intr 188 ## FFS, 1510 kb Kernel without fast filesystem support. -rwxr-xr-x 1 test test 1924948 2008-06-25 11:09 sys/arch/evbarm/compile/obj/TISDP2420/netbsd -rwxr-xr-x 1 test test 1546376 2008-06-25 11:09 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.bin -rw-r--r-- 1 test test 3015329 2008-06-25 11:09 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.map text data bss dec hex filename 1499248 39048 209056 1747352 1aa998 sys/arch/evbarm/compile/obj/TISDP2420/netbsd # dmesg|egrep "memory|gpm" total memory = 62464 KB avail memory = 58764 KB DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x8253a000, paddr=0x80660DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x82560000, paddr=0x80680DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x82586000, paddr=0x806b0gpmc0 at mainbus0 base 0x6800a000: General Purpose Memory Controller, rev 2.0 gpmc0: CS#0 valid, addr 0x04000000, size 64MB gpmc0: CS#1 valid, addr 0x08000000, size 16MB sm0 at gpmc0 addr 0x08000300 intr 188 ## MSDOSFS, 1509 kb Kernel without FAT filesystem support. -rwxr-xr-x 1 test test 1887407 2008-06-25 11:21 sys/arch/evbarm/compile/obj/TISDP2420/netbsd -rwxr-xr-x 1 test test 1546152 2008-06-25 11:21 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.bin -rw-r--r-- 1 test test 2955657 2008-06-25 11:21 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.map text data bss dec hex filename 1463132 38824 208672 1710628 1a1a24 sys/arch/evbarm/compile/obj/TISDP2420/netbsd # dmesg|egrep "memory|gpm" total memory = 62464 KB avail memory = 58764 KB DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x82538000, paddr=0x80660DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x8255e000, paddr=0x80680DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x82584000, paddr=0x806b0gpmc0 at mainbus0 base 0x6800a000: General Purpose Memory Controller, rev 2.0 gpmc0: CS#0 valid, addr 0x04000000, size 64MB gpmc0: CS#1 valid, addr 0x08000000, size 16MB sm0 at gpmc0 addr 0x08000300 intr 188 ## PTYFS, 1509 kb Kernel without pseudo TTY filesystem and pseudo-device pty. -rwxr-xr-x 1 test test 1883462 2008-06-25 11:37 sys/arch/evbarm/compile/obj/TISDP2420/netbsd -rwxr-xr-x 1 test test 1545916 2008-06-25 11:37 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.bin -rw-r--r-- 1 test test 2911049 2008-06-25 11:37 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.map text data bss dec hex filename 1446512 38588 208536 1693636 19d7c4 sys/arch/evbarm/compile/obj/TISDP2420/netbsd # dmesg|egrep "memory|gpm" total memory = 62464 KB avail memory = 58764 KB DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x82536000, paddr=0x80660DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x8255c000, paddr=0x80680DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x82582000, paddr=0x806b0gpmc0 at mainbus0 base 0x6800a000: General Purpose Memory Controller, rev 2.0 gpmc0: CS#0 valid, addr 0x04000000, size 64MB gpmc0: CS#1 valid, addr 0x08000000, size 16MB sm0 at gpmc0 addr 0x08000300 intr 188 ## bpfilter, 1445 kb Kernel without Berkeley packet filter pseudo-device support. -rwxr-xr-x 1 test test 1848968 2008-06-25 11:46 sys/arch/evbarm/compile/obj/TISDP2420/netbsd -rwxr-xr-x 1 test test 1480372 2008-06-25 11:46 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.bin -rw-r--r-- 1 test test 2896335 2008-06-25 11:46 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.map text data bss dec hex filename 1435984 38580 208272 1682836 19ad94 sys/arch/evbarm/compile/obj/TISDP2420/netbsd # dmesg|egrep "memory|gpm" total memory = 62464 KB avail memory = 58828 KB DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x82536000, paddr=0x80650DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x8255c000, paddr=0x80670DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x82582000, paddr=0x806a0gpmc0 at mainbus0 base 0x6800a000: General Purpose Memory Controller, rev 2.0 gpmc0: CS#0 valid, addr 0x04000000, size 64MB gpmc0: CS#1 valid, addr 0x08000000, size 16MB sm0 at gpmc0 addr 0x08000300 intr 188 ## INSECURE, 1445 kb Kernel without integrity protection. -rwxr-xr-x 1 test test 1848968 2008-06-25 11:54 sys/arch/evbarm/compile/obj/TISDP2420/netbsd -rwxr-xr-x 1 test test 1480372 2008-06-25 11:54 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.bin -rw-r--r-- 1 test test 2896335 2008-06-25 11:54 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.map text data bss dec hex filename 1435984 38580 208272 1682836 19ad94 sys/arch/evbarm/compile/obj/TISDP2420/netbsd # dmesg|egrep "memory|gpm" total memory = 62464 KB avail memory = 58828 KB DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x82536000, paddr=0x80650DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x8255c000, paddr=0x80670DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x82582000, paddr=0x806a0gpmc0 at mainbus0 base 0x6800a000: General Purpose Memory Controller, rev 2.0 gpmc0: CS#0 valid, addr 0x04000000, size 64MB gpmc0: CS#1 valid, addr 0x08000000, size 16MB sm0 at gpmc0 addr 0x08000300 intr 188 ## INET6, 1316 kb Kernel without IPv6 support. -rwxr-xr-x 1 test test 1666318 2008-06-25 11:57 sys/arch/evbarm/compile/obj/TISDP2420/netbsd -rwxr-xr-x 1 test test 1348532 2008-06-25 11:57 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.bin -rw-r--r-- 1 test test 2666559 2008-06-25 11:57 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.map text data bss dec hex filename 1271320 37812 199760 1508892 17061c sys/arch/evbarm/compile/obj/TISDP2420/netbsd # dmesg|egrep "memory|gpm" total memory = 62464 KB avail memory = 58960 KB DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x82537000, paddr=0x80630DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x8255d000, paddr=0x80650DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x82583000, paddr=0x80680gpmc0 at mainbus0 base 0x6800a000: General Purpose Memory Controller, rev 2.0 gpmc0: CS#0 valid, addr 0x04000000, size 64MB gpmc0: CS#1 valid, addr 0x08000000, size 16MB sm0 at gpmc0 addr 0x08000300 intr 188 ## MALLOC_NOINLINE, 1316 kb Kernel without inlined malloc functions. -rwxr-xr-x 1 test test 1666318 2008-06-25 12:01 sys/arch/evbarm/compile/obj/TISDP2420/netbsd -rwxr-xr-x 1 test test 1348532 2008-06-25 12:01 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.bin -rw-r--r-- 1 test test 2666559 2008-06-25 12:01 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.map text data bss dec hex filename 1271320 37812 199760 1508892 17061c sys/arch/evbarm/compile/obj/TISDP2420/netbsd # dmesg|egrep "memory|gpm" total memory = 62464 KB avail memory = 58960 KB DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x82537000, paddr=0x80630DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x8255d000, paddr=0x80650DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x82583000, paddr=0x80680gpmc0 at mainbus0 base 0x6800a000: General Purpose Memory Controller, rev 2.0 gpmc0: CS#0 valid, addr 0x04000000, size 64MB gpmc0: CS#1 valid, addr 0x08000000, size 16MB sm0 at gpmc0 addr 0x08000300 intr 188 ## PIPE_SOCKETPAIR, 1316 kb Kernel with smaller but slower pipe implementation. -rwxr-xr-x 1 test test 1665096 2008-06-25 12:04 sys/arch/evbarm/compile/obj/TISDP2420/netbsd -rwxr-xr-x 1 test test 1348468 2008-06-25 12:04 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.bin -rw-r--r-- 1 test test 2658886 2008-06-25 12:04 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.map text data bss dec hex filename 1264788 37748 199760 1502296 16ec58 sys/arch/evbarm/compile/obj/TISDP2420/netbsd # dmesg|egrep "memory|gpm" total memory = 62464 KB avail memory = 58964 KB DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x82537000, paddr=0x80630DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x8255d000, paddr=0x80650DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x82583000, paddr=0x80670gpmc0 at mainbus0 base 0x6800a000: General Purpose Memory Controller, rev 2.0 gpmc0: CS#0 valid, addr 0x04000000, size 64MB gpmc0: CS#1 valid, addr 0x08000000, size 16MB sm0 at gpmc0 addr 0x08000300 intr 188 ## VMSWAP, 1316 kb Kernel without swap support. Requires a build patch, but may still crash the kernel when uvm statistics are called from top, ps etc. programs. --- a/sys/uvm/uvm_pdpolicy_clock.c +++ b/sys/uvm/uvm_pdpolicy_clock.c @@ -262,12 +262,13 @@ uvmpdpol_balancequeue(int swap_shortage) /* * if there's a shortage of swap slots, try to free it. */ - +#if defined(VMSWAP) if (swap_shortage > 0 && (p->pqflags & PQ_SWAPBACKED) != 0) { if (uvmpd_trydropswap(p)) { swap_shortage--; } } +#endif /* defined(VMSWAP) */ /* * if there's a shortage of inactive pages, deactivate. -rwxr-xr-x 1 test test 1662968 2008-06-25 12:09 sys/arch/evbarm/compile/obj/TISDP2420/netbsd -rwxr-xr-x 1 test test 1348392 2008-06-25 12:09 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.bin -rw-r--r-- 1 test test 2642042 2008-06-25 12:09 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.map text data bss dec hex filename 1248392 37672 199248 1485312 16aa00 sys/arch/evbarm/compile/obj/TISDP2420/netbsd # dmesg|egrep "memory|gpm" total memory = 62464 KB avail memory = 58968 KB DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x82537000, paddr=0x80630DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x8255d000, paddr=0x80650DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x82583000, paddr=0x80670gpmc0 at mainbus0 base 0x6800a000: General Purpose Memory Controller, rev 2.0 gpmc0: CS#0 valid, addr 0x04000000, size 64MB gpmc0: CS#1 valid, addr 0x08000000, size 16MB sm0 at gpmc0 addr 0x08000300 intr 188 ## SYSVMSG, SYSVSEM, SYSVSHM, 1252 kb Kernel without system V message queues, semaphores and shared memory. -rwxr-xr-x 1 test test 1627140 2008-06-25 12:15 sys/arch/evbarm/compile/obj/TISDP2420/netbsd -rwxr-xr-x 1 test test 1282748 2008-06-25 12:15 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.bin -rw-r--r-- 1 test test 2612942 2008-06-25 12:15 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.map text data bss dec hex filename 1225204 37564 198904 1461672 164da8 sys/arch/evbarm/compile/obj/TISDP2420/netbsd # dmesg|egrep "memory|gpm" total memory = 62464 KB avail memory = 59032 KB DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x82537000, paddr=0x80620DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x8255d000, paddr=0x80640DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x82583000, paddr=0x80660gpmc0 at mainbus0 base 0x6800a000: General Purpose Memory Controller, rev 2.0 gpmc0: CS#0 valid, addr 0x04000000, size 64MB gpmc0: CS#1 valid, addr 0x08000000, size 16MB sm0 at gpmc0 addr 0x08000300 intr 188 ## rnd, 1252 kb Kernel without /dev/random pseudo-device. -rwxr-xr-x 1 test test 1625313 2008-06-25 12:25 sys/arch/evbarm/compile/obj/TISDP2420/netbsd -rwxr-xr-x 1 test test 1282684 2008-06-25 12:25 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.bin -rw-r--r-- 1 test test 2599464 2008-06-25 12:25 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.map text data bss dec hex filename 1219556 37500 197944 1455000 163398 sys/arch/evbarm/compile/obj/TISDP2420/netbsd # dmesg|egrep "memory|gpm" total memory = 62464 KB avail memory = 59032 KB DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x82537000, paddr=0x80620DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x8255d000, paddr=0x80640DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x82583000, paddr=0x80660gpmc0 at mainbus0 base 0x6800a000: General Purpose Memory Controller, rev 2.0 gpmc0: CS#0 valid, addr 0x04000000, size 64MB gpmc0: CS#1 valid, addr 0x08000000, size 16MB sm0 at gpmc0 addr 0x08000300 intr 188 ## md, 1252 kb Kernel without support for memory devices like ramdisks. -rwxr-xr-x 1 test test 1624021 2008-06-25 12:29 sys/arch/evbarm/compile/obj/TISDP2420/netbsd -rwxr-xr-x 1 test test 1282588 2008-06-25 12:29 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.bin -rw-r--r-- 1 test test 2581393 2008-06-25 12:29 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.map text data bss dec hex filename 1214024 37404 197752 1449180 161cdc sys/arch/evbarm/compile/obj/TISDP2420/netbsd # dmesg|egrep "memory|gpm" total memory = 62464 KB avail memory = 59032 KB DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x82537000, paddr=0x80620DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x8255d000, paddr=0x80640DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x82583000, paddr=0x80660gpmc0 at mainbus0 base 0x6800a000: General Purpose Memory Controller, rev 2.0 gpmc0: CS#0 valid, addr 0x04000000, size 64MB gpmc0: CS#1 valid, addr 0x08000000, size 16MB sm0 at gpmc0 addr 0x08000300 intr 188 ## maxusers 2, 1252 kb Kernel with maxusers set to two instead of the default 32. -rwxr-xr-x 1 test test 1624021 2008-06-25 12:34 sys/arch/evbarm/compile/obj/TISDP2420/netbsd -rwxr-xr-x 1 test test 1282588 2008-06-25 12:34 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.bin -rw-r--r-- 1 test test 2581393 2008-06-25 12:34 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.map text data bss dec hex filename 1214024 37404 197752 1449180 161cdc sys/arch/evbarm/compile/obj/TISDP2420/netbsd # dmesg|egrep "memory|gpm" total memory = 62464 KB avail memory = 59548 KB DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x824b6000, paddr=0x805a0DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x824dc000, paddr=0x805c0DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x82502000, paddr=0x805e0gpmc0 at mainbus0 base 0x6800a000: General Purpose Memory Controller, rev 2.0 gpmc0: CS#0 valid, addr 0x04000000, size 64MB gpmc0: CS#1 valid, addr 0x08000000, size 16MB sm0 at gpmc0 addr 0x08000300 intr 188 ## MFS, 1124 kb Kernel without memory filesystem. -rwxr-xr-x 1 test test 1483269 2008-06-25 12:37 sys/arch/evbarm/compile/obj/TISDP2420/netbsd -rwxr-xr-x 1 test test 1151292 2008-06-25 12:37 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.bin -rw-r--r-- 1 test test 2437216 2008-06-25 12:37 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.map text data bss dec hex filename 1106936 37180 197688 1341804 14796c sys/arch/evbarm/compile/obj/TISDP2420/netbsd # dmesg|egrep "memory|gpm" total memory = 62464 KB avail memory = 59672 KB DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x824b3000, paddr=0x80580DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x824d9000, paddr=0x805a0DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x824ff000, paddr=0x805d0gpmc0 at mainbus0 base 0x6800a000: General Purpose Memory Controller, rev 2.0 gpmc0: CS#0 valid, addr 0x04000000, size 64MB gpmc0: CS#1 valid, addr 0x08000000, size 16MB sm0 at gpmc0 addr 0x08000300 intr 188 ## COREDUMP, 1124 kb Kernel without core dump support. -rwxr-xr-x 1 test test 1482841 2008-06-25 13:34 sys/arch/evbarm/compile/obj/TISDP2420/netbsd -rwxr-xr-x 1 test test 1151292 2008-06-25 13:34 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.bin -rw-r--r-- 1 test test 2428587 2008-06-25 13:34 sys/arch/evbarm/compile/obj/TISDP2420/netbsd.map text data bss dec hex filename 1102708 37180 197688 1337576 1468e8 sys/arch/evbarm/compile/obj/TISDP2420/netbsd # dmesg|egrep "memory|gpm" total memory = 62464 KB avail memory = 59672 KB DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x824b3000, paddr=0x80580DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x824d9000, paddr=0x805a0DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x824ff000, paddr=0x805d0gpmc0 at mainbus0 base 0x6800a000: General Purpose Memory Controller, rev 2.0 gpmc0: CS#0 valid, addr 0x04000000, size 64MB gpmc0: CS#1 valid, addr 0x08000000, size 16MB sm0 at gpmc0 addr 0x08000300 intr 188 # Kernel memory consumption The kernel executable size is not the only measure of kernel memory consumption. A number of userspace tools are able to show how the kernel and it's executable threads use memory while the system is running. Following chapters list some example tool output on the ARM OMAP 2420 board. Using these tools is easy, but interpreting the numbers seems to require some indepth knowledge of the NetBSD kernel's memory management system uvm. More details on the meaning of these numbers as well as examples of found problems and countermeasures like kernel configurations and sysctl setting would be appreciated. ## top top shows overall memory usage in the system and an interesting process called system. # top load averages: 0.14, 0.03, 0.01 09:20:54 4 processes: 3 sleeping, 1 on CPU CPU states: % user, % nice, % system, % interrupt, % idle Memory: 1916K Act, 208K Wired, 996K Exec, 428K File, 52M Free Swap: PID USERNAME PRI NICE SIZE RES STATE TIME WCPU CPU COMMAND 51 root 43 0 976K 784K CPU 0:00 14.00% 0.68% top 0 root 125 0 0K 1008K schedule 0:00 0.00% 0.00% [system] 40 root 85 0 980K 956K wait 0:00 0.00% 0.00% sh 1 root 85 0 32K 500K wait 0:00 0.00% 0.00% init # dmesg|grep -i mem total memory = 62464 KB avail memory = 58032 KB DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x82536000, paddr=0x80710DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x8255c000, paddr=0x80740DISPC: omap2_lcd_alloc_fb(): memory allocated at vaddr=0x82582000, paddr=0x80760gpmc0 at mainbus0 base 0x6800a000: General Purpose Memory Controller, rev 2.0 # top -t load averages: 0.00, 0.00, 0.00 up 0 days, 1:54 11:14:37 23 threads: 5 idle, 17 sleeping, 1 on CPU CPU states: % user, % nice, % system, % interrupt, % idle Memory: 3844K Act, 208K Wired, 1308K Exec, 2044K File, 50M Free Swap: PID LID USERNAME PRI STATE TIME WCPU CPU COMMAND NAME 0 6 root 223 IDLE 0:00 0.00% 0.00% [system] softser/0 0 3 root 222 IDLE 0:00 0.00% 0.00% [system] softnet/0 0 4 root 221 IDLE 0:00 0.00% 0.00% [system] softbio/0 0 5 root 220 IDLE 0:00 0.00% 0.00% [system] softclk/0 0 2 root 0 IDLE 0:00 0.00% 0.00% [system] idle/0 77 1 root 43 CPU 0:00 0.00% 0.00% top - 0 7 root 127 xcall 0:00 0.00% 0.00% [system] xcall/0 0 25 root 126 pgdaemon 0:00 0.00% 0.00% [system] pgdaemon 0 1 root 125 schedule 0:00 0.00% 0.00% [system] swapper 0 28 root 125 vmem_reh 0:00 0.00% 0.00% [system] vmem_rehash 0 27 root 125 aiodoned 0:00 0.00% 0.00% [system] aiodoned 0 9 root 125 cachegc 0:00 0.00% 0.00% [system] cachegc 0 8 root 125 vrele 0:00 0.00% 0.00% [system] vrele 0 26 root 124 syncer 0:00 0.00% 0.00% [system] ioflush 0 11 root 96 iicintr 0:00 0.00% 0.00% [system] iic0 ## ps ps can show the system light weight processes/threads too with **-s** flag and lname displays a more sesible name: # ps -awxs -o uid,pid,lid,nlwp,pri,ni,vsz,rss,command,lname UID PID LID NLWP PRI NI VSZ RSS COMMAND LNAME 0 0 28 20 125 0 0 976 [system vmem_rehash 0 0 27 20 125 0 0 976 [system aiodoned 0 0 26 20 124 0 0 976 [system ioflush 0 0 25 20 126 0 0 976 [system pgdaemon 0 0 24 20 96 0 0 976 [system nfsio 0 0 23 20 96 0 0 976 [system nfsio 0 0 22 20 96 0 0 976 [system nfsio 0 0 21 20 96 0 0 976 [system nfsio 0 0 12 20 96 0 0 976 [system iic1 0 0 11 20 96 0 0 976 [system iic0 0 0 10 20 96 0 0 976 [system pmfevent 0 0 9 20 125 0 0 976 [system cachegc 0 0 8 20 125 0 0 976 [system vrele 0 0 7 20 127 0 0 976 [system xcall/0 0 0 6 20 223 0 0 976 [system softser/0 0 0 5 20 220 0 0 976 [system softclk/0 0 0 4 20 221 0 0 976 [system softbio/0 0 0 3 20 222 0 0 976 [system softnet/0 0 0 2 20 0 0 0 976 [system idle/0 0 0 1 20 125 0 0 976 [system swapper 0 1 1 1 85 0 32 500 init - 0 40 1 1 85 0 980 968 -sh - 0 62 1 1 42 0 976 676 ps -awx - 0 63 1 1 85 0 980 784 less -r - ## pmap Displaying process 0 memory map with pmap shows the kernel's memory map. # pmap 0 80000000 17680K read/write/exec [ anon ] 81144000 4K read/write/exec [ anon ] 81145000 20K read/write/exec [ anon ] 8114A000 7328K read/write/exec [ kmem_map ] 81872000 4096K read/write/exec [ pager_map ] 81C72000 304K read/write/exec [ anon ] 81CBE000 4K read/write/exec [ anon ] 81CBF000 580K read/write/exec [ anon ] 81D50000 4096K read/write/exec [ exec_map ] 82150000 1200K read/write/exec [ phys_map ] 8227C000 2096K read/write/exec [ mb_map ] 82488000 20K read/write/exec [ anon ] 8248D000 8K read/write/exec [ uvm_aobj ] 8248F000 524K read/write/exec [ anon ] 82512000 48K read/write/exec [ uvm_aobj ] 8251E000 32K read/write/exec [ anon ] 82526000 8K read/write/exec [ uvm_aobj ] 82528000 48K read/write/exec [ anon ] 82534000 8K read/write/exec [ uvm_aobj ] 82536000 456K read/write/exec [ anon ] 825A8000 16K read/write/exec [ uvm_aobj ] 825AC000 12K read/write/exec [ anon ] 825AF000 48K read/write/exec [ uvm_aobj ] 825BB000 4K read/write/exec [ anon ] 825BC000 16K read/write/exec [ uvm_aobj ] 825C0000 56K read/write/exec [ anon ] 825CE000 8192K read/write/exec [ ubc_pager ] 82DCE000 52K read/write/exec [ anon ] 82DDB000 8K read/write/exec [ uvm_aobj ] 82DDD000 20K read/write/exec [ anon ] 82DE2000 8K read/write/exec [ uvm_aobj ] 82DE4000 4K read/write/exec [ anon ] 82DE6000 4K read/write/exec [ anon ] 82DE9000 8K read/write/exec [ uvm_aobj ] 82DEB000 8K read/write/exec [ anon ] 82DED000 8K read/write/exec [ uvm_aobj ] 82DEF000 16K read/write/exec [ anon ] 82DF7000 16K read/write/exec [ uvm_aobj ] 82DFB000 12K read/write/exec [ anon ] 82DFE000 8K read/write/exec [ uvm_aobj ] 82E00000 400K read/write/exec [ anon ] 82E68000 12K read/write/exec [ anon ] 82E70000 576K read/write/exec [ anon ] total 48064K ## vmstat vmstat can display a lot of NetBSD memory management details and statistics, especially if the kernel is compiled with KMEMSTAT options. [Output of **vmstat -C -m**](http://wiki.netbsd.se/images/1/1b/vmstat_C_m.txt) on the ARM board. # References * options(4) * * [[Tuning NetBSD for performance]] * KMEMSTAT * [VM tuning from swap perspective](http://www.selonen.org/arto/netbsd/) * [NetBSD Internals: Memory Management](http://www.netbsd.org/docs/internals/en/chap-memory.html) * * [[How to reduce libc size]] *