File:  [NetBSD Developer Wiki] / wikisrc / tutorials / how_to_reduce_libc_size.mdwn
Revision 1.2: download - view: text, annotated - select for diffs
Sun Feb 5 07:14:36 2012 UTC (8 years, 9 months ago) by schmonz
Branches: MAIN
CVS tags: HEAD
dos2unix

    1: **Contents**
    2: 
    3: [[!toc levels=3]]
    4: 
    5: #  Introduction 
    6: 
    7: The NetBSD installation is not big, but it can be made smaller. Here are a few tricks to make the base of user space, the C standard library libc, a bit smaller for dynamically linked systems. These trials were done on a cross compiled NetBSD current ARMv6 branch. 
    8: 
    9: First the user space is built with default options. Then the cross compiler script **$TOOLDIR/bin/nbmake-evbarm** was used to clean and rebuild the libc with special options. The new libc binary was then copied to the target file system and a smoke test of booting the ARM development board was done. If /sbin/init and /bin/sh managed with the new libc, the test was a success while everything else was a failure. 
   10: 
   11: The result is a crippled libc and /lib/libc.so.12.159 file size reduction from 1164 to 692 kilobytes. Run time memory usage is harder to predict since it depends on what parts of in memory libc are actually used by the processes, but at least text and data sections reported by the size utility give some idea. 
   12: 
   13: #  Build options 
   14: 
   15: ##  Default -O2, file size 1164025 
   16: 
   17: -O2 optimization is used by default and the libc file size after a **./build.sh -U -m evbarm build** is: 
   18:     
   19:     -r--r--r-- 1 test test 1164025 2008-04-18 08:23 obj/destdir.evbarm/lib/libc.so.12.159
   20:     
   21: 
   22: Sections reported by size utility: 
   23:     
   24:        text    data     bss     dec     hex filename
   25:      931608   25728   64332 1021668   f96e4 obj/libc.so.12.159
   26:     
   27: 
   28: ##  -O1, file size 1159845 
   29: 
   30: If the libc is build with **CFLAGS=-O1**, the ELF shared object file size is: 
   31:     
   32:     -rw-r--r-- 1 test test 1159845 Apr 19 09:20 lib/libc.so.12.159
   33:     
   34: 
   35: Sections reported by size utility: 
   36:     
   37:        text    data     bss     dec     hex filename
   38:      927436   25728   64332 1017496   f8698 obj/libc.so.12.159
   39:     
   40: 
   41: ###  Linker strip, file size 1065624 
   42: 
   43: If the **-O1** build is stripped and size optimized by the linker with **LDFLAGS=-Wl,-O1\ -Wl,-s**, the file size reduces to: 
   44:     
   45:     -rwxr-xr-x 1 test test 1065624 2008-04-19 13:28 obj/libc.so.12.159 
   46:     
   47: 
   48: Sections reported by size utility: 
   49:     
   50:        text    data     bss     dec     hex filename
   51:      923316   25728   64332 1013376   f7680 obj/libc.so.12.159
   52:     
   53: 
   54: ##  -Os, file size 1094281 
   55: 
   56: The gcc compiler can optimize binaries for size with **CFLAGS=-Os**: 
   57:     
   58:     -rwxr-xr-x 1 test test 1094281 2008-04-19 10:56 obj/libc.so.12.159
   59:     
   60: 
   61: Sections reported by size utility: 
   62:     
   63:        text    data     bss     dec     hex filename
   64:      861864   25736   64332  951932   e867c obj/libc.so.12.159
   65:     
   66: 
   67: ###  Manual strip, file size 1004180 
   68: 
   69: The binary can then be stripped manually: 
   70:     
   71:     $ $TOOLDIR/bin/arm--netbsdelf-strip -s obj/libc.so.12.159 
   72:     $ ls -l obj/libc.so.12.159
   73:     -rwxr-xr-x 1 test test 1004180 2008-04-19 11:02 obj/libc.so.12.159
   74:     
   75: 
   76: Sections reported by size utility: 
   77:     
   78:        text    data     bss     dec     hex filename
   79:      861864   25736   64332  951932   e867c obj/libc.so.12.159
   80:     
   81: 
   82: ###  Linker strip, file size 1000060 
   83: 
   84: The **-Os** compiled binary is smaller with linker based strip and optimization where **LDFLAGS=-Wl,-O1\ -Wl,-s** than with a manual strip: 
   85:     
   86:     -rwxr-xr-x 1 test test 1000060 2008-04-19 12:07 obj/libc.so.12.159
   87:     
   88: 
   89: Sections reported by size utility: 
   90:     
   91:        text    data     bss     dec     hex filename
   92:      857744   25736   64332  947812   e7664 obj/libc.so.12.159
   93:     
   94: 
   95: #  Feature removal 
   96: 
   97: In addition to compiler flags **CFLAGS=-Os LDFLAGS=-Wl,-O1\ -Wl,-s**, special feature flags can be used to strip features out of libc and reduce its size. Some feature flags, as documented by **BUILDING** and **share/mk/bsd.README**, are supported for the whole user space. 
   98: 
   99: ##  SCCS version strings, file size 953136 
  100: 
  101: SCCS version strings are normally embedded into object file, but they be removed by following changes in lib/libc/Makefile: 
  102:     
  103:     --- lib/libc/Makefile.inc       3 Jun 2007 17:36:08 -0000       1.3 
  104:     +++ lib/libc/Makefile.inc       19 Apr 2008 11:01:23 -0000
  105:     @@ -24,7 +24,8 @@
  106:      .include <bsd.own.mk>
  107:     
  108:      WARNS=4
  109:     -CPPFLAGS+=     -D_LIBC -DLIBC_SCCS -DSYSLIBC_SCCS -D_REENTRANT
  110:     +#CPPFLAGS+=    -D_LIBC -DLIBC_SCCS -DSYSLIBC_SCCS -D_REENTRANT
  111:     +CPPFLAGS+=     -D_LIBC -D_REENTRANT
  112:     
  113:      .if (${USE_HESIOD} != "no")
  114:      CPPFLAGS+=     -DHESIOD
  115:     
  116: 
  117: The resulting libc binary finally goes below the one megabyte mark: 
  118:     
  119:     -rwxr-xr-x 1 test test 953136 2008-04-19 13:54 obj/libc.so.12.159
  120:     
  121: 
  122: Sections reported by size utility: 
  123:     
  124:        text    data     bss     dec     hex filename
  125:      857744   25736   64332  947812   e7664 obj/libc.so.12.159
  126:     
  127: 
  128: ##  Hesiod name service, file size 942468 
  129: 
  130: [Hesiod](http://en.wikipedia.org/wiki/Hesiod_(name_service)), a DNS based database service, support can be removed from libc with **USE_HESIOD=no MKHESIOD=no** build variables and result is: 
  131:     
  132: -rwxr-xr-x 1 test test 942468 2008-04-19 14:16 obj/libc.so.12.159
  133:     
  134: 
  135: Sections reported by size utility: 
  136:     
  137:        text    data     bss     dec     hex filename
  138:      847625   25252   62180  935057   e4491 obj/libc.so.12.159
  139:     
  140: 
  141: ##  Yellow Pages (YP), file size 917368 
  142: 
  143: [Yellow Pages (YP) or Network Information Service (NIS)](http://en.wikipedia.org/wiki/Network_Information_Service) directory service support can be removed with **USE_YP=no MKYP=no** variables: 
  144:     
  145: -rwxr-xr-x 1 test test 917368 2008-04-19 14:29 obj/libc.so.12.159
  146:     
  147: 
  148: Sections reported by size utility: 
  149:     
  150:        text    data     bss     dec     hex filename
  151:      824328   24488   58944  907760   dd9f0 obj/libc.so.12.159
  152:     
  153: 
  154: ##  IPv6 support, file size 909272 
  155: 
  156: [IPv6](http://en.wikipedia.org/wiki/IPv6) support can be removed with **USE_INET6=no**: 
  157:     
  158: -rwxr-xr-x 1 test test 909272 2008-04-19 14:48 obj/libc.so.12.159
  159:     
  160: 
  161: Sections reported by size utility: 
  162:     
  163:        text    data     bss     dec     hex filename
  164:      816537   24368   58944  899849   dbb09 obj/libc.so.12.159
  165:     
  166: 
  167: ##  Stack smashing protection (SSP), file size 894764 
  168: 
  169: [SSP](http://en.wikipedia.org/wiki/Stack-smashing_protection) buffer overflow protection from the GCC compiler can be disabled with **USE_SSP=no** and the libc binary size goes below 900k: 
  170:     
  171: -rwxr-xr-x 1 test test 894764 2008-04-19 15:02 obj/libc.so.12.159
  172:     
  173: 
  174: Sections reported by size utility: 
  175:     
  176:        text    data     bss     dec     hex filename
  177:      802029   24368   58944  885341   d825d obj/libc.so.12.159
  178:     
  179: 
  180: ##  Remote procedure call (RPC), file size 806036 
  181: 
  182: [RPC](http://en.wikipedia.org/wiki/ONC_RPC) support can be disabled with **MKRPC=no** variable and a patch like this: 
  183:     
  184: --- lib/libc/Makefile   9 Jan 2008 01:33:52 -0000       1.131.4.1 
  185:     +++ lib/libc/Makefile   23 Apr 2008 13:04:42 -0000
  186:     @@ -74,7 +80,10 @@
  187:      .endif
  188:      .include "${.CURDIR}/regex/Makefile.inc"
  189:      .include "${.CURDIR}/resolv/Makefile.inc"
  190:     +MKRPC?= yes
  191:     +.if (${MKRPC} != "no")
  192:      .include "${.CURDIR}/rpc/Makefile.inc"
  193:     +.endif
  194:      .include "${.CURDIR}/ssp/Makefile.inc"
  195:      .include "${.CURDIR}/stdio/Makefile.inc"
  196:      .include "${.CURDIR}/stdlib/Makefile.inc"
  197:     
  198: 
  199: As a result the libc size goes down to 806 kilobytes: 
  200:     
  201:     -rw-r--r-- 1 test test 806036 2008-04-23 16:00 lib/libc.so.12.159
  202:     
  203: 
  204: Sections reported by size utility: 
  205:     
  206:        text    data     bss     dec     hex filename
  207:      717964   22624   58500  799088   c3170 obj/libc.so.12.159
  208:     
  209: 
  210: ##  Execution profiling control, file size 801720 
  211: 
  212: [Profiling control](http://www.daemon-systems.org/man/moncontrol.3.html) can be removed from libc with **MKGMON=no** and a patch like: 
  213:     
  214: --- lib/libc/Makefile   9 Jan 2008 01:33:52 -0000       1.131.4.1 
  215:     +++ lib/libc/Makefile   24 Apr 2008 08:07:08 -0000
  216:     @@ -58,7 +58,10 @@
  217:      .include "${.CURDIR}/dlfcn/Makefile.inc"
  218:      .include "${.CURDIR}/gdtoa/Makefile.inc"
  219:      .include "${.CURDIR}/gen/Makefile.inc"
  220:     +MKGMON?= yes
  221:     +.if (${MKGMON} != "no")
  222:      .include "${.CURDIR}/gmon/Makefile.inc"
  223:     +.endif
  224:      .include "${.CURDIR}/hash/Makefile.inc"
  225:      .include "${.CURDIR}/iconv/Makefile.inc"
  226:      .include "${.CURDIR}/inet/Makefile.inc"
  227:     
  228: 
  229: And libc size goes down around 4 kilobytes: 
  230:     
  231:     -rwxr-xr-x 1 test test 801720 2008-04-24 10:57 obj/libc.so.12.159 
  232:     
  233: 
  234: Sections reported by size utility: 
  235:     
  236:        text    data     bss     dec     hex filename
  237:      713959   22500   58436  794895   c210f obj/libc.so.12.159
  238:     
  239: 
  240: ##  Citrus I18N, file size 767560 
  241: 
  242: [Citrus I18N](http://citrus.bsdclub.org/) support can be removed with **MKCITRUS=no**, [lib/libc/locale/setlocale.c patch](http://wiki.netbsd.se/images/d/d5/citrus_locale_patch01.txt) and a makefile patch: 
  243:     
  244: 
  245: 
  246: --- Makefile    9 Jan 2008 01:33:52 -0000       1.131.4.1 
  247:     +++ Makefile    25 Apr 2008 07:51:20 -0000
  248:     @@ -53,12 +53,18 @@
  249:     
  250:      .include "${.CURDIR}/../../common/lib/libc/Makefile.inc"
  251:      .include "${.CURDIR}/db/Makefile.inc"
  252:     +MKCITRUS?= yes
  253:     +.if (${MKCITRUS} != "no")
  254:      .include "${.CURDIR}/citrus/Makefile.inc"
  255:     +.endif
  256:      .include "${.CURDIR}/compat-43/Makefile.inc"
  257:      .include "${.CURDIR}/dlfcn/Makefile.inc"
  258:      .include "${.CURDIR}/gdtoa/Makefile.inc"
  259:     
  260: 
  261: The libc binary is now below 800 kilobytes: 
  262:     
  263:     -rwxr-xr-x 1 test test 767560 2008-04-25 10:01 obj/libc.so.12.159 
  264:     
  265: 
  266: Sections reported by size utility: 
  267:     
  268:        text    data     bss     dec     hex filename
  269:      685150   18696   56968  760814   b9bee obj/libc.so.12.159
  270:     
  271: 
  272: ##  MD2, RMD160, SHA1, SHA2, MD4 and MD5 cryptographic hash functions, file size 723780 
  273: 
  274: All [cryptographic hash functions](http://en.wikipedia.org/wiki/Cryptographic_hash_function) can be removed from the libc with **MKMD2=no MKRMD160=no MKSHA1=no MKSHA2=no MKMD=no** and lib/libc/hash/Makefile.inf and lib/libc/Makefile patches: 
  275:     
  276: 
  277: --- lib/libc/hash/Makefile.inc  27 Oct 2006 18:29:21 -0000      1.11
  278:     +++ lib/libc/hash/Makefile.inc  30 Apr 2008 09:10:21 -0000
  279:     @@ -4,8 +4,20 @@
  280:      # hash functions
  281:      .PATH: ${ARCHDIR}/hash ${.CURDIR}/hash
  282:     
  283:     +MKMD2?= yes
  284:     +.if (${MKMD2} != "no")
  285:      .include "${.CURDIR}/hash/md2/Makefile.inc"
  286:     +.endif
  287:     +MKRMD160?= yes
  288:     +.if (${MKRMD160} != "no")
  289:      .include "${.CURDIR}/hash/rmd160/Makefile.inc"
  290:     +.endif
  291:     +MKSHA1?= yes
  292:     +.if (${MKSHA1} != "no")
  293:      .include "${.CURDIR}/hash/sha1/Makefile.inc"
  294:     +.endif
  295:     +MKSHA2?= yes
  296:     +.if (${MKSHA2} != "no")
  297:      .include "${.CURDIR}/hash/sha2/Makefile.inc"
  298:     +.endif
  299:     
  300:     
  301:     --- lib/libc/Makefile   9 Jan 2008 01:33:52 -0000       1.131.4.1
  302:     +++ lib/libc/Makefile   30 Apr 2008 09:11:25 -0000
  303:     
  304:      .include "${.CURDIR}/inet/Makefile.inc"
  305:      .include "${.CURDIR}/isc/Makefile.inc"
  306:      .include "${.CURDIR}/locale/Makefile.inc"
  307:     +MKMD?= yes
  308:     +.if (${MKMD} != "no")
  309:      .include "${.CURDIR}/md/Makefile.inc"
  310:     +.endif
  311:      .include "${.CURDIR}/misc/Makefile.inc"
  312:      .include "${.CURDIR}/net/Makefile.inc"
  313:      .include "${.CURDIR}/nameser/Makefile.inc"
  314:     
  315: 
  316: libc size reduces to: 
  317:     
  318:     -rwxr-xr-x 1 test test 723780 2008-04-30 12:03 obj/libc.so.12.159
  319:     
  320: 
  321: Sections reported by size utility: 
  322:     
  323:        text    data     bss     dec     hex filename
  324:      642476   18456   56968  717900   af44c obj/libc.so.12.159
  325:     
  326: 
  327: ##  Misc, Native Language Support (NLS), Regular Expression and Stack Smashing Protection, file size 691884 
  328: 
  329: Indeed, misc object, [NLS](http://en.wikipedia.org/wiki/Internationalization_and_localization), [regular expression](http://en.wikipedia.org/wiki/Regular_expression) (IEEE Std 1003.2-1992 (“POSIX.2”) regular expressions) and [Stack Smashing Protection (SSP)](http://en.wikipedia.org/wiki/Stack-smashing_protection) support library are easily removed from the build process with **MKMISC=no MKNLS=no MKREGEX=no MKSSP=no** and an obvious patch to lib/libc/Makefile: 
  330:     
  331: 
  332:  .include "${.CURDIR}/md/Makefile.inc"
  333:     +.endif
  334:     +MKMISC?= yes
  335:     +.if (${MKMISC} != "no")
  336:      .include "${.CURDIR}/misc/Makefile.inc"
  337:     +.endif
  338:      .include "${.CURDIR}/net/Makefile.inc"
  339:      .include "${.CURDIR}/nameser/Makefile.inc"
  340:     +MKNLS?= yes
  341:     +.if (${MKNLS} != "no")
  342:      .include "${.CURDIR}/nls/Makefile.inc"
  343:     +.endif
  344:      .if (${MACHINE_ARCH} != "alpha") && (${MACHINE_ARCH} != "sparc64")
  345:      .include "${.CURDIR}/quad/Makefile.inc"
  346:      .endif
  347:     +MKREGEX?= yes
  348:     +.if (${MKREGEX} != "no")
  349:      .include "${.CURDIR}/regex/Makefile.inc"
  350:     +.endif
  351:      .include "${.CURDIR}/resolv/Makefile.inc"
  352:     ...
  353:      .include "${.CURDIR}/rpc/Makefile.inc"
  354:     +.endif
  355:     +MKSSP?= yes
  356:     +.if (${MKSSP} != "no")
  357:      .include "${.CURDIR}/ssp/Makefile.inc"
  358:     +.endif
  359:      .include "${.CURDIR}/stdio/Makefile.inc"
  360:      .include "${.CURDIR}/stdlib/Makefile.inc"
  361:      .include "${.CURDIR}/string/Makefile.inc"
  362:     
  363: 
  364: Boot with sbin/init still works as does bin/sh, but user friendly programs like bin/ps and bin/ls now fail due to missing symbols: 
  365:     
  366:     # ls -l lib/libc.so.12.159
  367:     /lib/libc.so.12: Undefined PLT symbol "__fgets_chk" (symnum = 156)
  368:     # ls lib/libc.so.12.159
  369:     lib/libc.so.12.159
  370:     # ps aux
  371:     /lib/libc.so.12: Undefined PLT symbol "__strcat_chk" (symnum = 207)
  372:     # ps
  373:     ps: warning: /var/run/dev.db: /lib/libc.so.12: Undefined PLT symbol "_catopen" )
  374:     
  375: 
  376: File size now: 
  377:     
  378:     -rwxr-xr-x 1 test test 691884 2008-04-30 14:18 obj/libc.so.12.159
  379:     
  380: 
  381: Segment sizes reported by size utility: 
  382:     
  383:        text    data     bss     dec     hex filename
  384:      614238   17284   56928  688450   a8142 obj/libc.so.12.159
  385:     
  386: 
  387: #  Ideas 
  388: 
  389: While a few compiler and feature options were a tried out, a number of new ideas were found. Some of these were quickly tried out, but they resulted in the build or smoke/boot test failures. 
  390: 
  391: ##  Compiler options 
  392: 
  393: ###  Thumb instruction set, file size 585440 
  394: 
  395: Compile to 16 bit THUMB instruction set instead of normal 32 bit 
  396: 
  397:   * Whole user space needs to be build with **-mthumb-interwork**
  398:   * CPUFLAGS build variable should contain **-mthumb** and **-mthumb-interwork**
  399:   * If some files (like atomic_init_testset.c) need arm32 code due to embedded ARM assembly or other tool chain issues, the CPUFLAGS build variable can be overridden: [[Per file build options override]]
  400:   * libc from matt-armv6 branch builds with -mthumb but fails to run with SIGILL: [Thumb compilation discussion on port-arm](http://mail-index.netbsd.org/port-arm/2008/04/30/msg000175.html)
  401: 
  402: 
  403: After a successfull compile with atomic_init_testset.o compiled with -mthumb-interwork only, the file size is: 
  404:     
  405:     -rwxr-xr-x 1 mira mira 585440 2008-05-12 11:20 obj/libc.so.12.159
  406:     
  407: 
  408: size utility reports: 
  409:     
  410:        text    data     bss     dec     hex filename
  411:      507462   17616   56928  582006   8e176 obj/libc.so.12.159
  412:     
  413: 
  414: ##  Feature removals 
  415: 
  416: ###  Database support from lib/libc/db 
  417: 
  418:   * sbin/init and possibly others depend on Berkeley DB support 
  419:   * a custom init and bin/sh propably work without it 
  420: 
  421: ###  Regular memory allocator instead of JEMALLOC 
  422: 
  423:   * USE_JEMALLOC: (share/mk/bsd.README) If "no", disables building the "jemalloc" allocator designed for improved performance with threaded applications. 
  424: 
  425:     * seems to require rebuild of user space, but even after that init dies with: 
  426:     
  427:     warning: no /dev/console 
  428:     panic: init died (signal 0, exit 12)
  429:     Stopped in pid 1.1 (init) at    netbsd:cpu_Debugger+0x4:        bx      r14
  430:     
  431: 
  432: ###  Reduce resolver size in lib/libc/resolver 
  433: 
  434:   * remove IPv6 support 
  435:   * remove debug features 
  436: 
  437: ###  Use libhack 
  438: 
  439:   * Use size optimized distrib/utils/libhack with/instead of libc 
  440: 
  441: #  Alternatives 
  442: 
  443: ##  Crunchgen 
  444: 
  445: crunchgen can be used to create statically (at compile time) linked executables, which include only those object files, which are really used by the executable. 
  446: 
  447:   * Use crunchgen or related build infra to build a dynamic library from only those object files that are actually required by the applications 
  448: 
  449: #  References 
  450: 
  451:   * The **BUILDING** and **share/mk/bsd.README** files in addition to libc specific makefiles and sources contain information on the different build and feature options. 
  452:   * gcc and ld manual pages 
  453:   * [Shrinking NetBSD - deep final distribution re-linker, discussion on tech-userlevel](http://thread.gmane.org/gmane.os.netbsd.devel.embedded/268)
  454:   * [Reducing libc size, discussion on tech-userlevel](http://thread.gmane.org/gmane.os.netbsd.devel.userlevel/9333)
  455: 

CVSweb for NetBSD wikisrc <wikimaster@NetBSD.org> software: FreeBSD-CVSweb