File:  [NetBSD Developer Wiki] / wikisrc / tutorials / pkgsrc / cross_compile_distcc.mdwn
Revision 1.5: download - view: text, annotated - select for diffs
Thu Oct 19 23:31:44 2017 UTC (2 years, 5 months ago) by khorben
Branches: MAIN
CVS tags: HEAD
Typo

    1: ## Distributing pkgsrc builds across different OSes
    2: 
    3: You may want to use several machines to speed up your *pkgsrc* builds, but
    4: as those computers are not running NetBSD, you may think they are useless as
    5: build-helpers. This is where enters NetBSD's *cross-compiling* system in
    6: conjunction with [distcc](http://code.google.com/p/distcc/).
    7: 
    8: ### A classic scenario
    9: 
   10: GNU/Linux is a very common OS you probably have on your network, in this
   11: tutorial, we will focus on how to use a Debian GNU/Linux system as a
   12: *distcc* node, but this procedure can apply to almost any pkgsrc-supported
   13: platform.
   14: 
   15: First things first, your GNU/Linux machine must have the following packages
   16: installed
   17: 
   18: * gcc
   19: * g++
   20: * zlib1g-dev
   21: * ncurses-base
   22: 
   23: ### Building the cross-compiling chain
   24: 
   25: Once done, download NetBSD's source tree. For example, if your target is a
   26: NetBSD 5.1.2 system
   27: 
   28: 	# pwd
   29: 	/home/netbsd
   30: 	# cvs -d anoncvs@anoncvs.fr.netbsd.org:/cvsroot co -rnetbsd-5-1-2-RELEASE src
   31: 
   32: We will then use the *build.sh* script in order to build the tools needed
   33: for cross-compiling
   34: 
   35: 	# cd src
   36: 	# ./build.sh -m amd64 tools
   37: 
   38: Do *not* use the *-u* flag, as we need to configure *nbcompat* for the tools
   39: to build correctly.
   40: 
   41: Once finished, you should have a directory like
   42: 
   43: 	tooldir.Linux-2.6.32-5-xen-amd64-x86_64
   44: 
   45: On the *src* directory. This is where the cross-compiling toolkit resides.
   46: 
   47: ### Installing and configuring distcc
   48: 
   49: It is mandatory to configure *distcc* so it uses our cross-comping tools to
   50: build binaries for another platform. On a Debian system, this is done in
   51: */etc/default/distcc*
   52: 
   53: 	STARTDISTCC="true"
   54: 	# [...]
   55: 	# Allowed networks
   56: 	ALLOWEDNETS="127.0.0.1 192.168.0.0/24"
   57: 	# IP where distcc will listen
   58: 	LISTENER="192.168.0.7"
   59: 	###
   60: 	# Here's the real trick, supersede $PATH so the firsts binaries
   61: 	# distcc will look for are NetBSD's ones
   62: 	###
   63: 	PATH=/home/netbsd/src/tooldir.Linux-2.6.32-5-xen-amd64-x86_64/x86_64--netbsd/bin:$PATH
   64: 
   65: After that, simply start *distcc*
   66: 
   67: 	/etc/init.d/distcc start
   68: 
   69: ### Testing the setup
   70: 
   71: On a NetBSD machine located in *distcc*'s allowed network, add the following to
   72: */etc/mk.conf*
   73: 
   74: 	PKGSRC_COMPILER=ccache distcc gcc
   75: 	MAKE_JOBS=4
   76: 	DISTCC_HOSTS=192.168.0.7 localhost
   77: 
   78: And fire up the *make* command in a *pkgsrc* subdirectory containing a C-based
   79: package. You should see something like this in helper's */var/log/distccd.log*
   80: file
   81: 
   82: 	distccd[20634] (dcc_job_summary) client: 192.168.0.2:64006 COMPILE_OK exit:0 sig:0 core:0 ret:0 time:57ms gcc /usr/pkgsrc/misc/tmux/work/.ccache-cache/tmp/unvis.tmp.bulkor.18298.i
   83: 	distccd[20734] (dcc_job_summary) client: 192.168.0.2:64005 COMPILE_OK exit:0 sig:0 core:0 ret:0 time:172ms gcc /usr/pkgsrc/misc/tmux/work/.ccache-cache/tmp/imsg-buffe.tmp.bulkor.20945.i
   84: 	distccd[20586] (dcc_job_summary) client: 192.168.0.2:64004 COMPILE_OK exit:0 sig:0 core:0 ret:0 time:40ms gcc /usr/pkgsrc/misc/tmux/work/.ccache-cache/tmp/strtonum.tmp.bulkor.4879.i
   85: 
   86: Of course, you may want to adjust the *MAKE_JOBS* according to the number of
   87: nodes your build-cluster has.
   88: 
   89: ### pbulk and distcc
   90: 
   91: If you intend to use distributed build while running
   92: [pbulk](http://wiki.netbsd.org/tutorials/pkgsrc/pbulk/), you **must** add the
   93: following to the */etc/mk.conf* of your *sandbox*
   94: 
   95: 	.for DISTCCDEPS in devel/ccache sysutils/checkperms pkgtools/digest devel/distcc devel/popt devel/libtool-base lang/f2c devel/gmake
   96: 	.	if ${PKGPATH} == ${DISTCCDEPS}
   97: 	IGNORE_DISTCC=	yes
   98: 	IGNORE_CCACHE=	yes
   99: 	.	endif
  100: 	.endfor
  101: 
  102: Or the scanning phase will end up with circular dependencies
  103: 
  104: 	Cyclic dependency for package:
  105: 	ccache-3.1.4nb1
  106: 	checkperms-1.11
  107: 	digest-20111104
  108: 	distcc-3.1nb1
  109: 	popt-1.16nb1
  110: 	libtool-base-2.2.6bnb5
  111: 	f2c-20100903
  112: 	ccache-3.1.4nb1
  113: 
  114: Enjoy your faster builds!

CVSweb for NetBSD wikisrc <wikimaster@NetBSD.org> software: FreeBSD-CVSweb