File:  [NetBSD Developer Wiki] / wikisrc / tutorials / pkgsrc / pbulk.mdwn
Revision 1.16: download - view: text, annotated - select for diffs
Fri Feb 7 02:22:35 2020 UTC (3 months, 2 weeks ago) by jnemeth
Branches: MAIN
CVS tags: HEAD
mention that a Bourne style shell must be used

    1: ## Using pbulk to create a pkgsrc binary repository
    2: 
    3: *pkgtools/pbulk* package consists in a set of tools designed to ease
    4: mass-building of *pkgsrc* packages, and the creation your own *pkgsrc* binaries
    5: repository.
    6: 
    7: Its setup needs a bit of work, here is how to prepare and run your bulk-build
    8: box. In this article we will only consider a one-node machine.
    9: 
   10: This documentation is based on the [The pkgsrc Guide](http://netbsd.org/docs/pkgsrc/bulk.html).
   11: 
   12: ### Prerequisites
   13: 
   14: These are the prerequisites needed by *pbulk*:
   15: 
   16: * A *pkgsrc* source tree
   17: * Possibly a *src* source tree, only some rare packages needs it. If you choose not to checkout *src*, simply create a */usr/src* directory so *mksandbox* (see below) doesn't complain about non-existing directory.
   18: * Possibly (not mandatory) a tool like *misc/screen* or *misc/tmux* as the full build process can take a **very** long time.
   19: 
   20: For example, to prepare a *pkgsrc-2011Q3* bulk build:
   21: 
   22: 	# cd /usr
   23: 	# cvs -d anoncvs@anoncvs.netbsd.org:/cvsroot co -rpkgsrc-2011Q3 pkgsrc
   24: 
   25: Avoid automatic update of *pkgsrc* tree (cron or such), if you're in the middle of a build, it could lead to unfortunate results.
   26: 
   27: ### Prepare a chroot
   28: 
   29: In order to isolate the bulk build, it is advised you run all the operations
   30: within a *chroot*. Running *pbulk* on your real environment would wipe all
   31: of your installed packages, and would modify your base system with lots of
   32: directories, users and groups you don't need.
   33: 
   34: Fortunately, a tool called *mksandbox* will simplify this process. *mksandbox*
   35: is located in the *pkgtools/mksandbox* package, and it is called like this:
   36: 
   37: 	# mksandbox [optionnal flags] /path/to/sandbox
   38: 
   39: For example, to create a sandbox in */home/bulk* without the X11 system, run:
   40: 
   41: 	# mksandbox --without-x /home/bulk
   42: 
   43: This command will prepare and mount most of the needed directories, and will
   44: place a shell script on top of the sandbox filesystem called *sandbox*. This
   45: script is used to *mount*/*umount* your sandbox. It is a good idea to add
   46: */var/spool* to the list of directories mounted as read/write in your sandbox
   47: so the email report is actually sent. Simply add:
   48: 
   49: 	/var/spool /var/spool rw \
   50: 
   51: to the list of directories in the *sandbox* script. *sandbox* script use is
   52: really straightforward:
   53: 
   54: 	# /path/to/your/sandbox/sandbox umount
   55: 
   56: Unmounts the sandbox
   57: 
   58: 	# /path/to/your/sandbox/sandbox mount
   59: 
   60: Mounts the sandbox
   61: 
   62: ### Prepare the pbulk environment
   63: 
   64: Now that our sandbox is available and mounted, we will chroot to it by executing
   65: the *sandbox* script without any parameter specified:
   66: 
   67: 	# /path/to/your/sandbox/sandbox
   68: 
   69: Create pbulk user (a Bourne style shell must be used)
   70: 
   71: 	# useradd pbulk -s /bin/sh
   72: 
   73: In a file, set the preferences to use when building packages (mk.conf fragment).
   74: Here is a sample mk.conf.frag file:
   75: 
   76: 	SKIP_LICENSE_CHECK=             yes
   77: 	ALLOW_VULNERABLE_PACKAGES=      yes
   78: 
   79: 	PKG_DEVELOPER?=         yes
   80: 
   81: 	WRKDIR=                 /tmp/work
   82: 
   83: 	# site specific changes
   84: 
   85: 	PKG_OPTIONS.irssi=      perl inet6
   86: 	PKG_OPTIONS.mplayer=    oss
   87: 
   88: 	DSPAM_STORAGE_DRIVER=   mysql
   89: 	PKG_OPTIONS.dspam+=     graphs
   90: 	PKG_OPTIONS.dovecot=    ssl ldap dovecot-sieve dovecot-managesieve
   91: 	PKG_OPTIONS.nagios-nrpe=ssl tcpwrappers
   92: 
   93: 	X11_TYPE=               modular
   94: 
   95: Deploy and configure pbulk tools
   96: 
   97: 	# sh /usr/pkgsrc/mk/pbulk/pbulk.sh -n -c mk.conf.frag
   98: 
   99: *pbulk* configuration file is */usr/pbulk/etc/pbulk.conf*.
  100: You may want to review and customize some parameters like
  101: "base_url" and "report_recipients".
  102: 
  103: Also, in order to avoid hangs, it might be a good idea to add the following to the top of *pbulk.conf*
  104: 
  105: 	ulimit -t 1800 # set the limit on CPU time (in seconds)
  106: 	ulimit -v 2097152 # limits process address space
  107: 
  108: ### Running the build
  109: 
  110: Now that everything's in place, we can fire up the build from the *chroot* using
  111: the following command:
  112: 
  113: 	# /usr/pbulk/bin/bulkbuild
  114: 
  115: It is recommended to run the build inside a tool like *misc/screen* or
  116: *misc/tmux* as it will take a lot of time.
  117: 
  118: If the build is stopped, it is possible to restart it by invoking:
  119: 
  120: 	# /usr/pbulk/bin/bulkbuild-restart
  121: 
  122: ### Hints
  123: 
  124: If you'd like to rebuild a single package, use the *bulkbuild-rebuild* command followed by the package name.

CVSweb for NetBSD wikisrc <wikimaster@NetBSD.org> software: FreeBSD-CVSweb