Using pbulk to create a pkgsrc binary repository
pkgtools/pbulk package consists in a set of tools designed to ease mass-building of pkgsrc packages, and the creation your own pkgsrc binaries repository.
Its setup needs a bit of work, here is how to prepare and run your bulk-build box. In this article we will only consider a one-node machine.
This documentation is based on the The pkgsrc Guide.
These are the prerequisites needed by pbulk:
- A pkgsrc source tree
- Possibly a src source tree, only some rare packages needs it. If you choose not to checkout src, simply create a /usr/src directory so mksandbox (see below) doesn't complain about non-existing directory.
- Possibly (not mandatory) a tool like misc/screen or misc/tmux as the full build process can take a very long time.
For example, to prepare a pkgsrc-2011Q3 bulk build:
# cd /usr # cvs -d firstname.lastname@example.org:/cvsroot co -rpkgsrc-2011Q3 pkgsrc
Avoid automatic update of pkgsrc tree (cron or such), if you're in the middle of a build, it could lead to unfortunate results.
Prepare a chroot
In order to isolate the bulk build, it is advised you run all the operations within a chroot. Running pbulk on your real environment would wipe all of your installed packages, and would modify your base system with lots of directories, users and groups you don't need.
Fortunately, a tool called mksandbox will simplify this process. mksandbox is located in the pkgtools/mksandbox package, and it is called like this:
# mksandbox [optionnal flags] /path/to/sandbox
For example, to create a sandbox in /home/bulk without the X11 system, run:
# mksandbox --without-x /home/bulk
This command will prepare and mount most of the needed directories, and will place a shell script on top of the sandbox filesystem called sandbox. This script is used to mount/umount your sandbox. It is a good idea to add /var/spool to the list of directories mounted as read/write in your sandbox so the email report is actually sent. Simply add:
/var/spool /var/spool rw \
to the list of directories in the sandbox script. sandbox script use is really straightforward:
# /path/to/your/sandbox/sandbox umount
Unmounts the sandbox
# /path/to/your/sandbox/sandbox mount
Mounts the sandbox
Prepare the pbulk environment
Now that our sandbox is available and mounted, we will chroot to it:
# chroot /home/bulk /bin/sh
Create pbulk user
# useradd bulk
Prepare configuration for packages build (mk.conf fragment). Here's sample mk.conf.frag file:
SKIP_LICENSE_CHECK= yes ALLOW_VULNERABLE_PACKAGES= yes PKG_DEVELOPER?= yes # site specific changes PKG_OPTIONS.irssi= perl inet6 PKG_OPTIONS.mplayer= oss DSPAM_STORAGE_DRIVER= mysql PKG_OPTIONS.dspam+= graphs PKG_OPTIONS.dovecot= ssl ldap dovecot-sieve dovecot-managesieve PKG_OPTIONS.nagios-nrpe=ssl tcpwrappers X11_TYPE= modular
Deploy and configure pbulk tools
# sh /usr/pkgsrc/mk/pbulk/pbulk.sh -n -c mk.conf.frag
pbulk configuration file is /usr/pbulk/etc/pbulk.conf. You may want to review and customize some parameters like "base_url" and "report_recipients".
Also, in order to avoid hangs, it might be a good idea to add the following to the top of pbulk.conf
ulimit -t 1800 # set the limit on CPU time (in seconds) ulimit -v 2097152 # limits process address space
Running the build
Now that everything's in place, we can fire up the build from the chroot using the following command:
It is recommended to run the build inside a tool like misc/screen or misc/tmux as it will take a lot of time.
If the build is stopped, it is possible to restart it by invoking:
If you'd like to rebuild a single package, use the bulkbuild-rebuild command followed by the package name.