Return to how_to_do_an_unprivileged_bulk_build_on_solaris.mdwn CVS log

Up to [NetBSD Developer Wiki] / wikisrc / pkgsrc

import a bunch of pkgsrc articles from the old pkgsrc.se wiki

First, I bootstrapped pkgsrc into the directory /usr/pkg/2006Q4.

    $HOME/proj/pkgsrc/bootstrap/bootstrap --prefix=/usr/pkg/2006Q4 --unprivileged --compiler=sunpro
    mkdir /usr/pkg/2006Q4/etc
    mv work/mk.conf.example /usr/pkg/2006Q4/etc/mk.conf

Then I made a backup of that directory. You can never know. :)

    cd /usr/pkg && gtar cfz 2006Q4.tar.gz 2006Q4

Since in a previous try, the pre-build program had removed the pkgsrc directory, I just commented it out by inserting an exit 0 in line 2. Since now there is no danger that anything in that directory is deleted, I could edit the mk.conf file and adjust some local settings.
#/usr/pkg/2006Q4/mk.conf
<pre><code>
# Example /usr/pkg/2006Q4/etc/mk.conf file produced by bootstrap-pkgsrc
# Tue Jan  9 13:01:49 CET 2007

.ifdef BSD_PKG_MK	# begin pkgsrc settings

PKGSRC_COMPILER=	sunpro
UNPRIVILEGED=		yes
PKG_DBDIR=		/usr/pkg/2006Q4/var/db/pkg
LOCALBASE=		/usr/pkg/2006Q4
VARBASE=		/usr/pkg/2006Q4/var
PKG_TOOLS_BIN=		/usr/pkg/2006Q4/sbin
PKGMANDIR=		man

TOOLS_PLATFORM.awk?=		/usr/pkg/2006Q4/bin/nawk
TOOLS_PLATFORM.sed?=		/usr/pkg/2006Q4/bin/nbsed
FETCH_CMD=			/usr/pkg/2006Q4/bin/ftp
TOOLS_PLATFORM.pax?=		/usr/pkg/2006Q4/bin/pax
TOOLS_PLATFORM.tar?=		/usr/pkg/2006Q4/bin/tar
TOOLS_PLATFORM.mtree?=		/usr/pkg/2006Q4/sbin/mtree

DISTDIR=		/usr/pkg/distfiles
PACKAGES=		/usr/pkg/2006Q4-packages

PKG_DEVELOPER=		yes

CC=	cc
CXX=	CC
CPP=	cc -E
CXXCPP=	CC -E

SUNWSPROBASE=		/local/SUNWspro
USE_LANGUAGES+=		c99

TOOLS_PLATFORM.nroff=	/opt/bin/groff

.endif			# end pkgsrc settings
</code></pre>
/usr/pkg/2006Q4/build.conf
<pre><code>
# $NetBSD: how_to_do_an_unprivileged_bulk_build_on_solaris.mdwn,v 1.1 2011/11/20 20:28:27 mspo Exp $
#

osrev=`uname -r`
arch=`uname -m`
USR_PKGSRC="$HOME/proj/pkgsrc"
MAKECONF="/usr/pkg/2006Q4/etc/mk.conf"

PRUNEDISTFILES=no
NICE_LEVEL="nice -n 20"
LINTPKGSRC_CACHE=no

ADMIN="1illig@informatik.uni-hamburg.de"
ADMINSIG="- Roland"

REPORTS_DIR="/usr/pkg/2006Q4-pkgstat"
REPORT_BASEDIR=`date +%Y%m%d.%H%M`
REPORT_HTML_FILE="report.html"
REPORT_TXT_FILE="report.txt"
REPORTS_URL="file://$REPORTS_DIR"

UPDATE_VULNERABILITY_LIST=yes
PRUNEPACKAGES=yes
MKSUMS=yes
MKSUMMARY=no
RSYNC_DST=ftp.NetBSD.org:/pub/NetBSD/packages/pkgsrc-200xQy/NetBSD-a.b.c/i386
RSYNC_OPTS='-e ssh'
</code></pre>
I began to setup the bulk build environment in a screen.

    $ screen -S <ulk-2006Q4
    $ sh ./proj/pkgsrc/mk/bulk/build -c /usr/pkg/2006Q4/etc/build.conf

It seems to work, and I'm waiting for the package databases to be generated. But since I have a T2000 here, I'm thinking about parallelizing the work. For generating the databases, the two "make" processes could be run completely in parallel, making use of 12000 processors. :)

In the main phase, all packages that have the exact same dependencies can be built in parallel, but must be still installed one after another.
#Help, my files are being removed

When I had started the bulk build, it complained that it couldn't find /usr/pkg/2006Q4/bin/nawk anymore. But looking at the backup I made above, I say that it had been there after bootstrapping. The simple solution to this was to change the tools definitions in mk.conf so that they point to tools outside ${LOCALBASE}:
<pre><code>
TOOLS_PLATFORM.awk=	/usr/pkg/current/bin/nawk
TOOLS_PLATFORM.sed?=	/usr/pkg/current/bin/nbsed
FETCH_CMD=		/usr/pkg/current/bin/ftp
TOOLS_PLATFORM.pax?=	/usr/pkg/current/bin/pax
TOOLS_PLATFORM.tar?=	/usr/pkg/current/bin/tar
TOOLS_PLATFORM.mtree?=	/usr/pkg/current/sbin/mtree
</code></pre>
Luckily, bin/bmake is not registered as belonging to any package, so it hasn't been removed yet.

The bulk build also tried to deinstall the infrastructure packages, so I had to protect them against that:
<pre><code>
BULK_PREREQ+=		pkgtools/bootstrap-mk-files
BULK_PREREQ+=		pkgtools/tnftp
BULK_PREREQ+=		pkgtools/mtree
BULK_PREREQ+=		pkgtools/pax
BULK_PREREQ+=		pkgtools/pkg_install
BULK_PREREQ+=		sysutils/mtree

BULK_PREREQ+=		sysutils/checkperms
</code></pre>