pkg comp pkg chk

Keeping packages up-to-date with pkg_comp and pkg_chk

Pkgsrc is a fantastic package management framework, but when it comes to upgrades, some use cases may lead to an unstable situation. Also, if by any chance you have 2 or more NetBSD machines to keep up-to-date, upgrading each one separately could be risky and a real waste of time. We'll see how to flawlessly keep your packages up-to-date with minimal risks.

pkg_comp

Under pkgsrc/pkgtools you will find a great utility called pkg_comp. This script permits to handle packages manipulation in a chrooted environment, thus keeping your real packages safe from any mistakes.

For pkg_comp 2.x, located under pkgsrc/pkgtools/pkg_comp, follow these tutorials:

For pkg_comp 1.x, located under pkgsrc/pkgtools/pkg_comp1, continue reading this page.

Let's install pkg_comp 1.x:

# cd /usr/pkgsrc/pkgtools/pkg_comp1
# make install clean

Once done, we will create the chrooted environment:

# mkdir -p /home/pkg_comp
# cd /home/pkg_comp
# pkg_comp -C test.conf maketemplate

This will create a template file, which will be used to build our fake NetBSD system, but first, we'll have to set up some information. Using your favourite editor, change the following variables to suit your needs:

DESTDIR, where the chroot will be built
DISTRIBDIR, where pkg_comp will find your NetBSD binary sets
SETS_X11 may be set to "no" if you do not intend to use the X Window system

This is my pkg_comp configuration:

...
DESTDIR="/home/pkg_comp/test"
DISTRIBDIR="/home/pkg_comp/dist/NetBSD-5.0.1"
SETS_X11="no"
...

If you don't yet have NetBSD's binary sets, download them from your favourite mirror and put them in /home/pkg_comp/dist/NetBSD-5.0.1/binary/sets. Also note that NetBSD's source directory (/usr/src in most cases) must exist.

We can now build the chroot using the following command:

# pkg_comp -C test.conf makeroot

From now on, we can enter our chroot using the chroot target:

# pkg_comp -C test.conf chroot
PKG_COMP ==> Mounting sandboxed filesystems
PKG_COMP ==> Entering sandbox `/home/pkg_comp/test'
pkg_comp:test.conf# exit
PKG_COMP ==> Unmounting sandboxed filesystems

A very simple method to build a package in the chroot is to use the build target: # pkg_comp -C test.conf build pkgtools/pkgfind

But as we want to keep a good control on our packages freshness and build method, we will use another tool: pkg_chk.

pkg_chk

pkg_chk is another tool available under pkgsrc/pkgtools. This script reads the content of the pkgsrc/pkgchk.conf file and checks if every listed package is up to date. You will have to install pkg_chk on the chroot as well as in the host.

Let's create a /usr/pkgsrc/pkgchk.conf file. Please note this must be done outside of the chroot, pkg_comp uses pkgsrc's directory to read content, but it mounts it as a read only partition. Here's an output of the mount command inside of the chroot:

/usr/src on /var/chroot/pkg_comp/default/usr/src type null (read-only, local)
/usr/pkgsrc on /var/chroot/pkg_comp/default/usr/pkgsrc type null (read-only, local)
/usr/pkgsrc/distfiles on /var/chroot/pkg_comp/default/pkg_comp/distfiles type null (local)
/usr/pkgsrc/packages on /var/chroot/pkg_comp/default/pkg_comp/packages type null (local)

As you can see, generated packages will be written to /usr/pkgsrc/packages and we are allowed to fetch distfiles to /usr/pkgsrc/distfiles, but /usr/pkgsrc and /usr/src are not writables.

# pkg_chk -g

This command will generate an initial pkgchk.conf file based upon the packages installed on the host machine.

Now, enter the chroot as we must configure its etc/mk.conf file:

# no X11
MKX11=no
# clean dependencies when the "clean" target is called
CLEANDEPENDS=yes
# everybody likes vim
ACCEPTABLE_LICENSES+=vim-license
# we want to build packages fo every software
DEPENDS_TARGET=package-install
UPDATE_TARGET=package-install

Everything is now ready. pkg_chk has many options, but we will keep its use very simple.

To see what operations are going to take place without actually doing anything, use the following switches:

pkg_comp:test.conf# pkg_chk -uan

When ready, call pkg_chk this way:

pkg_comp:test.conf# pkg_chk -ua

Depending on how many packages you must generate, this operation could be a rather long one.

Once the packages creation is finished, you may logout from pkg_comp and update your host's packages using binaries created by pkg_comp's pkg_chk:

# pkg_chk -uab

As pkg_chk manpage says:

-b  Use binary packages.  If -s is not set this allows pkg_chk to
    run without PKGSRCDIR.

Here we are! massive upgrade, no harm, no pain.

Upgrading more than one machine with pkgin

A convenient method to upgrade more than one machine is to use pkgtools/pkgin, a remote package installation and upgrade utility being able to handle packages dependencies.

In the machine hosting binary packages, install an HTTP or FTP server being able to access the directory where your binary packages are located. For example, using www/lighttpd:

...
dir-listing.activate = "enable"
...
$HTTP["host"] == "pkgsrc.home.imil.net" {
        server.document-root = "/usr/pkgsrc/packages"
}
...

In this directory, create a pkg_summary.bz2 file, where all packages, dependencies and descriptions will be available:

# cd /usr/pkgsrc/packages/All
# pkg_info -X * | bzip2 > pkg_summary.bz2

Then, on the machine to be upgraded, install pkgin :

# pkg_add -v ftp://ftp.fr.netbsd.org/pub/pkgsrc/packages/NetBSD/amd64/5.0/databases/sqlite3-3.6.17.tgz
# pkg_add -v ftp://ftp.fr.netbsd.org/pub/pkgsrc/packages/NetBSD/amd64/5.0/pkgtools/pkgin-0.2.5.tgz

And put your own repository in /usr/pkg/etc/pkgin/repositories.conf:

http://pkgsrc.home.imil.net/All

Update pkgin's database:

# pkgin up

And upgrade your packages:

# pkgin full-upgrade