NetBSD Wiki/pkgsrc/ An introduction to packaging

This guide should allow you to learn how to create a new port or simply fix a port that you need. There are three target demographics listed below:

- binary packages user with pkgin or pkg_add
    (you should be confident here)
- build from source, use options
    (you will know this after reading the guide)
- port developers
    (you should be able to get started here)

pkgsrc tree

You should have a copy of the pkgsrc tree sitting somewhere on your disk, already bootstrapped. The tree contains a Makefile, a README, distfiles, packages, category directories containing the ports, the bootstrap directory and some documentation.

The mk/* directory contains the pkgsrc framework Makefiles but also shell and Awk scripts

pkglocate is a script to find port names in the tree, though pkgtools/pkgfind is much faster.

use the right tools

If you want to get started working on ports like creating new ones or simply fix ones you need, you should know about these tools:

It contains very useful programs like:

port contents

A pkgsrc port should at least contain:

Here's how they would look like for a small port I submitted not long ago in pkgsrc-wip

Makefile:

# $NetBSD$

PKGNAME=      osxinfo-0.1
CATEGORIES=   misc
GHCOMMIT=     de74b8960f27844f7b264697d124411f81a1eab6
DISTNAME=     ${GHCOMMIT}
MASTER_SITES= https://github.com/yrmt/osxinfo/archive/

MAINTAINER=   youri.mout@gmail.com
HOMEPAGE=     http://github.com/yrmt/osxinfo
COMMENT=      Small Mac OS X Info Program
LICENSE=      isc

ONLY_FOR_PLATFORM= Darwin-*-*

DIST_SUBDIR= osxinfo
WRKSRC= ${WRKDIR}/osxinfo-${GHCOMMIT}

.include "../../databases/sqlite3/buildlink3.mk"
.include "../../mk/bsd.pkg.mk"

DESCR:

Small and fast Mac OS X info program written in C.

PLIST:

@comment $NetBSD$
bin/osxinfo

distinfo:

$NetBSD$

SHA1 (osxinfo/de74b8960f27844f7b264697d124411f81a1eab6.tar.gz) = 83a2838ad95ff73255bea7f496a8cc9aaa4e17ca
RMD160 (osxinfo/de74b8960f27844f7b264697d124411f81a1eab6.tar.gz) = 9102eb2a938be38c4adf8cfbf781c04d0844d09a
Size (osxinfo/de74b8960f27844f7b264697d124411f81a1eab6.tar.gz) = 5981 bytes

make

Now you know what kind of files you can see when you're in a port directory. The command used to compile it is the NetBSD make but often bmake on non NetBSD systems to avoid Makefile errors. Typing make alone will only compile the program but you can also use other command line arguments to make such as extract, patch, configure, install, package, ...

I'll try to list them and explain them in logical order. You can run them together.

You should be aware that there are many make options along with these targets, like

pkgsrc configuration

The framework uses an mk.conf file, usually found in /etc. Here's how mine looks:

# Tue Oct 15 21:21:46 CEST 2013

.ifdef BSD_PKG_MK          # begin pkgsrc settings

DISTDIR=                   /pkgsrc/distfiles
PACKAGES=                  /pkgsrc/packages
WRKOBJDIR=                 /pkgsc/work
ABI=                       64
PKGSRC_COMPILER=           clang
CC=                        clang
CXX=                       clang++
CPP=                       ${CC} -E

PKG_DBDIR=                 /var/db/pkg
LOCALBASE=                 /usr/pkg
VARBASE=                   /var
PKG_TOOLS_BIN=             /usr/pkg/sbin
PKGINFODIR=                info
PKGMANDIR=                 man
BINPKG_SITES=              http://pkgsrc.saveosx.org/Darwin/2013Q4/x86_64
DEPENDS_TARGET=            bin-install
X11_TYPE=                  modular
TOOLS_PLATFORM.awk?=       /usr/pkg/bin/nawk
TOOLS_PLATFORM.sed?=       /usr/pkg/bin/nbsed
ALLOW_VULNERABLE_PACKAGES= yes
MAKE_JOBS=                 8
SKIP_LICENSE_CHECK=        yes
PKG_DEVELOPER=             yes
SIGN_PACKAGES=             gpg
PKG_DEFAULT_OPTIONS+=      -pulseaudio -x264 -imlib2-amd64 -dconf
.endif                     # end pkgsrc settings

Keep in mind that there are many other available options.

creating a simple port

Let's create a little port using the tools we've talked about above. I will use a little window manager called 2bwm.

You'll be presented with a text editor like vim to enter basic Makefile options:

So now you have valid Makefile and distinfo files but you need to write a paragraph in DESCR. You can usually find inspiration on the program's homepage.

Here's how they look like at the moment:

Makefile:

# $NetBSD$

DISTNAME=       2bwm-0.1
CATEGORIES=     wm
MASTER_SITES=   http://pkgsrc.saveosx.org/Darwin/distfiles/

MAINTAINER=     yrmt@users.sourceforge.net
HOMEPAGE=       http://github.com/venam/2bwm/
COMMENT=        Fast floating WM written over the XCB library and derived from mcwm
LICENSE=        isc

.include "../../mk/bsd.pkg.mk"

distinfo:

$NetBSD$

SHA1 (2bwm-0.1.tar.gz) = e83c862dc1d9aa198aae472eeca274e5d98df0ad
RMD160 (2bwm-0.1.tar.gz) = d9a93a7d7ae7183f5921f9ad76abeb1401184ef9
Size (2bwm-0.1.tar.gz) = 38419 bytes

DESCR:

A fast floating WM, with the particularity of having 2 borders,
written over the XCB library and derived from mcwm written by
Michael Cardell. In 2bWM everything is accessible from the keyboard
but a pointing device can be used for move, resize and raise/lower.

But our PLIST file is still empty.

build stage

Let's try to build the port to see if things work but as soon as the build stage starts, we get this error:

2bwm.c:26:10: fatal error: 'xcb/randr.h' file not found

Let's find out which port provides this file !

$ pkgin se xcb

returns these possible packages:

xcb-util-wm-0.3.9nb1  Client and window-manager helpers for ICCCM and EWMH
xcb-util-renderutil-0.3.8nb1  Convenience functions for the Render extension
xcb-util-keysyms-0.3.9nb1  XCB Utilities
xcb-util-image-0.3.9nb1  XCB port of Xlib's XImage and XShmImage
xcb-util-0.3.9nb1 =  XCB Utilities
xcb-proto-1.9 =      XCB protocol descriptions (in XML)
xcb-2.4nb1           Extensible, multiple cut buffers for X

Package content inspection allowed me to find the right port

$ pkgin pc libxcb|grep randr.h

So we can add the libxcb buildlink3.mk file to the Makefile above the bsd.pkg.mk include:

.include "../../x11/libxcb/buildlink3.mk"

This allows the port to link 2bwm against the libxcb port. Let's try to build the port again!

$ make clean
$ make

Reports another error !

2bwm.c:27:10: fatal error: 'xcb/xcb_keysyms.h' file not found

It looks like this file is provided by xcb-util-keysyms, so let's add:

.include "../../x11/xcb-util-keysyms/buildlink3.mk"

in our Makefile.

Clean, build again, and add more dependencies until it passes the build stage. Here's how my Makefile ends up looking like:

# $NetBSD$

DISTNAME=       2bwm-0.1
CATEGORIES=     wm
MASTER_SITES=   http://pkgsrc.saveosx.org/Darwin/distfiles/

MAINTAINER=     yrmt@users.sourceforge.net
HOMEPAGE=       http://github.com/venam/2bwm/
COMMENT=        Fast floating WM written over the XCB library and derived from mcwm
LICENSE=        isc

.include "../../x11/libxcb/buildlink3.mk"
.include "../../x11/xcb-util-wm/buildlink3.mk"
.include "../../x11/xcb-util-keysyms/buildlink3.mk"
.include "../../x11/xcb-util/buildlink3.mk"
.include "../../mk/bsd.pkg.mk"

install phase

Geat ! We got our program to compile in pkgsrc. Now we must generate the PLIST file so we can actually install the program, but we must make stage-install to make sure that it installs in the right place.

$ find /pkgsrc/work/wm/2bwm/work/.destdir/

returns:

/pkgsrc/work/wm/2bwm/work/.destdir/
/pkgsrc/work/wm/2bwm/work/.destdir//usr
/pkgsrc/work/wm/2bwm/work/.destdir//usr/local
/pkgsrc/work/wm/2bwm/work/.destdir//usr/local/bin
/pkgsrc/work/wm/2bwm/work/.destdir//usr/local/bin/2bwm
/pkgsrc/work/wm/2bwm/work/.destdir//usr/local/bin/hidden
/pkgsrc/work/wm/2bwm/work/.destdir//usr/local/share
/pkgsrc/work/wm/2bwm/work/.destdir//usr/local/share/man
/pkgsrc/work/wm/2bwm/work/.destdir//usr/local/share/man/man1
/pkgsrc/work/wm/2bwm/work/.destdir//usr/local/share/man/man1/2bwm.1
/pkgsrc/work/wm/2bwm/work/.destdir//usr/local/share/man/man1/hidden.1
/pkgsrc/work/wm/2bwm/work/.destdir//usr/pkg

This doesn't look right since our LOCALBASE is /usr/pkg.

$ make print-PLIST

returns nothing, because 2bwm installs files in the wrong place so we need to fix 2bwm's own Makefile to use the right DESTDIR and PREFIX, that is set to the right place by pkgsrc. Let's inspect how 2bwm installs:

From 2bwm's Makefile:

install: $(TARGETS)
        test -d $(DESTDIR)$(PREFIX)/bin || mkdir -p $(DESTDIR)$(PREFIX)/bin
        install -pm 755 2bwm $(DESTDIR)$(PREFIX)/bin
        install -pm 755 hidden $(DESTDIR)$(PREFIX)/bin
        test -d $(DESTDIR)$(MANPREFIX)/man1 || mkdir -p $(DESTDIR)$(MANPREFIX)/man1
        install -pm 644 2bwm.man $(DESTDIR)$(MANPREFIX)/man1/2bwm.1
        install -pm 644 hidden.man $(DESTDIR)$(MANPREFIX)/man1/hidden.1

This looks fine since it installs in a DESTDIR/PREFIX but it sets

PREFIX=/usr/local

and

MANPREFIX=$(PREFIX)/share/man

In the beginning of the Makefile. We should remove the first line and edit the man prefix:

MANPREFIX=${PKGMANDIR}

so pkgsrc can install the program's files in the right place. We have two ways of modifying this file, either patch the Makefile or use sed substitution which is a builtin pkgsrc feature that allows you to change lines in files with a sed command before building the port.

I will show how to do both ways so you can get an introduction on how to generate patch files for pkgsrc.

patching the Makefile :

using the sed substitution framework

You should be able to fix the prefix error much quicker than with the patching explained above thanks to the sed substitution framework. Here's how it looks like in my port Makefile:

SUBST_CLASSES+=         makefile
SUBST_STAGE.makefile=   pre-build
SUBST_MESSAGE.makefile= Fixing makefile
SUBST_FILES.makefile=   Makefile
SUBST_SED.makefile=     -e 's,/usr/local,${PREFIX},g'
SUBST_SED.makefile+=    -e 's,share/man,${PKGMANDIR},g'

As you can see, you can do multiple commands on multiple files, it is very useful for very small fixes like this.

pkglint

Now that we have a working port, we must make sure it complies to the pkgsrc rules.

$ pkglint

Returns

ERROR: DESCR:4: File must end with a newline.
ERROR: patches/patch-Makefile:3: Comment expected.
2 errors and 0 warnings found. (Use -e for more details.)

Fix the things pkglint tells you to do until you get the glorious:

looks fine.

Then you should do some testing on the program itelf on at least two platforms such as NetBSD, Mac OS X. Other platforms supported by pkgsrc can be found at pkgsrc.org. If you would like to submit your pkgsrc upstream you can either subscribe to pkgsrc-wip or ask a NetBSD developer to add it for you.

You can find the 2bwm port I submitted in pkgsrc-wip.

pkgsrc and wip

If you want to submit your port for others to use you can either subscribe to pkgsrc-wip or ask a NetBSD developer to add it for you.

pkgsrc-wip is hosted on sourceforge and you can easily get cvs access to it if you create an account on there and send an email to NetBSD developer @wiz (Thomas Klausner) asking nicely for commit access.

the options framework

You can create port options with the options.mk file, like for wm/dwm

# $NetBSD$

PKG_OPTIONS_VAR=            PKG_OPTIONS.dwm
PKG_SUPPORTED_OPTIONS=  xinerama
PKG_SUGGESTED_OPTIONS=  xinerama

.include "../../mk/bsd.options.mk"

#
# Xinerama support
#
# If we don't want the Xinerama support we delete XINERAMALIBS and
# XINERAMAFLAGS lines, otherwise the Xinerama support is the default.
#
.if !empty(PKG_OPTIONS:Mxinerama)
.  include "../../x11/libXinerama/buildlink3.mk"
.else
SUBST_CLASSES+=         options
SUBST_STAGE.options=    pre-build
SUBST_MESSAGE.options=  Toggle the Xinerama support
SUBST_FILES.options=    config.mk
SUBST_SED.options+=     -e '/^XINERAMA/d'
.  include "../../x11/libX11/buildlink3.mk"
.endif

This file should be included in the Makefile:

.include "options.mk"

If you type make show-options, you should see this:

Any of the following general options may be selected:
xinerama     Enable Xinerama support.

These options are enabled by default:
    xinerama

These options are currently enabled:
    xinerama

You can select which build options to use by setting    PKG_DEFAULT_OPTIONS
or PKG_OPTIONS.dwm.

Running make PKG_OPTIONS="" should build without the xinerama dwm option enabled by default.

The options.mk file must contain these variables:

It allows you to change configure arguments and include other buildlinks, and various other settings.

hosting a package repo

Now that you've created a few ports, you might want to make precompiled packages available for testing. You will need pkgsrc's pkg_install on the host system. I host my packages on a FreeBSD server with a bootstrapped pkgsrc.

use this shell function to :

add () {
    # upload the package to remote server
    scp $1 youri@saveosx.org:/usr/local/www/saveosx/packages/Darwin/2013Q4/x86_64/All/ 2> /dev/null
    
    # update the package summary
    ssh youri@saveosx.org 'cd /usr/local/www/saveosx/packages/Darwin/2013Q4/x86_64/All/;
           rm pkg_summary.gz;
           /usr/pkg/sbin/pkg_info -X *.tgz | gzip -9 > pkg_summary.gz'
    
    # pkgin update
    sudo pkgin update
}

And this shell alias to upload all my built packages, but I still need to run add() mentionned above to update the pkg_summary

up='rsync -avhz --progress /pkgsrc/packages/ youri@saveosx.org:/usr/local/www/saveosx/packages/Darwin/2013Q4/x86_64/'

Then you should be able to set the url in repositories.conf to use your packages with pkgin. You can also install them directly with something like pkg_add http://pkgsrc.saveosx.org/Darwin/2013Q4/x86_64/All/9menu-1.8nb1.tgz of course.

build all packages

see jperkin's excellent blog posts about this.

faq

what if the port I'm making is a dependency for another one?

You should just generate the buildlink3.mk file we've talked about earlier like this:

$ createbuildlink > buildlink3.mk

what if the program is only hosted on GitHub ?

pkgsrc supports fetching archives from specific git commits on GitHub like this:

PKGNAME=           2bwm-0.1
CATEGORIES=        wm
GHCOMMIT=          52a097ca644eb571b22a135951c945fcca57a25c
DISTNAME=          ${GHCOMMIT}
MASTER_SITES=      https://github.com/venam/2bwm/archive/
DIST_SUBDIR=       2bwm
WRKSRC=            ${WRKDIR}/2bwm-${GHCOMMIT}

You can then easily update the git commit and the distinfo with it to update the program.

what if the program doesn't have a Makefile

You can do all Makefile operations directly from the port's Makefile like this:

post-extract:
    ${CHMOD} a-x ${WRKSRC}/elementary/apps/48/internet-mail.svg

do-install:
    ${INSTALL_DATA_DIR} ${DESTDIR}${PREFIX}/share/icons
    cd ${WRKSRC} && pax -rw -pe . ${DESTDIR}${PREFIX}/share/icons/

To install, but you can also build programs from the Makefile. This is what qt4-sqlite3 uses:

do-build:
    cd ${WRKSRC}/src/tools/bootstrap && env ${MAKE_ENV} ${GMAKE}
    cd ${WRKSRC}/src/tools/moc && env ${MAKE_ENV} ${GMAKE}
    cd ${WRKSRC}/src/plugins/sqldrivers/sqlite && env ${MAKE_ENV} ${GMAKE}

You can install the following type of files:

INSTALL_PROGRAM_DIR : directories that contain binaries

INSTALL_SCRIPT_DIR : directories that contain scripts

INSTALL_LIB_DIR : directories that contain shared and static libraries

INSTALL_DATA_DIR: directories that contain data files

INSTALL_MAN_DIR : directories that contain man pages

INSTALL_PROGRAM : binaries that can be stripped from debugging symbols

INSTALL_SCRIPT : binaries that cannot be stripped

INSTALL_GAME : game binaries

INSTALL_LIB : shared and static libraries

INSTALL_DATA : data files

INSTALL_GAME_DATA : data files for games

INSTALL_MAN : man pages

INSTALLATION_DIRS : A list of directories relative to PREFIX that are created by pkgsrc at the beginning of the install phase. The package is supposed to create all needed directories itself before installing files to it and list all other directories here.

common errors

This means you're not using the right make. On most systems, the make installed from the pkgsrc bootstrap is called bmake

Add a comment
Last edited at lunch time on Tuesday, November 2nd, 2021
Preferences | Logout