File:  [NetBSD Developer Wiki] / wikisrc / pkgsrc / intro_to_packaging.mdwn
Revision 1.1: download - view: text, annotated - select for diffs
Tue Jul 8 14:19:33 2014 UTC (9 years, 2 months ago) by schmonz
Branches: MAIN
CVS tags: HEAD
Add introductory packaging tutorial, written and submitted by Youri Mouton.

    1: This guide should allow you to learn how to create a new port or simply fix a port that you need. There are three target demographics listed below:
    2: 
    3: 	- binary packages user with pkgin or pkg_add 
    4:  		(you should be confident here)
    5: 	- build from source, use options 
    6:  		(you will know this after reading the guide)
    7: 	- port developers
    8: 		(you should be able to get started here)
    9: 
   10: 
   11: ## pkgsrc tree
   12: 
   13: You should have a copy of the pkgsrc tree sitting somewhere on your disk, already bootstrapped, see this [blog post](http://saveosx.org/pkgsrc-bootstrap/) on how to do this.
   14: 
   15: The tree contains a `Makefile`, a `README`, distfiles, packages, category directories containing the ports, the bootstrap directory and some documentation.
   16: 
   17: The `mk/*` directory contains the pkgsrc framework Makefiles but also shell and Awk scripts
   18: 
   19: `pkglocate` is a script to find port names in the tree, though `pkgtools/pkgfind` is much faster.
   20: 
   21: 
   22: ## use the right tools
   23: 
   24: If you want to get started working on ports like creating new ones or simply fix ones you need, you should know about these tools:
   25: 
   26:  - install package developer utilities: 
   27: 	
   28: 		pkgin -y in pkg_developer
   29: 
   30: It contains very useful programs like:
   31: 
   32:  - checkperms: 
   33:  		
   34: 		verify file permissions
   35:  - createbuildlink:
   36:  
   37: 		create buildlink3.mk files, which I'll explain later
   38:  - digest:
   39:  
   40: 		create hashes for messages with crypto algorithms such as sha512 and many others
   41:  - lintpkgsrc:
   42: 
   43: 		checks the whole pkgsrc tree, list all explicitly broken packages for example
   44:  - pkg_chk:
   45: 
   46: 		checks package versions and update if necessary
   47:  - pkg_tarup:
   48: 
   49: 		create archives of installed programs for later use on other machines or backups
   50:  - pkgdiff:
   51:  
   52: 		show diffs of patched files 
   53:  - pkglint:
   54: 
   55: 		verify the port you're creating for common mistakes (very useful!)
   56:  - revbump:
   57:  	
   58: 		update package version by one bump by increasing PKGREVISION
   59:  - url2pkg:
   60: 
   61: 		create a blank port from the software download link, it saves you some time by filling out a few basic Makefile settings
   62:  - verifypc:
   63: 
   64: 		sanity check for pkg-config in ports
   65: 
   66: 
   67: ## port contents
   68: 
   69: A pkgsrc port should at least contain:
   70: 
   71: - `Makefile` : a comment, developer info, software download site and lots of other possibilities
   72: - `DESCR` : a paragraph containing the description for the software of the port we're making
   73: - `PLIST` : the list of files to install, pkgsrc will only install the files listed here to your prefix
   74: - `distinfo` : hashes of the software archive and patches or files in the port
   75: 
   76: 
   77: Here's how they would look like for a small port I submitted not long ago in pkgsrc-wip
   78: 
   79: Makefile: 
   80: 	
   81: {% highlight make %}
   82: # $NetBSD: intro_to_packaging.mdwn,v 1.1 2014/07/08 14:19:33 schmonz Exp $
   83: 
   84: PKGNAME=      osxinfo-0.1
   85: CATEGORIES=   misc
   86: GHCOMMIT=     de74b8960f27844f7b264697d124411f81a1eab6
   87: DISTNAME=     ${GHCOMMIT}
   88: MASTER_SITES= https://github.com/yrmt/osxinfo/archive/
   89: 
   90: MAINTAINER=   youri.mout@gmail.com
   91: HOMEPAGE=     http://github.com/yrmt/osxinfo
   92: COMMENT=      Small Mac OS X Info Program
   93: LICENSE=      isc
   94: 
   95: ONLY_FOR_PLATFORM= Darwin-*-*
   96: 
   97: DIST_SUBDIR= osxinfo
   98: WRKSRC= ${WRKDIR}/osxinfo-${GHCOMMIT}
   99: 
  100: .include "../../databases/sqlite3/buildlink3.mk"
  101: .include "../../mk/bsd.pkg.mk"
  102: {% endhighlight %}
  103: 
  104: DESCR: 
  105: 	
  106: 	Small and fast Mac OS X info program written in C
  107: 	by Youri Mouton.
  108: 
  109: 
  110: PLIST:
  111: 	
  112: 	@comment $NetBSD: intro_to_packaging.mdwn,v 1.1 2014/07/08 14:19:33 schmonz Exp $
  113: 	bin/osxinfo
  114: 
  115: distinfo:
  116: 
  117: 	$NetBSD: intro_to_packaging.mdwn,v 1.1 2014/07/08 14:19:33 schmonz Exp $
  118: 
  119: 	SHA1 (osxinfo/de74b8960f27844f7b264697d124411f81a1eab6.tar.gz) = 83a2838ad95ff73255bea7f496a8cc9aaa4e17ca
  120: 	RMD160 (osxinfo/de74b8960f27844f7b264697d124411f81a1eab6.tar.gz) = 9102eb2a938be38c4adf8cfbf781c04d0844d09a
  121: 	Size (osxinfo/de74b8960f27844f7b264697d124411f81a1eab6.tar.gz) = 5981 bytes
  122: 
  123: 
  124: ## make
  125: 
  126: Now you know what kind of files you can see when you're in a port directory. The command used to compile it is the NetBSD `make` but often `bmake` on non NetBSD systems to avoid Makefile errors. Typing make alone will only compile the program but you can also use other command line arguments to make such as extract, patch, configure, install, package, ...
  127: 
  128: I'll try to list them and explain them in logical order. You can run them together.
  129: 
  130: - `make clean` will remove the source file from the work directory so you can restart with either new options, new patches, ...
  131: - `make fetch` will simply fetch the file and check if the hash corresponds. It will throw an error if it doesn't.
  132: - `make distinfo` or `make mdi` to update the file hashes in the `distinfo` file mentionned above.
  133: - `make extract` extracts the program source files from it's archive in the work directory
  134: - `make patch` applies the local pkgsrc patches to the source
  135: - `make configure` run the GNU configure script
  136: - `make` or `make build` or `make all` will stop after the program is compiled
  137: - `make stage-install` will install in the port destdir, where pkgsrc first installs program files to check if the files correspond with the `PLIST` contents before installing to your prefix. For `wget`, if you have a default WRKOBJDIR (I'll explain later), the program files will first be installed in `<path>/pkgsrc/net/wget/work/.destdir` then after a few checks, in your actual prefix like `/usr/pkg`
  138: - `make test` run package tests, if they have any
  139: - `make package` create a package without installing it, it will install dependencies though
  140: - `make replace` upgrade or reinstall the port if already installed
  141: - `make deinstall` deinstall the program
  142: - `make install` installs from the aforementionned `work/.destdir` to your prefix
  143: - `make bin-install` installs a package for the port, locally if previously built or remotely, as defined by BINPKG_SITES in `mk.conf`, you can make a port install dependencies from packages rather than building them with the DEPENDS_TARGET= bin-install in `mk.conf`
  144: - `make show-depends` show port dependencies
  145: - `make show-options` show various port options, as defined by `options.mk`
  146: - `make clean-depends` cleans all port dependencies
  147: - `make distclean` remove the source archive
  148: - `make package-clean` remove the package
  149: - `make distinfo` or `make mdi` to update the `distinfo` file containing file hashes if you have a new distfile or patch
  150: - `make print-PLIST` to generate a `PLIST` file from files found in `work/.destdir`
  151: 
  152: You should be aware that there are many make options along with these targets, like 
  153: 
  154: - `PKG_DEBUG_LEVEL`
  155: - `CHECK_FILES`
  156: - and many others described the the NetBSD pkgsrc guide
  157: 
  158: 
  159: ## pkgsrc configuration
  160: 
  161: The framework uses an `mk.conf` file, usually found in /etc. Here's how mine looks:
  162: 
  163: {% highlight make %}
  164: # Tue Oct 15 21:21:46 CEST 2013
  165: 
  166: .ifdef BSD_PKG_MK          # begin pkgsrc settings
  167: 
  168: DISTDIR=                   /pkgsrc/distfiles
  169: PACKAGES=                  /pkgsrc/packages
  170: WRKOBJDIR=                 /pkgsc/work
  171: ABI=                       64
  172: PKGSRC_COMPILER=           clang
  173: CC=                        clang
  174: CXX=                       clang++
  175: CPP=                       ${CC} -E
  176: 
  177: PKG_DBDIR=                 /var/db/pkg
  178: LOCALBASE=                 /usr/pkg
  179: VARBASE=                   /var
  180: PKG_TOOLS_BIN=             /usr/pkg/sbin
  181: PKGINFODIR=                info
  182: PKGMANDIR=                 man
  183: BINPKG_SITES=              http://pkgsrc.saveosx.org/Darwin/2013Q4/x86_64
  184: DEPENDS_TARGET=            bin-install
  185: X11_TYPE=                  modular
  186: TOOLS_PLATFORM.awk?=       /usr/pkg/bin/nawk
  187: TOOLS_PLATFORM.sed?=       /usr/pkg/bin/nbsed
  188: ALLOW_VULNERABLE_PACKAGES= yes
  189: MAKE_JOBS=                 8
  190: SKIP_LICENSE_CHECK=        yes
  191: PKG_DEVELOPER=             yes
  192: SIGN_PACKAGES=             gpg
  193: PKG_DEFAULT_OPTIONS+=      -pulseaudio -x264 -imlib2-amd64 -dconf
  194: .endif                     # end pkgsrc settings
  195: {% endhighlight %}
  196: 
  197: - I use `DISTDIR`, `PACKAGES`, `WRKOBJDIR` to move distfiles, packages and source files somewhere else to keep my pkgsrc tree clean
  198: - `PKGSRC_COMPILER`, `CC`, `CXX`, `CPP` and `ABI` are my compiler options. I'm using clang to create 64 bit binaries here
  199: - `PKG_DBDIR`, `VARBASE`, `LOCALBASE`, `PKG_TOOLS_BIN` are my prefix and package database path and package tools settings
  200: - `PKGINFODIR`, `PKGMANDIR` are the info and man directories 
  201: - `BINPKG_SITES` is the remote place where to get packages with the `bin-install` make target
  202: - `DEPENDS_TARGET` is the way port dependencies should be installed. `bin-install` will simply install a package instead of building the port
  203: - `X11_TYPE` sould be `native` or `modular`, the latter meaning we want X11 libraries from pkgsrc instead of using the `native` ones usually in `/usr/X11R7` in Linux or BSD systems and `/opt/X11` on Mac OS X with XQuartz
  204: - `TOOLS_PLATFORM.*` points to specific programs used by pkgsrc, here I use the one that was generated by pkgsrc bootstrap for maximum compatibility
  205: - `ALLOW_VULNERABLE_PACKAGES` allows you to disallow the installation of vulnerable packages in critical environments like servers
  206: - `MAKE_JOBS` the number of concurrent make jobs, I set it to 8 but it breaks some ports
  207: - `SKIP_LICENSE_CHECK` will skip the license check. If disabled you will have to define a list of licenses you find acceptable with `ACCEPTABLE_LICENSES`
  208: - `PKG_DEVELOPER` this option will show more details during the port building 
  209: - `SIGN_PACKAGES` allows you to `gpg` sign packages. More info in my [blog post](http://saveosx.org/signed-packages/) about it
  210: - `PKG_DEFAULT_OPTIONS` allows you to enable or disable specific options for all ports (as defined with ports' options.mk files), I disabled a few options so less ports would break, pulseaudio doesn't build on Mac OS X for example, neither do x264, dconf
  211: 
  212: Keep in mind that there are many other available options documented in the official pkgsrc guide.
  213: 
  214: 
  215: ## creating a simple port
  216: 
  217: Let's create a little port using the tools we've talked about above. I will use a little window manager called 2bwm.
  218: 
  219: - We need an url for the program source files archive. It can be a direct link to a tar or xz archive. Mine's `http://pkgsrc.saveosx.org/Darwin/distfiles/2bwm-0.1.tar.gz`
  220: 
  221: - Now that we have a proper link for our program source, create a directory for your port:
  222: 	
  223: 		$ mkdir ~/pkgsrc/wm/2bwm
  224: 
  225: - Use `url2pkg` to create the needed files automatically:
  226: 		
  227: 		$ url2pkg http://pkgsrc.saveosx.org/Darwin/distfiles/2bwm-0.1.tar.gz
  228: 
  229: You'll be presented with a text editor like `vim` to enter basic Makefile options:
  230: 
  231: - `DISTNAME`, `CATEGORIES`, `MASTER_SITES` should be set automatically 
  232: - enter your mail address for `MAINTAINER` so users know whom to contact if the port is broken
  233: - make sure the `HOMEPAGE` is set right, for 2bwm it is a github page
  234: - write a `COMMENT`, it should be a one-line description of the program
  235: - find out which license the program uses, in my case it is the `isc` license. You can find a list of licenses in `pkgsrc/mk/licenses.mk`.
  236: - Below you will see `.include "../../mk/bsd.pkg.mk"` at the end of the Makefile and above this should go the port's needed dependencies to build, we'll leave that empty at the moment and try to figure out what 2bwm needs
  237: - exit vim and it should fetch and update the file hashes for you. If it says `permission denied` you can just run `make mdi` to fetch and upadate the `distinfo` file
  238: 
  239: So now you have valid `Makefile` and `distinfo` files but you need to write a paragraph in `DESCR`. You can usually find inspiration on the program's homepage.
  240: 
  241: Here's how they look like at the moment: 
  242: 	
  243:  Makefile:
  244: {% highlight make %} 
  245: # $NetBSD: intro_to_packaging.mdwn,v 1.1 2014/07/08 14:19:33 schmonz Exp $
  246: 
  247: DISTNAME=       2bwm-0.1
  248: CATEGORIES=     wm
  249: MASTER_SITES=   http://pkgsrc.saveosx.org/Darwin/distfiles/
  250: 
  251: MAINTAINER=     yrmt@users.sourceforge.net
  252: HOMEPAGE=       http://github.com/venam/2bwm/
  253: COMMENT=        Fast floating WM written over the XCB library and derived from mcwm
  254: LICENSE=        isc
  255: 
  256: .include "../../mk/bsd.pkg.mk"
  257: {% endhighlight %}
  258: 
  259: distinfo:
  260: 
  261: 	
  262: 	$NetBSD: intro_to_packaging.mdwn,v 1.1 2014/07/08 14:19:33 schmonz Exp $
  263: 
  264: 	SHA1 (2bwm-0.1.tar.gz) = e83c862dc1d9aa198aae472eeca274e5d98df0ad
  265: 	RMD160 (2bwm-0.1.tar.gz) = d9a93a7d7ae7183f5921f9ad76abeb1401184ef9
  266: 	Size (2bwm-0.1.tar.gz) = 38419 bytes
  267: 
  268: DESCR:
  269: 
  270: 	A fast floating WM, with the particularity of having 2 borders,
  271: 	written over the XCB library and derived from mcwm written by
  272: 	Michael Cardell. In 2bWM everything is accessible from the keyboard
  273: 	but a pointing device can be used for move, resize and raise/lower.
  274: 
  275: But our PLIST file is still empty.
  276: 
  277: 
  278: #### build stage
  279: 
  280:  Let's try to build the port to see if things work but as soon as the build stage starts, we get this error:
  281: 
  282: > 2bwm.c:26:10: fatal error: 'xcb/randr.h' file not found
  283: 
  284: Let's find out which port provides this file ! 
  285: 
  286: 	$ pkgin se xcb 
  287: 
  288: returns these possible packages: 
  289: 
  290: 	xcb-util-wm-0.3.9nb1  Client and window-manager helpers for ICCCM and EWMH
  291: 	xcb-util-renderutil-0.3.8nb1  Convenience functions for the Render extension
  292: 	xcb-util-keysyms-0.3.9nb1  XCB Utilities
  293: 	xcb-util-image-0.3.9nb1  XCB port of Xlib's XImage and XShmImage
  294: 	xcb-util-0.3.9nb1 =  XCB Utilities
  295: 	xcb-proto-1.9 =      XCB protocol descriptions (in XML)
  296: 	xcb-2.4nb1           Extensible, multiple cut buffers for X
  297: 
  298: Package content inspection allowed me to find the right port 
  299: 
  300: 	$ pkgin pc libxcb|grep randr.h
  301: 
  302: So we can add the libxcb `buildlink3.mk` file to the Makefile above the bsd.pkg.mk include: 
  303: 
  304: 	.include "../../x11/libxcb/buildlink3.mk"
  305: 
  306: This allows the port to link 2bwm against the libxcb port. Let's try to build the port again!
  307: 
  308: 	$ make clean
  309: 	$ make
  310: 
  311: Reports another error !
  312: 
  313: > 2bwm.c:27:10: fatal error: 'xcb/xcb_keysyms.h' file not found
  314: 
  315: It looks like this file is provided by xcb-util-keysyms, so let's add:
  316: 
  317: 	.include "../../x11/xcb-util-keysyms/buildlink3.mk"
  318: 
  319: in our Makefile.
  320: 
  321: Clean, build again, and add more dependencies until it passes the build stage. Here's how my Makefile ends up looking like:
  322: 
  323: {% highlight make %}
  324: # $NetBSD: intro_to_packaging.mdwn,v 1.1 2014/07/08 14:19:33 schmonz Exp $
  325: 
  326: DISTNAME=       2bwm-0.1
  327: CATEGORIES=     wm
  328: MASTER_SITES=   http://pkgsrc.saveosx.org/Darwin/distfiles/
  329: 
  330: MAINTAINER=     yrmt@users.sourceforge.net
  331: HOMEPAGE=       http://github.com/venam/2bwm/
  332: COMMENT=        Fast floating WM written over the XCB library and derived from mcwm
  333: LICENSE=        isc
  334: 
  335: .include "../../x11/libxcb/buildlink3.mk"
  336: .include "../../x11/xcb-util-wm/buildlink3.mk"
  337: .include "../../x11/xcb-util-keysyms/buildlink3.mk"
  338: .include "../../x11/xcb-util/buildlink3.mk"
  339: .include "../../mk/bsd.pkg.mk"
  340: {% endhighlight %}
  341: 
  342: 
  343: #### install phase
  344: 
  345: Geat ! We got our program to compile in pkgsrc. Now we must generate the PLIST file so we can actually install the program, but we must `make stage-install` to make sure that it installs in the right place.
  346: 
  347: 	
  348: 	$ find /pkgsrc/work/wm/2bwm/work/.destdir/
  349: 
  350: returns:
  351: 
  352: 	/pkgsrc/work/wm/2bwm/work/.destdir/
  353: 	/pkgsrc/work/wm/2bwm/work/.destdir//usr
  354: 	/pkgsrc/work/wm/2bwm/work/.destdir//usr/local
  355: 	/pkgsrc/work/wm/2bwm/work/.destdir//usr/local/bin
  356: 	/pkgsrc/work/wm/2bwm/work/.destdir//usr/local/bin/2bwm
  357: 	/pkgsrc/work/wm/2bwm/work/.destdir//usr/local/bin/hidden
  358: 	/pkgsrc/work/wm/2bwm/work/.destdir//usr/local/share
  359: 	/pkgsrc/work/wm/2bwm/work/.destdir//usr/local/share/man
  360: 	/pkgsrc/work/wm/2bwm/work/.destdir//usr/local/share/man/man1
  361: 	/pkgsrc/work/wm/2bwm/work/.destdir//usr/local/share/man/man1/2bwm.1
  362: 	/pkgsrc/work/wm/2bwm/work/.destdir//usr/local/share/man/man1/hidden.1
  363: 	/pkgsrc/work/wm/2bwm/work/.destdir//usr/pkg
  364: 
  365: This doesn't look right since our `LOCALBASE` is `/usr/pkg`.
  366: 
  367: 
  368: 	$ make print-PLIST
  369: 
  370: returns nothing, because 2bwm installs files in the wrong place so we need to fix 2bwm's own Makefile to use the right `DESTDIR` and `PREFIX`, that is set to the right place by pkgsrc. Let's inspect how 2bwm installs:
  371: 
  372: From 2bwm's Makefile: 
  373: 
  374: {% highlight make %}
  375: install: $(TARGETS)
  376:         test -d $(DESTDIR)$(PREFIX)/bin || mkdir -p $(DESTDIR)$(PREFIX)/bin
  377:         install -pm 755 2bwm $(DESTDIR)$(PREFIX)/bin
  378:         install -pm 755 hidden $(DESTDIR)$(PREFIX)/bin
  379:         test -d $(DESTDIR)$(MANPREFIX)/man1 || mkdir -p $(DESTDIR)$(MANPREFIX)/man1
  380:         install -pm 644 2bwm.man $(DESTDIR)$(MANPREFIX)/man1/2bwm.1
  381:         install -pm 644 hidden.man $(DESTDIR)$(MANPREFIX)/man1/hidden.1
  382: {% endhighlight %}
  383: 
  384: This looks fine since it installs in a `DESTDIR`/`PREFIX` but it sets 
  385: 
  386: > PREFIX=/usr/local
  387: 
  388: and
  389: 
  390: > MANPREFIX=$(PREFIX)/share/man
  391: 
  392: In the beginning of the Makefile. We should remove the first line and edit the man prefix:
  393: 
  394: > MANPREFIX=${PKGMANDIR}
  395: 
  396: so pkgsrc can install the program's files in the right place. We have two ways of modifying this file, either patch the Makefile or use `sed` substitution which is a builtin pkgsrc feature that allows you to change lines in files with a sed command before building the port. 
  397: 
  398: I will show how to do both ways so you can get an introduction on how to generate patch files for pkgsrc.
  399: 
  400: #### patching the Makefile :
  401: 
  402: - edit the file you need to modify with `pkgvi`:
  403: 
  404: 	
  405: 		$ pkgvi /pkgsrc/work/wm/2bwm/work/2bwm-0.1/Makefile
  406: 
  407: 	which should return:
  408: 
  409: 	> pkgvi: File was modified. For a diff, type:
  410: pkgdiff "/Volumes/Backup/pkgsrc/work/wm/2bwm/work/2bwm-0.1/Makefile"
  411: 
  412: 	and this returns our diff.
  413: 	
  414: 
  415: - create the patch with `mkpatches`, it should create a `patches` directory in the port containing the patch and an original file removed with `mkpatches -c`. 
  416: 
  417: 		$ find patches/*
  418: 		patches/patch-Makefile
  419: 
  420: - now that the patch has been created, we need to add it's hash to distinfo otherwise pkgsrc won't pick it up:
  421: 
  422: 		$ make mdi
  423: you should get this new line:
  424: 
  425: 	> SHA1 (patch-Makefile) = 9f8cd00a37edbd3e4f65915aa666ebd0f3c04e04
  426: 
  427: 
  428: - you can now clean and `make patch` and `make stage-install CHECK_FILES=no` since we still haven't generated a proper PLIST. Let's see if 2wm files were installed in the right place this time:
  429: 
  430: 		$ find /pkgsrc/work/wm/2bwm/work/.destdir/
  431: 
  432: 		/pkgsrc/work/wm/2bwm/work/.destdir/
  433: 		/pkgsrc/work/wm/2bwm/work/.destdir//usr
  434: 		/pkgsrc/work/wm/2bwm/work/.destdir//usr/pkg
  435: 		/pkgsrc/work/wm/2bwm/work/.destdir//usr/pkg/bin
  436: 		/pkgsrc/work/wm/2bwm/work/.destdir//usr/pkg/bin/2bwm
  437: 		/pkgsrc/work/wm/2bwm/work/.destdir//usr/pkg/bin/hidden
  438: 
  439: 	It looks like it is alright ! Let's generate the PLIST:
  440: 
  441: 		$ make print-PLIST > PLIST
  442: 	
  443: 	containing:
  444: 
  445: 		@comment $NetBSD: intro_to_packaging.mdwn,v 1.1 2014/07/08 14:19:33 schmonz Exp $
  446: 		bin/2bwm
  447: 		bin/hidden
  448: 
  449: 	There you have a working port you can install normally with 
  450: 
  451: 		$ make install 
  452: 
  453: 
  454: #### using the sed substitution framework
  455: 
  456: You should be able to fix the prefix error much quicker than with the patching explained above thanks to the sed substitution framework. Here's how it looks like in my port Makefile:
  457: 
  458: {% highlight make %}
  459: SUBST_CLASSES+=         makefile
  460: SUBST_STAGE.makefile=   pre-build
  461: SUBST_MESSAGE.makefile= Fixing makefile
  462: SUBST_FILES.makefile=   Makefile
  463: SUBST_SED.makefile=     -e 's,/usr/local,${PREFIX},g'
  464: SUBST_SED.makefile+=    -e 's,share/man,${PKGMANDIR},g'
  465: {% endhighlight %}
  466: 
  467: As you can see, you can do multiple commands on multiple files, it is very useful for very small fixes like this.
  468: 
  469: 
  470: #### pkglint
  471: 
  472: Now that we have a working port, we must make sure it complies to the pkgsrc rules. 
  473: 
  474: 	$ pkglint
  475: 	
  476: Returns 
  477: 
  478: 	ERROR: DESCR:4: File must end with a newline.
  479: 	ERROR: patches/patch-Makefile:3: Comment expected.
  480: 	2 errors and 0 warnings found. (Use -e for more details.)
  481: 
  482: Fix the things pkglint tells you to do until you get the glorious:
  483: 
  484: > looks fine.
  485: 
  486: Then you should do some testing on the program itelf on at least two platforms such as NetBSD, Mac OS X. Other platforms supported by pkgsrc can be found at [pkgsrc.org](http://pkgsrc.org). If you would like to submit your pkgsrc upstream you can either subscribe to pkgsrc-wip or ask a NetBSD developer to add it for you.
  487: 
  488: You can find the 2bwm port I submitted in [pkgsrc-wip](http://pkgsrc-wip.cvs.sourceforge.net/viewvc/pkgsrc-wip/wip/2bwm/).
  489: 
  490: 
  491: ## pkgsrc and wip 
  492: 
  493: If you want to submit your port for others to use you can either subscribe to pkgsrc-wip or ask a NetBSD developer to add it for you which can be tough. Even though there are many IRC channels in which you can find nice developers, you will have to take the time to get to know them. The easiest way for beginners is to submit to pkgsrc-wip so other people can review and test it first. 
  494: 
  495: pkgsrc-wip is hosted on [sourceforge](https://sourceforge.net/projects/pkgsrc-wip/) and you can easily get cvs access to it if you create an account on there and send an email to NetBSD developer `@wiz` (Thomas Klausner) asking nicely for commit access. I got access fairly quickly and he even fixed a port to show me how to do it properly. 
  496: 
  497: You can also send me an email or talk to me on IRC so I can submit it for you.
  498: 
  499: 
  500: ## the options framework
  501: 
  502: You can create port options with the `options.mk` file, like for `wm/dwm`
  503: 
  504: 	
  505: {% highlight make %}
  506: # $NetBSD: intro_to_packaging.mdwn,v 1.1 2014/07/08 14:19:33 schmonz Exp $
  507: 
  508: PKG_OPTIONS_VAR=			PKG_OPTIONS.dwm
  509: PKG_SUPPORTED_OPTIONS=	xinerama
  510: PKG_SUGGESTED_OPTIONS=	xinerama
  511: 
  512: .include "../../mk/bsd.options.mk"
  513: 
  514: #
  515: # Xinerama support
  516: #
  517: # If we don't want the Xinerama support we delete XINERAMALIBS and
  518: # XINERAMAFLAGS lines, otherwise the Xinerama support is the default.
  519: #
  520: .if !empty(PKG_OPTIONS:Mxinerama)
  521: .  include "../../x11/libXinerama/buildlink3.mk"
  522: .else
  523: SUBST_CLASSES+=         options
  524: SUBST_STAGE.options=    pre-build
  525: SUBST_MESSAGE.options=  Toggle the Xinerama support
  526: SUBST_FILES.options=    config.mk
  527: SUBST_SED.options+=     -e '/^XINERAMA/d'
  528: .  include "../../x11/libX11/buildlink3.mk"
  529: .endif
  530: {% endhighlight %}
  531: 
  532: This file should be included in the Makefile:
  533: 
  534: 	.include "options.mk"
  535: 
  536: If you type `make show-options`, you should see this:
  537: 
  538: 	Any of the following general options may be selected:
  539: 	xinerama	 Enable Xinerama support.
  540: 
  541: 	These options are enabled by default:
  542: 		xinerama
  543: 
  544: 	These options are currently enabled:
  545: 		xinerama
  546: 
  547: 	You can select which build options to use by setting 	PKG_DEFAULT_OPTIONS
  548: 	or PKG_OPTIONS.dwm.
  549: 
  550: Running `make PKG_OPTIONS=""` should build without the `xinerama` dwm option enabled by default.
  551: 
  552: The options.mk file must contain these variables:
  553: 
  554: - `PKG_OPTIONS_VAR` sets the options variable name
  555: - `PKG_SUPPORTED_OPTIONS` lists all available options
  556: - `PKG_SUGGESTED_OPTIONS` lists options enabled by default
  557: 
  558: It allows you to change configure arguments and include other buildlinks, and various other settings.
  559: 
  560: 
  561: ## hosting a package repo
  562: 
  563: Now that you've created a few ports, you might want to make
  564: precompiled packages available for testing. You will need pkgsrc's `pkg_install` on the host system. I host my [packages](http://pkgsrc.saveosx.org/) on a FreeBSD server with a bootstrapped pkgsrc.
  565: 
  566: I use this `zsh` function to :
  567: 
  568: {% highlight bash %} 
  569: add () {
  570: 	# upload the package to remote server
  571: 	scp $1 yrmt@saveosx.org:/usr/local/www/saveosx/packages/Darwin/2013Q4/x86_64/All/ 2> /dev/null
  572: 	
  573: 	# update the package summary
  574: 	ssh yrmt@saveosx.org 'cd /usr/local/www/saveosx/packages/Darwin/2013Q4/x86_64/All/;
  575: 	        rm pkg_summary.gz;
  576:  	        /usr/pkg/sbin/pkg_info -X *.tgz | gzip -9 > pkg_summary.gz'
  577: 	
  578: 	# pkgin update
  579: 	sudo pkgin update
  580: }
  581: {% endhighlight %}
  582: 
  583: - upload a package 
  584: - update the package summary, which is an archive containing information about all present packages that will be picked up by pkg_install and pkgin. It looks like this for one package:
  585: 
  586: 		PKGNAME=osxinfo-0.1
  587: 		DEPENDS=sqlite3>=3.7.16.2nb1
  588: 		COMMENT=Small Mac OS X Info Program
  589: 		SIZE_PKG=23952
  590: 		BUILD_DATE=2014-06-29 12:45:08 +0200
  591: 		CATEGORIES=misc
  592: 		HOMEPAGE=http://github.com/yrmt/osxinfo
  593: 		LICENSE=isc
  594: 		MACHINE_ARCH=x86_64
  595: 		OPSYS=Darwin
  596: 		OS_VERSION=14.0.0
  597: 		PKGPATH=wip/osxinfo
  598: 		PKGTOOLS_VERSION=20091115
  599: 		REQUIRES=/System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation
  600: 		REQUIRES=/System/Library/Frameworks/Foundation.framework/Versions/C/Foundation
  601: 		REQUIRES=/System/Library/Frameworks/IOKit.framework/Versions/A/IOKit
  602: 		REQUIRES=/usr/lib/libSystem.B.dylib
  603: 		REQUIRES=/usr/pkg/lib/libsqlite3.0.dylib
  604: 		FILE_NAME=osxinfo-0.1.tgz
  605: 		FILE_SIZE=9710
  606: 		DESCRIPTION=Small and fast Mac OS X info program written in C 
  607: 		DESCRIPTION=by Youri Mouton.
  608: 		DESCRIPTION=
  609: 		DESCRIPTION=Homepage:
  610: 		DESCRIPTION=http://github.com/yrmt/osxinfo
  611: 
  612: 
  613: - update pkgin 
  614: 
  615: 
  616: And this shell alias to upload all my built packages, but I still need to run `add()` mentionned above to update the pkg_summary
  617: 
  618: {% highlight bash %} 
  619: up='rsync -avhz --progress /pkgsrc/packages/ root@saveosx.org:/usr/local/www/saveosx/packages/Darwin/2013Q4/x86_64/'
  620: {% endhighlight %} 
  621: 
  622: Then you should be able to set the url in repositories.conf to use your packages with pkgin. You can also install them directly with something like `pkg_add http://pkgsrc.saveosx.org/Darwin/2013Q4/x86_64/All/9menu-1.8nb1.tgz` of course. 
  623: 
  624: 
  625: ## build all packages
  626: 
  627: Bulk building pkgsrc packages is a topic for another post, see jperkin's excellent blog [posts](http://www.perkin.org.uk/posts/distributed-chrooted-pkgsrc-bulk-builds.html) about this.
  628: 
  629: 
  630: ## faq
  631: 
  632: #### what if the port I'm making is a dependency for another one?
  633: 
  634: You should just generate the buildlink3.mk file we've talked about earlier like this:
  635: 
  636: 	$ createbuildlink > buildlink3.mk
  637: 
  638: #### what if the program is only hosted on GitHub ?
  639: 
  640: pkgsrc supports fetching archives from specific git commits on GitHub like this:
  641: {% highlight make %}
  642: PKGNAME=           2bwm-0.1
  643: CATEGORIES=        wm
  644: GHCOMMIT=          52a097ca644eb571b22a135951c945fcca57a25c
  645: DISTNAME=          ${GHCOMMIT}
  646: MASTER_SITES=      https://github.com/venam/2bwm/archive/
  647: DIST_SUBDIR=       2bwm
  648: WRKSRC=            ${WRKDIR}/2bwm-${GHCOMMIT}
  649: {% endhighlight %}
  650: 
  651: You can then easily update the git commit and the distinfo with it to update the program. 
  652: 
  653: #### what if the program doesn't have a Makefile
  654: 
  655: You can do all Makefile operations directly from the port's Makefile like this: 
  656: 
  657: 
  658: {% highlight make %}
  659: post-extract:
  660: 	${CHMOD} a-x ${WRKSRC}/elementary/apps/48/internet-mail.svg
  661: 
  662: do-install:
  663: 	${INSTALL_DATA_DIR} ${DESTDIR}${PREFIX}/share/icons
  664: 	cd ${WRKSRC} && pax -rw -pe . ${DESTDIR}${PREFIX}/share/icons/
  665: {% endhighlight %}
  666: 
  667: To install, but you can also build programs from the Makefile. This is what qt4-sqlite3 uses:
  668: 
  669: {% highlight make %}
  670: do-build:
  671: 	cd ${WRKSRC}/src/tools/bootstrap && env ${MAKE_ENV} ${GMAKE}
  672: 	cd ${WRKSRC}/src/tools/moc && env ${MAKE_ENV} ${GMAKE}
  673: 	cd ${WRKSRC}/src/plugins/sqldrivers/sqlite && env ${MAKE_ENV} ${GMAKE}
  674: {% endhighlight %}
  675: 
  676: 
  677: You can install the following type of files: 
  678: 
  679: `INSTALL_PROGRAM_DIR` : directories that contain binaries
  680: 
  681: `INSTALL_SCRIPT_DIR` : directories that contain scripts
  682: 
  683: `INSTALL_LIB_DIR` : directories that contain shared and static libraries
  684: 
  685: `INSTALL_DATA_DIR`: directories that contain data files
  686: 
  687: `INSTALL_MAN_DIR` : directories that contain man pages
  688: 
  689: `INSTALL_PROGRAM` : binaries that can be stripped from debugging symbols
  690: 
  691: `INSTALL_SCRIPT` : binaries that cannot be stripped
  692: 
  693: `INSTALL_GAME` : game binaries
  694: 
  695: `INSTALL_LIB` : shared and static libraries
  696: 
  697: `INSTALL_DATA` : data files
  698: 
  699: `INSTALL_GAME_DATA` : data files for games
  700: 
  701: `INSTALL_MAN` : man pages
  702: 
  703: 
  704: `INSTALLATION_DIRS` : A list of directories relative to PREFIX that are created by pkgsrc at the beginning of the install phase. The package is supposed to create all needed directories itself before installing files to it and list all other directories here.
  705: 
  706: #### common errors
  707: 
  708: - > Makefile:19: *** missing separator.  Stop.
  709: 
  710: This means you're not using the right `make`. On most systems, the make installed from the pkgsrc bootstrap is called `bmake`
  711: 
  712: - If you have a feeling a port is stuck in the building stage, disable make jobs in your mk.conf
  713: 
  714: - Please contribute here :)
  715: 
  716: 
  717: ## links
  718: - [Jonathan Perkin's excellent blog](http://www.perkin.org.uk/)
  719: - [NetBSD's very extensive pkgsrc guide](http://www.netbsd.org/docs/pkgsrc/)
  720: - [NetBSD's pkgsrc wiki](http://wiki.netbsd.org/pkgsrc/)
  721: - Other blog posts here :)
  722: 
  723: ## where to find me
  724: 
  725: - yrmt@edgebsd.org
  726: - irc.oftc.net
  727: 	
  728: 	`#saveosx`
  729: 
  730: [[!meta title="title: An introduction to pkgsrc"]]
  731: [[!meta author="Youri Mouton"]]

CVSweb for NetBSD wikisrc <wikimaster@NetBSD.org> software: FreeBSD-CVSweb