File:  [NetBSD Developer Wiki] / wikisrc / pkgsrc / intro_to_packaging.mdwn
Revision 1.3: download - view: text, annotated - select for diffs
Tue Jul 8 14:44:38 2014 UTC (9 years, 4 months ago) by schmonz
Branches: MAIN
CVS tags: HEAD
Split RCS Id into two cut-paste strings to avoid expansion.

    1: This guide should allow you to learn how to create a new port or
    2: simply fix a port that you need. There are three target demographics
    3: listed below:
    4: 
    5: 	- binary packages user with pkgin or pkg_add
    6:  		(you should be confident here)
    7: 	- build from source, use options
    8:  		(you will know this after reading the guide)
    9: 	- port developers
   10: 		(you should be able to get started here)
   11: 
   12: 
   13: ## pkgsrc tree
   14: 
   15: You should have a copy of the pkgsrc tree sitting somewhere on your
   16: disk, already bootstrapped, see this [blog
   17: post](http://saveosx.org/pkgsrc-bootstrap/) on how to do this.
   18: 
   19: The tree contains a `Makefile`, a `README`, distfiles, packages,
   20: category directories containing the ports, the bootstrap directory
   21: and some documentation.
   22: 
   23: The `mk/*` directory contains the pkgsrc framework Makefiles but
   24: also shell and Awk scripts
   25: 
   26: `pkglocate` is a script to find port names in the tree, though
   27: `pkgtools/pkgfind` is much faster.
   28: 
   29: 
   30: ## use the right tools
   31: 
   32: If you want to get started working on ports like creating new ones
   33: or simply fix ones you need, you should know about these tools:
   34: 
   35:  - install package developer utilities:
   36: 	
   37: 		pkgin -y in pkg_developer
   38: 
   39: It contains very useful programs like:
   40: 
   41:  - checkperms:
   42:  		
   43: 		verify file permissions
   44:  - createbuildlink:
   45: 
   46: 		create buildlink3.mk files, which I'll explain later
   47:  - digest:
   48: 
   49: 		create hashes for messages with crypto algorithms such as sha512 and many others
   50:  - lintpkgsrc:
   51: 
   52: 		checks the whole pkgsrc tree, list all explicitly broken packages for example
   53:  - pkg_chk:
   54: 
   55: 		checks package versions and update if necessary
   56:  - pkg_tarup:
   57: 
   58: 		create archives of installed programs for later use on other machines or backups
   59:  - pkgdiff:
   60: 
   61: 		show diffs of patched files
   62:  - pkglint:
   63: 
   64: 		verify the port you're creating for common mistakes (very useful!)
   65:  - revbump:
   66:  	
   67: 		update package version by one bump by increasing PKGREVISION
   68:  - url2pkg:
   69: 
   70: 		create a blank port from the software download link, it saves you some time by filling out a few basic Makefile settings
   71:  - verifypc:
   72: 
   73: 		sanity check for pkg-config in ports
   74: 
   75: 
   76: ## port contents
   77: 
   78: A pkgsrc port should at least contain:
   79: 
   80: - `Makefile` : a comment, developer info, software download site
   81:   and lots of other possibilities
   82: - `DESCR` : a paragraph containing the description for the software
   83:   of the port we're making
   84: - `PLIST` : the list of files to install, pkgsrc will only install
   85:   the files listed here to your prefix
   86: - `distinfo` : hashes of the software archive and patches or files
   87:   in the port
   88: 
   89: 
   90: Here's how they would look like for a small port I submitted not
   91: long ago in pkgsrc-wip
   92: 
   93: Makefile:
   94: 	
   95: {% highlight make %}
   96: # [[!paste id=rcsid1]][[!paste id=rcsid2]]
   97: 
   98: PKGNAME=      osxinfo-0.1
   99: CATEGORIES=   misc
  100: GHCOMMIT=     de74b8960f27844f7b264697d124411f81a1eab6
  101: DISTNAME=     ${GHCOMMIT}
  102: MASTER_SITES= https://github.com/yrmt/osxinfo/archive/
  103: 
  104: MAINTAINER=   youri.mout@gmail.com
  105: HOMEPAGE=     http://github.com/yrmt/osxinfo
  106: COMMENT=      Small Mac OS X Info Program
  107: LICENSE=      isc
  108: 
  109: ONLY_FOR_PLATFORM= Darwin-*-*
  110: 
  111: DIST_SUBDIR= osxinfo
  112: WRKSRC= ${WRKDIR}/osxinfo-${GHCOMMIT}
  113: 
  114: .include "../../databases/sqlite3/buildlink3.mk"
  115: .include "../../mk/bsd.pkg.mk"
  116: {% endhighlight %}
  117: 
  118: DESCR:
  119: 	
  120: 	Small and fast Mac OS X info program written in C
  121: 	by Youri Mouton.
  122: 
  123: 
  124: PLIST:
  125: 	
  126: 	@comment [[!paste id=rcsid1]][[!paste id=rcsid2]]
  127: 	bin/osxinfo
  128: 
  129: distinfo:
  130: 
  131: 	[[!paste id=rcsid1]][[!paste id=rcsid2]]
  132: 
  133: 	SHA1 (osxinfo/de74b8960f27844f7b264697d124411f81a1eab6.tar.gz) = 83a2838ad95ff73255bea7f496a8cc9aaa4e17ca
  134: 	RMD160 (osxinfo/de74b8960f27844f7b264697d124411f81a1eab6.tar.gz) = 9102eb2a938be38c4adf8cfbf781c04d0844d09a
  135: 	Size (osxinfo/de74b8960f27844f7b264697d124411f81a1eab6.tar.gz) = 5981 bytes
  136: 
  137: 
  138: ## make
  139: 
  140: Now you know what kind of files you can see when you're in a port
  141: directory. The command used to compile it is the NetBSD `make` but
  142: often `bmake` on non NetBSD systems to avoid Makefile errors. Typing
  143: make alone will only compile the program but you can also use other
  144: command line arguments to make such as extract, patch, configure,
  145: install, package, ...
  146: 
  147: I'll try to list them and explain them in logical order. You can run them together.
  148: 
  149: - `make clean` will remove the source file from the work directory
  150:   so you can restart with either new options, new patches, ...
  151: - `make fetch` will simply fetch the file and check if the hash
  152:   corresponds. It will throw an error if it doesn't.
  153: - `make distinfo` or `make mdi` to update the file hashes in the
  154:   `distinfo` file mentionned above.
  155: - `make extract` extracts the program source files from it's archive
  156:   in the work directory
  157: - `make patch` applies the local pkgsrc patches to the source
  158: - `make configure` run the GNU configure script
  159: - `make` or `make build` or `make all` will stop after the program
  160:   is compiled
  161: - `make stage-install` will install in the port destdir, where
  162:   pkgsrc first installs program files to check if the files correspond
  163:   with the `PLIST` contents before installing to your prefix. For
  164:   `wget`, if you have a default WRKOBJDIR (I'll explain later), the
  165:   program files will first be installed in
  166:   `<path>/pkgsrc/net/wget/work/.destdir` then after a few checks,
  167:   in your actual prefix like `/usr/pkg`
  168: - `make test` run package tests, if they have any
  169: - `make package` create a package without installing it, it will
  170:   install dependencies though
  171: - `make replace` upgrade or reinstall the port if already installed
  172: - `make deinstall` deinstall the program
  173: - `make install` installs from the aforementionned `work/.destdir`
  174:   to your prefix
  175: - `make bin-install` installs a package for the port, locally if
  176:   previously built or remotely, as defined by BINPKG_SITES in
  177:   `mk.conf`, you can make a port install dependencies from packages
  178:   rather than building them with the DEPENDS_TARGET= bin-install
  179:   in `mk.conf`
  180: - `make show-depends` show port dependencies
  181: - `make show-options` show various port options, as defined by `options.mk`
  182: - `make clean-depends` cleans all port dependencies
  183: - `make distclean` remove the source archive
  184: - `make package-clean` remove the package
  185: - `make distinfo` or `make mdi` to update the `distinfo` file
  186:   containing file hashes if you have a new distfile or patch
  187: - `make print-PLIST` to generate a `PLIST` file from files found
  188:   in `work/.destdir`
  189: 
  190: You should be aware that there are many make options along with
  191: these targets, like
  192: 
  193: - `PKG_DEBUG_LEVEL`
  194: - `CHECK_FILES`
  195: - and many others described the the NetBSD pkgsrc guide
  196: 
  197: 
  198: ## pkgsrc configuration
  199: 
  200: The framework uses an `mk.conf` file, usually found in /etc. Here's
  201: how mine looks:
  202: 
  203: {% highlight make %}
  204: # Tue Oct 15 21:21:46 CEST 2013
  205: 
  206: .ifdef BSD_PKG_MK          # begin pkgsrc settings
  207: 
  208: DISTDIR=                   /pkgsrc/distfiles
  209: PACKAGES=                  /pkgsrc/packages
  210: WRKOBJDIR=                 /pkgsc/work
  211: ABI=                       64
  212: PKGSRC_COMPILER=           clang
  213: CC=                        clang
  214: CXX=                       clang++
  215: CPP=                       ${CC} -E
  216: 
  217: PKG_DBDIR=                 /var/db/pkg
  218: LOCALBASE=                 /usr/pkg
  219: VARBASE=                   /var
  220: PKG_TOOLS_BIN=             /usr/pkg/sbin
  221: PKGINFODIR=                info
  222: PKGMANDIR=                 man
  223: BINPKG_SITES=              http://pkgsrc.saveosx.org/Darwin/2013Q4/x86_64
  224: DEPENDS_TARGET=            bin-install
  225: X11_TYPE=                  modular
  226: TOOLS_PLATFORM.awk?=       /usr/pkg/bin/nawk
  227: TOOLS_PLATFORM.sed?=       /usr/pkg/bin/nbsed
  228: ALLOW_VULNERABLE_PACKAGES= yes
  229: MAKE_JOBS=                 8
  230: SKIP_LICENSE_CHECK=        yes
  231: PKG_DEVELOPER=             yes
  232: SIGN_PACKAGES=             gpg
  233: PKG_DEFAULT_OPTIONS+=      -pulseaudio -x264 -imlib2-amd64 -dconf
  234: .endif                     # end pkgsrc settings
  235: {% endhighlight %}
  236: 
  237: - I use `DISTDIR`, `PACKAGES`, `WRKOBJDIR` to move distfiles,
  238:   packages and source files somewhere else to keep my pkgsrc tree
  239:   clean
  240: - `PKGSRC_COMPILER`, `CC`, `CXX`, `CPP` and `ABI` are my compiler
  241:   options. I'm using clang to create 64 bit binaries here
  242: - `PKG_DBDIR`, `VARBASE`, `LOCALBASE`, `PKG_TOOLS_BIN` are my prefix
  243:   and package database path and package tools settings
  244: - `PKGINFODIR`, `PKGMANDIR` are the info and man directories
  245: - `BINPKG_SITES` is the remote place where to get packages with the
  246:   `bin-install` make target
  247: - `DEPENDS_TARGET` is the way port dependencies should be installed.
  248:   `bin-install` will simply install a package instead of building
  249:   the port
  250: - `X11_TYPE` sould be `native` or `modular`, the latter meaning we
  251:   want X11 libraries from pkgsrc instead of using the `native` ones
  252:   usually in `/usr/X11R7` in Linux or BSD systems and `/opt/X11`
  253:   on Mac OS X with XQuartz
  254: - `TOOLS_PLATFORM.*` points to specific programs used by pkgsrc,
  255:   here I use the one that was generated by pkgsrc bootstrap for
  256:   maximum compatibility
  257: - `ALLOW_VULNERABLE_PACKAGES` allows you to disallow the installation
  258:   of vulnerable packages in critical environments like servers
  259: - `MAKE_JOBS` the number of concurrent make jobs, I set it to 8 but
  260:   it breaks some ports
  261: - `SKIP_LICENSE_CHECK` will skip the license check. If disabled you
  262:   will have to define a list of licenses you find acceptable with
  263:   `ACCEPTABLE_LICENSES`
  264: - `PKG_DEVELOPER` this option will show more details during the port building
  265: - `SIGN_PACKAGES` allows you to `gpg` sign packages. More info in
  266:   my [blog post](http://saveosx.org/signed-packages/) about it
  267: - `PKG_DEFAULT_OPTIONS` allows you to enable or disable specific
  268:   options for all ports (as defined with ports' options.mk files),
  269:   I disabled a few options so less ports would break, pulseaudio
  270:   doesn't build on Mac OS X for example, neither do x264, dconf
  271: 
  272: Keep in mind that there are many other available options documented
  273: in the official pkgsrc guide.
  274: 
  275: 
  276: ## creating a simple port
  277: 
  278: Let's create a little port using the tools we've talked about above.
  279: I will use a little window manager called 2bwm.
  280: 
  281: - We need an url for the program source files archive. It can be a
  282: direct link to a tar or xz archive. Mine's
  283: `http://pkgsrc.saveosx.org/Darwin/distfiles/2bwm-0.1.tar.gz`
  284: 
  285: - Now that we have a proper link for our program source, create a
  286:   directory for your port:
  287: 
  288: 		$ mkdir ~/pkgsrc/wm/2bwm
  289: 
  290: - Use `url2pkg` to create the needed files automatically:
  291: 		
  292: 		$ url2pkg http://pkgsrc.saveosx.org/Darwin/distfiles/2bwm-0.1.tar.gz
  293: 
  294: You'll be presented with a text editor like `vim` to enter basic
  295: Makefile options:
  296: 
  297: - `DISTNAME`, `CATEGORIES`, `MASTER_SITES` should be set automatically
  298: - enter your mail address for `MAINTAINER` so users know whom to
  299:   contact if the port is broken
  300: - make sure the `HOMEPAGE` is set right, for 2bwm it is a github page
  301: - write a `COMMENT`, it should be a one-line description of the program
  302: - find out which license the program uses, in my case it is the
  303:   `isc` license. You can find a list of licenses in `pkgsrc/mk/licenses.mk`.
  304: - Below you will see `.include "../../mk/bsd.pkg.mk"` at the end
  305:   of the Makefile and above this should go the port's needed
  306:   dependencies to build, we'll leave that empty at the moment and
  307:   try to figure out what 2bwm needs
  308: - exit vim and it should fetch and update the file hashes for you.
  309:   If it says `permission denied` you can just run `make mdi` to
  310:   fetch and upadate the `distinfo` file
  311: 
  312: So now you have valid `Makefile` and `distinfo` files but you need
  313: to write a paragraph in `DESCR`. You can usually find inspiration
  314: on the program's homepage.
  315: 
  316: Here's how they look like at the moment:
  317: 	
  318: Makefile:
  319: {% highlight make %}
  320: # [[!paste id=rcsid1]][[!paste id=rcsid2]]
  321: 
  322: DISTNAME=       2bwm-0.1
  323: CATEGORIES=     wm
  324: MASTER_SITES=   http://pkgsrc.saveosx.org/Darwin/distfiles/
  325: 
  326: MAINTAINER=     yrmt@users.sourceforge.net
  327: HOMEPAGE=       http://github.com/venam/2bwm/
  328: COMMENT=        Fast floating WM written over the XCB library and derived from mcwm
  329: LICENSE=        isc
  330: 
  331: .include "../../mk/bsd.pkg.mk"
  332: {% endhighlight %}
  333: 
  334: distinfo:
  335: 
  336: 	
  337: 	[[!paste id=rcsid1]][[!paste id=rcsid2]]
  338: 
  339: 	SHA1 (2bwm-0.1.tar.gz) = e83c862dc1d9aa198aae472eeca274e5d98df0ad
  340: 	RMD160 (2bwm-0.1.tar.gz) = d9a93a7d7ae7183f5921f9ad76abeb1401184ef9
  341: 	Size (2bwm-0.1.tar.gz) = 38419 bytes
  342: 
  343: DESCR:
  344: 
  345: 	A fast floating WM, with the particularity of having 2 borders,
  346: 	written over the XCB library and derived from mcwm written by
  347: 	Michael Cardell. In 2bWM everything is accessible from the keyboard
  348: 	but a pointing device can be used for move, resize and raise/lower.
  349: 
  350: But our PLIST file is still empty.
  351: 
  352: 
  353: #### build stage
  354: 
  355: Let's try to build the port to see if things work but as soon as
  356: the build stage starts, we get this error:
  357: 
  358: > 2bwm.c:26:10: fatal error: 'xcb/randr.h' file not found
  359: 
  360: Let's find out which port provides this file !
  361: 
  362: 	$ pkgin se xcb
  363: 
  364: returns these possible packages:
  365: 
  366: 	xcb-util-wm-0.3.9nb1  Client and window-manager helpers for ICCCM and EWMH
  367: 	xcb-util-renderutil-0.3.8nb1  Convenience functions for the Render extension
  368: 	xcb-util-keysyms-0.3.9nb1  XCB Utilities
  369: 	xcb-util-image-0.3.9nb1  XCB port of Xlib's XImage and XShmImage
  370: 	xcb-util-0.3.9nb1 =  XCB Utilities
  371: 	xcb-proto-1.9 =      XCB protocol descriptions (in XML)
  372: 	xcb-2.4nb1           Extensible, multiple cut buffers for X
  373: 
  374: Package content inspection allowed me to find the right port
  375: 
  376: 	$ pkgin pc libxcb|grep randr.h
  377: 
  378: So we can add the libxcb `buildlink3.mk` file to the Makefile above
  379: the bsd.pkg.mk include:
  380: 
  381: 	.include "../../x11/libxcb/buildlink3.mk"
  382: 
  383: This allows the port to link 2bwm against the libxcb port. Let's
  384: try to build the port again!
  385: 
  386: 	$ make clean
  387: 	$ make
  388: 
  389: Reports another error !
  390: 
  391: > 2bwm.c:27:10: fatal error: 'xcb/xcb_keysyms.h' file not found
  392: 
  393: It looks like this file is provided by xcb-util-keysyms, so let's add:
  394: 
  395: 	.include "../../x11/xcb-util-keysyms/buildlink3.mk"
  396: 
  397: in our Makefile.
  398: 
  399: Clean, build again, and add more dependencies until it passes the
  400: build stage. Here's how my Makefile ends up looking like:
  401: 
  402: {% highlight make %}
  403: # [[!paste id=rcsid1]][[!paste id=rcsid2]]
  404: 
  405: DISTNAME=       2bwm-0.1
  406: CATEGORIES=     wm
  407: MASTER_SITES=   http://pkgsrc.saveosx.org/Darwin/distfiles/
  408: 
  409: MAINTAINER=     yrmt@users.sourceforge.net
  410: HOMEPAGE=       http://github.com/venam/2bwm/
  411: COMMENT=        Fast floating WM written over the XCB library and derived from mcwm
  412: LICENSE=        isc
  413: 
  414: .include "../../x11/libxcb/buildlink3.mk"
  415: .include "../../x11/xcb-util-wm/buildlink3.mk"
  416: .include "../../x11/xcb-util-keysyms/buildlink3.mk"
  417: .include "../../x11/xcb-util/buildlink3.mk"
  418: .include "../../mk/bsd.pkg.mk"
  419: {% endhighlight %}
  420: 
  421: 
  422: #### install phase
  423: 
  424: Geat ! We got our program to compile in pkgsrc. Now we must generate
  425: the PLIST file so we can actually install the program, but we must
  426: `make stage-install` to make sure that it installs in the right
  427: place.
  428: 
  429: 	
  430: 	$ find /pkgsrc/work/wm/2bwm/work/.destdir/
  431: 
  432: returns:
  433: 
  434: 	/pkgsrc/work/wm/2bwm/work/.destdir/
  435: 	/pkgsrc/work/wm/2bwm/work/.destdir//usr
  436: 	/pkgsrc/work/wm/2bwm/work/.destdir//usr/local
  437: 	/pkgsrc/work/wm/2bwm/work/.destdir//usr/local/bin
  438: 	/pkgsrc/work/wm/2bwm/work/.destdir//usr/local/bin/2bwm
  439: 	/pkgsrc/work/wm/2bwm/work/.destdir//usr/local/bin/hidden
  440: 	/pkgsrc/work/wm/2bwm/work/.destdir//usr/local/share
  441: 	/pkgsrc/work/wm/2bwm/work/.destdir//usr/local/share/man
  442: 	/pkgsrc/work/wm/2bwm/work/.destdir//usr/local/share/man/man1
  443: 	/pkgsrc/work/wm/2bwm/work/.destdir//usr/local/share/man/man1/2bwm.1
  444: 	/pkgsrc/work/wm/2bwm/work/.destdir//usr/local/share/man/man1/hidden.1
  445: 	/pkgsrc/work/wm/2bwm/work/.destdir//usr/pkg
  446: 
  447: This doesn't look right since our `LOCALBASE` is `/usr/pkg`.
  448: 
  449: 
  450: 	$ make print-PLIST
  451: 
  452: returns nothing, because 2bwm installs files in the wrong place so
  453: we need to fix 2bwm's own Makefile to use the right `DESTDIR` and
  454: `PREFIX`, that is set to the right place by pkgsrc. Let's inspect
  455: how 2bwm installs:
  456: 
  457: From 2bwm's Makefile:
  458: 
  459: {% highlight make %}
  460: install: $(TARGETS)
  461:         test -d $(DESTDIR)$(PREFIX)/bin || mkdir -p $(DESTDIR)$(PREFIX)/bin
  462:         install -pm 755 2bwm $(DESTDIR)$(PREFIX)/bin
  463:         install -pm 755 hidden $(DESTDIR)$(PREFIX)/bin
  464:         test -d $(DESTDIR)$(MANPREFIX)/man1 || mkdir -p $(DESTDIR)$(MANPREFIX)/man1
  465:         install -pm 644 2bwm.man $(DESTDIR)$(MANPREFIX)/man1/2bwm.1
  466:         install -pm 644 hidden.man $(DESTDIR)$(MANPREFIX)/man1/hidden.1
  467: {% endhighlight %}
  468: 
  469: This looks fine since it installs in a `DESTDIR`/`PREFIX` but it sets
  470: 
  471: > PREFIX=/usr/local
  472: 
  473: and
  474: 
  475: > MANPREFIX=$(PREFIX)/share/man
  476: 
  477: In the beginning of the Makefile. We should remove the first line
  478: and edit the man prefix:
  479: 
  480: > MANPREFIX=${PKGMANDIR}
  481: 
  482: so pkgsrc can install the program's files in the right place. We
  483: have two ways of modifying this file, either patch the Makefile or
  484: use `sed` substitution which is a builtin pkgsrc feature that allows
  485: you to change lines in files with a sed command before building the
  486: port.
  487: 
  488: I will show how to do both ways so you can get an introduction on
  489: how to generate patch files for pkgsrc.
  490: 
  491: #### patching the Makefile :
  492: 
  493: - edit the file you need to modify with `pkgvi`:
  494: 
  495: 	
  496: 		$ pkgvi /pkgsrc/work/wm/2bwm/work/2bwm-0.1/Makefile
  497: 
  498: 	which should return:
  499: 
  500: 	> pkgvi: File was modified. For a diff, type:
  501: pkgdiff "/Volumes/Backup/pkgsrc/work/wm/2bwm/work/2bwm-0.1/Makefile"
  502: 
  503: 	and this returns our diff.
  504: 	
  505: 
  506: - create the patch with `mkpatches`, it should create a `patches`
  507:   directory in the port containing the patch and an original file
  508:   removed with `mkpatches -c`.
  509: 
  510: 		$ find patches/*
  511: 		patches/patch-Makefile
  512: 
  513: - now that the patch has been created, we need to add it's hash to
  514:   distinfo otherwise pkgsrc won't pick it up:
  515: 
  516: 		$ make mdi
  517: you should get this new line:
  518: 
  519: 	> SHA1 (patch-Makefile) = 9f8cd00a37edbd3e4f65915aa666ebd0f3c04e04
  520: 
  521: 
  522: - you can now clean and `make patch` and `make stage-install
  523:   CHECK_FILES=no` since we still haven't generated a proper PLIST.
  524:   Let's see if 2wm files were installed in the right place this
  525:   time:
  526: 
  527: 		$ find /pkgsrc/work/wm/2bwm/work/.destdir/
  528: 
  529: 		/pkgsrc/work/wm/2bwm/work/.destdir/
  530: 		/pkgsrc/work/wm/2bwm/work/.destdir//usr
  531: 		/pkgsrc/work/wm/2bwm/work/.destdir//usr/pkg
  532: 		/pkgsrc/work/wm/2bwm/work/.destdir//usr/pkg/bin
  533: 		/pkgsrc/work/wm/2bwm/work/.destdir//usr/pkg/bin/2bwm
  534: 		/pkgsrc/work/wm/2bwm/work/.destdir//usr/pkg/bin/hidden
  535: 
  536: 	It looks like it is alright ! Let's generate the PLIST:
  537: 
  538: 		$ make print-PLIST > PLIST
  539: 	
  540: 	containing:
  541: 
  542: 		@comment [[!paste id=rcsid1]][[!paste id=rcsid2]]
  543: 		bin/2bwm
  544: 		bin/hidden
  545: 
  546: 	There you have a working port you can install normally with
  547: 
  548: 		$ make install
  549: 
  550: 
  551: #### using the sed substitution framework
  552: 
  553: You should be able to fix the prefix error much quicker than with
  554: the patching explained above thanks to the sed substitution framework.
  555: Here's how it looks like in my port Makefile:
  556: 
  557: {% highlight make %}
  558: SUBST_CLASSES+=         makefile
  559: SUBST_STAGE.makefile=   pre-build
  560: SUBST_MESSAGE.makefile= Fixing makefile
  561: SUBST_FILES.makefile=   Makefile
  562: SUBST_SED.makefile=     -e 's,/usr/local,${PREFIX},g'
  563: SUBST_SED.makefile+=    -e 's,share/man,${PKGMANDIR},g'
  564: {% endhighlight %}
  565: 
  566: As you can see, you can do multiple commands on multiple files, it
  567: is very useful for very small fixes like this.
  568: 
  569: 
  570: #### pkglint
  571: 
  572: Now that we have a working port, we must make sure it complies to the pkgsrc rules.
  573: 
  574: 	$ pkglint
  575: 	
  576: Returns
  577: 
  578: 	ERROR: DESCR:4: File must end with a newline.
  579: 	ERROR: patches/patch-Makefile:3: Comment expected.
  580: 	2 errors and 0 warnings found. (Use -e for more details.)
  581: 
  582: Fix the things pkglint tells you to do until you get the glorious:
  583: 
  584: > looks fine.
  585: 
  586: Then you should do some testing on the program itelf on at least
  587: two platforms such as NetBSD, Mac OS X. Other platforms supported
  588: by pkgsrc can be found at [pkgsrc.org](http://pkgsrc.org). If you
  589: would like to submit your pkgsrc upstream you can either subscribe
  590: to pkgsrc-wip or ask a NetBSD developer to add it for you.
  591: 
  592: You can find the 2bwm port I submitted in
  593: [pkgsrc-wip](http://pkgsrc-wip.cvs.sourceforge.net/viewvc/pkgsrc-wip/wip/2bwm/).
  594: 
  595: 
  596: ## pkgsrc and wip
  597: 
  598: If you want to submit your port for others to use you can either
  599: subscribe to pkgsrc-wip or ask a NetBSD developer to add it for you
  600: which can be tough. Even though there are many IRC channels in which
  601: you can find nice developers, you will have to take the time to get
  602: to know them. The easiest way for beginners is to submit to pkgsrc-wip
  603: so other people can review and test it first.
  604: 
  605: pkgsrc-wip is hosted on
  606: [sourceforge](https://sourceforge.net/projects/pkgsrc-wip/) and you
  607: can easily get cvs access to it if you create an account on there
  608: and send an email to NetBSD developer `@wiz` (Thomas Klausner)
  609: asking nicely for commit access. I got access fairly quickly and
  610: he even fixed a port to show me how to do it properly.
  611: 
  612: You can also send me an email or talk to me on IRC so I can submit it for you.
  613: 
  614: 
  615: ## the options framework
  616: 
  617: You can create port options with the `options.mk` file, like for `wm/dwm`
  618: 
  619: 	
  620: {% highlight make %}
  621: # [[!paste id=rcsid1]][[!paste id=rcsid2]]
  622: 
  623: PKG_OPTIONS_VAR=			PKG_OPTIONS.dwm
  624: PKG_SUPPORTED_OPTIONS=	xinerama
  625: PKG_SUGGESTED_OPTIONS=	xinerama
  626: 
  627: .include "../../mk/bsd.options.mk"
  628: 
  629: #
  630: # Xinerama support
  631: #
  632: # If we don't want the Xinerama support we delete XINERAMALIBS and
  633: # XINERAMAFLAGS lines, otherwise the Xinerama support is the default.
  634: #
  635: .if !empty(PKG_OPTIONS:Mxinerama)
  636: .  include "../../x11/libXinerama/buildlink3.mk"
  637: .else
  638: SUBST_CLASSES+=         options
  639: SUBST_STAGE.options=    pre-build
  640: SUBST_MESSAGE.options=  Toggle the Xinerama support
  641: SUBST_FILES.options=    config.mk
  642: SUBST_SED.options+=     -e '/^XINERAMA/d'
  643: .  include "../../x11/libX11/buildlink3.mk"
  644: .endif
  645: {% endhighlight %}
  646: 
  647: This file should be included in the Makefile:
  648: 
  649: 	.include "options.mk"
  650: 
  651: If you type `make show-options`, you should see this:
  652: 
  653: 	Any of the following general options may be selected:
  654: 	xinerama	 Enable Xinerama support.
  655: 
  656: 	These options are enabled by default:
  657: 		xinerama
  658: 
  659: 	These options are currently enabled:
  660: 		xinerama
  661: 
  662: 	You can select which build options to use by setting 	PKG_DEFAULT_OPTIONS
  663: 	or PKG_OPTIONS.dwm.
  664: 
  665: Running `make PKG_OPTIONS=""` should build without the `xinerama` dwm option enabled by default.
  666: 
  667: The options.mk file must contain these variables:
  668: 
  669: - `PKG_OPTIONS_VAR` sets the options variable name
  670: - `PKG_SUPPORTED_OPTIONS` lists all available options
  671: - `PKG_SUGGESTED_OPTIONS` lists options enabled by default
  672: 
  673: It allows you to change configure arguments and include other buildlinks, and various other settings.
  674: 
  675: 
  676: ## hosting a package repo
  677: 
  678: Now that you've created a few ports, you might want to make precompiled
  679: packages available for testing. You will need pkgsrc's `pkg_install`
  680: on the host system. I host my [packages](http://pkgsrc.saveosx.org/)
  681: on a FreeBSD server with a bootstrapped pkgsrc.
  682: 
  683: I use this `zsh` function to :
  684: 
  685: {% highlight bash %}
  686: add () {
  687: 	# upload the package to remote server
  688: 	scp $1 yrmt@saveosx.org:/usr/local/www/saveosx/packages/Darwin/2013Q4/x86_64/All/ 2> /dev/null
  689: 	
  690: 	# update the package summary
  691: 	ssh yrmt@saveosx.org 'cd /usr/local/www/saveosx/packages/Darwin/2013Q4/x86_64/All/;
  692: 	        rm pkg_summary.gz;
  693:  	        /usr/pkg/sbin/pkg_info -X *.tgz | gzip -9 > pkg_summary.gz'
  694: 	
  695: 	# pkgin update
  696: 	sudo pkgin update
  697: }
  698: {% endhighlight %}
  699: 
  700: - upload a package
  701: - update the package summary, which is an archive containing
  702:   information about all present packages that will be picked up by
  703:   pkg_install and pkgin. It looks like this for one package:
  704: 
  705: 		PKGNAME=osxinfo-0.1
  706: 		DEPENDS=sqlite3>=3.7.16.2nb1
  707: 		COMMENT=Small Mac OS X Info Program
  708: 		SIZE_PKG=23952
  709: 		BUILD_DATE=2014-06-29 12:45:08 +0200
  710: 		CATEGORIES=misc
  711: 		HOMEPAGE=http://github.com/yrmt/osxinfo
  712: 		LICENSE=isc
  713: 		MACHINE_ARCH=x86_64
  714: 		OPSYS=Darwin
  715: 		OS_VERSION=14.0.0
  716: 		PKGPATH=wip/osxinfo
  717: 		PKGTOOLS_VERSION=20091115
  718: 		REQUIRES=/System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation
  719: 		REQUIRES=/System/Library/Frameworks/Foundation.framework/Versions/C/Foundation
  720: 		REQUIRES=/System/Library/Frameworks/IOKit.framework/Versions/A/IOKit
  721: 		REQUIRES=/usr/lib/libSystem.B.dylib
  722: 		REQUIRES=/usr/pkg/lib/libsqlite3.0.dylib
  723: 		FILE_NAME=osxinfo-0.1.tgz
  724: 		FILE_SIZE=9710
  725: 		DESCRIPTION=Small and fast Mac OS X info program written in C
  726: 		DESCRIPTION=by Youri Mouton.
  727: 		DESCRIPTION=
  728: 		DESCRIPTION=Homepage:
  729: 		DESCRIPTION=http://github.com/yrmt/osxinfo
  730: 
  731: 
  732: - update pkgin
  733: 
  734: 
  735: And this shell alias to upload all my built packages, but I still
  736: need to run `add()` mentionned above to update the pkg_summary
  737: 
  738: {% highlight bash %}
  739: up='rsync -avhz --progress /pkgsrc/packages/ root@saveosx.org:/usr/local/www/saveosx/packages/Darwin/2013Q4/x86_64/'
  740: {% endhighlight %}
  741: 
  742: Then you should be able to set the url in repositories.conf to use
  743: your packages with pkgin. You can also install them directly with
  744: something like `pkg_add
  745: http://pkgsrc.saveosx.org/Darwin/2013Q4/x86_64/All/9menu-1.8nb1.tgz` of
  746: course.
  747: 
  748: 
  749: ## build all packages
  750: 
  751: Bulk building pkgsrc packages is a topic for another post, see
  752: jperkin's excellent blog
  753: [posts](http://www.perkin.org.uk/posts/distributed-chrooted-pkgsrc-bulk-builds.html)
  754: about this.
  755: 
  756: 
  757: ## faq
  758: 
  759: #### what if the port I'm making is a dependency for another one?
  760: 
  761: You should just generate the buildlink3.mk file we've talked about
  762: earlier like this:
  763: 
  764: 	$ createbuildlink > buildlink3.mk
  765: 
  766: #### what if the program is only hosted on GitHub ?
  767: 
  768: pkgsrc supports fetching archives from specific git commits on
  769: GitHub like this:
  770: {% highlight make %}
  771: PKGNAME=           2bwm-0.1
  772: CATEGORIES=        wm
  773: GHCOMMIT=          52a097ca644eb571b22a135951c945fcca57a25c
  774: DISTNAME=          ${GHCOMMIT}
  775: MASTER_SITES=      https://github.com/venam/2bwm/archive/
  776: DIST_SUBDIR=       2bwm
  777: WRKSRC=            ${WRKDIR}/2bwm-${GHCOMMIT}
  778: {% endhighlight %}
  779: 
  780: You can then easily update the git commit and the distinfo with it
  781: to update the program.
  782: 
  783: #### what if the program doesn't have a Makefile
  784: 
  785: You can do all Makefile operations directly from the port's Makefile
  786: like this:
  787: 
  788: 
  789: {% highlight make %}
  790: post-extract:
  791: 	${CHMOD} a-x ${WRKSRC}/elementary/apps/48/internet-mail.svg
  792: 
  793: do-install:
  794: 	${INSTALL_DATA_DIR} ${DESTDIR}${PREFIX}/share/icons
  795: 	cd ${WRKSRC} && pax -rw -pe . ${DESTDIR}${PREFIX}/share/icons/
  796: {% endhighlight %}
  797: 
  798: To install, but you can also build programs from the Makefile. This
  799: is what qt4-sqlite3 uses:
  800: 
  801: {% highlight make %}
  802: do-build:
  803: 	cd ${WRKSRC}/src/tools/bootstrap && env ${MAKE_ENV} ${GMAKE}
  804: 	cd ${WRKSRC}/src/tools/moc && env ${MAKE_ENV} ${GMAKE}
  805: 	cd ${WRKSRC}/src/plugins/sqldrivers/sqlite && env ${MAKE_ENV} ${GMAKE}
  806: {% endhighlight %}
  807: 
  808: 
  809: You can install the following type of files:
  810: 
  811: `INSTALL_PROGRAM_DIR` : directories that contain binaries
  812: 
  813: `INSTALL_SCRIPT_DIR` : directories that contain scripts
  814: 
  815: `INSTALL_LIB_DIR` : directories that contain shared and static libraries
  816: 
  817: `INSTALL_DATA_DIR`: directories that contain data files
  818: 
  819: `INSTALL_MAN_DIR` : directories that contain man pages
  820: 
  821: `INSTALL_PROGRAM` : binaries that can be stripped from debugging symbols
  822: 
  823: `INSTALL_SCRIPT` : binaries that cannot be stripped
  824: 
  825: `INSTALL_GAME` : game binaries
  826: 
  827: `INSTALL_LIB` : shared and static libraries
  828: 
  829: `INSTALL_DATA` : data files
  830: 
  831: `INSTALL_GAME_DATA` : data files for games
  832: 
  833: `INSTALL_MAN` : man pages
  834: 
  835: 
  836: `INSTALLATION_DIRS` : A list of directories relative to PREFIX that
  837: are created by pkgsrc at the beginning of the install phase. The
  838: package is supposed to create all needed directories itself before
  839: installing files to it and list all other directories here.
  840: 
  841: #### common errors
  842: 
  843: - > Makefile:19: *** missing separator.  Stop.
  844: 
  845: This means you're not using the right `make`. On most systems, the
  846: make installed from the pkgsrc bootstrap is called `bmake`
  847: 
  848: - If you have a feeling a port is stuck in the building stage,
  849:   disable make jobs in your mk.conf
  850: 
  851: - Please contribute here :)
  852: 
  853: 
  854: ## links
  855: - [Jonathan Perkin's excellent blog](http://www.perkin.org.uk/)
  856: - [NetBSD's very extensive pkgsrc guide](http://www.netbsd.org/docs/pkgsrc/)
  857: - [NetBSD's pkgsrc wiki](http://wiki.netbsd.org/pkgsrc/)
  858: - Other blog posts here :)
  859: 
  860: ## where to find me
  861: 
  862: - yrmt@edgebsd.org
  863: - irc.oftc.net
  864: 	
  865: 	`#saveosx`
  866: 
  867: [[!cut id=rcsid1 text="$Net"]]
  868: [[!cut id=rcsid2 text="BSD$"]]
  869: [[!meta title="An introduction to packaging"]]
  870: [[!meta author="Youri Mouton"]]

CVSweb for NetBSD wikisrc <wikimaster@NetBSD.org> software: FreeBSD-CVSweb