Annotation of wikisrc/pkgsrc/how_to_use_pkgsrc_on_aix.mdwn, revision 1.2
1.2 ! schmonz 1: **This page should use cross references to avoid duplicate content. Please help us by cleaning it up.**
! 3: **Contents**
! 5: [[!toc levels=3]]
! 7: #Introduction
! 9: 'pkgsrc' on AIX must be a fairly uncommon occurance considering the general state of the documentation around getting it working. This is probably partly due to the advent of the 'AIX toolkit' which allows RPMs to be installed with minimum hassle on an AIX box. On the plus side, this toolkit also makes what appears to have been a fernickity bootstrap procedure pretty simple.
! 11: Due to limited resources I've only attempted this on AIX 5.2 and AIX 5.3 but both appear to work smoothly. Notes, from previous versions regarding AIX 4.1 have been left.
! 13: #Setup the base system
! 15: For the majority of systems the following will be a non-issue as, more often than not, the entire base system will have been installed on the box and thus the 'bos.adt' packages will be present. To verify your system has those tools run the following command
! 17: TODO: installp <check for 'bos.adt'>
! 19: you'll notice the ABC output. If it's not there then you need to add those packages by sticking in the relevant CD (first install disk, generally) and running your favourite admin tool or the following command.
! 21: TODO: installp <install the package>
! 23: Precisely which set of packages is the minimum required set of 'bos.adt' packages to bootstrap is left as an exercise for the reader.
! 25: #Install a compiler
! 27: As you'll probably realise, the one thing that 'pkgsrc' can't do without is a complier. The simplest option is to use the 'gcc' complier which (I think) is available on most versions of AIX (perhaps someone else could clarify these options). There's no particular reason that you can't or shouldn't use another complier but you might want to note that many of the packages within 'pkgsrc' will be dependent on GNU extensions and may cause you problems. Mixing compliers will probably wreak havoc unless you are extremely careful and have an excellent understanding of the dependancies and technologies involved.
! 29: ##Option 1: Using 'gcc' from the AIX toolkit
! 31: I believe that 'gcc' is available in AIX but if you don't have it you should be able to download it from IBM like I did. However I'd send a word of warning about having the complete 'AIX linux toolkit' installed because you will inevitably get caught trying to avoid conflicts with the 'pkgsrc' versions of libraries. Consequently, I'd recommend you *only* get the complier you want and remove everything else via the IBM supplied 'rpm' command.
! 33: The first step is to ensure that you have the 'rpm' manager
! 35: installp <show rpm>
! 37: and install it (in a similar manner to above) if you find it's missing or has been stripped out by an enthusiastic security officer.
! 39: Thus, if you follow the above advice, running the command 'rpm -qa' should produce something similar to the following
! 41: # rpm -qa
! 42: libgcc
! 43: gcc
! 44: gcc-c++
! 46: if you don't, you'll have a much longer list but the above items should be included in it.
! 48: P.S: i'm using 'gcc4' from the AIX toolkit, however, it would probably be more prudent to use 'gcc3' which is also available as 'pkgsrc' does not appear to be particularly 'gcc4' friendly.
! 50: P.P.S: in achieving the above I removed the 'mkisofs' and 'cdrecord' packages from the system. This suited me fine, however, you may wish to verify whether that's going to impact some of the base system's backup option (i.e. backup to DVD) and/or whether re-instatement of those utilities via 'pkgsrc' will solve those issues.
! 52: ##Option 2: Use a-n-other compiler
! 54: I cannot recommend or caution against this option; I'm simply not able to afford the IBM compiler (or other, if there is one). Should anyone wish to give me a license I'll be happy to try it. However, apart from the problems you'll no doubt have with the general adherence to GCC and it's extensions within the 'pkgsrc' packages, it *should* work.
! 56: That's a heavily conditioned *should*, for those that didn't get it the first time.
! 58: #Bootstrap away, bootstrap away, bootstrap away
! 60: Generally, I like to mirror the native system layout (it's one of the primary reasons I like 'pkgsrc') and so generally I put 'pkgsrc' into '/opt' and use the following bootstrap options
! 62: * --prefix=/opt/pkg
! 63: * --pkgdbdir=/var/opt/pkg
! 65: whilst leaving the others with their default 'pkgsrc' setting.
! 67: Where you put it is, of course, entirely up to yourself but remember to set the 'CC' environment variable before bootstrap or you'll get into a bit of a mess when the build wrappers fail to find it later. This
! 69: # CC=/opt/freeware/bin/gcc ; export CC
! 71: is probably what you want but, if you're using another complier, you'll need to change it correspondingly.
! 73: Go bootstrap.
! 75: # cd /opt/pkgsrc/bootstrap
! 76: # ./bootstrap --prefix=/opt/pkg --pkgdbdir=/var/opt/pkg
! 77: [ ... ]
! 78: Complete success and happiness has been achieved
! 79: [ ... ]
! 80: Fulfillment of other life ambitions imminient
! 81: [ ... ]
! 82: Bootstrap successful
! 83: #
! 85: Hopefully, that's the worst of it over.
! 87: #Pack-up and wagons roll
! 89: Now you need to complete your local configuration
! 91: * set library locations
! 92: * set path
! 93: * set build environment (i.e. mk.conf)
! 94: * set other variables
! 96: And ensure that you double check 'README.AIX' for important changes.
! 98: The last decision you have to make (for now at least) is whether to use one of the 'pkgsrc' compilers instead of the AIX linux toolkit you just used. Personally, I see little reason, particularly as the latest compiler in 'pkgsrc' is 'gcc3' and the AIX toolkit give me a shiny gcc4.2 version. N.B: as noted above building with 'gcc4' may not be as resilient as 'pkgsrc' seems to be more settled on 'gcc3' at present.
! 100: The only thing left is for someone to re-generate the binary bootstrap package for the other AIX souls out there so that the above article is completely useless.
! 102: #Known issues
! 104: The following section outlines the various problems encountered whilst trying to get the system working, note (to self) these should only really appear here if they are specific to AIX. Find somewhere friendlier for more generic notes.
! 106: ##Packages
! 108: This is a very limited selection of the packages problems that have been encountered if anyone does a complete build, power to them, perhaps they'll add some info here.
! 110: ###sysutils/fam
! 112: It will just not compile - The situation is a bit like for OSF1 as described in PR [#31489](http://www.netbsd.org/cgi-bin/query-pr-single.pl?number=31489) for AIX as well. After trying to fix the first errors I decided to add FAM_DEFAULT=gamin in my mk.conf. I've posted a PR for setting this permanently on AIX: [#41815](http://www.netbsd.org/cgi-bin/query-pr-single.pl?number=41815)
! 114: ###devel/gettext
! 116: See bug (todo: look this up)
! 118: ###devel/pcre
! 120: Just does not compile (the C++ part cannot be linked) - a [bug report](http://bugs.exim.org/show_bug.cgi?id=874) in the PCRE project was opened.
! 122: ###lang/perl5
! 124: The Configure and Makefile.SH scripts delivered by perl are not properly working when compiling with gcc. They hand over -b arguments to gcc without prefixing them -Wl, which needs to be done because they are meant for ld. I've raised PR [#41814](http://www.netbsd.org/cgi-bin/query-pr-single.pl?number=41814) with a patch file included to fix this. The fix was tested on a Power2 System running AIX 5.1 with gcc 3.4 from pkgsrc. [OliverLehmann](file:///index.php?title=User:OliverLehmann&action=edit&redlink=1)
! 126: ###security/openssh
! 128: Another package that thinks it needs 'perl'; OK it does but we don't want to build it so lets just hack out the tools again.
! 130: ###security/openssl
! 132: One of those packages that depends on 'perl' simply hack the Makefile to remove 'perl' from the required build tools. This should allow it to build using the base system 'perl'.
! 134: Also had trouble with the linking of the shared libraries, not sure if this points to my use of 'gcc4' but manually hacked 'Makefile.shared' (think this is generated so you may need to hack this after the error) to include '--shared' with the linking command. You'll find this in the LINK_SO definition; look for the SHAREDLINK command. -- [ttw](file:///User:Ttw)
! 136: P.S: remember to set the 'PERL5' variable above
! 138: ###security/gnupg2
! 140: This isn't actually a problem with the build of 'gnupg2' but it is a problem with the fact that it depends on a bizillion packages. I had issues with 'curl' and 'pinentry'. I should really log bugs for these, and I'll try but I need to get this @$%# completed and get work done too. Anyway, can't remember 'what the problem with 'curl' was but it must have been on of the standard ones here, probably my base perl. 'pinentry' posed more of a problem. Apparently there is a dependancy on 'getopt' missing from the build. I didn't actually fix this correctly, once I finally tracked it down i simply did the following
! 142: # ln -s /opt/pkg/include/getopt.h work/.buildlink/include/getopt.h
! 143: # for f in /opt/pkg/lib/libgetopt* ; do ln -s $f work/.buildlink/lib/`basename $f` ; done
! 145: Next we hit a block with the definition of FD_SETSIZE on AIX which is 65534 but the GNU portable threads that this package uses fixes a maximum of 1024 (although the change log appears to contradict this). Either way I hacked the 'Makefile' such that the static inclusion of the 'pth' stuff (under 'pre-configure' action) includes the '--with-fdsetsize=1024' option.
! 147: Current advice would be try 'security/gnupg' instead.
! 149: PS: Odd that the build still shows the '-L/opt/pkg/include' but cannot find it. Makes tracking down issues difficult. Need to investigate the 'pkgsrc' mechanics further to understand how/why this works. --ttw 23:45, 17 May 2009 (UTC)
! 151: ###lang/python25
! 153: This is a new one by me but the configure script for python explicitly overrides the 'CC' variable we defined when setting up pkgsrc. I've hacked this (once more, i know i should correct these things but i'm busy scratching other itches ... anyway) by adding a 'CONFIGURE_ARGS' to the 'Makefile' with the value '--with-gcc'. this conveniently avoids the evil configure code and allows the default 'CC' to be uses sanely.
! 155: NB: The above is only one issue. This doesn't work for me as is.
! 157: ###lang/python24
! 159: This is slightly less ugly than 'lang/python25' primarily because it doesn't appear dependent on GNU 'pth' but I didn't really track this down too hard either. Major grip is that IPv6 doesn't work due to a failure in the 'configure' script. That should be hackable but I've currently got it built with '--disable-ipv6', although unsatisfactory.
! 161: Further down the line there is a problem with the 'bsddb' package which has some threading issues that I've not investigated. I wasn't interested in it anyway so I simply disabled it (by hacking it out of the Makefile) and we have a build.
! 163: I'm not sure that counts as success but I'm ignoring it in favour of my own needs.
! 165: ###mail/dovecot
! 167: this is pretty nice all in all but I couldn't get this to build due to issues with 'rquota' in the 'quotas' plugins. the solution was to hack 'quota-fs.c' and change the local 'rquota.h' to the system version (<rpcsvc/rquota.h>) and add the other necessaries (i.e. <rpc/rpc.h> and <rpc/clnt.h>).
! 169: Unfortunately, there's a little more incompatability that needs hacking and that's more edits to 'quota-fs.c' where we hack the result.status' to the corresponding 'result.gqr_status' and the 'result.getquota_rslt_u.status' to 'result.gqr_rquota'.
! 171: I'm sure all this is "wrong" but it appears to build. Only thing required after that is to ensure you add the 'dovecot' user before doing the install. Happy days.
! 173: ##Operating system
! 175: As we all know, AIX isn't anymore singular than other systems and various problems arise on various versions, particularly as both systems are organic entities.
! 177: Feel free to add to that knowledge quotient here.
! 179: ##AIX 5L
! 180: **'undef'd stdio functions within c++ library**
! 182: the c++ standard libraries appear have a small problem wherein the 'cstdio' 'undef's the macros for 'fgetpos', 'fsetpos', 'fopen', 'freopen'. unfortunately, this is incorrect when using _LARGE_FILE extensions as these macros alias the 64 versions of the functions. if you alter your '/opt/freeware/lib/gcc/powerpc-ibm-aix220.127.116.11/4.2.0/include/c++/cstdio' file to correct this everything should start to flow smoothly again. the following is what i did
! 184: # cd /opt/freeware/lib/gcc/powerpc-ibm-aix18.104.22.168/4.2.0/include/c++
! 185: # ln cstdio cstdio,v22.214.171.124
! 186: # sed -e '/fgetpos/ d' -e '/fsetpos/ d' \
! 187: -e '/fopen/ d' -e '/freopen/ d' \
! 188: cstdio,v126.96.36.199 >cstdio,v1.2
! 190: then we add the following to the new file
! 192: #ifndef _LARGE_FILES
! 193: #undef fgetpos
! 194: #undef fsetpos
! 195: #undef fopen
! 196: #undef freopen
! 197: #endif /* _LARGE_FILES */
! 199: and replace the active one
! 201: # ln -f cstdio,v1.2 cstdio
! 203: we can now compile away happily again
! 205: **'bos.net.nfs.adt' required for various builds**
! 207: i would imagine that this issue will be transparent to most users and can probably be resolved by a quick search on google, however, i put it here for posterity and completeness and for those whom hacking at any level is alien. basically, you need 'rpcgen' for various builds and that's only included as part of the 'bos.net.nfs.adt' package from the base system. you'll probably have it already.
! 209: --ttw 15:18, 7 June 2009 (UTC): this may be incorrect, perhaps 'bos.net.nfs.server' package
! 211: ##AIX 4.1
! 213: AIX 4.1 is a pretty old system at this stage, however, AIX systems are renowned for their longevity, although I'd guess there are very few being used as development platforms these days.
! 215: Still if you can contribute to clarifying any of the following and/or find some issues of your own feel free to help shuffle the rest of the world round that blockade.
! 217: For older AIX releases like this, mirrors of the AIX Public Domain Software Library (aixpdslib) may prove useful to get started.
! 219: **Conflicting type declarations**
! 221: Following the instructions to bootstrap pkgsrc-2006Q3 did not work on my older 4.1 system. After several minutes the bootstrap process aborted with the following error:
! 223: sh makelist -bc ./vi.c ./emacs.c ./common.c > help.c
! 224: sh makelist -bh ./vi.c ./emacs.c ./common.c > help.h
! 225: gcc -g -O2 -I. -I./.. -I. -I.. -I./../libedit -I./../libnetbsd -c chared.c
! 226: In file included from sys.h:142,
! 227: from chared.c:37:
! 228: /usr/include/regex.h:172: conflicting types for `regex'
! 229: /usr/include/libgen.h:31: previous declaration of `regex'
! 230: /usr/include/regex.h:173: conflicting types for `regcmp'
! 231: /usr/include/libgen.h:30: previous declaration of `regcmp'
! 232: *** Error code 1
! 233: Stop.
! 234: bmake: stopped in /usr/pkgsrc/bootstrap/work/tnftp/libedit
! 235: *** Error code 1
! 236: Stop.
! 237: bmake: stopped in /usr/pkgsrc/bootstrap/work/tnftp
! 238: ===> exited with status 1
! 239: aborted.
! 241: I found an [explanation of this error](http://lists.ibiblio.org/pipermail/freetds/2004q4/017290.html) with help from Google:
! 243: * Old Unix versions define these functions in libgen.h, newer in regex.h. It seems that AIX define in two places but with different prototypes.
! 245: Not having any skills in C programming, I was unable to resolve the issue by correcting the conflicting definitions and had to resort to sledgehammer tactics; I removed libgen.h but the bootstrap process then failed as it was not able to locate the file. I then overwrote the libgen.h file with a copy of regex.h. This sorted out the conflicting type declaration problem, but I am not sure if this will have any adverse effect on my system! The plan was just to get the bootstrap process to complete then re-instate the original libgen.h file.
! 247: >You should never change your operating system's files just to make any third-party software run. The proper solution is to tell the author of tnftp (see pkgsrc/net/tnftp/Makefile.common) to have a look at it. --Rillig 13:11, 17 December 2006 (CET)
! 249: >A workaround -- remove check for libgen.h from tnftp/configure. (Verified on AIX 188.8.131.52, pkgsrc-2007Q1.) Shattered 21:10, 9 July 2007 (CEST)
! 251: **Undeclared variable**
! 253: After restarting the bootstrap process, it failed again with the following error:
! 255: gcc -g -O2 -I. -I./.. -I. -I.. -I./../libedit -I./../libnetbsd -c inet_ntop.c
! 256: gcc -g -O2 -I. -I./.. -I. -I.. -I./../libedit -I./../libnetbsd -c inet_pton.c
! 257: inet_pton.c: In function `inet_pton4':
! 258: inet_pton.c:92: `uint32_t' undeclared (first use in this function)
! 259: inet_pton.c:92: (Each undeclared identifier is reported only once
! 260: inet_pton.c:92: for each function it appears in.)
! 261: inet_pton.c:92: parse error before `val'
! 262: inet_pton.c:108: `val' undeclared (first use in this function)
! 263: *** Error code 1
! 264: Stop.
! 265: bmake: stopped in /usr/pkgsrc/bootstrap/work/tnftp/libnetbsd
! 266: *** Error code 1
! 267: Stop.
! 268: bmake: stopped in /usr/pkgsrc/bootstrap/work/tnftp
! 269: ===> exited with status 1
! 270: aborted.
! 272: This is as far as I have managed to get at the moment. I will update this page as and when I have a solution to this problem.
! 274: >I think adding #include <inttypes.h> at line 25 of pkgsrc/net/tnftp/files/tnftp.h helps. --Rillig 14:03, 17 December 2006 (CET)
! 276: >AIX 4.1.5 appears not to have inttypes.h. ? ChristTrekker 14:30, 22 April 2009 (UTC)
! 278: >Which makes sense, since 4.1.5 predates C99. ? ChristTrekker 04:12, 29 April 2009 (UTC)
! 280: **missing termcap library**
! 282: Using AIX 4.1.5 and gcc 2.95.2, I get this far...
! 284: ===> running: (cd /usr/pkgsrc/net/tnftp && /usr/pkgsrc/bootstrap/work/bin/bmake -DPKG_PRESERVE MAKECONF=/usr/pkgsrc/bootstrap/work/mk.conf install)
! 285: ERROR: This package has set PKG_FAIL_REASON:
! 286: ERROR: No usable termcap library found on the system.
! 287: *** Error code 1
! 289: Stop.
! 290: bmake: stopped in /usr/pkgsrc/net/tnftp
! 291: ===> exited with status 1
! 292: aborted.
! 294: >Updates will be posted here as progress is made. ? ChristTrekker 18:57, 17 March 2009 (UTC)
! 296: >It appears that AIX has a libtermcap.a but doesn't provide termcap.h. ? ChristTrekker 16:00, 18 March 2009 (UTC)
! 298: **working bootstrap**
! 300: I don't remember what I did now, but I managed to get bootstrapped. The problem now is that very few packages build, because when trying to make libtool-base I get configure: error: C compiler cannot create executables, which is clearly a bogus problem since I've obviously created executables. I've submitted some patches with PRs, so I really need to wipe out my pkgsrc installation and try bootstrapping from scratch again to see if the process is smoother. ? ChristTrekker 21:24, 9 June 2009 (UTC)
! 302: >I think fixing this is just a matter of specifying CC on the command line. ? ChristTrekker 18:09, 21 July 2009 (UTC)
! 304: #See also
! 306: [README.AIX](ftp://ftp.netbsd.org/pub/pkgsrc/current/pkgsrc/bootstrap/README.AIX)
! 308: #External links
1.1 mspo 310: [Open Source with AIX](http://www.ibm.com/developerworks/wikis/display/WikiPtype/aixopen)
CVSweb for NetBSD wikisrc <wikimaster@NetBSD.org> software: FreeBSD-CVSweb