--- wikisrc/pkgsrc/gcc.mdwn 2017/11/26 01:07:36 1.1 +++ wikisrc/pkgsrc/gcc.mdwn 2017/12/31 17:15:37 1.9 @@ -1,8 +1,6 @@ -Using gcc in pkgsrc - On many systems pkgsrc supports, gcc is the standard compiler. In general, different versions of each OS have different gcc versions, -and some packages require newer GCC versions, in order to support +and some packages require newer gcc versions, in order to support newer language standards (e.g. c++11, written in the style of USE_LANGUAGES), or because older versions don't work (infrequently). @@ -20,11 +18,15 @@ build gcc versions (typically newer vers in a compiler within ${PREFIX}, e.g. /usr/pkg/gcc6/bin/gcc. This compiler can then be used to compile other packages. -Issues with using base system gcc are typically that it is too old, -such as gcc 4.5 with NetBSD 6, which cannot compile c++11. +The issue with using base system gcc is typically that it is too old, +such as gcc 4.5 with NetBSD 6, which cannot compile c++11. Another +example is gcc 4.8 with NetBSD 7. While this can compile most c++11 +programs, it cannot be used for firefox or glibmm (and therefore any +package that links against glibmm). Issues when using pkgsrc gcc are that + - on some platforms, pkgsrc gcc does not build and work - it must be bootstrapped, requiring compiling a number of packages with the system compiler - C++ packages that are linked together should be built with the @@ -45,41 +47,60 @@ This section attempts to gather all the - The set of packages that are needed when building a bootstrap compiler should be minimized. - - All packages that use C++ should be built with the same compiler version. - - All packages that use C should have final linking with the highest version used in any included library. + - All packages that use C++ should be built with the same compiler + version. Because these in the general case may include C, the + version used for C++ must be at least as new as the version used + for any used C package. + - pkgsrc should avoid building gcc unless it is more or less necessary to build packges. (As an example, if the base system gcc can build c99 but not c++11, building a c99-only program should not trigger building a gcc version adequate for c++11.) - - The compiler selection logic should work on NetBSD 6, and in-use - (including LTS) GNU/Linux systems. It is desirable for this logic - to work on NetBSD 5. + - The compiler selection logic should work on NetBSD 6 and newer, + and other systems currently supported by pkgsrc, including in-use + LTS GNU/Linux systems. It should work on systems that default to + clang, when set to use GCC, at least as well as the current + scheme. It is desirable for this logic to work on NetBSD 5. + + - All systems should work at least as well as they do before + implementation of new compiler selection logic. - The compiler selection logic should be understandable and not brittle. ## Design The above requirements could in theory be satisfied in many ways, but -most of them are too complicated. +most of them are too complicated. We present a design that aims to be +sound while mimimizing complexity. - Packages declare what languages they need, with c++, c++11, and - c++14 being expressed differently. + c++14 being expressed differently. (This is exactly current + practice and just noted for completeness.) - The package-settable variable GCC_REQD will be used only when a compiler that generally can compile the declared language version - is insufficient. These cases are expected to be relatively rare. + is insufficient. These cases are expected to be relatively rare; + an example is firefox that is in c++ (but not c+11) and needs gcc + 4.9. - A user-settable variable PKGSRC_GCC_VERSION will declare the - version of gcc to be used for C programs, with an OS- and - version--specific default. + version of gcc to be used for C programs, with an OS-, + version- and architeture- specific default. - - A user-settable variable PKGSRC_GXX_VERSION will declare the version of gcc to - be used for all C++ programs, again with an OS- and - version-specific default. It must be at least PKGSRC_GCC_VERSION. + - A user-settable variable PKGSRC_GXX_VERSION will declare the + version of gcc to be used for all C++ programs, again with an OS-, + version- and architeture-specific default. It must be at least + PKGSRC_GCC_VERSION. + + - If PKGSRC_GCC_VERSION and PKGSRC_GXX_VERSION are not set, the + system will behave much as before. As a possible exception, + builds may still fail if the required version is greater than the + base system version. So far the only known reason to avoid + setting these variable is if pkgsrc gcc cannot be built. - Each of c99, c++, c++11, and c++14 will be associated with a minimum gcc version, such that almost all programs declaring that @@ -95,8 +116,8 @@ most of them are too complicated. package building will fail. We call the resulting PKGSRC_GCC_VERSION or PKGSRC_GXX_VERSION the chosen version. - - When building a program using C or C++, the chosen version is not - provided by the base system, and the chosen version is not + - When building a program using C or C++, if the chosen version is + not provided by the base system, and the chosen version is not installed via pkgsrc, then it (and its dependencies) will be built from pkgsrc in a special bootstrap mode. When building in bootstrap mode, the version selection logic is ignored and the @@ -121,13 +142,23 @@ most of them are too complicated. approach is possible inconsistency with gcc's dependencies being built with the base compiler and used later. + As an alternative, we store lists of bootstrap packages in a + variable, because it will vary with OS and version, and with + PREFER_PKGSRC settings. + + As a third alternative, we pass a GCC_BOOTSTRAPPING variable + recursively. This is easier but less consistent. + + - We hope that the chosen version can be built using the base system + version, and hope to avoid multi-stage bootstrapping. + - We expect that any program containing C++ will undergo final linking with a C++ compiler. This is not a change from the current situation. ## Remaining issues -### gcc dependencies +### gcc dependencies introduction Because gcc can have dependencies, there could be packages built with the system compiler that are then later used with the chosen version. @@ -135,30 +166,98 @@ For now, we defer worrying about these p will be less serious than the current situation where all c++11 programs fail to build on NetBSD 6). +\todo: Perhaps change gcc 4.8 and 4.9 to enable gcc-inplace-math by +default. Perhaps decide that if we want to build gcc, we want to +build 5 or 6, and 4.9 is no longer of interest as a bootstrap target. + \todo: Analyze what build-time and install-time dependencies actually -exist. +exist. Include old GNU/Linux in this analysis. + +\todo: Consider if dropping nls would help. (On NetBSD, it seems that +base system libraries are used, so it would not help.) + +\todo: Consider failing if optins that we want one way are another, +when bootstrapping. + +### managing gcc dependencies + +There are multiple paths forward. + +\todo Choose one. Straw proposal is "Don't worry" and recursive +variable for the initial implementation. + +#### Separate prefix + +Build compilers in a separate prefix, or a subprefix, so that the +compiler and the packages needed to build it will not be used by any +normal packages. This completely avoids the issue of building a +package one way in bootstrap and another not in bootstrap, at the cost +of two builds and writing the separate-prefix code. + +#### Don't worry + +Don't worry that packages used to bootstrap the needed compiler are +compiled with an older compiler. Don't worry that they might be +different depending on build order. If we have an actual problem, +deal with it. This requires choosing an approach to omit compiler +selection logic when building the compiler: + +##### Mark bootstrap packages -\todo: Discuss adjusting options to minimize dependencies, including -gcc-inplace-math and nls. +Mark packages used to build gcc as PKGSRC_GCC_BOOTSTRAP=yes. +Conditionalize this on OPSYS if necessary. Don't force the compiler +if this is set. + +Alternatively, manage a per-OS list of packages in a central mk file. + +##### Pass a recursive variable + +As above, but set PKGSRC_GCC_BOOTSTRAP=yes in the evniroment of the +call to build the compiler, so that all dependencies inherit +permission to skip compiler selection logic. (Alternatively, use some +other mechanism such as passing a make variable explicitly.) + +### Differing GCC and GXX versions + +Perhaps it is a mistake to allow the chosen GCC and GXX versions to +differ. If we require them to be the same, then essentially all +systems with a base system compiler older than gcc 5 will have to +bootstrap the compiler. For now, we allow them to differ and will +permit the defaults to differ. + +### gcc versions and number of buildable packages + +A gcc version that is too old will not build a number of packages. +Anything older than 4.8 fails for c++11. 4.8 fails on some c++11 +packages, such as firefox and glibmm. + +A version that is too new also fails to build packages. Analyses +posted to tech-pkg indicate that 5 is close to 4.9 in the number of +packages built, and that moving to 6 causes hundreds of additional +failures. + +Therefore, the current answer to "What is the best version to use" is +5. ### Default versions for various systems -Note that if any particular system (or bulk build), a newer gcc has to -be built, it does not hurt incrementally to have built it earlier. +Note that if for any particular system's set of installed packages (or +bulk build), a newer gcc has to be built, it does not hurt to have +built it earlier. When the base system is old (e.g., gcc 4.5 in NetBSD 6, or 4.1, in NetBSD 5), then it is clear that a newer version must be built. For these, PKGSRC_GXX_VERSION should default to a newish gcc, avoiding -being so new as to cause building issues. Currently, gcc6 is probably -a good choice. PKGSRC_GCC_VERSION should probably default to the -system version if it can build C99, or match PKGSRC_GXX_VERSION, if -the system version is too old. Perhaps gcc 4.5 would be used, but 4.1 -not used. \todo Discuss. +being so new as to cause building issues. Currently, gcc5 is probably +a good choice, with gcc6 compiling significantly but not vastly fewer +packages. PKGSRC_GCC_VERSION should probably default to the system +version if it can build all C99 programs, or match PKGSRC_GXX_VERSION, +if the system version is too old. Perhaps gcc 4.5 would be used, but +4.1 not used. \todo Discuss. When the base system is almost new enough, the decision about the default is more complicated. A key example is gcc 4.8, found in -NetBSD 7. Firefox requires gcc 4.9 (\todo because the c++11 support -in 4.8 is not quite good enough), and all programs using c++14 also +NetBSD 7. Firefox requires gcc 4.9, and all programs using c++14 also need a newer version. One options is to choose 4.8, resulting in firefox failing, as well as all c++14 programs. Another is to choose 4.9, but this makes little sense because c++14 programs will still @@ -166,22 +265,55 @@ fail, and the general rule of moving to generally-acceptable version applies, which currently leads to gcc6. This is in effect a declaration that "almost new enough" does not count as new enough. Thus the plan for NetBSD 7 is to set -PKGSRC_GCC_VERSION to 4.8 and PKGSRC_GXX_VERSION to 6. +PKGSRC_GCC_VERSION to 4.8 and PKGSRC_GXX_VERSION to 5. -When the base system is new, e.g. gcc 5 or gcc 6 it should simply be -used. By "new enough", we mean that almost no programs in pkgsrc fail -to build with it, which implies that it supports (almost all) C++14 -programs. Our current definiton of new enough is gcc 5. +When the base system is new enough, e.g. gcc 5, 6 or 7 it should +simply be used. By "new enough", we mean that almost no programs in +pkgsrc fail to build with it (because it is too old), which implies +that it supports (almost all) C++14 programs. Our current definiton +of new enough is gcc 5. + +### Limited mixed versions + +One approach would be to allow limited mixed versions, where +individual programs could force a specific version to be bootstrapped +and used, so that e.g. firefox could use 4.9 even though most programs +use 4.8, which is what happens now on NetBSD 7. This would rely on +being able to link c++ with 4.9 including some things built with 4.8 +(which is done presently). However, this approach would become +unsound with a library rather than an end program. We reject this as +too much complexity for avoiding building a newer compiler in limited +situations. ### Fortran Fortran support is currently somewhat troubled.. It seems obvious to extend to PGKSRC_GFORTRAN_VERSION, and have that match PKGSRC_GCC_VERSION or PKGSRC_GXX_VERSION, but the Fortran situation is -not worsened by the above design. \todo Discuss. +not worsened by the above design. + +When building a gcc version, we get gfortran. Perhaps, because of +fortran, we should require a single version, vs a C and a C++ version. + +\todo Discuss. + +### C++ libraries used by C programs + +The choice of one version for C++ and one for C (e.g. 5, 4.8 on +netbsd-7) breaks down if a C program links against a library that is +written in C++ but provides a C API, because we still need the C++ +version's stdlib. + +\todo Define a variable for such packages to have in their buildlink3, +which will not add c++ to USE_LANGUAGES but will force +PKGSRC_GXX_VERSION to be used. Or decide that this is a good reason +to really just have one compiler version. ## Path forward +(This assumes per-package marking of bootstrap packages, but is +reasonably obviously extended to the other schemes.) + - Modify all gcc packages to have minimal dependencies, and to add PKGSRC_GCC_BOOTSTRAP. @@ -191,10 +323,16 @@ not worsened by the above design. \todo - Modify the compiler selection logic for LANGUAGES= to fail if PKGSRC_GCC_VERSION/PKGSRC_GXX_VERSION is not new enough. - - Modify the compiler selection logic for GCC_REQD to fail if the - version of GCC/GXX is not new enough. + - Modify the compiler selection logic for GCC_REQD to fail if + PKGSRC_GCC_VERSION/PKGSRC_GXX_VERSION is not new enough. - Decide on defaults. The straw proposal is that PKGSRC_GCC_VERSION - is the base system version if >= 4.5 (or 4.4?), and otherwise 6, + is the base system version if >= 4.5 (or 4.4?), and otherwise 5, and that PKGSRC_GXX_VERSION is the base system version if >= 5, and - otherwise 6. + otherwise 5. Implement these in platform.mk as they are tested. + +### Later steps + + - Address fortran. Probably add PKGSRC_GFORTRAN_VERSION, after + determining how Fortran, C and C++ interact with library ABI + compatibility.