Return to gcc.mdwn CVS log

Up to [NetBSD Developer Wiki] / wikisrc / pkgsrc

Add discussion of gcc version selection

    1: Using gcc in pkgsrc
    2: 
    3: On many systems pkgsrc supports, gcc is the standard compiler.  In
    4: general, different versions of each OS have different gcc versions,
    5: and some packages require newer GCC versions, in order to support
    6: newer language standards (e.g. c++11, written in the style of
    7: USE_LANGUAGES), or because older versions don't work (infrequently).
    8: 
    9: This page discusses issues related to version selection, and intends
   10: to be a design document for how pkgsrc should address this problem, to
   11: be converted into historical design rationale once implemented.  It
   12: freely takes content from extensive mailinglist discussions, and
   13: attempts to follow the rough consensus that has emerged.
   14: 
   15: ## Base system gcc vs pkgsrc gcc
   16: 
   17: Systems using gcc (e.g. NetBSD) have a compiler as /usr/bin/gcc, and
   18: this is usable by pkgsrc without any bootstrapping activity.  One can
   19: build gcc versions (typically newer versions) from pkgsrc, resulting
   20: in a compiler within ${PREFIX}, e.g. /usr/pkg/gcc6/bin/gcc.  This
   21: compiler can then be used to compile other packages.
   22: 
   23: Issues with using base system gcc are typically that it is too old,
   24: such as gcc 4.5 with NetBSD 6, which cannot compile c++11.
   25: 
   26: Issues when using pkgsrc gcc are that
   27: 
   28:   - it must be bootstrapped, requiring compiling a number of packages
   29:     with the system compiler
   30:   - C++ packages that are linked together should be built with the
   31:     same compiler, because the standard library ABI is not necessarily
   32:     the same for each compiler version
   33:   - While C packages can be built with mixed versions, the binary
   34:     should be linked with the higher version because the support
   35:     library is backwards compatible but not forward compatible.
   36: 
   37: ## Specific constraints and requirements
   38: 
   39: This section attempts to gather all the requirements.
   40: 
   41:   - By default, pkgsrc should be able to build working packages, even
   42:     for packages that need a newer compiler than that provided in the
   43:     base system.
   44: 
   45:   - The set of packages that are needed when building a bootstrap
   46:     compiler should be minimized.
   47: 
   48:   - All packages that use C++ should be built with the same compiler version.
   49: 
   50:   - All packages that use C should have final linking with the highest
   51:     version used in any included library.
   52: 
   53:   - pkgsrc should avoid building gcc unless it is more or less
   54:     necessary to build packges.  (As an example, if the base system
   55:     gcc can build c99 but not c++11, building a c99-only program
   56:     should not trigger building a gcc version adequate for c++11.)
   57: 
   58:   - The compiler selection logic should work on NetBSD 6, and in-use
   59:     (including LTS) GNU/Linux systems.  It is desirable for this logic
   60:     to work on NetBSD 5.
   61: 
   62:   - The compiler selection logic should be understandable and not brittle.
   63: 
   64: ## Design
   65: 
   66: The above requirements could in theory be satisfied in many ways, but
   67: most of them are too complicated.
   68: 
   69:   - Packages declare what languages they need, with c++, c++11, and
   70:     c++14 being expressed differently.
   71: 
   72:   - The package-settable variable GCC_REQD will be used only when a
   73:     compiler that generally can compile the declared language version
   74:     is insufficient.  These cases are expected to be relatively rare.
   75: 
   76:   - A user-settable variable PKGSRC_GCC_VERSION will declare the
   77:     version of gcc to be used for C programs, with an OS- and
   78:     version--specific default.
   79: 
   80:   - A user-settable variable PKGSRC_GXX_VERSION will declare the version of gcc to
   81:     be used for all C++ programs, again with an OS- and
   82:     version-specific default.  It must be at least PKGSRC_GCC_VERSION.
   83: 
   84:   - Each of c99, c++, c++11, and c++14 will be associated with a
   85:     minimum gcc version, such that almost all programs declaring that
   86:     language can be built with that version.  (This avoids issues of
   87:     strict compliance with c++11, which requires a far higher version
   88:     of gcc than the version required to compile almost all actual
   89:     programs in c++11.)
   90: 
   91:   - The minimum version inferred from the language tag will be
   92:     combined with any GCC_REQD declarations to find a minimum version
   93:     for a specific package.  If that is greater than
   94:     PKGSRC_GCC_VERSION (programs using only C) or PKGSRC_GXX_VERSION,
   95:     package building will fail.  We call the resulting
   96:     PKGSRC_GCC_VERSION or PKGSRC_GXX_VERSION the chosen version.
   97: 
   98:   - When building a program using C or C++, the chosen version is not
   99:     provided by the base system, and the chosen version is not
  100:     installed via pkgsrc, then it (and its dependencies) will be built
  101:     from pkgsrc in a special bootstrap mode.  When building in
  102:     bootstrap mode, the version selection logic is ignored and the
  103:     base system compiler is used.  Consistency and reproducible builds
  104:     require that a package built with the normal prefix must be the
  105:     same whether built because of compiler bootstrapping or normal
  106:     use.
  107: 
  108:     There are thus two choices for dealing with bootstrapping.  One is
  109:     to use a distinct prefix, which will ensure that all packages that
  110:     are part of the compiler bootstrap will not be linked into normal
  111:     pkgsrc programs.  This implies that any dependencies of gcc may
  112:     exist twice, once in bootstrap mode and once if built normally.  A
  113:     gcc version itself will be built twice, if it is desired for
  114:     regular use.  This double building and the complexity of a second
  115:     prefix are the negatives of this approach.
  116: 
  117:     The other choice is to mark gcc and all depending packages as used
  118:     for compiler bootstrapping, and to always build those with the
  119:     base compiler.  We use the package-settable variable
  120:     PKGSRC_GCC_BOOTSTRAP=yes to denote this.  The negative with this
  121:     approach is possible inconsistency with gcc's dependencies being
  122:     built with the base compiler and used later.
  123: 
  124:   - We expect that any program containing C++ will undergo final
  125:     linking with a C++ compiler.  This is not a change from the
  126:     current situation.
  127: 
  128: ## Remaining issues
  129: 
  130: ### gcc dependencies
  131: 
  132: Because gcc can have dependencies, there could be packages built with
  133: the system compiler that are then later used with the chosen version.
  134: For now, we defer worrying about these problems (judging that they
  135: will be less serious than the current situation where all c++11
  136: programs fail to build on NetBSD 6).
  137: 
  138: \todo: Analyze what build-time and install-time dependencies actually
  139: exist.
  140: 
  141: \todo: Discuss adjusting options to minimize dependencies, including
  142: gcc-inplace-math and nls.
  143: 
  144: ### Default versions for various systems
  145: 
  146: Note that if any particular system (or bulk build), a newer gcc has to
  147: be built, it does not hurt incrementally to have built it earlier.
  148: 
  149: When the base system is old (e.g., gcc 4.5 in NetBSD 6, or 4.1, in
  150: NetBSD 5), then it is clear that a newer version must be built.  For
  151: these, PKGSRC_GXX_VERSION should default to a newish gcc, avoiding
  152: being so new as to cause building issues.  Currently, gcc6 is probably
  153: a good choice.  PKGSRC_GCC_VERSION should probably default to the
  154: system version if it can build C99, or match PKGSRC_GXX_VERSION, if
  155: the system version is too old.  Perhaps gcc 4.5 would be used, but 4.1
  156: not used.  \todo Discuss.
  157: 
  158: When the base system is almost new enough, the decision about the
  159: default is more complicated.  A key example is gcc 4.8, found in
  160: NetBSD 7.  Firefox requires gcc 4.9 (\todo because the c++11 support
  161: in 4.8 is not quite good enough), and all programs using c++14 also
  162: need a newer version.  One options is to choose 4.8, resulting in
  163: firefox failing, as well as all c++14 programs.  Another is to choose
  164: 4.9, but this makes little sense because c++14 programs will still
  165: fail, and the general rule of moving to the most recent
  166: generally-acceptable version applies, which currently leads to gcc6.
  167: This is in effect a declaration that "almost new enough" does not
  168: count as new enough.  Thus the plan for NetBSD 7 is to set
  169: PKGSRC_GCC_VERSION to 4.8 and PKGSRC_GXX_VERSION to 6.
  170: 
  171: When the base system is new, e.g. gcc 5 or gcc 6 it should simply be
  172: used.  By "new enough", we mean that almost no programs in pkgsrc fail
  173: to build with it, which implies that it supports (almost all) C++14
  174: programs.   Our current definiton of new enough is gcc 5.
  175: 
  176: ### Fortran
  177: 
  178: Fortran support is currently somewhat troubled..  It seems obvious to
  179: extend to PGKSRC_GFORTRAN_VERSION, and have that match
  180: PKGSRC_GCC_VERSION or PKGSRC_GXX_VERSION, but the Fortran situation is
  181: not worsened by the above design.  \todo Discuss.
  182: 
  183: ## Path forward
  184: 
  185:  - Modify all gcc packages to have minimal dependencies, and to add
  186:    PKGSRC_GCC_BOOTSTRAP.
  187: 
  188:  - Modify the compiler selection logic to do nothing if
  189:    PKGSRC_GCC_BOOTSTRAP is set.
  190: 
  191:  - Modify the compiler selection logic for LANGUAGES= to fail if
  192:    PKGSRC_GCC_VERSION/PKGSRC_GXX_VERSION is not new enough.
  193: 
  194:  - Modify the compiler selection logic for GCC_REQD to fail if the
  195:    version of GCC/GXX is not new enough.
  196: 
  197:  - Decide on defaults.  The straw proposal is that PKGSRC_GCC_VERSION
  198:    is the base system version if >= 4.5 (or 4.4?), and otherwise 6,
  199:    and that PKGSRC_GXX_VERSION is the base system version if >= 5, and
  200:    otherwise 6.