[[!meta title="Symbol Versions in NetBSD Libraries"]]

NetBSD implements various standard C language interfaces such as the
[[!template id=man name="time" section="3"]] function in POSIX in
`libc`, which has a prototype like this:

    time_t time(time_t *);

However, between NetBSD 5 and NetBSD 6, the definition of the type
`time_t` in NetBSD changed on many architectures from 32-bit to 64-bit
to avoid the
[year 2038 problem](https://en.wikipedia.org/wiki/Year_2038_problem).
So programs compiled in NetBSD<=5 saw a declaration like

    int time(int *);

which on most architectures is 32-bit, while programs compiled in
NetBSD>=6 see a declaration like

    int64_t time(int64_t *);

These declarations are not compatible -- consider a program with a
fragment like:

    int before, after;

    time(&before);
    ...
    time(&after);

This would work in NetBSD<=5, but in NetBSD>=6, the calls to
[[!template id=man name="time" section="3"]] might overwrite adjacent
positions on the stack, or crash altogether because the argument is
misaligned.

Programs written and compiled on older versions of NetBSD are supposed
to continue to work -- with suitable emulators/compatNN packages and
compatNN.kmod modules or COMPAT_NN kernel options -- on newer versions
of NetBSD.

To make this work, NetBSD's `libc` provides _two_ symbols:

- `time`, which still implements the legacy prototype as before; and
- `__time50` (yes, this is not a typo for `__time60`), which implements
  the new 64-bit prototype.

The declaration in newer NetBSD
[time.h](https://nxr.netbsd.org/xref/src/include/time.h) is actually:

    time_t time(time_t *) __RENAME(__time50);

where `__RENAME(__time50)` is a
[macro](https://nxr.netbsd.org/search?q=&project=src&defs=__RENAME&refs=&path=&hist=)
expanding to `__asm("__time50")`, which has the effect that the
compiler will use the symbol `__time50` for calls to the C function
this declares.
Thus, old programs with calls to the symbol `time` using the 32-bit
prototype will continue to work, and new programs will be compiled to
call the symbol `__time50` using the 64-bit prototype.
([Details on how the symbols are implemented in `libc`.](https://nxr.netbsd.org/xref/src/lib/libc/README))

# [[!template id=man name="dlsym" section="3"]] and symbol interposition

**Programs that use
[[!template id=man name="dlsym" section="3"]],
such as C foreign function interfaces in dynamic languages like Python,
need to know that if they want the legacy 32-bit time() function, they
must use the symbol `time`, and if they want the modern 64-bit time()
function, they must use the symbol `__time50`.**

**Similarly, programs that use `LD_PRELOAD` (see
[[!template id=man name="ld.elf_so" section="3"]])
to interpose their own definitions of symbols, such as
[[!template id=man name="rumphijack" section="3"]]
and
[torsocks](https://gitlab.torproject.org/legacy/trac/-/wikis/doc/torsocks),
must know to define `__time50` if they want to replace the new
semantics in new programs, or `time` if they want to replace the old
semantics in old programs.**

The same applies to many other standard C functions, such as
[[!template id=man name="clock_gettime" section="3"]]
(`__clock_gettime50`) and
[[!template id=man name="socket" section="3"]]
(`__socket30`), which have all had their prototypes or semantics
revised at some point.

Symbol interposition is very difficult to get right, and it is hard to
make programs that do it reliably.
On NetBSD, it should be reserved for certain standard library functions
like `malloc` and `free` (and `calloc` and everything else in that
family), and some system call stubs; except for the `__...50`
pseudo-versioned renames of public functions, you should not try to
interpose your own definition of any symbol beginning with ‘`_`’ (a
single underscore), which is reserved to the implementation in C.

# Appendix: ELF symbol versions

The renaming scheme of `__time50` is informal -- any symbol can be
renamed the same way, and NetBSD uses it for some other purposes too,
such as exposing a slightly different
[[!template id=man name="rename" section="2"]]
function via the symbol `__posix_rename` in programs that define
`_POSIX_C_SOURCE` but not `_NETBSD_SOURCE`.

The GNU ELF toolchain (gcc, ld, &c.) supports a formal concept of
‘symbol versions’ with sections called `.gnu.version` (associating
versions with symbols), `.gnu.version_d` (versions defined in an
object), and `.gnu.version_n` (versions needed in an object).
As of 2020, NetBSD does not use ELF symbol versions, although the
linker and loader support them for libraries developed outside NetBSD.

The semantics is:

- When creating a library, a version map may be specified like so:

      NetBSD_BASE {
              global:
                      __time50;
                      free;
                      malloc;
                      time;
              local:
                      *;
      };

      NetBSD_6 {
              global:
                      time;
      };

  The library can specify what versioned symbol each definition in the
  library is exposed with:

      __asm(".symver time_legacy,time@NetBSD_BASE");
      int time_legacy(int *t) { ... }

      __asm(".symver time64,time@@NetBSD_6");         /* default version */
      int64_t time64(int64_t *t) { ... }

      __asm(".symver __time50,__time50@NetBSD_BASE");
      __typeof(time) __time50 __attribute__((__alias__("time64")));

  Versions marked with `@@` are _default_ versions; versions marked
  with `@` are non-default.

- When running a program that was linked _without_ ELF symbol versions,
  from before the library had ELF symbol versions (like `libc` today),
  the first version in the map is used to resolve symbols:

  - Old programs calling the legacy `time` symbol will get
    `time@NetBSD_BASE`, which is defined via `time_legacy` above.

  - Programs calling `__time50` will get `__time50@NetBSD_BASE`, which
    is defined via `time64` above.

- When linking a program against a library with symbol versions, the
  linker will record what the default version was; when later running
  the program, the stored symbol version will be used.
  If there is no default version, and the program did not request a
  specific version with `.symver`, then the linker refuses to link, so
  obsolete symbols can be ‘removed’ by giving them only non-default
  versions -- thus old programs continue to work but new programs can't
  be made that use the obsolete symbols.

  For example, if [[!template id=man name="time" section="3"]] is
  declared in a header file as simply

      typedef int64_t time_t;
      time_t time(time_t *);

  then new programs will be linked against `time@NetBSD_6`, which is
  the default version for the symbol name `time`.
  If NetBSD ever changed the prototype of
  [[!template id=man name="time" section="3"]]
  again, and defined a `time@NetBSD_11` as the new default version,
  existing programs compiled with `time@NetBSD_6` would continue to get
  the semantics they were built against.

  Unfortunately, setting only a non-default version doesn't work to
  compatibly remove obsolete symbols that never had versions in the
  first place.
  If a program was already linked with a reference to an unversioned
  symbol, the runtime loader will refuse to resolve that reference by
  the non-default version.

- When a program uses
  [[!template id=man name="dlsym" section="3"]],
  it always gets the default version, if any.
  Programs can request specific versions with
  [[!template id=man name="dlvsym" section="3"]].

## ELF symbol versions versus `__...50` pseudo-versions

ELF symbol versions and NetBSD's `__time50` pseudo-version renaming
scheme both try to address the same problem: making sure old programs
that were built under the assumption of the old semantics continue to
run unmodified with new libraries.

Both of them run into problems with
[[!template id=man name="dlsym" section="3"]]
and symbol interposition:

- A program written _today_ that expects to find the function time() in
  `libc`, such as a C foreign function interface for a dynamic language
  like Python, needs to know to call `dlsym("__time50")`; otherwise it
  will get an obsolete definition that does not match the semantics of
  the current definition of `time_t`, possibly leading to data
  corruption, crashes, or worse.

- If `libc` used used ELF symbol versions, then `dlsym("time")` would
  return the modern symbol.

  But any _old_ programs that used `dlsym("time")` assuming it
  returned the legacy definition (which was the ‘modern’ definition at
  the time the programs were written and built) will break if it
  instead returns the 64-bit definition.

  And if we ever modified
  [[!template id=man name="time" section="3"]]
  again (hypothetically, to extend it to 128-bit galactic-scale times),
  programs written assuming that `dlsym("time")` returns the 64-bit
  definition will break if it begins to return the 128-bit definition.
  Programs could future-proof themselves by using `dlsym("time",
  "NetBSD_6")` explicitly, but this is no better than writing
  `dlsym("__time50")` explicitly.

**Thus, switching from the pseudo-versions we use to ELF symbol
versions doesn't improve the
[[!template id=man name="dlsym" section="3"]]
situation -- in fact, it makes the situation _worse_, by breaking old
programs and providing no way for new programs to bind to the name of
the current version.**

Perhaps we could create a compiler builtin `__builtin_asm_name` which
would expand to the `__asm("...")` name by which a C identifier has
been declared -- then programs could instead do:

    __typeof(time) *timep = dlsym(dso, __builtin_asm_name(time));

This way the text of the program is the same no matter how
[[!template id=man name="time" section="3"]]
is declared in the header file, but it will continue to work across
changes to the signature of the
[[!template id=man name="time" section="3"]]
function in newer releases of NetBSD.

# References

- Jörg Sonnenberger,
  [How to break long-term compatibility in NetBSD](https://www.NetBSD.org/gallery/presentations/joerg/asiabsdcon2016/asiabsdcon2016.pdf),
  AsiaBSDcon 2016.

- Ulrich Drepper,
  [How To Write Shared Libraries](https://akkadia.org/drepper/dsohowto.pdf),
  2011-12-10.

- Ulrich Drepper,
  [ELF Symbol Versioning](https://akkadia.org/drepper/symbol-versioning)