# Updating an existing system from sources

A common mechanism for upgrading a NetBSD system to a newer version is by
rebuilding the system from sources and installing the results. This works both
for stable releases such as [NetBSD 5.0](/releases/formal-5/) and for
NetBSD-current. In particular, if you are running a stable NetBSD release in a
production environment, you are encouraged to perform this procedure regularly
in order to incorporate any security fixes that have been applied to the branch
since its release.

There are a variety of ways of achieving the goal of rebuilding NetBSD from
source, and this chapter will guide you through the variety of options that are
available. The chapter starts by showing first what the manual procedure looks
like, and proceeds to describe some of automation tools that simplify the
process.

**Note**: Please remember to check
[src/UPDATING](http://cvsweb.NetBSD.org/bsdweb.cgi/src/UPDATING) for the latest
changes and special instructions that may be involved in upgrading the system.*

## Manual build and update procedure

Most of the following steps can be done as ordinary user. Only the installation
of a new kernel and the userland will require root privileges. Although `/usr`
is choosen as the working directory in the following examples, the procedure
can also take place in a user's home directory. Ordinary users have normally
not the permissions to make changes in `/usr`, but this can be changed by root.

Having up-to-date sources is a prerequisite for the following steps.
[[Fetching by CVS|guide/fetch#cvs] informs about the ways to retrieve or update
the sources for a release, stable or current branch (using CVS).

Please always refer to the output of **build.sh -h** and the files `UPDATING`
and `BUILDING` for details - it's worth it, there are *many* options that can
be set on the command line or in `/etc/mk.conf`

### Building a new userland

The first step is to build the userland:

    $ cd /usr/src
    $ ./build.sh -O ../obj -T ../tools -U distribution

### Building a new kernel

The next step will build the kernel:

    $ cd /usr/src
    $ ./build.sh -O ../obj -T ../tools kernel=<KERNEL>

### Installing the kernel and userland

Installing the new kernel, rebooting (to ensure that the new kernel works) and
installing the new userland are the final steps of the updating procedure:

    $ cd /usr/src
    $ su
    # mv /netbsd /netbsd.old
    # mv /usr/obj/sys/arch/<ARCH>/compile/<KERNEL>/netbsd /
    # shutdown -r now
     ...
    $ cd /usr/src
    $ su
    # ./build.sh -O ../obj -T ../tools -U install=/

If the new kernel `netbsd` does not boot successfully, you can fall back on
booting the `netbsd.old` kernel.

### Updating the system configuration files

Run the `etcupdate` script
([etcupdate(8)](http://netbsd.gw.com/cgi-bin/man-cgi?etcupdate+8+NetBSD-5.0.1+i386))
and follow the instructions in the output for fixing obsolete files:

    # /usr/sbin/etcupdate -s /usr/src

Optionally reboot to ensure all running services are using the new binaries:

    # shutdown -r now

### Summary

 1. From the root of the source tree:

        $ cd /usr/src

 2. Build the userland:

        $ ./build.sh -O ../obj -T ../tools -U -u distribution

 3. Build the kernel:

        $ ./build.sh -O ../obj -T ../tools -U -u kernel=GENERIC

 4. Install the kernel:

        $ cd ../obj/sys/arch/<ARCH>/compile/GENERIC
        $ su
        # mv /netbsd /netbsd.old
        # cp netbsd /netbsd

 5. Reboot into the new kernel:

        # shutdown -r now

 6. Install the new userland:

        $ cd /usr/src
        $ su
        # ./build.sh -O ../obj -T ../tools -U install=/

 7. Update the system and configuration files;:

        #  /usr/sbin/etcupdate -s /usr/src

**Note**: In the procedure above, the `-u` option indicates an update process,
and that a `make clean` operation should not be run before starting the build.
This is useful when doing an update from a previous build and/or a fresh build.
The `-U` option allows the entire build by a non-root user followed with an
install by root.

## Using sysinst

It is also possible to use `sysinst` to install a freshly built system. The
steps are as follows:

 1. Build a complete release:

        $ ./build.sh -O ../obj -T ../tools -U -u -x release

 2. The resulting install sets will be in the `/usr/obj/releasedir/` directory.
 3. Copy the install kernel to the root directory of your NetBSD system, reboot
    from it, and upgrade with `sysinst` (see
	[[Upgrading NetBSD|guide/upgrading]]).


## Using sysbuild and sysupgrade

The sysbuild and sysupgrade tools (currently available in
`pkgsrc/sysutils/sysbuild` and `pkgsrc/sysutils/sysupgrade` respectively)
automate the full process of rebuilding NetBSD from sources (*including the
retrieval of the sources from a CVS repository*) and installing the results
with minimal effort.

Both of these tools have configuration files to determine how to build a
release and how to install it. Among other things, these specify the CVS
repository to use, what architecture to build for, where to place the build
files and what steps to perform during an upgrade. The files can be found in
`/usr/pkg/etc/sysbuild/default.conf` and `/usr/pkg/etc/sysupgrade.conf`. The
default configuration of both tools should let you get started with minimal
effort.

In their simplest form, you can do a full NetBSD build and upgrade your system
to it by running these commands:

    # sysbuild build
    # sysupgrade auto ~/sysbuild/release/$(uname -m)

And that's all that it takes. These invocations will do the following:

 1. Download the source trees from CVS into `/usr/src` and `/usr/xsrc`. The
    latter is only fetched if your system has X11. And, if you already have
    the sources in your system, this will only update them to the newest
	version.
 2. Build a new release into `~/sysbuild/<machine>/`. This per-machine
    directory will include subdirectories like `obj`, `destdir`, etc. The
    build results will be left in `~/sysbuild/release/<machine>/`.
 3. Install a new kernel and unpack the new sets using the just-built release
    files.
 4. Run both etcupdate and postinstall to aid you in merging new configuration
    changes into your system.

For more details, please see the included `sysbuild(1)` and `sysupgrade(8)`
manual pages, as well as the comments in the referenced configuration files.

### Tweak: Building as non-root

The commands above depict the most basic and simple invocation of the tools
using the *default configuration files*. One drawback is that you require root
access during the build of the source tree so that sysbuild can upgrade the
source trees under `/usr/src` and `/usr/xsrc`. It is recommended that you avoid
building as root once you are familiar with the procedure, and this section
show what is needed to do so with sysbuild.

In order to build as non-root, you can either choose to store your source trees
out of `/usr` (easiest) or give permissions to your user to modify the trees
under `/usr` (good if you want to share the source tree with more than one
user).

If you want to store the source trees under your home directory, which is
convenient for development purposes, simply edit `/usr/pkg/etc/sysbuild.conf`
and add these settings:

    SRCDIR="${HOME}/sysbuild/src"
    [ ! -f /etc/mtree/set.xbase ] || XSRCDIR="${HOME}/sysbuild/xsrc"

Once this is done, the `sysbuild build` invocation show above should just work
under your unprivileged user. The upgrade procedure then becomes:

    $ sysbuild build
    ... become root ...
    # sysupgrade auto ~/sysbuild/release/$(uname -m)

The other alternative, in case you want to maintain your source trees in the
locations described by
[hier(7)](http://netbsd.gw.com/cgi-bin/man-cgi?hier+7+NetBSD-5.0.1+i386), is
to do the following as root:

    # mkdir -p /usr/src /usr/xsrc
    # chown -R <your-user>:wsrc /usr/src /usr/xsrc
    ... and optionally add <your-user> to wsrc in /etc/group ...

After this, the default configuration file of sysbuild will let you place the
files in these locations and let you do unprivileged builds.

**Note**: If you have an an encrypted home partition, or another "special"
filesystem you store your sources on, you should backup them somehwere you can
easily access them! In case of a failed build you might want to rebuild without
being able to access an encrypted partition.

### Tweak: Setting up nightly builds

The `pkgsrc/sysutils/sysbuild-user` package can be used to configure and
maintain an unprivileged system user to perform periodic (e.g. nightly) builds
from source. This can come in very handy to closely track NetBSD-current.

The installed user is appropriately named sysbuild, and is configured by
default to run a full system build overnight. The results are left in
`/home/sysbuild/release/<machine>/`, which is the convenient default of
sysupgrade's release directory. Any build failures will be reported to you by
email.

The behavior of sysbuild for this unprivileged user is configured in
`/home/sysbuild/default.conf`.

You can interact with sysbuild under this unprivileged user by running
commands of the form:

    # su - sysbuild /usr/pkg/bin/sysbuild ...

## More details about the updating of configuration and startup files

[etcupdate(8)](http://netbsd.gw.com/cgi-bin/man-cgi?etcupdate+8+NetBSD-current)
is a script to help users compare, merge and install new configuration and
startup files (files found in the `etc.tgz` distribution set) in `/dev`, `/etc`
and `/root` after performing an operating system upgrade. The upgrade of the
operating system could have been performed either by compiling sources or by
extracting the distribution binaries.

### Using etcupdate with source files

In case where the sources are in `/usr/src` the following command should be enough:

    # etcupdate

But what if your NetBSD sources are in an alternative location, such as in
`/home/jdoe/netbsd/src`? Don't worry, tell etcupdate the location of your
source tree with `-s srcdir` and it will work just fine:

    # etcupdate -s /home/jdoe/netbsd/src

### Using etcupdate with binary distribution sets

Sometimes it's not convenient to have the sources around but you still want to
update the configuration and startup files. The solution is to feed `etc.tgz`
(or `xetc.tgz`) to etcupdate via the `-s tgzfile` switch:

    # etcupdate -s /some/where/etc.tgz

### Using etcmanage instead of etcupdate

The `etcmanage` perl script (available from
[pkgsrc/sysutils/etcmanage](http://pkgsrc.se/sysutils/etcmanage) or as binary
package) is an alternative to etcupdate(8). It should be used in the following
way, in combination with
[postinstall(8)](http://netbsd.gw.com/cgi-bin/man-cgi?postinstall+8+NetBSD-5.0.1+i386):

    # /usr/pkg/bin/etcmanage
    # /usr/sbin/postinstall