Tracking NetBSD-current

Contents

  1. Tracking NetBSD-current
    1. Why track NetBSD-current?
    2. Things you need to remember
    3. Updating an existing system from a current snapshot
    4. Downloading current source
    5. Building a release from source
    6. Updating an existing system from source
    7. Updating the configuration and startup files
    8. What if I get an error?
    9. Tracking NetBSD-current with anoncvs
    10. Importing and merging sources.
    11. Tagging a successful build
    12. Getting the whole repository

Why track NetBSD-current?

The developers of NetBSD have made the current development sources available to the public for several reasons. Overall, providing NetBSD-current helps us to create a more stable, accessible system.

It makes it easier for people to become involved in the development of NetBSD. Distributing the current development sources allows a greater number of people to see where the system is going, and to become involved with new features as they are implemented.

It also makes changes from users easier to integrate. If users make changes against the current development sources, then virtually no integration is needed to get them into the master source tree.

It also allows wider testing of the software as it is developed. Users of NetBSD-current are encouraged to send in bug reports about the current sources, and that helps find and fix bugs. Because people are testing the software soon after it's written, more bugs can be found and eliminated.

Things you need to remember

Updating an existing system from a current snapshot

Please remember to check src/UPDATING for quirks around certain specific changes.

To quickly begin using current, start with a snapshot generated by release engineering. The current status of each platform can be seen at NetBSD Autobuild and the corresponding releases found in by date and platform.

  1. Hunt down to the desired binary/sets directory, and mget *.tgz files into your favorite local administrative directory (for example, $HOME/current); when limited by disk space and/or time, only kern-GENERIC, etc, base, and comp (if you want a compiler) are essential.
  2. Extract the desired kernel (usually GENERIC), copy it into (root) directory.

    # cd /root
    # tar -zxpf ~/kern-GENERIC.tgz
    # ln -fh /netbsd /netbsd.old
    # cp netbsd /netbsd.new
    # ln -fh /netbsd.new /netbsd
    

    Warning: Don't extract any userland binary sets before rebooting your machine with the new kernel. Newer binaries might use new system calls an old running kernel doesn't support.

  3. Check if there are any other files which might also be required by a new kernel. Again, src/UPDATING might mention possible quirks on daily changes.

    The following items are typical files that possibly need to be updated:

    1. bootloader

      Usually a machine specific bootloader passes several parameters to a loaded kernel. If some new parameters have been added or some existing APIs between bootloader and kernel are changed you might also have to install new bootloader files for a new kernel to handle new features. A method to update bootloader files is quite machine dependent, so check boot(8) and installboot(8) man pages for details.

      On i386 and amd64, if you are using FFSv1 for root file system on wd0a (i.e. first ATA drive), typical commands to update bootloaders are:

      # tar -C /tmp -zxf ~/base.tgz ./usr/mdec
      # cp /tmp/usr/mdec/boot /
      # installboot -v /dev/rwd0a /tmp/usr/mdec/bootxx_ffsv1
      

      If you are using FFSv2 for root file system use the following commands instead:

      # tar -C /tmp -zxf ~/base.tgz ./usr/mdec
      # cp /tmp/usr/mdec/boot /
      # installboot -v /dev/rwd0a /tmp/usr/mdec/bootxx_ffsv2
      

      Note /usr/mdec/bootxx_ffsv1 and /usr/mdec/bootxx_ffsv2 are primary bootloaders which are file system dependent. /usr/mdec/boot is secondary loader and it's file system independenet.

      If you forget your root file system type (FFSv1 or FFSv2), you can check it by dumpfs(8) command:

      # dumpfs /dev/rwd0a | head -3
      file system: /dev/rwd0a
      format  FFSv2
      endian  little-endian
      
    2. kernel modules

      A new framework kernel modules has been introduced after netbsd-5 was branched, and GENERIC kernel on i386 port has been switched to using the kernel module files since November 2008. The kernel module files will be loaded dynamically by the kernel to support various kernel options (including file systems) on demand, rather than linking all necessary (but possibly unused) object files into the kernel binary. This means if you are trying to boot a new GENERIC kernel, you also have to prepare new kernel module files for the new kernel.

      To prepare new kernel module files, you can simply use a new modules set file which has been prepared since September 2009:

      # cd /
      # tar -zxpf ~/modules.tgz
      

      Note i386 port also provides MONOLITHIC kernel binary in kern-MONOLITHIC.tgz set file since October 2009. The MONOLITHIC kernel includes all necessary options in its kernel as well as 5.0 and prior GENERIC kernels and it doesn't depend on kernel module files at all. If you would just like to test new features of a new kernel without updating kernel modules, using MONOLITHIC kernel is easier way for the first and quick trial.

      It's also a good idea to put an old MONOLITHIC kernel into / (root) directory for emergency and recovery because if newer modules have some fatal issue there is no easy way to specify an alternative path of old module files to a modular'ized kernel (and you can't rename directories without a working kernel).

      Warning: The infrastructure of kernel module files mentioned here is still under discussion on -current development. It could be changed at some point before the next 6.0 release and in that case the description in this section will be obsolete. Again, check src/UPDATING and currenet-users mailing list for updated information.

      There is a possible alternative structure for kernel modules which was proposed on May 2009, but we have not got any conclusion yet. This would be because most -current users build their own custom kernels from sources, but kernel modules might be rather useful for users who don't want to bother to compile their own kernels from sources to just try to use optinal functions. Anyway, any feedback about this brandnew kernel modules is quite appreciated.

  4. Reboot machine with the new kernel:

    # shutdown -r now
    
  5. Make sure the new kernel boots and works properly. If your new kernel has any trouble, you can recover it by loading the renamed old one. If you are using modular'ized GENERIC kernel mentioned above, you might also have to restore old kernel module files.

  6. Extract the matching base, and any other desirable feature sets except etc:

    # cd /
    # tar -zxpf ~/base.tgz
    # tar -zxpf ~/comp.tgz
    # ...
    

    Don't forget to specify "p" option (preserve permissions) on tar(1) command otherwise setuid'ed commands (like su(1)) won't work.

    Warning: Extracting etc.tgz on the installed system will overwrite your local settings.

  7. Update /etc as the last step: postinstall(8) will first check and fix most things that can be automated, and etcupdate(8) in the second step will ask on what to merge:

    # /usr/sbin/postinstall -s ~/etc.tgz check
    # /usr/sbin/postinstall -s ~/etc.tgz fix
    # /usr/sbin/etcupdate -s ~/etc.tgz
    # shutdown -r now
    

    If you have the X sets installed (xbase, ...), you can repeat the postinstall and etcupdate steps with xetc.tgz as argument before rebooting.

At this point, you are relatively current and ready to build your own current source.

Downloading current source

See the Obtaining the sources section in the NetBSD Guide.

Building a release from source

See the Crosscompiling NetBSD section in the NetBSD Guide.

Updating an existing system from source

See the Updating an existing system from sources section in the NetBSD Guide.

Updating the configuration and startup files

See the More details about the updating of configuration and startup files section in the NetBSD Guide.

What if I get an error?

If you try to build -current, either from a snapshot or an earlier -current, and it doesn't work, don't panic. Try these steps:

  1. Read the src/UPDATING file from the release you're trying to build.

  2. Read the current-users archive for hints.

  3. Update again. You may have caught the repository in the middle of a commit to several related files, or the problem might have already been fixed.

  4. If all else fails, send email to current-users explaining the problem. Include the date, time, and method you used to get your -current sources, as well as any local changes you've made. Then put in a short script that includes the error messages you're getting. Somebody will probably fix the problem momentarily.

Tracking NetBSD-current with anoncvs

See the Fetching by CVS section in the NetBSD Guide.

To check out the sources from a certain date

$ cvs checkout -D 20020501-UTC src

To check out the sources from a certain branch

$ cvs checkout -rnetbsd-5-0 src

See src/doc/BRANCHES for a description of the branches in the CVS repository.

Useful hints

Importing and merging sources.

Sources are imported as follows:

$ cvs -d /misc/cvsrep import -I ! -I CVS netbsd netbsd current-date

date is replaced by the date of the SUP for tracking purposes. The -I ! -I CVS options ensure that no file in the source tree is ignored except 'CVS' directories. This is because some NetBSD source files have extensions which are normally ignored by CVS. If there are any conflicts with local patches the import command will report them and will describe a command to merge the conflicts something like:

$ cvs checkout -jnetbsd:yesterday -jnetbsd netbsd

This merge command will correctly merge the imported NetBSD sources but it will not handle the removal of files locally which have already been removed by the SUP process. To do this the merge command would be:

$ cvs update -jprevious_import_tag -j current-date

previous_import_tag should be replaced with the name of the tag used for the previous cvs import. date should be replaced with the current date to yield the same tag used on the current import that has just been merged.

The conflicts reported by the import command are potential conflicts. These are usually merged by the update command but in some cases a real conflict occurs. In these cases a manual merge of the conflicting lines will be required. A real conflict will be reported in the cvs update output as a C followed by a filename.

Merging conflicts manually is not a simple process but in most cases it should be resolved by removing the local changes and making the file like the original NetBSD source code.

CVS marks conflicts as follows:

<<<<<<
  code from local file
======
  code from imported file
>>>>>> local revision number of newly imported revision

If the import reports no conflicts the checked out copy of the tree should be updated in exactly the same way as for the conflicts case.

All update and checkout commands should be done in the directory where the sources have been checked out. On my system this is /usr/src/netbsd.

If this is the first import then there will be no sources checked out. Assuming you wish to create the source tree in /usr/src/netbsd The following commands will check out the source and no merge step is required.

$ cd /usr/src
$ cvs -d /misc/cvsrep checkout netbsd

Tagging a successful build

If the build completes successfully, and produces a working set of binaries, it can be useful to tag the working sources. This allows rewinding to a working build tree with a single CVS command in the event that the current tree becomes unbuildable for any reason. This can be performed by issuing the following command:

$ cvs tag successful-build-BUILD date

Notes

If you have any comments or suggestions please send them to Mike Pumford (who maintains this entry) or www@NetBSD.org.

Getting the whole repository

All the procedures described above allow you keeping your own changes in your repository, which has its advantages if you develop your own software based on NetBSD. If you don't want to maintain your own CVS repository, but just want to mirror NetBSD's CVS repository, there are three ways to do so.

Each of the methods described briefly below will get you a copy of the NetBSD CVS repository (i.e. the RCS ,v files, not the checked out files!). You can then setup your own anoncvs server or check out to a local harddisk. It's also useful for fast access to the history information stored in the repository.

The methods to retrieve the whole repository are:

Add a comment