Annotation of wikisrc/tracking_current.mdwn, revision 1.3
1.1 jdf 1: # Tracking NetBSD-current
5: [[!toc levels=2]]
7: ## Why track NetBSD-current?
9: The developers of NetBSD have made the current development sources
10: available to the public for several reasons. Overall, providing
11: NetBSD-current helps us to create a more stable, accessible system.
13: It makes it easier for people to become involved in the development of
14: NetBSD. Distributing the current development sources allows a greater
15: number of people to see where the system is going, and to become
16: involved with new features as they are implemented.
18: It also makes changes from users easier to integrate. If users make
19: changes against the current development sources, then virtually no
20: integration is needed to get them into the master source tree.
22: It also allows wider testing of the software as it is developed. Users
23: of NetBSD-current are encouraged to send in
24: [bug reports](http://netbsd.org/support/send-pr.html) about the current sources,
25: and that helps find and fix bugs. Because people are testing the software soon
26: after it's written, more bugs can be found and eliminated.
28: ## Things you need to remember
30: * People using NetBSD-current are strongly encouraged to subscribe to
31: the [current-users](http://netbsd.org/mailinglists/#current-users) mailing
32: list. The [source-changes](http://netbsd.org/mailinglists/#source-changes)
33: mailing list is also of interest.
35: * When upgrading to a more recent version of -current you should
1.2 jdf 36: *always* install and boot a new kernel before installing any new libs (unless
37: you are certain there have been no new system calls added, but do it anyway;
38: it's safer).In general the best approach is to try the new kernel before
39: anything else, and if you hit any problems see the entry in the
40: [Kernel FAQ](http://netbsd.org/docs/kernel/#problems_compiling_a_current_kernel).
1.1 jdf 41:
42: * When compiling a -current kernel, always remember to include the
1.3 ! sevan 43: `COMPAT_<lastrelease>` option (e.g., `COMPAT_80`). As current diverges from
! 44: the last stable release, compatibility code will be added, but it will only
1.1 jdf 45: be enabled if this option is present. At a bare minimum, you will need this
46: compatibility code for the time between booting the new kernel and finishing
47: your build via `build.sh`
49: ## Updating an existing system from a current snapshot
51: *Please remember to check
52: [src/UPDATING](http://ftp.netbsd.org/pub/NetBSD/NetBSD-current/src/UPDATING) for
53: quirks around certain specific changes.*
55: To quickly begin using current, start with a snapshot generated by release
56: engineering. The current status of each platform can be seen at
57: [NetBSD Autobuild](http://releng.NetBSD.org/cgi-bin/builds.cgi) and the
58: corresponding releases found in by date and platform.
60: 1. Hunt down to the desired `binary/sets` directory, and `mget *.tgz` files
61: into your favorite local administrative directory (for example,
62: `$HOME/current`); when limited by disk space and/or time, only
63: `kern-GENERIC`, `etc`, `base`, and `comp` (if you want a compiler) are
65: 2. Extract the desired kernel (usually `GENERIC`), copy it into (root) directory.
67: # cd /root
68: # tar -zxpf ~/kern-GENERIC.tgz
69: # ln -fh /netbsd /netbsd.old
70: # cp netbsd /netbsd.new
71: # ln -fh /netbsd.new /netbsd
73: > **Warning**: Don't extract any userland binary sets before rebooting your
74: > machine with the new kernel. Newer binaries might use new system calls an
75: > old running kernel doesn't support.
77: 3. Check if there are any other files which might also be required by a new
78: kernel. Again,
80: might mention possible quirks on daily changes.
82: The following items are typical files that possibly need to be updated:
84: 1. bootloader
86: Usually a machine specific bootloader passes several parameters to a
87: loaded kernel. If some new parameters have been added or some existing
88: APIs between bootloader and kernel are changed you might also have to
89: install new bootloader files for a new kernel to handle new features. A
90: method to update bootloader files is quite machine dependent, so check
91: boot(8) and installboot(8) man pages for details.
94: On i386 and amd64, if you are using FFSv1 for root file system on `wd0a`
95: (i.e. first ATA drive), typical commands to update bootloaders are:
97: # tar -C /tmp -zxf ~/base.tgz ./usr/mdec
98: # cp /tmp/usr/mdec/boot /
99: # installboot -v /dev/rwd0a /tmp/usr/mdec/bootxx_ffsv1
101: If you are using FFSv2 for root file system use the following commands
104: # tar -C /tmp -zxf ~/base.tgz ./usr/mdec
105: # cp /tmp/usr/mdec/boot /
106: # installboot -v /dev/rwd0a /tmp/usr/mdec/bootxx_ffsv2
108: Note `/usr/mdec/bootxx_ffsv1` and `/usr/mdec/bootxx_ffsv2` are primary
109: bootloaders which are file system dependent. `/usr/mdec/boot` is
110: secondary loader and it's file system independenet.
112: If you forget your root file system type (FFSv1 or FFSv2), you can check
113: it by dumpfs(8) command:
115: # dumpfs /dev/rwd0a | head -3
116: file system: /dev/rwd0a
117: format FFSv2
118: endian little-endian
120: 2. kernel modules
1.3 ! sevan 122: A new framework *kernel modules* was been introduced after netbsd-5 was
! 123: branched.
! 124: The kernel module files
1.1 jdf 125: will be loaded dynamically by the kernel to support various kernel
126: options (including file systems) on demand, rather than linking all
127: necessary (but possibly unused) object files into the kernel binary.
128: This means if you are trying to boot a new `GENERIC` kernel, you also
129: have to prepare new kernel module files for the new kernel.
131: To prepare new kernel module files, you can simply use a new `modules`
132: set file which has been prepared since September 2009:
134: # cd /
135: # tar -zxpf ~/modules.tgz
137: Note i386 port also provides `MONOLITHIC` kernel binary in
138: `kern-MONOLITHIC.tgz` set file since October 2009. The `MONOLITHIC`
139: kernel includes all necessary options in its kernel as well as 5.0 and
140: prior `GENERIC` kernels and it doesn't depend on kernel module files at
141: all. If you would just like to test new features of a new kernel without
142: updating kernel modules, using `MONOLITHIC` kernel is easier way for the
143: first and quick trial.
145: It's also a good idea to put an old `MONOLITHIC` kernel into `/` (root)
146: directory for emergency and recovery because if newer modules have some
147: fatal issue there is no easy way to specify an alternative path of old
148: module files to a modular'ized kernel (and you can't rename directories
149: without a working kernel).
151: 4. Reboot machine with the new kernel:
153: # shutdown -r now
155: 5. Make sure the new kernel boots and works properly. If your new kernel has
156: any trouble, you can recover it by loading the renamed old one. If you are
1.3 ! sevan 157: using a modularised GENERIC kernel as mentioned above, you might also have to
1.1 jdf 158: restore old kernel module files.
160: 6. Extract the matching base, and any other desirable feature sets **except
163: # cd /
164: # tar -zxpf ~/base.tgz
165: # tar -zxpf ~/comp.tgz
166: # ...
169: Don't forget to specify "p" option (preserve permissions) on tar(1) command
170: otherwise setuid'ed commands (like su(1)) won't work.
172: **Warning**: Extracting `etc.tgz` on the installed system will overwrite your
173: local settings.
175: 7. [Update](http://netbsd.org/docs/current/index.html#etcupdate) `/etc` as the
176: last step: postinstall(8) will first check and fix most things that can be
177: automated, and etcupdate(8) in the second step will ask on what to merge:
179: # /usr/sbin/postinstall -s ~/etc.tgz check
180: # /usr/sbin/postinstall -s ~/etc.tgz fix
181: # /usr/sbin/etcupdate -s ~/etc.tgz
182: # shutdown -r now
184: If you have the X sets installed (xbase, ...), you can repeat the
185: postinstall and etcupdate steps with xetc.tgz as argument before rebooting.
187: At this point, you are relatively current and ready to build your own current
190: ## Downloading current source
192: See the [Obtaining the sources](http://netbsd.org/docs/guide/en/chap-fetch.html)
193: section in the [[NetBSD Guide|guide/index]].
195: ## Building a release from source
197: See the [Crosscompiling NetBSD](http://netbsd.org/docs/guide/en/chap-build.html)
198: section in the [[NetBSD Guide|guide/index]].
200: ## Updating an existing system from source
202: See the [[Updating an existing system from sources|guide/updating]] section in
203: the [[NetBSD Guide|guide/index]].
205: ## Updating the configuration and startup files
1.2 jdf 207: See the
208: [[More details about the updating of configuration and startup files|guide/updating]]
209: section in the [[NetBSD Guide|guide/index]].
1.1 jdf 210:
211: ## What if I get an error?
213: If you try to build -current, either from a snapshot or an earlier -current, and
214: it doesn't work, don't panic. Try these steps:
216: 1. Read the
218: file from the release you're trying to build.
220: 2. Read the [current-users archive](http://mail-index.netbsd.org/current-users/)
221: for hints.
223: 3. Update again. You may have caught the repository in the middle of a commit
224: to several related files, or the problem might have already been fixed.
226: 4. If all else fails, send email to current-users explaining the problem.
227: Include the date, time, and method you used to get your -current sources, as
228: well as any local changes you've made. Then put in a **short** script that
229: includes the error messages you're getting. Somebody will probably fix the
230: problem momentarily.
232: ## Tracking NetBSD-current with anoncvs
234: See the
235: [Fetching by CVS](http://netbsd.org/docs/guide/en/chap-fetch.html#chap-fetch-cvs)
236: section in the [[NetBSD Guide|guide/index]].
238: ### To check out the sources from a certain date
240: $ cvs checkout -D 20020501-UTC src
242: ### To check out the sources from a certain branch
1.3 ! sevan 244: $ cvs checkout -rnetbsd-8-0 src
1.1 jdf 245:
248: for a description of the branches in the CVS repository.
250: ### Useful hints
252: * Do not use the cvs `-z` flag. The data stream gets out of sync,
253: leading to corruption on the client, or causing the client to hang
254: completely. The additional load is also hard on the cvs server.
255: * If you want to check out a certain branch of the tree, you may want
256: to take caution not to overwrite any existing directories by
257: creating a new directory for this branch:
259: $ cd /parent/dir/to/checkout/into
260: $ mkdir NewName-temp
261: $ cd NewName-temp
262: $ cvs checkout ... src
263: $ mv src ../NewName
264: $ cd ..
265: $ rmdir NewName-temp
267: * You will have to use objdirs in order for cvs updates to work correctly. If
268: you happen to get errors from cvs saying things like:
270: cvs [update aborted]: could not chdir to gnu/usr.bin/gdb/gdb: Not a directory
272: You should do a make cleandir and try again. Make sure to run make obj after
273: the cvs update.
275: * You can put switches for specific commands in a `.cvsrc` in your home
276: directory, and they will be automatically used. A sample `.cvsrc` would be:
1.3 ! sevan 278: cvs -q
! 279: checkout -P
! 280: update -dP
! 281: diff -upN
! 282: rdiff -u
! 283: release -d
1.1 jdf 284:
285: ## Importing and merging sources.
287: Sources are imported as follows:
289: $ cvs -d /misc/cvsrep import -I ! -I CVS netbsd netbsd current-date
291: `date` is replaced by the date of the SUP for tracking purposes. The
292: `-I ! -I CVS` options ensure that no file in the source tree is ignored
293: except 'CVS' directories. This is because some NetBSD source files have
294: extensions which are normally ignored by CVS. If there are any conflicts
295: with local patches the import command will report them and will describe
296: a command to merge the conflicts something like:
298: $ cvs checkout -jnetbsd:yesterday -jnetbsd netbsd
300: This merge command will correctly merge the imported NetBSD sources but
301: it will not handle the removal of files locally which have already been
302: removed by the SUP process. To do this the merge command would be:
304: $ cvs update -jprevious_import_tag -j current-date
306: `previous_import_tag` should be replaced with the name of the tag used
307: for the previous cvs import. `date` should be replaced with the current
308: date to yield the same tag used on the current import that has just been
311: The conflicts reported by the import command are potential conflicts.
312: These are usually merged by the update command but in some cases a real
313: conflict occurs. In these cases a manual merge of the conflicting lines
314: will be required. A real conflict will be reported in the cvs update
315: output as a `C` followed by a filename.
317: Merging conflicts manually is not a simple process but in most cases it
318: should be resolved by removing the local changes and making the file
319: like the original NetBSD source code.
321: CVS marks conflicts as follows:
324: code from local file
326: code from imported file
327: >>>>>> local revision number of newly imported revision
329: If the import reports no conflicts the checked out copy of the tree
330: should be updated in exactly the same way as for the conflicts case.
332: All update and checkout commands should be done in the directory where
333: the sources have been checked out. On my system this is
336: If this is the first import then there will be no sources checked out.
337: Assuming you wish to create the source tree in `/usr/src/netbsd` The
338: following commands will check out the source and no merge step is
341: $ cd /usr/src
342: $ cvs -d /misc/cvsrep checkout netbsd
344: ## Tagging a successful build
346: If the build completes successfully, and produces a working set of binaries, it
347: can be useful to tag the working sources. This allows rewinding to a working
348: build tree with a single CVS command in the event that the current tree becomes
349: unbuildable for any reason. This can be performed by issuing the following
352: $ cvs tag successful-build-BUILD date
354: ### Notes
1.2 jdf 355: * If the NetBSD customised version of CVS, which recognises *\$NetBSD\$*
1.1 jdf 356: markers in files, is not used, the NetBSD revision number of the file is
357: available for reference purposes when build problems occur.
358: * The sup/import/merge sequence described above is quite easily
359: automatable. The following Perl script automates this process.
363: # Script to SUP NetBSD-current, import it into CVS and merge it with
364: # any local changes.
366: # NOTES:
367: # This script does no error handling so is not really suitable for
368: # non-interactive use.
370: # This script has only been test with cvs-1.10.1 and cvs-1.9.18.
375: #run the sup into a perl stream
376: system "/usr/sbin/sup -zsv" ; # This may need to change for none
377: # current systems
378: # now import the new files into CVS
380: chdir $IMPORTROOT or die "Could not cd to $IMPORTROOT\n";
382: ($sec,$min,$hour,$mday,$mon,$year,$wday,$yday,$isdst) = localtime;
383: $date = localtime;
384: $shortdate = sprintf "%02d%02d%04d",$mday,$mon+1,1900+$year;
385: system "/usr/local/bin/cvs -d$CVSROOT import -I ! -m\"SUP Import $date\" netbsd netbsd current-$shortdate ";
387: # make the working directory the local NetBSD Tree
388: chdir $SRCROOT or die "Could not change to $SRCROOT directory\n";
390: # Now do the import.
391: $lastimport = `cat /usr/src/netbsd/.tag`; # `s are backquotes
392: $lastimport =~ s/\n//; # strip off any trailing newline in the string
393: system "/usr/local/bin/cvs update -j $lastimport -j
394: current-$shortdate ";
395: # Now write the current file into tag save file
396: open TAG,">$SRCROOT/.tag" or die "Could not open new tag file";
397: print TAG "current-$shortdate";
398: close TAG;
400: This script was written in Perl since it is the scripting tool which the
401: author has the most experience with. It should be fairly straightforward to
402: write a shell script to perform the same task.
404: * Techniques for tracking current with CVS have been discussed several times on
405: the NetBSD current-users mailing list. For alternative techniques try
406: searching the NetBSD mailing lists.
408: If you have any comments or suggestions please send them to
409: [Mike Pumford](mailto:email@example.com) (who maintains this
410: entry) or <www@NetBSD.org>.
412: ## Getting the whole repository
414: All the procedures described above allow you keeping your own changes in your
415: repository, which has its advantages if you develop your own software based on
416: NetBSD. If you don't want to maintain your own CVS repository, but just want to
417: mirror NetBSD's CVS repository, there are three ways to do so.
419: Each of the methods described briefly below will get you a copy of the NetBSD
420: CVS repository (i.e. the RCS ,v files, not the checked out files!). You can then
421: setup your own anoncvs server or check out to a local harddisk. It's also useful
422: for fast access to the history information stored in the repository.
424: The methods to retrieve the whole repository are:
426: * **sup**
428: If you use sup already to mirror other parts of the NetBSD source,
429: you will want to add the following lines to your sup config file:
431: anoncvs release=all host=sup.NetBSD.org hostbase=/ftp/pub \
432: base=/usr prefix=/usr backup use-rel-suffix compress
434: After that, run `sup /path/to/supfile anoncvs` to retrieve the files.
1.3 ! sevan 436: Some example sup files are available in `/usr/share/examples/supfiles`.
1.1 jdf 437:
438: * **rsync**
440: Note that rsync puts quite a heavy load on our rsync server, and as such the
441: number of concurrent rsync users is restricted. If you still want to try
442: rsync, the command to retrieve the repository is:
444: $ rsync -v -a --delete --exclude '#cvs.lock' rsync://anoncvs.NetBSD.org/cvsroot/src .
446: Please see our [list of rsync mirrors](http://netbsd.org/mirrors/#rsync)!
CVSweb for NetBSD wikisrc <wikimaster@NetBSD.org> software: FreeBSD-CVSweb