# Releng workflow for pullups, and policies
## Overview
This is one possible releng workflow. Yours, once you've
settled in, may differ. The goal is to illustrate the process of going
from a pullup request (generally formatted as a number of commit
emails, or links to commit emails, with diffs when the original
commits don't apply cleanly to the branch) to a completed pullup. Much
of the information in here is taken (often verbatim) from an email I (riz)
received from jmc@ when I joined releng in 2005, and has been updated
somewhat to reflect recent situations.
### Requirements
* NetBSD
* perl
* localsrc
(OK, so this should work on non-NetBSD. You really want to go there?)
## General operating policies
1. You do not process your own pullup requests. This is for sanity so another
set of eyes sees it. This is fairly major deal and we all need to abide by
it.
2. All pullup requests must come from an active developer. If you're not sure
if the person is a developer, check the People page on netbsd.org or ask
them. If they aren't a developer, kick it back to them (and close out
the request) and politely explain this must come through a developer. See
item 7 below for how to close a request.
3. Pullups need to conform to the guidelines given at
<http://www.netbsd.org/developers/releng/pullups.html>.
Properly formatted ones should be worked and improper ones can be worked
at your own discretion/mood (or kicked back to the sender for fixes).
4. The burden of basic testing (to verify the change is correct on the
branch for which it is requested) falls upon the requestor. In
those cases where it is clear the requestor has requested large
changes without adequate testing, don't be afraid to push back
politely. It's almost always easier to fix problems before
committing on the branch than it is to back out bad changes later.
5. Pullups should, when possible, be processed using the centrally-maintained
scripts in localsrc/releng/pullups/scripts, to keep the format as
standard as possible, and to minimize human error. It is not always
possible to use the scripts for all pullups; when a patch is required,
please conform as much as possible to standard formats for the
changelist (appended to doc/CHANGES-<rev>) and for the commit message.
Very large, intrusive, and non-automated pullups often require a lot
of manual intervention. If special circumstances arise where it's not
possible (or a ridiculous amount of work is required) to follow
convention, your best bet is to ask the other members of releng for
a consensus on how to proceed.
6. Changelist entries, even when generated by the scripts, should be edited
for brevity. No more than two or three lines per change in the usual
case, and please try to make it descriptive of the effect of the change.
In some cases, you may need to ask for help from the requestor if
you don't fully understand the effect yourself. Use complete
sentences, proper capitalization, note PRs in the format "PR#XXXX", and
SAs (when we have them) as "SA#XXXX".
7. When replying back to the pullup requestor, include their CHANGES entry
back in the email and then close out the request by adding
*REQ: resolve
to the email so req closes the ticket out. (note that any additional emails
referencing that ticket number will re-open it, so be careful if someone
replies and says "thanks" back to the pullup list). Note that you can
use req directly to close a ticket without sending email. See item 1 in
the workflow section below.
8. If a pullup doesn't work, needs more info, etc make sure to put it in the
"stalled" state so it's obvious it's been looked at but more work needs to
be done. (Also, when we're about to cut a minor release all open requests
that don't apply are stalled). Please make sure you note *why* a change
was stalled; when unstalling, note reasons as well.
9. In general, take ownership of a request using req before working on it,
so effort isn't duplicated. (With very straightforward, quick-to-handle
requests, this isn't strictly necessary, but it's good practice anyway)
## Workflow
I ([[riz]]) will describe my working setup; you may want to alter this to suit your
needs.
1. Req is the ticketing system used to track the releases we're doing and it's
installed on releng.netbsd.org in <code>/home/releng/req</code>. The important tools are in
<code>/home/releng/req/bin</code>:
<rev>-q - Show the current queue
<rev>-req - The main command processor to stall, resolve, etc
a request. Run <rev>-req -? for examples.
The current revs are:
9 - NetBSD 9 queue
8 - NetBSD 8 queue
pkgsrc - Current pkgsrc branch
Get the "req" commands set up to your liking. They are run on
releng.netbsd.org, and may either be run directly there, or via ssh.
What I generally do is set up a script in my path on the machine I
use for releng work that looks like this:
#!/bin/sh
ssh releng.netbsd.org $(basename $0) $*
...and link it to the various commands I wish to run, such as "9-req".
2. Create two directories: one for source trees (I like to keep one
for each supported branch checked out, to speed things up, but if space
is at a premium, you may want to adjust this) and one for the
files generated by the releng scripts. **NOTE**: the directory where
the generated files go is emptied every time the r-start script is
run, so DON'T put anything you might need later in there.
I use: <code>/usr/releng</code> and <code>~/releng</code> respectively. Whatever you choose,
make sure you adapt the instructions.
3. The <code>r-start</code> script (<code>localsrc/releng/pullups/scripts/r-start</code>) is the
one which actually gets called. It needs perl, and it needs to be
edited to set the values of <code>RELENG_TMPDIR</code> and <code>RELENG_SCRIPTS</code> to wherever
you locate them. I like to make a copy of <code>r-start</code> in my <code>/usr/releng</code>
dir and run it from there as <code>./r-start</code>; you may want it in your <code>$PATH</code>.
4. Check out your source trees with the proper branch tags.
$ cvs -q -d cvs.netbsd.org:/cvsroot co -P -r<BRANCH> src xsrc
(I generally rename these to "src7" and "xsrc7" (etc), and maintain
other branches similarly. YMMV)
5. Use (for example) "9-q" to list the queue for pullup-9; Take ownership
of the ticket you're going to work on. Let's use "9999" as the ticket
number:
$ 9-q
<lots of output; I want to work on 9999>
$ 9-req take 9999
6. Get the text of the ticket.
$ 9-req show 9999 > 9999 # put it in a file "9999" in /usr/releng
7. Find the login name of the developer *requesting the pullup* (not
necessarily the one who made the original change, though they're often
the same). Let's say it's me (riz). Assuming this is a standard pullup
(one or more commit emails, no patches necessary), use 'r-start' to
generate the working files:
$ ./r-start riz 9999 < 9999 # script wants the commit email on stdin
This generates a number of files in RELENG_TMPDIR (~/releng, in my case).
Look at these files; there's various useful stuff in there. Usually,
what you'll need are three files: cl, r-pullup, and r-commit. "cl" is
the changelist file which gets appended to doc/CHANGES-<rev>; it will
need to be edited for whitespace and content, but should have the basic
framework there. "r-pullup" is a script which uses 'cvs update -j' to
apply the changes to the branch, and "r-commit" performs the commit and
has the log message. You should check these over, but often they don't
require editing.
***NOTE***: with the current version of the scripts,
revisions can get pulled up out of order. If there are many commit
emails in the pullup request, you should edit r-pullup to make sure
that the ordering is correct, otherwise you may get unnecessary conflicts.
8. Assuming all is well, run the 'r-pullup' script FROM THE TOP LEVEL
SOURCE DIRECTORY OF THE BRANCH YOU'RE APPLYING IT TO. If this is
for netbsd-9, I would cd to "/usr/releng/src9" and run it there.
For netbsd-8, "/usr/releng/src8", etc. Make sure your sticky tags are
correct! Pullups in xsrc need to be treated separately; they're fairly
rare, so just keep an eye out for them, and handle them as seems
appropriate from what is generated in the r-pullup script. The only
exception is that r-pullup doesn't generate the "xsrc" prefix in
the "cl" file, and you will need to add that yourself.
You need to observe the output of r-pullup; if there are conflicts,
you should bounce the request, clean up your src dir (a manual process,
sorry) and stall the ticket until the requestor provides a patch.
**NOTE**: if the pullup request says that it doesn't apply cleanly,
it should include a patch. These need to be handled on a case-by-case
basis, but often it's enough to run 'r-start' to get the files
created to form a baseline; edit them as appropriate, but instead of
running the r-pullup script, just apply the patch instead. Be sure to
add "via patch" to revisions listed in the commit message and CHANGES
entry.
9. Assuming there are no conflicts (you may want to run a 'cvs -q update -dP'
to check, especially early on), you can commit the changes. In the
basic case, it's just a matter of running the "r-commit" script.
If you commit anything by hand, make sure the commit log has a
well-formed message.
10. Append the changes file ("cl") to doc/CHANGES-<rev>. For example,
doc/CHANGES-9.1 is the currently-in-use log for post-9.0 but pre-9.1
changes that we're working on. Each branch has a different file.
11. Commit the changes file. In the log message, note which ticket you
applied.
12. Reply to the requestor, and close the ticket.
That's basically it; if you perform a complicated pullup (or series of them),
you may want to keep an eye on the [autobuild](http://releng.netbsd.org/cgi-bin/builds.cgi), to see if there were any
problems introduced, and correct them.