File:  [NetBSD Developer Wiki] / wikisrc / releng / workflow.mdwn
Revision 1.1: download - view: text, annotated - select for diffs
Sat Mar 16 22:13:24 2013 UTC (13 months ago) by riz
Branches: MAIN
CVS tags: HEAD
Move workflow.mdwn to releng directory.

# Releng workflow for pullups, and policies

## Overview

This is one possible releng workflow.  Yours, once you've
settled in, may differ.  The goal is to illustrate the process of going
from a pullup request (generally formatted as a number of commit
emails, or links to commit emails, with diffs when the original
commits don't apply cleanly to the branch) to a completed pullup.  Much
of the information in here is taken (often verbatim) from an email I (riz)
received from jmc@ when I joined releng in 2005, and has been updated
somewhat to reflect recent situations.

### Requirements

* NetBSD
* perl
* localsrc

(OK, so this should work on non-NetBSD.  You really want to go there?)

## General operating policies

1. You do not process your own pullup requests. This is for sanity so another
   set of eyes sees it. This is fairly major deal and we all need to abide by
   it.

2. All pullup requests must come from an active developer. If you're not sure
   if the person is a developer, check the People page on netbsd.org or ask  
   them. If they aren't a developer, kick it back to them (and close out     
   the request) and politely explain this must come through a developer.  See
   item 7 below for how to close a request.

3. Pullups need to conform to the guidelines given at
   <http://www.netbsd.org/developers/releng/pullups.html>.
Properly formatted ones should be worked and improper ones can be worked
at your own discretion/mood (or kicked back to the sender for fixes).

4. The burden of basic testing (to verify the change is correct on the
   branch for which it is requested) falls upon the requestor.  In
   those cases where it is clear the requestor has requested large
   changes without adequate testing, don't be afraid to push back
   politely.  It's almost always easier to fix problems before
   committing on the branch than it is to back out bad changes later.

5. Pullups should, when possible, be processed using the centrally-maintained
   scripts in localsrc/releng/pullups/scripts, to keep the format as
   standard as possible, and to minimize human error.  It is not always
   possible to use the scripts for all pullups;  when a patch is required,
   please conform as much as possible to standard formats for the
   changelist (appended to doc/CHANGES-<rev>) and for the commit message.
   Very large, intrusive, and non-automated pullups often require a lot
   of manual intervention.   If special circumstances arise where it's not
   possible (or a ridiculous amount of work is required) to follow
   convention, your best bet is to ask the other members of releng for
   a consensus on how to proceed.

6. Changelist entries, even when generated by the scripts, should be edited
   for brevity.  No more than two or three lines per change in the usual
   case, and please try to make it descriptive of the effect of the change.
   In some cases, you may need to ask for help from the requestor if
   you don't fully understand the effect yourself.  Use complete
   sentences, proper capitalization, note PRs in the format "PR#XXXX", and
   SAs (when we have them) as "SA#XXXX".

7. When replying back to the pullup requestor, include their CHANGES entry
   back in the email and then close out the request by adding

        *REQ: resolve
   to the email so req closes the ticket out. (note that any additional emails
   referencing that ticket number will re-open it, so be careful if someone
   replies and says "thanks" back to the pullup list).  Note that you can
   use req directly to close a ticket without sending email.  See item 1 in
   the workflow section below.

8. If a pullup doesn't work, needs more info, etc make sure to put it in the
   "stalled" state so it's obvious it's been looked at but more work needs to
   be done. (Also, when we're about to cut a minor release all open requests
   that don't apply are stalled).  Please make sure you note *why* a change
   was stalled;  when unstalling, note reasons as well.

9. In general, take ownership of a request using req before working on it,
   so effort isn't duplicated.  (With very straightforward, quick-to-handle
   requests, this isn't strictly necessary, but it's good practice anyway)

## Workflow

I ([[riz]]) will describe my working setup;  you may want to alter this to suit your
needs.

1. Req is the ticketing system used to track the releases we're doing and it's
   installed on releng.netbsd.org in <code>/home/releng/req</code>.   The important tools are in
   <code>/home/releng/req/bin</code>:

        <rev>-q              - Show the current queue
        <rev>-req            - The main command processor to stall, resolve, etc
                               a request. Run <rev>-req -? for examples.

   The current revs are:

        5                    - NetBSD 5 queue
        6                    - NetBSD 6 queue
        pkgsrc            - Current pkgsrc branch

   Get the "req" commands set up to your liking.  They are run on 
   releng.netbsd.org, and may either be run directly there, or via ssh.
   What I generally do is set up a script in my path on the machine I
   use for releng work that looks like this:
        #!/bin/sh

        ssh releng.netbsd.org $(basename $0) $*

   ...and link it to the various commands I wish to run, such as "5-req".

2. Create two directories:  one for source trees (I like to keep one
   for each supported branch checked out, to speed things up, but if space
   is at a premium, you may want to adjust this) and one for the
   files generated by the releng scripts.  **NOTE**:  the directory where
   the generated files go is emptied every time the r-start script is
   run, so DON'T put anything you might need later in there.

   I use:  <code>/usr/releng</code> and <code>~/releng</code> respectively.  Whatever you choose,
   make sure you adapt the instructions.

3. The <code>r-start</code> script (<code>localsrc/releng/pullups/scripts/r-start</code>) is the
   one which actually gets called.  It needs perl, and it needs to be
   edited to set the values of <code>RELENG_TMPDIR</code> and <code>RELENG_SCRIPTS</code> to wherever
   you locate them.  I like to make a copy of <code>r-start</code> in my <code>/usr/releng</code>
   dir and run it from there as <code>./r-start</code>;  you may want it in your <code>$PATH</code>.

4. Check out your source trees with the proper branch tags.  

        $ cvs -q -d cvs.netbsd.org:/cvsroot co -P -r<BRANCH> src xsrc

   (I generally rename these to "src6" and "xsrc6" (etc), and maintain
    "src51" "src52"  and "src60" as well.  YMMV)

5. Use (for example) "6-q" to list the queue for pullup-6;  Take ownership
   of the ticket you're going to work on.  Let's use "9999" as the ticket
   number:

        $ 6-q
        <lots of output;  I want to work on 9999>
        $ 6-req take 9999
   
6. Get the text of the ticket.

        $ 6-req show 9999 > 9999   # put it in a file "9999" in /usr/releng
   
7. Find the login name of the developer *requesting the pullup* (not
   necessarily the one who made the original change, though they're often
   the same).  Let's say it's me (riz).  Assuming this is a standard pullup
   (one or more commit emails, no patches necessary), use 'r-start' to
   generate the working files:

        $ ./r-start riz 9999 < 9999    # script wants the commit email on stdin

   This generates a number of files in RELENG_TMPDIR (~/releng, in my case).
   Look at these files;  there's various useful stuff in there.  Usually,
   what you'll need are three files:  cl, r-pullup, and r-commit.  "cl" is
   the changelist file which gets appended to doc/CHANGES-<rev>;  it will
   need to be edited for whitespace and content, but should have the basic
   framework there.  "r-pullup" is a script which uses 'cvs update -j' to
   apply the changes to the branch, and "r-commit" performs the commit and
   has the log message. You should check these over, but often they don't
   require editing. 

   ***NOTE***:  with the current version of the scripts,
   revisions can get pulled up out of order.  If there are many commit
   emails in the pullup request, you should edit r-pullup to make sure
   that the ordering is correct, otherwise you may get unnecessary conflicts.

8. Assuming all is well, run the 'r-pullup' script FROM THE TOP LEVEL
   SOURCE DIRECTORY OF THE BRANCH YOU'RE APPLYING IT TO.  If this is
   for netbsd-5, I would cd to "/usr/releng/src5" and run it there.
   For netbsd-5-1, "/usr/releng/src51", etc.  Make sure your sticky tags are
   correct!  Pullups in xsrc need to be treated separately;  they're fairly
   rare, so just keep an eye out for them, and handle them as seems
   appropriate from what is generated in the r-pullup script.  The only
   exception is that r-pullup doesn't generate the "xsrc" prefix in
   the "cl" file, and you will need to add that yourself.

   You need to observe the output of r-pullup;  if there are conflicts,
   you should bounce the request, clean up your src dir (a manual process,
   sorry) and stall the ticket until the requestor provides a patch. 

   **NOTE**:  if the pullup request says that it doesn't apply cleanly, 
   it should include a patch.  These need to be handled on a case-by-case
   basis, but often it's enough to run 'r-start' to get the files 
   created to form a baseline;  edit them as appropriate, but instead of
   running the r-pullup script, just apply the patch instead.  Be sure to
   add "via patch" to revisions listed in the commit message and CHANGES
   entry.

9.  Assuming there are no conflicts (you may want to run a 'cvs -q update -dP'
    to check, especially early on), you can commit the changes.  In the
    basic case, it's just a matter of running the "r-commit" script.
    If you commit anything by hand, make sure the commit log has a
    well-formed message.

10. Append the changes file ("cl") to doc/CHANGES-<rev>.  For example,
    doc/CHANGES-5.2 is the currently-in-use log for post-5.1 but pre-5.2
    changes that we're working on.  Each branch has a different file.

11.  Commit the changes file.  In the log message, note which ticket you
     applied.

12.  Reply to the requestor, and close the ticket.


That's basically it;  if you perform a complicated pullup (or series of them),
you may want to keep an eye on the [autobuild](http://releng.netbsd.org/cgi-bin/builds.cgi), to see if there were any
problems introduced, and correct them.

CVSweb for NetBSD wikisrc <wikimaster@NetBSD.org> software: FreeBSD-CVSweb