Annotation of wikisrc/users/dholland/mercurial.mdwn, revision 1.2

1.1       dholland    1: ## Using Mercurial and mq to work on NetBSD
                      2: 
                      3: This page contains directions for using Mercurial as a commit buffer
                      4: for NetBSD.
                      5: 
                      6: (It will not do you much good if you're trying to convert the master
                      7: NetBSD tree to Mercurial or to work with such a converted tree.)
                      8: 
                      9: ### What it is
                     10: 
                     11: Mercurial is a distributed version control system ("DVCS").
                     12: mq is an extension to Mercurial for handling patch queues.
                     13: The concept of patch queues was introduced by Quilt some years back.
                     14: 
                     15: This document assumes you already know more or less how to use
                     16: Mercurial but may not have used mq before.
                     17: 
                     18: ### The model we're using here
                     19: 
                     20: What we're going to do is commit a NetBSD CVS working tree into a
                     21: Mercurial repository.
                     22: You can then use Mercurial to merge; it is better at this than CVS.
                     23: You can also commit changes locally and ship them back to the CVS
                     24: master later; this is useful in a variety of ways.
                     25: You can potentially also clone the Mercurial tree and work jointly
                     26: with other people, but there are limits to this as we'll discuss in a
                     27: moment.
                     28: 
                     29: Because the NetBSD tree is rather large, you will find that if you
                     30: commit the whole thing into Mercurial that a number of operations
                     31: (anything that scans the working tree for changes) become annoyingly
                     32: slow.
                     33: It isn't slow enough to be unusable, and it's quite a bit faster than a
                     34: comparable CVS option (like running cvs update from the top level),
                     35: but it's slow enough to be annoying.
                     36: 
                     37: For this reason, in most cases, I recommend committing only part of
                     38: the tree into Mercurial and telling it to ignore the rest. This means
                     39: you can't in general usefully clone the resulting Mercurial repo
                     40: (although that depends on exactly what you leave out) but this is not
                     41: a major problem unless you're specifically trying to work with someone
                     42: else.
                     43: 
                     44: So the basic model here is that you check out a CVS working tree and
                     45: use Mercurial to manage local changes to part of it, then later on
                     46: commit those changes back to the master CVS repository.
                     47: 
                     48: ### Branches vs. patches
                     49: 
                     50: There are two ways you can manage your changes: as a branch, or as a
                     51: patch queue.
                     52: The advantage of a patch queue is that you can easily commit each
                     53: patch individually back to CVS, and you can go back and forth between
                     54: them and debug and polish each one separately.
                     55: The disadvantage is that the merge facilities are (as far as I know
                     56: anyway) relatively limited.
                     57: 
                     58: Conversely, if you commit your changes to a branch, you get all the
                     59: native merging support in Mercurial.
                     60: However, it is painful to try to commit anything other than one big
                     61: diff for the whole branch back to CVS.
                     62: (You might be able to do it via bookmarks and rebasing, but I've never
                     63: tried and have no desire to figure out how.)
                     64: 
                     65: If you don't want to keep the incremental history of your local
                     66: commits, use a branch.
                     67: If you do, use a patch queue.
                     68: 
                     69: It is possible to use multiple branches to allow you to commit back in
                     70: several stages.
                     71: However, managing this is a major pain and I don't recommend it -- you
                     72: might get away with two branches but more than that is probably a bad
                     73: idea.
                     74: 
                     75: There's a Mercurial extension called the "patch branch extension" that
                     76: lets you manage a whole graph of patches using branches.
                     77: I haven't tried using it in some years; at the time it had scaling
                     78: problems such that it became horrifyingly slow once you had more than
                     79: a handful of such branches.
                     80: That might have been improved in the meantime; if you find yourself
                     81: wanting to use both branches and patches, it might be worth looking
                     82: into.
                     83: 
                     84: It is also fairly probable that there is now a solution for merging
                     85: with patch queues; it's been a while since I had time to look closely.
                     86: 
                     87: ### Setting up
                     88: 
                     89: First, check out a CVS working tree.
                     90: You probably want to use a different one for each project, because
                     91: different projects require changing different parts of the tree and so
                     92: you will probably want to have Mercurial ignore different subtrees for
                     93: different projects.
                     94: (At least, I find it so; it depends on what you're working on.)
                     95: 
1.2     ! dholland   96:        % cvs -d cvs.netbsd.org:/cvsroot checkout -P src
1.1       dholland   97: 
                     98: Now create a Mercurial repository at the top level.
                     99: (If you are working only in a subtree and you are *sure* that you will
                    100: never need to change anything in other parts of the tree, you can
                    101: create the Mercurial repository in a subtree.
                    102: But unless you're absolutely certain, don't take the risk.)
                    103: 
                    104:        % cd src
                    105:        % hg init
                    106: 
                    107: If you're going to be using a patch queue, now enable mq.
                    108: 
                    109:        % vi .hg/hgrc
                    110: and add
                    111:        [extensions]
                    112:        hgext.mq =
                    113: (Since the extension is built into Mercurial, that's all you need.)
                    114: You can if you prefer also put this in your .hgrc so mq is always on.
                    115: Then do
                    116:        hg qinit -c
                    117: The -c option tells mq that you'll be checkpointing your patches,
                    118: which is usually a good idea.
                    119: 
                    120: Now prepare a .hgignore file.
                    121: This file contains one regular expression per line; Mercurial ignores
                    122: files (and subdirectories) whose paths from the repository root match
                    123: one of the regexps.
                    124: Add at least:
                    125:        ^CVS$
                    126:        /CVS$
                    127: to ignore all the CVS control directories in the CVS checkout.
                    128: While you can commit these to Mercurial, there's no point and it gets
                    129: awkward if owing to mistakes later you end up having to merge them.
                    130: 
                    131: If you aren't arranging to put the tree's object directories somewhere
                    132: else, then also add
                    133:        ^obj\.[0-9a-z]$
                    134:        /obj\.[0-9a-z]$
                    135: and you might want
                    136:        ^sys/arch/[0-9a-z]*/compile/[A-Z]
                    137: to ignore kernel build directories.
                    138: 
                    139: Ignore subtrees that you aren't working in.
                    140: You don't have to bother to be very selective; the goal is to rapidly
                    141: rule out a few large subtrees that you definitely don't care about, in
                    142: order to avoid wasting time scanning them for changes.
                    143: Unless you plan to be working with 3rd-party software,
                    144: 
                    145:        ^external$
                    146:        ^gnu/dist$
                    147: 
                    148: is a good starting point.
                    149: Alternatively, if you aren't going to be working on MD kernel stuff or
                    150: bootloaders,
                    151: 
                    152:        ^sys/arch$
                    153: 
                    154: is a good choice as it's also large.
                    155: 
                    156: You can always unignore stuff later, so don't worry about remote
                    157: possibilities.
                    158: 
                    159: Now commit the .hgignore file:
                    160: 
                    161:        % hg add .hgignore
                    162:        % hg commit -m 'add .hgignore file' .hgignore
                    163: 
                    164: Now add and commit the contents of the working tree:
                    165: 
                    166:        % hg add
                    167:        % hg commit -m 'HEAD of 20130101'
                    168: (or whatever date)
                    169: 
                    170: You are now in business.
                    171: 
                    172: ### Working
                    173: 
                    174: If you're using a branch, remember to change branches before you
                    175: commit anything:
                    176:        % hg branch mystuff
                    177: You want to keep the default branch an untouched CVS tree so you can
                    178: use Mercurial to merge.
                    179: (And also so you can use Mercurial to extract diffs against CVS HEAD
                    180: and so forth.)
                    181: 
                    182: Similarly, if you're using a patch queue, put everything in patches
                    183: and don't commit.
                    184: (There's a section below about working with mq if you aren't familiar
                    185: with it.)
                    186: 
                    187: You can edit and build and test as normal.
                    188: Use hg commit or hg qrefresh to sync stuff into Mercurial.
                    189: 
                    190: If you're using mq, it's a good idea to checkpoint your patch queue
                    191: periodically.
                    192: This is done as follows:
                    193:        % hg qcommit
                    194: The patches directory (.hg/patches) is stored in its own Mercurial
                    195: repository, and this commits the patches to that repository.
                    196: If necessary you can then fetch older versions of the patches back and
                    197: so forth.
                    198: 
                    199: ### Updating from CVS
                    200: 
                    201: First, make sure all your changes are committed.
                    202: (If you have unfinished changes that aren't ready to commit, there's a
                    203: Mercurial extension for stashing them temporarily.
                    204: If you have stuff that you don't want to commit at all, like debugging
                    205: printouts or quick hacks, it's often convenient to keep those in their
                    206: own mq patch, even if you aren't using mq for development.)
                    207: 
                    208: Now go back to a clean CVS tree.
                    209: If using branches, go back to the default branch:
                    210:        % hg update -r default
                    211: If using mq, pop all the patches:
                    212:        % hg qpop -a
                    213: 
                    214: DO NOT run cvs update until/unless you have done this; it will make a
                    215: mess.
                    216: When you eventually do this by accident, see the section below on
                    217: recovering from mistakes.
                    218: 
                    219: Now run cvs update from the top of the source tree:
                    220:        % cvs -q update -dP
                    221: 
                    222: You should get no conflicts from CVS and nothing should show as
                    223: modified.
                    224: (It is usually a good habit to save the cvs update output to a file to
                    225: be able to check this.)
                    226: 
                    227: Tell hg to sync up:
                    228:        % hg addremove
                    229: 
                    230: Use hg to check what it thinks has changed:
                    231:        % hg status
                    232: 
                    233: Commit the changes to Mercurial:
                    234:        % hg commit -m 'Updated to 20130202"
                    235: 
                    236: Now you get to merge.
                    237: 
                    238: If you're using a branch, you want to merge the changes into your
                    239: branch rather than merge your branch into the changes:
                    240:        % hg update -r mystuff
                    241:        % hg merge default
                    242:        (edit and resolve as needed)
                    243:        % hg commit -m 'sync with HEAD'
                    244: 
                    245: If it tells you "update crosses branches" when trying to update back
                    246: to your branch, update to the parent changeset (the previous version
                    247: from CVS) first, as that's an ancestor of your branch.
                    248: 
                    249: If you're using mq, the thing to do now is to push all your patches,
                    250: and if any reject, clean up the mess and refresh them.
                    251: 
                    252: If patch tells you "hunk N succeeded at offset MMM with fuzz Q", it's
                    253: a good idea to manually inspect the results -- patch being what it is,
                    254: sometimes this means it's done the wrong thing.
                    255: Edit if needed.
                    256: Then (even if you didn't edit) refresh the patch so it won't happen
                    257: again.
                    258: 
                    259: As I said above, it's quite likely that by now there's a better scheme
                    260: for merging with mq that I don't know about yet.
                    261: 
                    262: ### Pushing back to CVS
                    263: 
                    264: When you're ready to push your changes back to CVS (so they're really
                    265: committed), first (unless you're absolutely sure it's not necessary)
                    266: update from CVS as above and merge.
                    267: Then:
                    268: 
                    269: If you're using a branch, go back to the default branch and merge your
                    270: changes into it:
                    271:        % hg update -r default
                    272:        % hg merge mystuff
                    273:        % hg commit -m "prepare to commit back to cvs"
                    274: Now cvs add any new directories and files; be sure not to forget this.
                    275: It is a good idea to crosscheck with cvs diff and/or cvs update:
                    276:        % cvs diff -up | less
                    277:        % cvs -nq update -dP
                    278: Then you can cvs commit:
                    279:        % cvs commit
                    280: Because of RCSIDs, committing into cvs changes the source files.
                    281: So now you need to do:
                    282:        % hg commit -m 'cvs committed'
                    283: and if you intend to keep working in this tree, you want to merge that
                    284: changeset back into your branch to avoid having it cause merge
                    285: conflicts later.
                    286: Do that as above.
                    287: 
                    288: 
                    289: If you're using a patch queue, usually it's because you want to commit
                    290: each patch back to CVS individually.
                    291: First pop all the patches:
                    292:        % hg qpop -a
                    293: Now, for each patch:
                    294:        % hg qpush
                    295:        % hg qfinish -a
                    296:        % cvs commit
                    297:        % hg commit -m "cvs committed previous"
                    298: With a long patch queue, you'll want to use the patch comments as the
                    299: CVS commit messages.
                    300: Also, running cvs commit from the top for every patch is horribly slow.
                    301: Both these problems can be fixed by putting the following in a script:
                    302:        hg log -v -r. | sed '1,/^description:$/d' > patch-message
                    303:        cat patch-message
                    304:        echo -n 'cvs commit -F patch-message '
                    305:        hg log -v -r. | grep '^files:' | sed 's/^files://'
                    306: (I call this "dogetpatch.sh") and then the procedure is:
                    307:        % hg qpop -a
                    308: then for each patch:
                    309:        % hg qpush && hg qfinish -a && dogetpatch.sh
                    310:        % cvs commit [as directed]
                    311:        % hg commit -m "cvs committed previous"
                    312: (This could be automated further but doing so seems unwise.)
                    313: 
                    314: ### Using CVS within Mercurial
                    315: 
                    316: You can successfully do any read-only CVS operation in the hybrid
                    317: tree: diff, annotate, log, update -p, etc.
                    318: Read-write operations should be avoided; if you mix upstream changes
                    319: with your changes you will find it much harder to commit upstream
                    320: later, and you may get weird merge conflicts or even accidentally
                    321: revert other people's changes and cause problems.
                    322: 
                    323: If you clone the Mercurial tree and you didn't include the CVS control
                    324: files in it, you won't be able to do CVS operations from clones.
                    325: Including the CVS control files in the Mercurial tree is one way
                    326: around that.
                    327: 
                    328: You will find that any large CVS operation on a clone is horribly
                    329: slow.
                    330: This is because making a clone causes CVS to think all the files in
                    331: the clone have been modified since you last ran it; it then re-fetches
                    332: every file you ask it about so it can update its own information.
                    333: For this reason cloning the Mercurial tree usually isn't worthwhile
                    334: and even when it is, including the CVS files in the Mercurial tree
                    335: isn't.
                    336: 
                    337: Another consequence of this: do not try to cvs update in a cloned
                    338: Mercurial repository; use only the original.
                    339: Updating a clone basically downloads the entire tree over again from
                    340: the CVS server.
                    341: 
                    342: DO NOT CVS COMMIT FROM A CLONE.
                    343: It is known that some operations that muck with the timestamps in a
                    344: CVS working tree can cause CVS to lose data.
                    345: It is not clear if hg clone is such an operation; don't be the person
                    346: who finds out the hard way.
                    347: 
                    348: ### Recovering from mistakes
                    349: 
                    350: The most common mistake is CVS updating when the Mercurial tree is not
                    351: in the proper state from that; e.g. onto your branch or while you have
                    352: patches applied.
                    353: 
                    354: The basic strategy for this is to use hg revert to restore the part of
                    355: the tree it knows about, then go back to CVS, clean up the mess there,
                    356: and update properly.
                    357: 
                    358: If you're using a branch:
                    359:        % hg revert -C
                    360:        % hg update -r default
                    361: If you're using a patch queue:
                    362:        % hg revert -C
                    363:        % hg qpop -a
                    364: 
                    365: The problem is, CVS will now think you've changed every file that
                    366: Mercurial is managing, and the modifications are to revert all the
                    367: changes that have happened since your previous update.
                    368: You do *not* want that to turn into reality.
                    369: Hunt down (with cvs -n update) any files that CVS thinks are modified,
                    370: then rm them and run cvs update on them.
                    371: CVS will print "Warning: foo was lost" and restore an unmodified copy.
                    372: 
                    373: When you have no files left that CVS thinks are modified, do a CVS
                    374: update on the whole tree and merge it as described above.
                    375: (You must do this, as the parts of the tree that Mercurial is ignoring
                    376: will otherwise be out of sync with the parts it's managing.)
                    377: 
                    378: If you stored the CVS control files in Mercurial, then the revert will
                    379: restore them, but your tree will still be inconsistent so you still
                    380: need to do a proper update and merge immediately.
                    381: 
                    382: ### mq
                    383: 
                    384: The basic idea of mq (like quilt) is it maintains a series of patches
                    385: against the source tree, that are to be applied in order.
                    386: By applying them or removing them one at a time, you can move the tree
                    387: to any intermediate state; and then you can update the topmost patch,
                    388: insert a new patch, or whatever.
                    389: 
                    390: To see the list of patches:
                    391:        % hg qseries
                    392: 
                    393: To apply the next patch:
                    394:        % hg qpush
                    395: 
                    396: To remove the current patch:
                    397:        % hg qpop
                    398: 
                    399: To merge current working tree changes into the current patch:
                    400:        % hg qrefresh
                    401: 
                    402: To also update the current patch's change comment:
                    403:        % hg qrefresh -e
                    404: 
                    405: To collect current working tree changes (if any) into a new patch:
                    406:        % hg qnew PATCHNAME 
                    407: 
                    408: When there's an mq patch applied, you can't commit.
                    409: (Doing qrefresh is basically equivalent to committing the current
                    410: patch.)
                    411: Diff will show the changes against the last refreshed version of the
                    412: current patch; to see the complete changes for the current patch
                    413: (including current changes), use "hg qdiff".
                    414: 
                    415: You can delete patches with "hg qrm" and rename them with "hg qmv".
                    416: 
                    417: Patches are applied with patch, unfortunately, which means that if
                    418: they don't apply (which can happen if you or someone else changes
                    419: something under one) you get .rej files you have to clean up by hand
                    420: rather than a Mercurial merge.
                    421: 
                    422: When a patch is ready to be committed for real, you do "hg qfinish" on
                    423: it.
                    424: This removes it from the patch queue and converts it to a normal
                    425: Mercurial changeset.
                    426: 
                    427: To change the ordering of patches, you edit the file
                    428: .hg/patches/series.
                    429: If the patches aren't orthogonal you'll have to fix the rejections
                    430: when you next apply them.
                    431: (Don't do this with patches that are currently applied.)
                    432: 
                    433: Use "hg help mq" to see the full list of mq-related commands.
                    434: 
                    435: I'm sure there are better mq tutorials out there.
                    436: 
                    437: ### Using mq
                    438: 
                    439: The basic process when using mq is that you start a new patch, edit
                    440: and hack for a while, use hg qrefresh to commit it (once or many
                    441: times), and when you're done go on to the next one.
                    442: 
                    443: If you find a bug in an earlier patch, you can go back to the patch
                    444: that introduced it and fix the bug there, creating a new version of
                    445: the offending patch that no longer contains the bug.
                    446: (Or you can create a new patch that fixes the bug, but insert it
                    447: immediately after the patch that created the bug.)
                    448: 
                    449: When a patch is ready to be seen by other people, you "finish" it and
                    450: then it becomes a normal immutable changeset.
                    451: 
                    452: One catch is that you can't push or pop the patch queue while you have
                    453: unsynced (uncommitted) changes.
                    454: There are two ways around this; there's a separate "stash" extension
                    455: that lets you put unfinished changes aside while you do something else.
                    456: Or, alternatively, you can create a new temporary patch holding your
                    457: unfinished changes, and then later use hg qfold to combine that with
                    458: the patch you originally meant this for.
                    459: 
                    460: A variant of this problem is when you discover a bug, open an editor,
                    461: fix it, and then realize that you wanted to make the edit in an
                    462: earlier patch.
                    463: Then you go to pop the queue and it complains that you have a modified
                    464: file.
                    465: If the modification in question is the only uncommitted change, the
                    466: best way to deal with this is to create a new patch for it, then pop
                    467: to where you wanted it to go and use hg qfold to apply it there.

CVSweb for NetBSD wikisrc <wikimaster@NetBSD.org> software: FreeBSD-CVSweb