File:  [NetBSD Developer Wiki] / wikisrc / users / dholland / mercurial.mdwn
Revision 1.3: download - view: text, annotated - select for diffs
Mon Sep 2 04:22:32 2013 UTC (8 years ago) by kim
Branches: MAIN
CVS tags: HEAD
Add empty lines around code blocks for correctly formatting as such.
Add '*' to the obj directory regexps.

    1: ## Using Mercurial and mq to work on NetBSD
    2: 
    3: This page contains directions for using Mercurial as a commit buffer
    4: for NetBSD.
    5: 
    6: (It will not do you much good if you're trying to convert the master
    7: NetBSD tree to Mercurial or to work with such a converted tree.)
    8: 
    9: ### What it is
   10: 
   11: Mercurial is a distributed version control system ("DVCS").
   12: mq is an extension to Mercurial for handling patch queues.
   13: The concept of patch queues was introduced by Quilt some years back.
   14: 
   15: This document assumes you already know more or less how to use
   16: Mercurial but may not have used mq before.
   17: 
   18: ### The model we're using here
   19: 
   20: What we're going to do is commit a NetBSD CVS working tree into a
   21: Mercurial repository.
   22: You can then use Mercurial to merge; it is better at this than CVS.
   23: You can also commit changes locally and ship them back to the CVS
   24: master later; this is useful in a variety of ways.
   25: You can potentially also clone the Mercurial tree and work jointly
   26: with other people, but there are limits to this as we'll discuss in a
   27: moment.
   28: 
   29: Because the NetBSD tree is rather large, you will find that if you
   30: commit the whole thing into Mercurial that a number of operations
   31: (anything that scans the working tree for changes) become annoyingly
   32: slow.
   33: It isn't slow enough to be unusable, and it's quite a bit faster than a
   34: comparable CVS option (like running cvs update from the top level),
   35: but it's slow enough to be annoying.
   36: 
   37: For this reason, in most cases, I recommend committing only part of
   38: the tree into Mercurial and telling it to ignore the rest. This means
   39: you can't in general usefully clone the resulting Mercurial repo
   40: (although that depends on exactly what you leave out) but this is not
   41: a major problem unless you're specifically trying to work with someone
   42: else.
   43: 
   44: So the basic model here is that you check out a CVS working tree and
   45: use Mercurial to manage local changes to part of it, then later on
   46: commit those changes back to the master CVS repository.
   47: 
   48: ### Branches vs. patches
   49: 
   50: There are two ways you can manage your changes: as a branch, or as a
   51: patch queue.
   52: The advantage of a patch queue is that you can easily commit each
   53: patch individually back to CVS, and you can go back and forth between
   54: them and debug and polish each one separately.
   55: The disadvantage is that the merge facilities are (as far as I know
   56: anyway) relatively limited.
   57: 
   58: Conversely, if you commit your changes to a branch, you get all the
   59: native merging support in Mercurial.
   60: However, it is painful to try to commit anything other than one big
   61: diff for the whole branch back to CVS.
   62: (You might be able to do it via bookmarks and rebasing, but I've never
   63: tried and have no desire to figure out how.)
   64: 
   65: If you don't want to keep the incremental history of your local
   66: commits, use a branch.
   67: If you do, use a patch queue.
   68: 
   69: It is possible to use multiple branches to allow you to commit back in
   70: several stages.
   71: However, managing this is a major pain and I don't recommend it -- you
   72: might get away with two branches but more than that is probably a bad
   73: idea.
   74: 
   75: There's a Mercurial extension called the "patch branch extension" that
   76: lets you manage a whole graph of patches using branches.
   77: I haven't tried using it in some years; at the time it had scaling
   78: problems such that it became horrifyingly slow once you had more than
   79: a handful of such branches.
   80: That might have been improved in the meantime; if you find yourself
   81: wanting to use both branches and patches, it might be worth looking
   82: into.
   83: 
   84: It is also fairly probable that there is now a solution for merging
   85: with patch queues; it's been a while since I had time to look closely.
   86: 
   87: ### Setting up
   88: 
   89: First, check out a CVS working tree.
   90: You probably want to use a different one for each project, because
   91: different projects require changing different parts of the tree and so
   92: you will probably want to have Mercurial ignore different subtrees for
   93: different projects.
   94: (At least, I find it so; it depends on what you're working on.)
   95: 
   96: 	% cvs -d cvs.netbsd.org:/cvsroot checkout -P src
   97: 
   98: Now create a Mercurial repository at the top level.
   99: (If you are working only in a subtree and you are *sure* that you will
  100: never need to change anything in other parts of the tree, you can
  101: create the Mercurial repository in a subtree.
  102: But unless you're absolutely certain, don't take the risk.)
  103: 
  104: 	% cd src
  105: 	% hg init
  106: 
  107: If you're going to be using a patch queue, now enable mq.
  108: 
  109: 	% vi .hg/hgrc
  110: 
  111: and add
  112: 
  113: 	[extensions]
  114: 	hgext.mq =
  115: 
  116: (Since the extension is built into Mercurial, that's all you need.)
  117: You can if you prefer also put this in your .hgrc so mq is always on.
  118: Then do
  119: 
  120: 	% hg qinit -c
  121: 
  122: The -c option tells mq that you'll be checkpointing your patches,
  123: which is usually a good idea.
  124: 
  125: Now prepare a .hgignore file.
  126: This file contains one regular expression per line; Mercurial ignores
  127: files (and subdirectories) whose paths from the repository root match
  128: one of the regexps.
  129: Add at least:
  130: 
  131: 	^CVS$
  132: 	/CVS$
  133: 
  134: to ignore all the CVS control directories in the CVS checkout.
  135: While you can commit these to Mercurial, there's no point and it gets
  136: awkward if owing to mistakes later you end up having to merge them.
  137: 
  138: If you aren't arranging to put the tree's object directories somewhere
  139: else, then also add
  140: 
  141: 	^obj\.[0-9a-z]*$
  142: 	/obj\.[0-9a-z]*$
  143: 
  144: and you might want
  145: 
  146: 	^sys/arch/[0-9a-z]*/compile/[A-Z]
  147: 
  148: to ignore kernel build directories.
  149: 
  150: Ignore subtrees that you aren't working in.
  151: You don't have to bother to be very selective; the goal is to rapidly
  152: rule out a few large subtrees that you definitely don't care about, in
  153: order to avoid wasting time scanning them for changes.
  154: Unless you plan to be working with 3rd-party software,
  155: 
  156: 	^external$
  157: 	^gnu/dist$
  158: 
  159: is a good starting point.
  160: Alternatively, if you aren't going to be working on MD kernel stuff or
  161: bootloaders,
  162: 
  163: 	^sys/arch$
  164: 
  165: is a good choice as it's also large.
  166: 
  167: You can always unignore stuff later, so don't worry about remote
  168: possibilities.
  169: 
  170: Now commit the .hgignore file:
  171: 
  172: 	% hg add .hgignore
  173: 	% hg commit -m 'add .hgignore file' .hgignore
  174: 
  175: Now add and commit the contents of the working tree:
  176: 
  177: 	% hg add
  178: 	% hg commit -m 'HEAD of 20130101'
  179: 
  180: (or whatever date)
  181: 
  182: You are now in business.
  183: 
  184: ### Working
  185: 
  186: If you're using a branch, remember to change branches before you
  187: commit anything:
  188: 
  189: 	% hg branch mystuff
  190: 
  191: You want to keep the default branch an untouched CVS tree so you can
  192: use Mercurial to merge.
  193: (And also so you can use Mercurial to extract diffs against CVS HEAD
  194: and so forth.)
  195: 
  196: Similarly, if you're using a patch queue, put everything in patches
  197: and don't commit.
  198: (There's a section below about working with mq if you aren't familiar
  199: with it.)
  200: 
  201: You can edit and build and test as normal.
  202: Use hg commit or hg qrefresh to sync stuff into Mercurial.
  203: 
  204: If you're using mq, it's a good idea to checkpoint your patch queue
  205: periodically.
  206: This is done as follows:
  207: 
  208: 	% hg qcommit
  209: 
  210: The patches directory (.hg/patches) is stored in its own Mercurial
  211: repository, and this commits the patches to that repository.
  212: If necessary you can then fetch older versions of the patches back and
  213: so forth.
  214: 
  215: ### Updating from CVS
  216: 
  217: First, make sure all your changes are committed.
  218: (If you have unfinished changes that aren't ready to commit, there's a
  219: Mercurial extension for stashing them temporarily.
  220: If you have stuff that you don't want to commit at all, like debugging
  221: printouts or quick hacks, it's often convenient to keep those in their
  222: own mq patch, even if you aren't using mq for development.)
  223: 
  224: Now go back to a clean CVS tree.
  225: If using branches, go back to the default branch:
  226: 
  227: 	% hg update -r default
  228: 
  229: If using mq, pop all the patches:
  230: 
  231: 	% hg qpop -a
  232: 
  233: DO NOT run cvs update until/unless you have done this; it will make a
  234: mess.
  235: When you eventually do this by accident, see the section below on
  236: recovering from mistakes.
  237: 
  238: Now run cvs update from the top of the source tree:
  239: 
  240: 	% cvs -q update -dP
  241: 
  242: You should get no conflicts from CVS and nothing should show as
  243: modified.
  244: (It is usually a good habit to save the cvs update output to a file to
  245: be able to check this.)
  246: 
  247: Tell hg to sync up:
  248: 
  249: 	% hg addremove
  250: 
  251: Use hg to check what it thinks has changed:
  252: 
  253: 	% hg status
  254: 
  255: Commit the changes to Mercurial:
  256: 
  257: 	% hg commit -m 'Updated to 20130202"
  258: 
  259: Now you get to merge.
  260: 
  261: If you're using a branch, you want to merge the changes into your
  262: branch rather than merge your branch into the changes:
  263: 
  264: 	% hg update -r mystuff
  265: 	% hg merge default
  266: 	(edit and resolve as needed)
  267: 	% hg commit -m 'sync with HEAD'
  268: 
  269: If it tells you "update crosses branches" when trying to update back
  270: to your branch, update to the parent changeset (the previous version
  271: from CVS) first, as that's an ancestor of your branch.
  272: 
  273: If you're using mq, the thing to do now is to push all your patches,
  274: and if any reject, clean up the mess and refresh them.
  275: 
  276: If patch tells you "hunk N succeeded at offset MMM with fuzz Q", it's
  277: a good idea to manually inspect the results -- patch being what it is,
  278: sometimes this means it's done the wrong thing.
  279: Edit if needed.
  280: Then (even if you didn't edit) refresh the patch so it won't happen
  281: again.
  282: 
  283: As I said above, it's quite likely that by now there's a better scheme
  284: for merging with mq that I don't know about yet.
  285: 
  286: ### Pushing back to CVS
  287: 
  288: When you're ready to push your changes back to CVS (so they're really
  289: committed), first (unless you're absolutely sure it's not necessary)
  290: update from CVS as above and merge.
  291: Then:
  292: 
  293: If you're using a branch, go back to the default branch and merge your
  294: changes into it:
  295: 
  296: 	% hg update -r default
  297: 	% hg merge mystuff
  298: 	% hg commit -m "prepare to commit back to cvs"
  299: 
  300: Now cvs add any new directories and files; be sure not to forget this.
  301: It is a good idea to crosscheck with cvs diff and/or cvs update:
  302: 
  303: 	% cvs diff -up | less
  304: 	% cvs -nq update -dP
  305: 
  306: Then you can cvs commit:
  307: 
  308: 	% cvs commit
  309: 
  310: Because of RCSIDs, committing into cvs changes the source files.
  311: So now you need to do:
  312: 
  313: 	% hg commit -m 'cvs committed'
  314: 
  315: and if you intend to keep working in this tree, you want to merge that
  316: changeset back into your branch to avoid having it cause merge
  317: conflicts later.
  318: Do that as above.
  319: 
  320: 
  321: If you're using a patch queue, usually it's because you want to commit
  322: each patch back to CVS individually.
  323: First pop all the patches:
  324: 
  325: 	% hg qpop -a
  326: 
  327: Now, for each patch:
  328: 
  329: 	% hg qpush
  330: 	% hg qfinish -a
  331: 	% cvs commit
  332: 	% hg commit -m "cvs committed previous"
  333: 
  334: With a long patch queue, you'll want to use the patch comments as the
  335: CVS commit messages.
  336: Also, running cvs commit from the top for every patch is horribly slow.
  337: Both these problems can be fixed by putting the following in a script:
  338: 
  339: 	hg log -v -r. | sed '1,/^description:$/d' > patch-message
  340: 	cat patch-message
  341: 	echo -n 'cvs commit -F patch-message '
  342: 	hg log -v -r. | grep '^files:' | sed 's/^files://'
  343: 
  344: (I call this "dogetpatch.sh") and then the procedure is:
  345: 
  346: 	% hg qpop -a
  347: 
  348: then for each patch:
  349: 
  350: 	% hg qpush && hg qfinish -a && dogetpatch.sh
  351: 	% cvs commit [as directed]
  352: 	% hg commit -m "cvs committed previous"
  353: 
  354: (This could be automated further but doing so seems unwise.)
  355: 
  356: ### Using CVS within Mercurial
  357: 
  358: You can successfully do any read-only CVS operation in the hybrid
  359: tree: diff, annotate, log, update -p, etc.
  360: Read-write operations should be avoided; if you mix upstream changes
  361: with your changes you will find it much harder to commit upstream
  362: later, and you may get weird merge conflicts or even accidentally
  363: revert other people's changes and cause problems.
  364: 
  365: If you clone the Mercurial tree and you didn't include the CVS control
  366: files in it, you won't be able to do CVS operations from clones.
  367: Including the CVS control files in the Mercurial tree is one way
  368: around that.
  369: 
  370: You will find that any large CVS operation on a clone is horribly
  371: slow.
  372: This is because making a clone causes CVS to think all the files in
  373: the clone have been modified since you last ran it; it then re-fetches
  374: every file you ask it about so it can update its own information.
  375: For this reason cloning the Mercurial tree usually isn't worthwhile
  376: and even when it is, including the CVS files in the Mercurial tree
  377: isn't.
  378: 
  379: Another consequence of this: do not try to cvs update in a cloned
  380: Mercurial repository; use only the original.
  381: Updating a clone basically downloads the entire tree over again from
  382: the CVS server.
  383: 
  384: DO NOT CVS COMMIT FROM A CLONE.
  385: It is known that some operations that muck with the timestamps in a
  386: CVS working tree can cause CVS to lose data.
  387: It is not clear if hg clone is such an operation; don't be the person
  388: who finds out the hard way.
  389: 
  390: ### Recovering from mistakes
  391: 
  392: The most common mistake is CVS updating when the Mercurial tree is not
  393: in the proper state from that; e.g. onto your branch or while you have
  394: patches applied.
  395: 
  396: The basic strategy for this is to use hg revert to restore the part of
  397: the tree it knows about, then go back to CVS, clean up the mess there,
  398: and update properly.
  399: 
  400: If you're using a branch:
  401: 
  402: 	% hg revert -C
  403: 	% hg update -r default
  404: 
  405: If you're using a patch queue:
  406: 
  407: 	% hg revert -C
  408: 	% hg qpop -a
  409: 
  410: The problem is, CVS will now think you've changed every file that
  411: Mercurial is managing, and the modifications are to revert all the
  412: changes that have happened since your previous update.
  413: You do *not* want that to turn into reality.
  414: Hunt down (with cvs -n update) any files that CVS thinks are modified,
  415: then rm them and run cvs update on them.
  416: CVS will print "Warning: foo was lost" and restore an unmodified copy.
  417: 
  418: When you have no files left that CVS thinks are modified, do a CVS
  419: update on the whole tree and merge it as described above.
  420: (You must do this, as the parts of the tree that Mercurial is ignoring
  421: will otherwise be out of sync with the parts it's managing.)
  422: 
  423: If you stored the CVS control files in Mercurial, then the revert will
  424: restore them, but your tree will still be inconsistent so you still
  425: need to do a proper update and merge immediately.
  426: 
  427: ### mq
  428: 
  429: The basic idea of mq (like quilt) is it maintains a series of patches
  430: against the source tree, that are to be applied in order.
  431: By applying them or removing them one at a time, you can move the tree
  432: to any intermediate state; and then you can update the topmost patch,
  433: insert a new patch, or whatever.
  434: 
  435: To see the list of patches:
  436: 
  437: 	% hg qseries
  438: 
  439: To apply the next patch:
  440: 
  441: 	% hg qpush
  442: 
  443: To remove the current patch:
  444: 
  445: 	% hg qpop
  446: 
  447: To merge current working tree changes into the current patch:
  448: 
  449: 	% hg qrefresh
  450: 
  451: To also update the current patch's change comment:
  452: 
  453: 	% hg qrefresh -e
  454: 
  455: To collect current working tree changes (if any) into a new patch:
  456: 
  457: 	% hg qnew PATCHNAME 
  458: 
  459: When there's an mq patch applied, you can't commit.
  460: (Doing qrefresh is basically equivalent to committing the current
  461: patch.)
  462: Diff will show the changes against the last refreshed version of the
  463: current patch; to see the complete changes for the current patch
  464: (including current changes), use "hg qdiff".
  465: 
  466: You can delete patches with "hg qrm" and rename them with "hg qmv".
  467: 
  468: Patches are applied with patch, unfortunately, which means that if
  469: they don't apply (which can happen if you or someone else changes
  470: something under one) you get .rej files you have to clean up by hand
  471: rather than a Mercurial merge.
  472: 
  473: When a patch is ready to be committed for real, you do "hg qfinish" on
  474: it.
  475: This removes it from the patch queue and converts it to a normal
  476: Mercurial changeset.
  477: 
  478: To change the ordering of patches, you edit the file
  479: .hg/patches/series.
  480: If the patches aren't orthogonal you'll have to fix the rejections
  481: when you next apply them.
  482: (Don't do this with patches that are currently applied.)
  483: 
  484: Use "hg help mq" to see the full list of mq-related commands.
  485: 
  486: I'm sure there are better mq tutorials out there.
  487: 
  488: ### Using mq
  489: 
  490: The basic process when using mq is that you start a new patch, edit
  491: and hack for a while, use hg qrefresh to commit it (once or many
  492: times), and when you're done go on to the next one.
  493: 
  494: If you find a bug in an earlier patch, you can go back to the patch
  495: that introduced it and fix the bug there, creating a new version of
  496: the offending patch that no longer contains the bug.
  497: (Or you can create a new patch that fixes the bug, but insert it
  498: immediately after the patch that created the bug.)
  499: 
  500: When a patch is ready to be seen by other people, you "finish" it and
  501: then it becomes a normal immutable changeset.
  502: 
  503: One catch is that you can't push or pop the patch queue while you have
  504: unsynced (uncommitted) changes.
  505: There are two ways around this; there's a separate "stash" extension
  506: that lets you put unfinished changes aside while you do something else.
  507: Or, alternatively, you can create a new temporary patch holding your
  508: unfinished changes, and then later use hg qfold to combine that with
  509: the patch you originally meant this for.
  510: 
  511: A variant of this problem is when you discover a bug, open an editor,
  512: fix it, and then realize that you wanted to make the edit in an
  513: earlier patch.
  514: Then you go to pop the queue and it complains that you have a modified
  515: file.
  516: If the modification in question is the only uncommitted change, the
  517: best way to deal with this is to create a new patch for it, then pop
  518: to where you wanted it to go and use hg qfold to apply it there.

CVSweb for NetBSD wikisrc <wikimaster@NetBSD.org> software: FreeBSD-CVSweb