--- wikisrc/users/dholland/mercurial.mdwn 2013/09/01 23:17:45 1.1 +++ wikisrc/users/dholland/mercurial.mdwn 2016/02/15 23:02:07 1.4 @@ -1,467 +1,3 @@ ## Using Mercurial and mq to work on NetBSD -This page contains directions for using Mercurial as a commit buffer -for NetBSD. - -(It will not do you much good if you're trying to convert the master -NetBSD tree to Mercurial or to work with such a converted tree.) - -### What it is - -Mercurial is a distributed version control system ("DVCS"). -mq is an extension to Mercurial for handling patch queues. -The concept of patch queues was introduced by Quilt some years back. - -This document assumes you already know more or less how to use -Mercurial but may not have used mq before. - -### The model we're using here - -What we're going to do is commit a NetBSD CVS working tree into a -Mercurial repository. -You can then use Mercurial to merge; it is better at this than CVS. -You can also commit changes locally and ship them back to the CVS -master later; this is useful in a variety of ways. -You can potentially also clone the Mercurial tree and work jointly -with other people, but there are limits to this as we'll discuss in a -moment. - -Because the NetBSD tree is rather large, you will find that if you -commit the whole thing into Mercurial that a number of operations -(anything that scans the working tree for changes) become annoyingly -slow. -It isn't slow enough to be unusable, and it's quite a bit faster than a -comparable CVS option (like running cvs update from the top level), -but it's slow enough to be annoying. - -For this reason, in most cases, I recommend committing only part of -the tree into Mercurial and telling it to ignore the rest. This means -you can't in general usefully clone the resulting Mercurial repo -(although that depends on exactly what you leave out) but this is not -a major problem unless you're specifically trying to work with someone -else. - -So the basic model here is that you check out a CVS working tree and -use Mercurial to manage local changes to part of it, then later on -commit those changes back to the master CVS repository. - -### Branches vs. patches - -There are two ways you can manage your changes: as a branch, or as a -patch queue. -The advantage of a patch queue is that you can easily commit each -patch individually back to CVS, and you can go back and forth between -them and debug and polish each one separately. -The disadvantage is that the merge facilities are (as far as I know -anyway) relatively limited. - -Conversely, if you commit your changes to a branch, you get all the -native merging support in Mercurial. -However, it is painful to try to commit anything other than one big -diff for the whole branch back to CVS. -(You might be able to do it via bookmarks and rebasing, but I've never -tried and have no desire to figure out how.) - -If you don't want to keep the incremental history of your local -commits, use a branch. -If you do, use a patch queue. - -It is possible to use multiple branches to allow you to commit back in -several stages. -However, managing this is a major pain and I don't recommend it -- you -might get away with two branches but more than that is probably a bad -idea. - -There's a Mercurial extension called the "patch branch extension" that -lets you manage a whole graph of patches using branches. -I haven't tried using it in some years; at the time it had scaling -problems such that it became horrifyingly slow once you had more than -a handful of such branches. -That might have been improved in the meantime; if you find yourself -wanting to use both branches and patches, it might be worth looking -into. - -It is also fairly probable that there is now a solution for merging -with patch queues; it's been a while since I had time to look closely. - -### Setting up - -First, check out a CVS working tree. -You probably want to use a different one for each project, because -different projects require changing different parts of the tree and so -you will probably want to have Mercurial ignore different subtrees for -different projects. -(At least, I find it so; it depends on what you're working on.) - - % cvs -d cvs.netbsd.org:/cvsroot checkout -dP src - -Now create a Mercurial repository at the top level. -(If you are working only in a subtree and you are *sure* that you will -never need to change anything in other parts of the tree, you can -create the Mercurial repository in a subtree. -But unless you're absolutely certain, don't take the risk.) - - % cd src - % hg init - -If you're going to be using a patch queue, now enable mq. - - % vi .hg/hgrc -and add - [extensions] - hgext.mq = -(Since the extension is built into Mercurial, that's all you need.) -You can if you prefer also put this in your .hgrc so mq is always on. -Then do - hg qinit -c -The -c option tells mq that you'll be checkpointing your patches, -which is usually a good idea. - -Now prepare a .hgignore file. -This file contains one regular expression per line; Mercurial ignores -files (and subdirectories) whose paths from the repository root match -one of the regexps. -Add at least: - ^CVS$ - /CVS$ -to ignore all the CVS control directories in the CVS checkout. -While you can commit these to Mercurial, there's no point and it gets -awkward if owing to mistakes later you end up having to merge them. - -If you aren't arranging to put the tree's object directories somewhere -else, then also add - ^obj\.[0-9a-z]$ - /obj\.[0-9a-z]$ -and you might want - ^sys/arch/[0-9a-z]*/compile/[A-Z] -to ignore kernel build directories. - -Ignore subtrees that you aren't working in. -You don't have to bother to be very selective; the goal is to rapidly -rule out a few large subtrees that you definitely don't care about, in -order to avoid wasting time scanning them for changes. -Unless you plan to be working with 3rd-party software, - - ^external$ - ^gnu/dist$ - -is a good starting point. -Alternatively, if you aren't going to be working on MD kernel stuff or -bootloaders, - - ^sys/arch$ - -is a good choice as it's also large. - -You can always unignore stuff later, so don't worry about remote -possibilities. - -Now commit the .hgignore file: - - % hg add .hgignore - % hg commit -m 'add .hgignore file' .hgignore - -Now add and commit the contents of the working tree: - - % hg add - % hg commit -m 'HEAD of 20130101' -(or whatever date) - -You are now in business. - -### Working - -If you're using a branch, remember to change branches before you -commit anything: - % hg branch mystuff -You want to keep the default branch an untouched CVS tree so you can -use Mercurial to merge. -(And also so you can use Mercurial to extract diffs against CVS HEAD -and so forth.) - -Similarly, if you're using a patch queue, put everything in patches -and don't commit. -(There's a section below about working with mq if you aren't familiar -with it.) - -You can edit and build and test as normal. -Use hg commit or hg qrefresh to sync stuff into Mercurial. - -If you're using mq, it's a good idea to checkpoint your patch queue -periodically. -This is done as follows: - % hg qcommit -The patches directory (.hg/patches) is stored in its own Mercurial -repository, and this commits the patches to that repository. -If necessary you can then fetch older versions of the patches back and -so forth. - -### Updating from CVS - -First, make sure all your changes are committed. -(If you have unfinished changes that aren't ready to commit, there's a -Mercurial extension for stashing them temporarily. -If you have stuff that you don't want to commit at all, like debugging -printouts or quick hacks, it's often convenient to keep those in their -own mq patch, even if you aren't using mq for development.) - -Now go back to a clean CVS tree. -If using branches, go back to the default branch: - % hg update -r default -If using mq, pop all the patches: - % hg qpop -a - -DO NOT run cvs update until/unless you have done this; it will make a -mess. -When you eventually do this by accident, see the section below on -recovering from mistakes. - -Now run cvs update from the top of the source tree: - % cvs -q update -dP - -You should get no conflicts from CVS and nothing should show as -modified. -(It is usually a good habit to save the cvs update output to a file to -be able to check this.) - -Tell hg to sync up: - % hg addremove - -Use hg to check what it thinks has changed: - % hg status - -Commit the changes to Mercurial: - % hg commit -m 'Updated to 20130202" - -Now you get to merge. - -If you're using a branch, you want to merge the changes into your -branch rather than merge your branch into the changes: - % hg update -r mystuff - % hg merge default - (edit and resolve as needed) - % hg commit -m 'sync with HEAD' - -If it tells you "update crosses branches" when trying to update back -to your branch, update to the parent changeset (the previous version -from CVS) first, as that's an ancestor of your branch. - -If you're using mq, the thing to do now is to push all your patches, -and if any reject, clean up the mess and refresh them. - -If patch tells you "hunk N succeeded at offset MMM with fuzz Q", it's -a good idea to manually inspect the results -- patch being what it is, -sometimes this means it's done the wrong thing. -Edit if needed. -Then (even if you didn't edit) refresh the patch so it won't happen -again. - -As I said above, it's quite likely that by now there's a better scheme -for merging with mq that I don't know about yet. - -### Pushing back to CVS - -When you're ready to push your changes back to CVS (so they're really -committed), first (unless you're absolutely sure it's not necessary) -update from CVS as above and merge. -Then: - -If you're using a branch, go back to the default branch and merge your -changes into it: - % hg update -r default - % hg merge mystuff - % hg commit -m "prepare to commit back to cvs" -Now cvs add any new directories and files; be sure not to forget this. -It is a good idea to crosscheck with cvs diff and/or cvs update: - % cvs diff -up | less - % cvs -nq update -dP -Then you can cvs commit: - % cvs commit -Because of RCSIDs, committing into cvs changes the source files. -So now you need to do: - % hg commit -m 'cvs committed' -and if you intend to keep working in this tree, you want to merge that -changeset back into your branch to avoid having it cause merge -conflicts later. -Do that as above. - - -If you're using a patch queue, usually it's because you want to commit -each patch back to CVS individually. -First pop all the patches: - % hg qpop -a -Now, for each patch: - % hg qpush - % hg qfinish -a - % cvs commit - % hg commit -m "cvs committed previous" -With a long patch queue, you'll want to use the patch comments as the -CVS commit messages. -Also, running cvs commit from the top for every patch is horribly slow. -Both these problems can be fixed by putting the following in a script: - hg log -v -r. | sed '1,/^description:$/d' > patch-message - cat patch-message - echo -n 'cvs commit -F patch-message ' - hg log -v -r. | grep '^files:' | sed 's/^files://' -(I call this "dogetpatch.sh") and then the procedure is: - % hg qpop -a -then for each patch: - % hg qpush && hg qfinish -a && dogetpatch.sh - % cvs commit [as directed] - % hg commit -m "cvs committed previous" -(This could be automated further but doing so seems unwise.) - -### Using CVS within Mercurial - -You can successfully do any read-only CVS operation in the hybrid -tree: diff, annotate, log, update -p, etc. -Read-write operations should be avoided; if you mix upstream changes -with your changes you will find it much harder to commit upstream -later, and you may get weird merge conflicts or even accidentally -revert other people's changes and cause problems. - -If you clone the Mercurial tree and you didn't include the CVS control -files in it, you won't be able to do CVS operations from clones. -Including the CVS control files in the Mercurial tree is one way -around that. - -You will find that any large CVS operation on a clone is horribly -slow. -This is because making a clone causes CVS to think all the files in -the clone have been modified since you last ran it; it then re-fetches -every file you ask it about so it can update its own information. -For this reason cloning the Mercurial tree usually isn't worthwhile -and even when it is, including the CVS files in the Mercurial tree -isn't. - -Another consequence of this: do not try to cvs update in a cloned -Mercurial repository; use only the original. -Updating a clone basically downloads the entire tree over again from -the CVS server. - -DO NOT CVS COMMIT FROM A CLONE. -It is known that some operations that muck with the timestamps in a -CVS working tree can cause CVS to lose data. -It is not clear if hg clone is such an operation; don't be the person -who finds out the hard way. - -### Recovering from mistakes - -The most common mistake is CVS updating when the Mercurial tree is not -in the proper state from that; e.g. onto your branch or while you have -patches applied. - -The basic strategy for this is to use hg revert to restore the part of -the tree it knows about, then go back to CVS, clean up the mess there, -and update properly. - -If you're using a branch: - % hg revert -C - % hg update -r default -If you're using a patch queue: - % hg revert -C - % hg qpop -a - -The problem is, CVS will now think you've changed every file that -Mercurial is managing, and the modifications are to revert all the -changes that have happened since your previous update. -You do *not* want that to turn into reality. -Hunt down (with cvs -n update) any files that CVS thinks are modified, -then rm them and run cvs update on them. -CVS will print "Warning: foo was lost" and restore an unmodified copy. - -When you have no files left that CVS thinks are modified, do a CVS -update on the whole tree and merge it as described above. -(You must do this, as the parts of the tree that Mercurial is ignoring -will otherwise be out of sync with the parts it's managing.) - -If you stored the CVS control files in Mercurial, then the revert will -restore them, but your tree will still be inconsistent so you still -need to do a proper update and merge immediately. - -### mq - -The basic idea of mq (like quilt) is it maintains a series of patches -against the source tree, that are to be applied in order. -By applying them or removing them one at a time, you can move the tree -to any intermediate state; and then you can update the topmost patch, -insert a new patch, or whatever. - -To see the list of patches: - % hg qseries - -To apply the next patch: - % hg qpush - -To remove the current patch: - % hg qpop - -To merge current working tree changes into the current patch: - % hg qrefresh - -To also update the current patch's change comment: - % hg qrefresh -e - -To collect current working tree changes (if any) into a new patch: - % hg qnew PATCHNAME - -When there's an mq patch applied, you can't commit. -(Doing qrefresh is basically equivalent to committing the current -patch.) -Diff will show the changes against the last refreshed version of the -current patch; to see the complete changes for the current patch -(including current changes), use "hg qdiff". - -You can delete patches with "hg qrm" and rename them with "hg qmv". - -Patches are applied with patch, unfortunately, which means that if -they don't apply (which can happen if you or someone else changes -something under one) you get .rej files you have to clean up by hand -rather than a Mercurial merge. - -When a patch is ready to be committed for real, you do "hg qfinish" on -it. -This removes it from the patch queue and converts it to a normal -Mercurial changeset. - -To change the ordering of patches, you edit the file -.hg/patches/series. -If the patches aren't orthogonal you'll have to fix the rejections -when you next apply them. -(Don't do this with patches that are currently applied.) - -Use "hg help mq" to see the full list of mq-related commands. - -I'm sure there are better mq tutorials out there. - -### Using mq - -The basic process when using mq is that you start a new patch, edit -and hack for a while, use hg qrefresh to commit it (once or many -times), and when you're done go on to the next one. - -If you find a bug in an earlier patch, you can go back to the patch -that introduced it and fix the bug there, creating a new version of -the offending patch that no longer contains the bug. -(Or you can create a new patch that fixes the bug, but insert it -immediately after the patch that created the bug.) - -When a patch is ready to be seen by other people, you "finish" it and -then it becomes a normal immutable changeset. - -One catch is that you can't push or pop the patch queue while you have -unsynced (uncommitted) changes. -There are two ways around this; there's a separate "stash" extension -that lets you put unfinished changes aside while you do something else. -Or, alternatively, you can create a new temporary patch holding your -unfinished changes, and then later use hg qfold to combine that with -the patch you originally meant this for. - -A variant of this problem is when you discover a bug, open an editor, -fix it, and then realize that you wanted to make the edit in an -earlier patch. -Then you go to pop the queue and it complains that you have a modified -file. -If the modification in question is the only uncommitted change, the -best way to deal with this is to create a new patch for it, then pop -to where you wanted it to go and use hg qfold to apply it there. +This page has been [[moved|http://www.netbsd.org/~dholland/notes/hg-mq-on-cvs.html]].