[BACK]Return to mercurial.mdwn CVS log [TXT] [DIR] Up to [NetBSD Developer Wiki] / wikisrc / users / dholland

Diff for /wikisrc/users/dholland/mercurial.mdwn between versions 1.1 and 1.4

version 1.1, 2013/09/01 23:17:45 version 1.4, 2016/02/15 23:02:07
Line 1 Line 1
 ## Using Mercurial and mq to work on NetBSD  ## Using Mercurial and mq to work on NetBSD
   
 This page contains directions for using Mercurial as a commit buffer  This page has been [[moved|http://www.netbsd.org/~dholland/notes/hg-mq-on-cvs.html]].
 for NetBSD.  
   
 (It will not do you much good if you're trying to convert the master  
 NetBSD tree to Mercurial or to work with such a converted tree.)  
   
 ### What it is  
   
 Mercurial is a distributed version control system ("DVCS").  
 mq is an extension to Mercurial for handling patch queues.  
 The concept of patch queues was introduced by Quilt some years back.  
   
 This document assumes you already know more or less how to use  
 Mercurial but may not have used mq before.  
   
 ### The model we're using here  
   
 What we're going to do is commit a NetBSD CVS working tree into a  
 Mercurial repository.  
 You can then use Mercurial to merge; it is better at this than CVS.  
 You can also commit changes locally and ship them back to the CVS  
 master later; this is useful in a variety of ways.  
 You can potentially also clone the Mercurial tree and work jointly  
 with other people, but there are limits to this as we'll discuss in a  
 moment.  
   
 Because the NetBSD tree is rather large, you will find that if you  
 commit the whole thing into Mercurial that a number of operations  
 (anything that scans the working tree for changes) become annoyingly  
 slow.  
 It isn't slow enough to be unusable, and it's quite a bit faster than a  
 comparable CVS option (like running cvs update from the top level),  
 but it's slow enough to be annoying.  
   
 For this reason, in most cases, I recommend committing only part of  
 the tree into Mercurial and telling it to ignore the rest. This means  
 you can't in general usefully clone the resulting Mercurial repo  
 (although that depends on exactly what you leave out) but this is not  
 a major problem unless you're specifically trying to work with someone  
 else.  
   
 So the basic model here is that you check out a CVS working tree and  
 use Mercurial to manage local changes to part of it, then later on  
 commit those changes back to the master CVS repository.  
   
 ### Branches vs. patches  
   
 There are two ways you can manage your changes: as a branch, or as a  
 patch queue.  
 The advantage of a patch queue is that you can easily commit each  
 patch individually back to CVS, and you can go back and forth between  
 them and debug and polish each one separately.  
 The disadvantage is that the merge facilities are (as far as I know  
 anyway) relatively limited.  
   
 Conversely, if you commit your changes to a branch, you get all the  
 native merging support in Mercurial.  
 However, it is painful to try to commit anything other than one big  
 diff for the whole branch back to CVS.  
 (You might be able to do it via bookmarks and rebasing, but I've never  
 tried and have no desire to figure out how.)  
   
 If you don't want to keep the incremental history of your local  
 commits, use a branch.  
 If you do, use a patch queue.  
   
 It is possible to use multiple branches to allow you to commit back in  
 several stages.  
 However, managing this is a major pain and I don't recommend it -- you  
 might get away with two branches but more than that is probably a bad  
 idea.  
   
 There's a Mercurial extension called the "patch branch extension" that  
 lets you manage a whole graph of patches using branches.  
 I haven't tried using it in some years; at the time it had scaling  
 problems such that it became horrifyingly slow once you had more than  
 a handful of such branches.  
 That might have been improved in the meantime; if you find yourself  
 wanting to use both branches and patches, it might be worth looking  
 into.  
   
 It is also fairly probable that there is now a solution for merging  
 with patch queues; it's been a while since I had time to look closely.  
   
 ### Setting up  
   
 First, check out a CVS working tree.  
 You probably want to use a different one for each project, because  
 different projects require changing different parts of the tree and so  
 you will probably want to have Mercurial ignore different subtrees for  
 different projects.  
 (At least, I find it so; it depends on what you're working on.)  
   
         % cvs -d cvs.netbsd.org:/cvsroot checkout -dP src  
   
 Now create a Mercurial repository at the top level.  
 (If you are working only in a subtree and you are *sure* that you will  
 never need to change anything in other parts of the tree, you can  
 create the Mercurial repository in a subtree.  
 But unless you're absolutely certain, don't take the risk.)  
   
         % cd src  
         % hg init  
   
 If you're going to be using a patch queue, now enable mq.  
   
         % vi .hg/hgrc  
 and add  
         [extensions]  
         hgext.mq =  
 (Since the extension is built into Mercurial, that's all you need.)  
 You can if you prefer also put this in your .hgrc so mq is always on.  
 Then do  
         hg qinit -c  
 The -c option tells mq that you'll be checkpointing your patches,  
 which is usually a good idea.  
   
 Now prepare a .hgignore file.  
 This file contains one regular expression per line; Mercurial ignores  
 files (and subdirectories) whose paths from the repository root match  
 one of the regexps.  
 Add at least:  
         ^CVS$  
         /CVS$  
 to ignore all the CVS control directories in the CVS checkout.  
 While you can commit these to Mercurial, there's no point and it gets  
 awkward if owing to mistakes later you end up having to merge them.  
   
 If you aren't arranging to put the tree's object directories somewhere  
 else, then also add  
         ^obj\.[0-9a-z]$  
         /obj\.[0-9a-z]$  
 and you might want  
         ^sys/arch/[0-9a-z]*/compile/[A-Z]  
 to ignore kernel build directories.  
   
 Ignore subtrees that you aren't working in.  
 You don't have to bother to be very selective; the goal is to rapidly  
 rule out a few large subtrees that you definitely don't care about, in  
 order to avoid wasting time scanning them for changes.  
 Unless you plan to be working with 3rd-party software,  
   
         ^external$  
         ^gnu/dist$  
   
 is a good starting point.  
 Alternatively, if you aren't going to be working on MD kernel stuff or  
 bootloaders,  
   
         ^sys/arch$  
   
 is a good choice as it's also large.  
   
 You can always unignore stuff later, so don't worry about remote  
 possibilities.  
   
 Now commit the .hgignore file:  
   
         % hg add .hgignore  
         % hg commit -m 'add .hgignore file' .hgignore  
   
 Now add and commit the contents of the working tree:  
   
         % hg add  
         % hg commit -m 'HEAD of 20130101'  
 (or whatever date)  
   
 You are now in business.  
   
 ### Working  
   
 If you're using a branch, remember to change branches before you  
 commit anything:  
         % hg branch mystuff  
 You want to keep the default branch an untouched CVS tree so you can  
 use Mercurial to merge.  
 (And also so you can use Mercurial to extract diffs against CVS HEAD  
 and so forth.)  
   
 Similarly, if you're using a patch queue, put everything in patches  
 and don't commit.  
 (There's a section below about working with mq if you aren't familiar  
 with it.)  
   
 You can edit and build and test as normal.  
 Use hg commit or hg qrefresh to sync stuff into Mercurial.  
   
 If you're using mq, it's a good idea to checkpoint your patch queue  
 periodically.  
 This is done as follows:  
         % hg qcommit  
 The patches directory (.hg/patches) is stored in its own Mercurial  
 repository, and this commits the patches to that repository.  
 If necessary you can then fetch older versions of the patches back and  
 so forth.  
   
 ### Updating from CVS  
   
 First, make sure all your changes are committed.  
 (If you have unfinished changes that aren't ready to commit, there's a  
 Mercurial extension for stashing them temporarily.  
 If you have stuff that you don't want to commit at all, like debugging  
 printouts or quick hacks, it's often convenient to keep those in their  
 own mq patch, even if you aren't using mq for development.)  
   
 Now go back to a clean CVS tree.  
 If using branches, go back to the default branch:  
         % hg update -r default  
 If using mq, pop all the patches:  
         % hg qpop -a  
   
 DO NOT run cvs update until/unless you have done this; it will make a  
 mess.  
 When you eventually do this by accident, see the section below on  
 recovering from mistakes.  
   
 Now run cvs update from the top of the source tree:  
         % cvs -q update -dP  
   
 You should get no conflicts from CVS and nothing should show as  
 modified.  
 (It is usually a good habit to save the cvs update output to a file to  
 be able to check this.)  
   
 Tell hg to sync up:  
         % hg addremove  
   
 Use hg to check what it thinks has changed:  
         % hg status  
   
 Commit the changes to Mercurial:  
         % hg commit -m 'Updated to 20130202"  
   
 Now you get to merge.  
   
 If you're using a branch, you want to merge the changes into your  
 branch rather than merge your branch into the changes:  
         % hg update -r mystuff  
         % hg merge default  
         (edit and resolve as needed)  
         % hg commit -m 'sync with HEAD'  
   
 If it tells you "update crosses branches" when trying to update back  
 to your branch, update to the parent changeset (the previous version  
 from CVS) first, as that's an ancestor of your branch.  
   
 If you're using mq, the thing to do now is to push all your patches,  
 and if any reject, clean up the mess and refresh them.  
   
 If patch tells you "hunk N succeeded at offset MMM with fuzz Q", it's  
 a good idea to manually inspect the results -- patch being what it is,  
 sometimes this means it's done the wrong thing.  
 Edit if needed.  
 Then (even if you didn't edit) refresh the patch so it won't happen  
 again.  
   
 As I said above, it's quite likely that by now there's a better scheme  
 for merging with mq that I don't know about yet.  
   
 ### Pushing back to CVS  
   
 When you're ready to push your changes back to CVS (so they're really  
 committed), first (unless you're absolutely sure it's not necessary)  
 update from CVS as above and merge.  
 Then:  
   
 If you're using a branch, go back to the default branch and merge your  
 changes into it:  
         % hg update -r default  
         % hg merge mystuff  
         % hg commit -m "prepare to commit back to cvs"  
 Now cvs add any new directories and files; be sure not to forget this.  
 It is a good idea to crosscheck with cvs diff and/or cvs update:  
         % cvs diff -up | less  
         % cvs -nq update -dP  
 Then you can cvs commit:  
         % cvs commit  
 Because of RCSIDs, committing into cvs changes the source files.  
 So now you need to do:  
         % hg commit -m 'cvs committed'  
 and if you intend to keep working in this tree, you want to merge that  
 changeset back into your branch to avoid having it cause merge  
 conflicts later.  
 Do that as above.  
   
   
 If you're using a patch queue, usually it's because you want to commit  
 each patch back to CVS individually.  
 First pop all the patches:  
         % hg qpop -a  
 Now, for each patch:  
         % hg qpush  
         % hg qfinish -a  
         % cvs commit  
         % hg commit -m "cvs committed previous"  
 With a long patch queue, you'll want to use the patch comments as the  
 CVS commit messages.  
 Also, running cvs commit from the top for every patch is horribly slow.  
 Both these problems can be fixed by putting the following in a script:  
         hg log -v -r. | sed '1,/^description:$/d' > patch-message  
         cat patch-message  
         echo -n 'cvs commit -F patch-message '  
         hg log -v -r. | grep '^files:' | sed 's/^files://'  
 (I call this "dogetpatch.sh") and then the procedure is:  
         % hg qpop -a  
 then for each patch:  
         % hg qpush && hg qfinish -a && dogetpatch.sh  
         % cvs commit [as directed]  
         % hg commit -m "cvs committed previous"  
 (This could be automated further but doing so seems unwise.)  
   
 ### Using CVS within Mercurial  
   
 You can successfully do any read-only CVS operation in the hybrid  
 tree: diff, annotate, log, update -p, etc.  
 Read-write operations should be avoided; if you mix upstream changes  
 with your changes you will find it much harder to commit upstream  
 later, and you may get weird merge conflicts or even accidentally  
 revert other people's changes and cause problems.  
   
 If you clone the Mercurial tree and you didn't include the CVS control  
 files in it, you won't be able to do CVS operations from clones.  
 Including the CVS control files in the Mercurial tree is one way  
 around that.  
   
 You will find that any large CVS operation on a clone is horribly  
 slow.  
 This is because making a clone causes CVS to think all the files in  
 the clone have been modified since you last ran it; it then re-fetches  
 every file you ask it about so it can update its own information.  
 For this reason cloning the Mercurial tree usually isn't worthwhile  
 and even when it is, including the CVS files in the Mercurial tree  
 isn't.  
   
 Another consequence of this: do not try to cvs update in a cloned  
 Mercurial repository; use only the original.  
 Updating a clone basically downloads the entire tree over again from  
 the CVS server.  
   
 DO NOT CVS COMMIT FROM A CLONE.  
 It is known that some operations that muck with the timestamps in a  
 CVS working tree can cause CVS to lose data.  
 It is not clear if hg clone is such an operation; don't be the person  
 who finds out the hard way.  
   
 ### Recovering from mistakes  
   
 The most common mistake is CVS updating when the Mercurial tree is not  
 in the proper state from that; e.g. onto your branch or while you have  
 patches applied.  
   
 The basic strategy for this is to use hg revert to restore the part of  
 the tree it knows about, then go back to CVS, clean up the mess there,  
 and update properly.  
   
 If you're using a branch:  
         % hg revert -C  
         % hg update -r default  
 If you're using a patch queue:  
         % hg revert -C  
         % hg qpop -a  
   
 The problem is, CVS will now think you've changed every file that  
 Mercurial is managing, and the modifications are to revert all the  
 changes that have happened since your previous update.  
 You do *not* want that to turn into reality.  
 Hunt down (with cvs -n update) any files that CVS thinks are modified,  
 then rm them and run cvs update on them.  
 CVS will print "Warning: foo was lost" and restore an unmodified copy.  
   
 When you have no files left that CVS thinks are modified, do a CVS  
 update on the whole tree and merge it as described above.  
 (You must do this, as the parts of the tree that Mercurial is ignoring  
 will otherwise be out of sync with the parts it's managing.)  
   
 If you stored the CVS control files in Mercurial, then the revert will  
 restore them, but your tree will still be inconsistent so you still  
 need to do a proper update and merge immediately.  
   
 ### mq  
   
 The basic idea of mq (like quilt) is it maintains a series of patches  
 against the source tree, that are to be applied in order.  
 By applying them or removing them one at a time, you can move the tree  
 to any intermediate state; and then you can update the topmost patch,  
 insert a new patch, or whatever.  
   
 To see the list of patches:  
         % hg qseries  
   
 To apply the next patch:  
         % hg qpush  
   
 To remove the current patch:  
         % hg qpop  
   
 To merge current working tree changes into the current patch:  
         % hg qrefresh  
   
 To also update the current patch's change comment:  
         % hg qrefresh -e  
   
 To collect current working tree changes (if any) into a new patch:  
         % hg qnew PATCHNAME   
   
 When there's an mq patch applied, you can't commit.  
 (Doing qrefresh is basically equivalent to committing the current  
 patch.)  
 Diff will show the changes against the last refreshed version of the  
 current patch; to see the complete changes for the current patch  
 (including current changes), use "hg qdiff".  
   
 You can delete patches with "hg qrm" and rename them with "hg qmv".  
   
 Patches are applied with patch, unfortunately, which means that if  
 they don't apply (which can happen if you or someone else changes  
 something under one) you get .rej files you have to clean up by hand  
 rather than a Mercurial merge.  
   
 When a patch is ready to be committed for real, you do "hg qfinish" on  
 it.  
 This removes it from the patch queue and converts it to a normal  
 Mercurial changeset.  
   
 To change the ordering of patches, you edit the file  
 .hg/patches/series.  
 If the patches aren't orthogonal you'll have to fix the rejections  
 when you next apply them.  
 (Don't do this with patches that are currently applied.)  
   
 Use "hg help mq" to see the full list of mq-related commands.  
   
 I'm sure there are better mq tutorials out there.  
   
 ### Using mq  
   
 The basic process when using mq is that you start a new patch, edit  
 and hack for a while, use hg qrefresh to commit it (once or many  
 times), and when you're done go on to the next one.  
   
 If you find a bug in an earlier patch, you can go back to the patch  
 that introduced it and fix the bug there, creating a new version of  
 the offending patch that no longer contains the bug.  
 (Or you can create a new patch that fixes the bug, but insert it  
 immediately after the patch that created the bug.)  
   
 When a patch is ready to be seen by other people, you "finish" it and  
 then it becomes a normal immutable changeset.  
   
 One catch is that you can't push or pop the patch queue while you have  
 unsynced (uncommitted) changes.  
 There are two ways around this; there's a separate "stash" extension  
 that lets you put unfinished changes aside while you do something else.  
 Or, alternatively, you can create a new temporary patch holding your  
 unfinished changes, and then later use hg qfold to combine that with  
 the patch you originally meant this for.  
   
 A variant of this problem is when you discover a bug, open an editor,  
 fix it, and then realize that you wanted to make the edit in an  
 earlier patch.  
 Then you go to pop the queue and it complains that you have a modified  
 file.  
 If the modification in question is the only uncommitted change, the  
 best way to deal with this is to create a new patch for it, then pop  
 to where you wanted it to go and use hg qfold to apply it there.  
Removed from v.1.1  
changed lines
  Added in v.1.4

CVSweb for NetBSD wikisrc <wikimaster@NetBSD.org> software: FreeBSD-CVSweb