[[!meta title="Using Git with the NetBSD Packages Collection"]]

*This document is a proposed draft. Please feel free to send comments to cjep@*.

[[!toc ]]

# Background

In April 2024, the [NetBSD Packages Collection](https://pkgsrc.org) management team (pkgsrc-pmc) decided to migrate from [CVS](https://www.nongnu.org/cvs/) to [Git](https://git-scm.com/) for managing the source code of the NetBSD Packages Collection. This document covers the basic commands to manipulate the sources using Git.

A key difference between Git and CVS is that your machine maintains a full local copy of the source code repository. Changes are maintained locally and are *pushed* back to the remote repository. Changes from the remote repository can be *pulled* and merged with your local copy.

[[!img repositories.svg size=490x align=right]]

Initially the repository will be accessible via SSH by NetBSD developers only. To submit patches please see [below](#index10h1).

# Initial Setup

Install ```pkgsrc/devel/git-base``` for a minimal installation of ```git```. To do this from source, you will need to [download pkgsrc first](http://cdn.netbsd.org/pub/pkgsrc/current/pkgsrc.tar.xz). For non-NetBSD platforms you will need to bootstrap - please see [the documentation](http://www.pkgsrc.org/#index1h1) for more details

```
# From pkgsrc
cd pkgsrc/devel/git-base
make install

# From pkgsrc on non-NetBSD platforms
cd pkgsrc/devel/git-base
bmake install

# Or pkgin
pkgin install git-base
```

Alternatively use ```pkgsrc/devel/git``` for ```git```, it's documentation and other contributed tools.

Move your existing ```pkgsrc``` directory out of the way. 

```
mv pkgsrc pkgsrc.old
```

Use ```git clone``` to obtain the source tree via ```git``` as follows:

```
git clone xxxhosttbaxxx.netbsd.org:/home/git/pkgsrc.git 
```


Set up your user and e-mail address. Please use your pkgsrc.org e-mail address and not your netbsd.org one.

```
cd pkgsrc
git config --local user.name "My Name"
git config --local user.email "myuserid@pkgsrc.org"
```

We only allow "rebasing" (see [Syncing with the remote repository](#index5h1) below). Please set this in your local configuration:

```
git config pull.rebase true
```

# Basic usage

To add, remove or move files:

```
git add new-file.c
git rm old-file.c
git mv old-file.c new-file.c
```

To inspect the state of your tree or examine changes since the last commit:

```
git diff
git status
```

# Committing changes

New files need to be added with ```git add```. 

```
git add newfile
# next commit will automatically include newfile
git commit
```

You can either stage existing files with ```git add``` for your next commit or specify them on the command line:

```
git add README.md
git commit

# or
git commit README.md
```

```git``` will open your file editor to edit the commit message. However it can be specified directly on the command line:

```
git commit -m "Make cool changes to the documentation" README.md
```

You can stage and commit all changes with ```-a``` but the preferred method
is to specify the filenames explicitly.

```
git commit -m "Make lots of cool changes at once" file1 file2 file3 dir1
```

# Syncing with the remote repository

You can synchronise your current branch from the remote repository with ```git pull```.  To synchronise your local repository fully with the remote repository use ```git fetch```. 
You can submit your local changes to the remote repository with ```git push```.

Before you use ```git push```, you can examine the change that will be pushed with ```git log -p upstream/main```.

When you ```git pull```, you may have to resolve conflicts between the remote changes and your local changes before continuing. Additionally when you ```git push```, the remote repository may have changes that you do not have. You will need to ```git pull``` first.

Git will attempt to include the changes from the remote repository. You may receive this error message the first time you use ```git pull``` if you have not set up git already:


```
warning: Pulling without specifying how to reconcile divergent branches is
discouraged. You can squelch this message by running one of the following
commands sometime before your next pull:

  git config pull.rebase false  # merge (the default strategy)
  git config pull.rebase true   # rebase
  git config pull.ff only       # fast-forward only

You can replace "git config" with "git config --global" to set a default
preference for all repositories. You can also pass --rebase, --no-rebase,
or --ff-only on the command line to override the configured default per
invocation.
```

The ```rebase``` and ```merge``` methods are ways of integrating changes from divergent branches. Please see the git-merge(1) and git-rebase(1) manual pages for more details. The "Fast-Forward only" method is useful if you are just following the remote repository but are not making local changes. We use (and enforce) the ```rebase``` method for the NetBSD Packages Collection. You can set this as follows:

```
git config pull.rebase true
```

If the local repository and branch contain conflicting changes, you will need to fix the conflicts by hand, re-add the files and commit them to the local repository. For example, here we make a local change that conflicts when we attempt to push our repository. We fetch the changes, resolve the conflicts and push our changes again. The ```rebase``` method requires us to re-add the conflicting files and use ```git rebase --continue```. Please see the git-rebase(1) manual page for the detail.

```
$ git commit -m "mychange" test
[master a013462] mychange
 1 file changed, 1 insertion(+)
$ git push
...
error: failed to push some refs to 'githut.co.uk:randomrepo/repo.git'
hint: Updates were rejected because the remote contains work that you do
hint: not have locally. This is usually caused by another repository pushing
hint: to the same ref. You may want to first integrate the remote changes
hint: (e.g., 'git pull ...') before pushing again.
hint: See the 'Note about fast-forwards' in 'git push --help' for details.
$ git pull

...
Auto-merging conflict
CONFLICT (content): Merge conflict in conflict
error: could not apply 716ce63... test
hint: Resolve all conflicts manually, mark them as resolved with
hint: "git add/rm <conflicted_files>", then run "git rebase --continue".
hint: You can instead skip this commit: run "git rebase --skip".
hint: To abort and get back to the state before "git rebase", run "git rebase --abort".
Could not apply 716ce63... test
$ vi conflict
$ git add conflict
$ git rebase --continue
$ git push
```

# Adding a new package

To add a new package, ensure that your tree is up to date. The process
is similar to the one using CVS.

```
$ git pull
$ cd .../pkgsrc/category
$ mkdir pkgname
$ cd pkgname
# setup DESCR, Makefile, PLIST, distinfo, etc and add them
$ git add DESCR Makefile PLIST distinfo buildlink3.mk
$ git add patches/p*
$ cd ..
# add "SUBDIR+=pkgname" line to the parent Makefile
$ vi Makefile
$ git commit -m "category/pkgname: add new package" Makefile pkgname
$ cd pkgname
$ make CTYPE=Added PKG_DEVELOPER=yes commit-changes-entry
$ git push
```

# Branches

Any changes you make with git are done to the current branch in your local copy of the repository. The *main* branch is essentially the trunk of the repository. Although you can make local branches, you will not be able to push them to the remote repository. We will only use branches to maintain quarterly releases. Pkgsrc developers should continue to commit to *main* in the same way they have been committing to *head* with CVS.

# Quarterly Releases

The NetBSD Packages Collection has [quarterly stable releases](http://www.pkgsrc.org/quarterly/). The release comprises of a branch so that a consistent set of packages can be built and managed. The naming convention of the branch is ```pkgsrc-YYYYQN``` where *YYYY* is the year and *N* is the quarter number.

To obtain the release branch for Q1 2024 use:

```
git fetch
git checkout pkgsrc-2024Q1
```

Never commit directly onto a release branch. Always commit onto *main*. If you need a change in a release branch please refer to the next section.

# Submitting a pull-up request to a release branch

Please refer to the [developer's pull-up guide](https://www.netbsd.org/developers/releng/pullups.html). Pull-ups for pkgsrc should be sent to the ```pullup-pkgsrc``` e-mail group. You can send a pull-up request either by:

- Sending a commit hash to ```git cherry-pick```
- Sending a unified diff file if a more complicated patch is required.

The commit hash can be found in the output of ```git commit```:

```
$ git commit -m "fix off-by-one security problem in lang/nawk." -a
[main ba3116fda897] spurious change
1 file changed, 1 insertion(+)
```

or ```git log```:

```
$ git log
 commit ba3116fda897a387ca12c5f4abcc9f9b49a85c9b (HEAD -> main, origin/main, origin/HEAD)
Author: A. Hacker <ahacker@mydomain.org>
Date:   Sun May 19 19:26:24 2024 +0100

    fix off-by-one security problem in lang/nawk.
```

An example of a good pull-up request is:

```
From: A. Hacker <ahacker@mydomain.org>
To: Package Source Pull-up requests <pullup-pkgsrc>
Subject: Urgent nawk fix

	Hello,

"lang/nawk" needs a security fix. Please use the following commit 
ba3116fda897a387ca12c5f4abcc9f9b49a85c9b to incorporate the fixes on pkgsrc-2023Q4.

Yours sincerely
A. Hacker
```

The release engineer can apply the change using ```git cherry-pick``` as follows:

```
$ git fetch
$ git checkout pkgsrc-2023Q4
$ git cherry-pick ba3116fda897a387ca12c5f4abcc9f9b49a85c9b -e -x
# Accept or change commit messages, make any changes or fix conflicts
$ git push
```

# Submitting patches

We are in the early stages of our migration to Git. We expect to offer public pullup requests at some point in the future. But for now, if you do not have access to the NetBSD Packages Collection source tree directly, you can still submit patches. For example:

```
git fetch
cd lang/nawk
# Make changes to the nawk package
git diff -u > /tmp/_mypatch.txt
```

You can submit *_mypatch.txt* directly to the maintainer listed in the package *Makefile*, discuss it on a mailing list or [file it as a problem report](https://www.netbsd.org/cgi-bin/sendpr.cgi?gndb=netbsd).