ARM Linux Developer - rmk's GIT tree

This is a guide concerning Russell's GIT tree. It is not meant to be a general guide about GIT trees, or how to even run a GIT tree. Please seek such advice elsewhere.

About GIT Trees

GIT is an distributed source control system, created by Linus Torvalds, and is widely used by kernel developers and others to manage source trees. For the kernel, anyone with an up-to-date copy of Linus' GIT tree has the complete kernel history back to v2.6.12-rc2 stored locally.

A commit in the GIT sense is a container for a set of changes to a set of files, which gives an overall explanation of the change and various attributions. It will normally encompass one logical change. In the classical emailed patch sense, the commit text is equivalent to the textual description at the beginning of the email, and the set of changes are equivalent to the appended patch.

GIT trees can be merged together via a 'git pull', which can fetch the history and changes from another git tree into your local tree.

GIT allows the creation of multiple branches of development, sometimes called 'topic branches' which can then be merged together.

A full description of GIT is available from other sources; please refer to the git documentation for a more complete description.

Stephen Rothwell's linux-next Tree

Stephen Rothwell runs a project called linux-next, which is a testing ground for GIT trees. Stephen merges lots of GIT trees together from different parts of the Linux community, and then tries to build the result for several kernel configurations. Full details can be found on the  linux-next project site.

As part of this activity, Stephen generates reports on merge conflicts between trees, copying them to the conflicting tree owners and people mentioned in the responsible conflicting commits, as well as reporting some build failures.

RMK's GIT Tree - General Operation

Russell uses git as a means of storing and sorting patches, and queueing up work for submission to mainline. These patches are organised as a set of `topic' branches, which are created as necessary.

It is important to note that there are two types of branches - stable and unstable - but there is no way to know by looking at a GIT tree whether a any particular branch is stable or unstable. This depends entirely on the behaviour of the tree owner, and how the tree owner operates that tree.

• Stable branches guarantee that any commit which has been published becomes immutable. This branch can be relied upon as a basis for further work, and can be pulled into other git trees without fear that the commits may become invalidated.
• Unstable branches do not give that guarantee, and the commits found on such branches may be deleted, updated, re-based, or even the entire branch thrown away. Commit messages may be updated to include the 'acked-by', 'reviewed-by' and 'tested-by' attributions which are a normal part of the way the Linux community reviews and handles patches. For this reason, unstable branches must never be used as a basis for further work.

It is possible through agreement that an unstable branch can be converted to being a stable branch - but this requires discussion with the tree owner.

Examples of topic branches are: `aaci' for ARMs AC'97 audio codec interface, `mmci' for ARMs multimedia card interface, `ep93xx' for Cirrus EP93xx related patches and `p2v' for the dynamic physical to virtual mapping patches. These branches generally start off as 'unstable' as comments are received and attributions added, and become more 'stable' as they settle down.

There are three main visible branches:

• master (unstable)
  This branch generally contains Linus' head of tree, and occasionally may have the fixes branch merged at its head. This is an unstable branch.
• fixes
  This branch contains minor fixes for -rc kernels, and is regularly requested to be merged into mainline kernels. This may also include trees from 3rd parties.
• devel-stable (stable)
  When Russell pulls git trees from 3rd parties, he pulls them into the devel-stable branch, which as the name suggests, this is a stable branch. Russell guarantees that any commit in the 'devel-stable' branch will never change.

  Some topic branches which have become stable are also merged into 'devel-stable' as necessary.

• for-next (unstable) [1]
  The for-next branch is a testing branch, created by regularly merging a set of branches together, including the fixes and devel-stable branches. It is primarily published for inclusion into Stephen Rothwell's linux-next tree, but may be used by others for a basis of their own testing. As this branch is regularly destroyed and re-merged, sometimes several times a day, it is 'unstable'.

These branches are available in several forms:

• Overall combined patches
  Overall combined patches are generated where possible from the GIT branch, and contain a header with each commit text. These are available from the FTP site with a suffix of ".diff". The files are named arm.diff for the master branch, or arm:<branch-name>.diff for individual branches.
• Mailbox archive
  Mailbox archive format contains one commit per message, and contains a thread structure which reflects the branches in the GIT tree. For simple branches, it's possible to re-apply these to a git tree using git am. For branches containing merges, it's possible to directly recreate the GIT tree structure from these mailbox archives. The mailbox archives are available from the FTP site with a suffix of ".mbox". The files are named arm.mbox for the master branch, or arm:<branch-name>.mbox for individual branches.
• GIT tree
  Russell's git tree is available via the GIT URL: http://ftp.arm.linux.org.uk/pub/linux/arm/kernel/git-cur/linux-2.6-arm.git and the URL should be followed by the branch name. Eg,

  git fetch http://ftp.arm.linux.org.uk/pub/linux/arm/kernel/git-cur/linux-2.6-arm.git fixes

  If you are using a version of Git older than v1.6.6 which has no support for the smart GIT protocol over HTTP then Russell recommends that you ensure that the tree into which you're fetching is first up to date with Linus' GIT tree before you fetch from his tree. This will avoid lengthy fetches caused by git downloading (currently) 450MB of data when Linus' tree is 'repacked'. This size will only get bigger over time.

  The other issue to watch out for is that an aborted fetch will leave a partially transferred pack file or index file in .git/objects/pack, which will cause problems when you next try to fetch. If git complains about missing objects, this is most likely what has happened.

  You could set up your repository like this:

  git remote add linus git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6
  git remote add rmk http://ftp.arm.linux.org.uk/pub/linux/arm/kernel/git-cur/linux-2.6-arm.git

  and then, to periodically bring your local copy of those remote branches up to date:

  git fetch linus
  git fetch rmk

  It is then easy to see what's in Russell's branches which has not reached Linus' tree, simply by looking at the branch of interest such as rmk/fixes, rmk/devel-stable, merging those together in a test branch with your own local branches, etc.

• Gitweb
  Gitweb allows on-line viewing of a GIT tree. Russell's GIT tree can be viewed via gitweb here.

RMK's GIT Tree - Timing

Immediately after a final kernel release (such as v2.6.38), a merge window of up to two weeks opens to allow new development into the main kernel. The end of this merge window is marked by the -rc1 release, and then there are a series of -rc releases until the next final kernel release. The cycle then starts over.

The number of -rc releases is indeterminate, but the last -rc release is usually somewhere between -rc6 and -rc9. It is a good idea to keep an eye on Linus' -rc announcements to get a feel of how the cycle will pan out.

As the merge window is relatively short, merge issues and tree ordering needs to be resolved prior to the merge window opening. This requires all git-based changes for a merge window to be in their respective git trees prior to the merge window opening - and preferably in linux-next so that the merge conflicts can be flagged. Russell would prefer to allow about a week for things to settle before the merge window opens.

New development which has not been included in linux-next from other trees and new patches will not be merged into Russell's git tree until after the merge window has closed. However, regression fixes and obvious bug fixes are welcome at any time and will be merged irrespective of the merge window timing. Bug fixes are welcome up until the last couple of -rc releases, depending on complexity.

RMK's GIT tree - Pull Requests

When a 3rd party requests a tree to be pulled into Russell's tree, they send Russell a 'pull request'. This pull request is an email message containing certain basic information:

• The git-compatible URL to fetch the tree from. This should be on a single line.
• The base commit which both trees should have.
• The expected diffstat for the resulting merge. This is so that Russell can check that what was intended to be pulled was what was pulled.
• A summary of the included changes.

git request-pull is an ideal way to generate suitably formatted messages, which ensures that all the above information is included. Custom pull messages are also acceptable provided the basic information is included. Please refer to GIT documentation for further information.

It is also good practice to include an overall explanation for the pull, which will be passed on to Linus when Russell sends his pull request.

Once a pull request has been sent, the requested branch should not be altered until it has been pulled.

Patches corresponding to the commits should have been sent to the relevant mailing lists for review, and attributions added prior to the pull request being sent.

RMK's GIT Tree - Patches

Emailed patches tend to be a problem for Russell for two reasons. They tend to get buried beneath lots of other ARM related email, resulting in them being missed. Secondly, Russell reads email on a different machine to the one which the GIT tree resides upon, and this requires additional steps to copy patches between machines. Thirdly, email clients tend to corrupt patch files, making them impossible to apply.

The overall result of this means that Russell has a very poor record for applying patches sent by email to his addresses. To work around this, Russell setup a patch system many years ago which allows contributors to submit their patches to a system which won't accidentally forget them. (Russell has said that sometimes the patch system is sometimes too good at remembering patches.)

Full details can be found in the patch system's extensive documentation. Essentially, the process is similar to that used to send patches to mailing lists, with the exception that the patch system requires a kernel version tag.

Notes

1- This used to be called 'devel', but Russell has recently decided to rename this branch to make its purpose clear.

FROM:http://www.arm.linux.org.uk/developer/git-arm.php