Git: Initial clone should use "--depth 1" option

[tfogal]

The git repositories that buildbot needs could be considered "leaf" repositories. That is, one doesn't expect that the repository buildbot checks out would be used as an upstream for anybody else. As such, in the common case, buildbot only needs "the latest" revision in the repository.

To reduce disk load and processing time, but mostly network bandwidth, it would be nice if buildbot only grabbed a "shallow" clone. Such clones are not useful for browsing history, but buildbot does not do that regularly.

This all breaks down if the user requests a specific revision. Presumably, we could fetch only that revision and later, but the effort involved in figuring that out seems difficult.

[tfogal]

(Incomplete?) patch to implement shallow clones.

[tfogal]

The attached patch is the start of an implementation. Unfortunately it suffers from an issue: the initial clone implicitly does a git checkout, and then git reset --hard FETCH_HEAD that runs afterwards ends up doing another checkout. This might be avoidable if we switch to "git clone ... ; git checkout -f branch; git clean ..." instead of the fetch business that is going on now.

Then again, it might just be an artifact of my test repository. "git clone" on this repo does not give the branch I specify in master.cfg, so it could be this works fine && it's only my configuration being suboptimal.

[dustin]

If I understand your worry correctly, it's hat 'git fetch' is happening twice. But it isn't downloading the commits twice, right? I'm worried about changing the behavior of any of the VC modes - a git clean is anathema to mode=update, for example.

[marcusl]

IIUC, this is similar to some discussion we had on Mercurial, where we decided to keep the full repo on disk between update, but for clobber mode, just retrieve the specified. For git, cloning shallow in clobber mode makes sense, in update, not as much.

(I also think that it'd be cool to write something that would allow a buildslave to cache DVCS:es locally, outside the build.dir, then do hard-link clone into the buld dir. That would reduce bandwidth usage between slave and repo servers, but still have access to the full history locally.)

[tfogal]

This was actually fixed in:

5c269c04ab20ad82b3ca9921d34f7a6a1fe6baf0 , 8b6e8c3436762b9d45f6a1f749f180ed2553b44e , and 7d2ccda6386f7f0cce0bc7dc101621ba114e7f1c

(sort of, user-configurable, not default) but the bug was never closed.