buildbot try not working with mercurial

[bgunnison]

Several issues, some may be in my misunderstanding how to use it.

.buildbot/options:

try_connect = 'pb'
try_vc = 'hg'
try_master = '....'
try_username = '...'
try_password = '...'
try_builders = ['... Smoke Build']

1. try client calls "hg diff" in working repository. If there are no uncommitted changes, there is no output. Should "try" fail at the client if there are no changes? Or at least inform the user?

1. The build step that try invokes is:

hgUpdateStep = Mercurial(repourl = RepoURL)

RepoURL is the master repository that the working repository was cloned from.

This fails at the buildslave at:

hg.exe update --clean --repository <build_slave_path>uild --rev <workdir_revision_hash>
...
abort: unknown revision '<workdir_revision_hash>'!

It seems we are not applying the patch as the hg command above expects the build slave repository (which was cloned from the master repo) to be update-able to the working directory. Something is missing or not working.

1. So if I have the use-case correct, only uncommitted changes get "tried" in the above scenario. Our workflow is such that we clone from the master repo and commit many times in the working repo. Then a "hg push" is done from the working to the master after smoke test passes. I need "buildbot try" to interrogate the master repo, then "hg diff -r master_rev -r working_tip" to get the difference between the master and working repos. Then try those changes. Is this possible?

Thanks,

Brian

[marcusl]

Correct. The try was written before DVCS:es, so it doesn't check for non-pushed commits, which it should.

We need to decide whether to:

• send the missing changesets + patch, and optionally clean/strip the slave-repo afterwards
• or, just send one large patch between the user's repo and the slave's 'nearest' revision.

Patches welcome. I think this is a good thing to get fixed. :)

[bgunnison]

Replying to marcusl:

Correct. The try was written before DVCS:es, so it doesn't check for non-pushed commits, which it should.

We need to decide whether to:

• send the missing changesets + patch, and optionally clean/strip the slave-repo afterwards
• or, just send one large patch between the user's repo and the slave's 'nearest' revision.

Patches welcome. I think this is a good thing to get fixed. :)

I was thinking of using hg bundle/unbundle

1) Determine Master rep (MR) common ancestor with Working Repo (WR). This produces a CAR (Common Ancestor Revision) 2) do "hg bundle" in WR to package changes since CAR 3) Send CAR and bundle to Build Slave (BS) 4) BS Clones MR to CAR then applies bundle (this is just like a push) and will exactly replicate WR.

I like the bundle approach as a compressed set of diffs is reliable over networks. An error in text diffs can be very bad.

Thoughts?

[marcusl]

Replying to bgunnison:

I was thinking of using hg bundle/unbundle

1. Determine Master rep (MR) common ancestor with Working Repo (WR). This produces a CAR (Common Ancestor Revision)
2. do "hg bundle" in WR to package changes since CAR
3. Send CAR and bundle to Build Slave (BS)
4. BS Clones MR to CAR then applies bundle (this is just like a push) and will exactly replicate WR.

I like the bundle approach as a compressed set of diffs is reliable over networks. An error in text diffs can be very bad.

Thoughts?

Sounds good!

Having a persisting local clone of the MR would make this even more efficient (less network traffic etc), but we don't need to worry about that in this context. (That requires some global support from buildbot, since we probably want to do something similar for all VCS:es)

[pepsiman]

Replying to marcusl:

We need to decide whether to:

• send the missing changesets + patch, and optionally clean/strip the slave-repo afterwards
• or, just send one large patch between the user's repo and the slave's 'nearest' revision.

Sending one large patch seems the simplest option to me.

The diff between the parent of the first draft changeset on the current branch and the user's working directory can be obtained using

hg diff -r 'parents(first(draft() and branch(parents())))'

This also works when using mercurial queues (ticket #180).

[pepsiman]

 https://github.com/pepsiman/buildbot/compare/pepsiman:master...pepsiman:mercurialtry

[Malcolm Parsons]

try: For mercurial, create a patch containing all outgoing changesets on the current branch. (fixes #865, #180)

Changeset: d867a203b2cf613422a0971f5e31789c9259c339

[Dustin J. Mitchell]

Merge branch 'mercurialtry' of  git://github.com/pepsiman/buildbot

• 'mercurialtry' of  git://github.com/pepsiman/buildbot: try: For mercurial, create a patch containing all outgoing changesets on the current branch. (fixes #865, #180) Mercurial: Remove any added files before running update --clean Mercurial: Support applying a patch for Try builds Changeset: 5aee01d01da16de441ccd43af5806f1db087ed91

[Malcolm Parsons]

try: For mercurial, create a patch containing all outgoing changesets on the current branch. (fixes #865, #180)

Changeset: d867a203b2cf613422a0971f5e31789c9259c339

[Dustin J. Mitchell]

Merge branch 'mercurialtry' of  git://github.com/pepsiman/buildbot

• 'mercurialtry' of  git://github.com/pepsiman/buildbot: try: For mercurial, create a patch containing all outgoing changesets on the current branch. (fixes #865, #180) Mercurial: Remove any added files before running update --clean Mercurial: Support applying a patch for Try builds Changeset: 5aee01d01da16de441ccd43af5806f1db087ed91

[Dustin J. Mitchell]

Merge branch 'master' into nine

• master: (80 commits) Force defaultBranch if repoURL is set. tabs to spaces upgrade to use the new getProcessOutput expectations framework fix pyflakes Doc fixes try: For mercurial, create a patch containing all outgoing changesets on the current branch. (fixes #865, #180) Mercurial: Remove any added files before running update --clean Mercurial: Support applying a patch for Try builds Add some test to doc and release-notes about HTPasswdAprAuth. Fix master-side SVN step. skip test_test_util_gpo on py27+tw900 Updated a partly wrong docstring in hgpoller Moved os.environment change to setUp/tearDown in hgpoller tests Move libaprutil password checking to an new subclass Skip instead of hide md5 and sha tests if libaprutil is missing. Use ctypes.util.find_library to resolve library name. Remove old GetProcessOutputMixin?. Fix gitpoller tests of environment passing. Fix p4 poller tests to use new gpo mixin. Fix svn poller tests to use new gpo mixin. ...

Conflicts:

master/buildbot/test/unit/test_changes_gitpoller.py master/buildbot/test/unit/test_changes_p4poller.py master/buildbot/test/unit/test_changes_svnpoller.py master/docs/manual/cfg-global.rst

Semantic Conflicts:

master/buildbot/changes/hgpoller.py + tests updated to use data API

Changeset: ece5f76ae1953974684f5573a3718e35c1864d9e