Opened 6 years ago

Last modified 5 years ago

#2121 new enhancement

Allow limited-scope Perforce checkouts

Reported by: gmcnew Owned by:
Priority: patches-accepted Milestone: 0.9.+
Version: 0.8.4p2 Keywords: p4
Cc:

Description

I use Buildbot for projects stored in Perforce, and it works great. However, our builds require one's clientspec to span hundreds of gigabytes of data, very small portions of which may be synced during the build.

You're probably wondering, "Why do your builds need to do that?" I could provide more detail, but it's complicated, proprietary, and, most importantly, out of my control. <sigh> Sorry.

The default behavior of a P4 sync step is to sync everything in the clientspec. However, this simply does not work for me, because the clientspec is far too broad.

As a workaround, I've hacked my Buildbot slaves' p4.py to only sync p4base, rather than everything in my clientspec. This works fine, but I'd really love similar functionality to be in Buildbot proper so my coworkers and I won't have to hack Buildbot's code every time we install it.

I'll be happy to write and test a patch for this.

Mostly I'd like a little feedback to figure out:

  1. the best way to implement this
  2. whether a patch would have a chance of actually being mainlined

My first instinct would be to add an optional "p4sync" argument to P4(). The value would be a Perforce filespec (example: "//depot/path/to/sync/..."). If supplied, P4._doP4Sync() would pass this string verbatim to "p4 sync" to limit the scope of the sync. If not supplied, nothing would change -- the entire clientspec would be synced, as before.

Thoughts?

Change History (6)

comment:1 Changed 6 years ago by fcrestois

I started long time ago like you, after a string, I was passing an array of workspace line, quickly I let the slave creating (and updating) the workspace client view and starting to manage the exclusion (-) , at the end, most of the change was on the master.cfg was to modify workspaces views.

It’s why now I use only one line description (no more restart of the master) The location of the p4s file (p4 client –o) in perforce p4sync("//depot/project/project.p4s")

I use also this argument to “p4scheduler” the builder only if the changed files map in the workspace view.

Last edited 6 years ago by fcrestois (previous) (diff)

comment:2 follow-up: Changed 6 years ago by dustin

To speak to your two questions:

  1. This should be implemented in a master-side P4 step. We've ported several other VC's to the master side, but since Perforce is proprietary, we weren't able to do so for P4. Still, there are sufficient examples to make this clear. Once the port is complete, I expect it will be fairly trivial to add some new step-constructor parameters (or alter existing parameters, since the master-side implementation represents a clean break) to support the functionality you're looking for.
  2. As for being maintained - I can only guarantee that any unit tests you write will continue to pass. Perforce is currently listed as "Orphaned" in MAINTAINERS.txt:
    Perforce VC
        S: Orphaned
        U: http://trac.buildbot.net/wiki/p4
    
    Of course, you're welcome to step up as a maintainer of this component!

comment:3 in reply to: ↑ 2 Changed 6 years ago by gmcnew

Thanks Dustin! I'll look into the master/slave-side stuff.

Replying to dustin:

  1. As for being maintained - I can only guarantee that any unit tests you write will continue to pass. ...

I think you misread "mainlined" as "maintained" in my description above. =) What I was really asking was "would a patch for this feature have a chance of being accepted, or is this use case too rare?"

comment:4 Changed 6 years ago by dustin

I did misread it :)

Honestly I don't know enough about perforce to understand what you're suggesting, but if it's written, tested, and documented, I'm happy to merge it.

comment:5 Changed 6 years ago by dustin

  • Milestone changed from undecided to 0.9.+
  • Type changed from undecided to enhancement

comment:6 Changed 5 years ago by dustin

  • Priority changed from major to patches-accepted
Note: See TracTickets for help on using tickets.