wiki:Meeting17December2010

Version 2 (modified by rutsky, 2 years ago) (diff)

update "slave" with "worker"

This was primarily an organizational summit, although technical topics are certainly welcome as well.

Agenda

  • Purpose
    • Agree on practices and policies that we can carry out
      • clarity avoids ambiguity and potential conflict
      • new and existing users can know what to expect
      • baseline for discussions about making changes
    • Discuss ways to broaden the developer base for Buildbot
      • lots of contributors (yay!), relatively few maintainers
      • limited capacity for work on big projects
      • project direction follows maintainer's interests
      • bus factor
    • do we somehow block potential maintainers?
      • how can we better recognize contributions?
    • can we establish intermediate roles for devs?
  • 0.8.3 news
  • Developer Recruitment
    • define components (see tags/keywords)
      • In VC; jettison unmaintained, buggy components
    • developer roles
      • core maintainer, component maintainer, tester, packager, infrastructure, security
    • developer permissions
      • commit, root, trac admin, IRC +v
    • promotion process
      • how does a contributor know when to ask for commit rights?
    • Google Summer of Code?
      • needs someone to drive it
  • Policies
    • security incident response
      • team membership
    • version requirements for dependencies
      • Python, Twisted, Jinja, SQLAlchemy, etc.
    • review
    • licensing
      • stuck with GPLv2; can we make new components?
  • Release Policies
    • release pace/timing (current: every 2-3mo)
    • support (fixes) for old versions (any? how long?)
    • release process changes
      • currently rc's for packagers, then full release
    • downstream communication
      • better communication with packagers?
  • Brand upgrade
    • boilerplate "about Buildbot" text needs work
    • consistent visual design for docs, Metabuildbot, Trac
  • Technical Futurey Stuff!
    • 0.9.0: out of process status serving
      • statusdb (history)
      • statuspush (events)
    • 0.10.0: buildbot as a framework
      • well-defined subclassing APIs
      • test harnesses for custom classes
      • message broker structure
      • dynamic reconfig via graceful restart
      • simplified workers

Dustin's Notes

These are Dustin's notes from the meeting

  • components
    • assign maintainers to components, and track it officially in git
    • need a loose definition of "maintainer" that can include "concerned user" as a domain expert
    • another option: if you like $component, offer a worker to test it. This is especially relevant for version control
  • commit rights and other developer permissions
    • I should offer, rather than wait for developers to ask -- push instead of waiting
  • for security: committers *are* the response team
    • use a private github repository, where we can track issues
    • write reporting guidelines for those who discover a vulnerability (email an alias)
    • write down the response process - who to email, what to patch, etc.
    • continue to operate in firefighting mode for now, rather than trying to take a comprehensive approach without comprehensive APIs
  • Dependency versions
    • versions are much more of a worry for workers, rather than masters, so the policy approach should be different
    • python version compatibility is different from prerequisite packages (which can be installed easily if necessary)
    • at a minimum, we should support prerequisite versions shipped with the most recent distro
    • when bumping a requirement, name the feature necessary, and if possible allow older versions of the prerequisite for configs not using that feature
  • Licensing
    • Licensing the JS UI separately would help Apple and others both use and contribute to the project, so it's a win
  • Support for old versions (LTS)
    • in general, we should only support the most recent version, except for security patches
    • we got a number of users stuck on 0.7.{10,11,12} during the 0.8.0 transition. Why?
      • x.0 is scary -- let it settle down first
      • performance was a worry with a database backend
      • observed breakage in new versions, so upgrading was a bigger project
      • 0.8.0 didn't introduce features, so there was no drive to upgrade
    • takeaways:
      • plan ahead for major upgrades - do a stability release of e.g., 0.8.last before starting 0.9.1
      • do not add major features in 0.x.0, so that early adopters can shake out bugs first
  • Version numbering: drop the leading 0 in 0.10, so we'll have 0.8.x, 0.9.x, then 10.x

Roundtable

  • Log/artifact Storage
    • separate log server?
      • push logs (artifacts) to it directly from worker
      • use a unique identifier (hash?) for the artifact; perhaps pre-generate that unique id before getting the data, if that makes coordination easier
    • tailing is cool, but should be treated as a special case, as the important thing is getting the artifacts where they need to be
    • we have lots of models for how data should move: to central server? between two workers? into a cluster of servers?
    • ddunbar is interested in working on / driving this
  • Separate server processes are helpful, but can't be required
    • for performance monitoring - "oh, the scheduler process is pegged"
    • for distribution - "my artifact server deals with a lot of data, so I moved it off to chunky hardware"
  • Supporting "Easy" configurations
    • sample config should work - and does now
    • Hosted config generator??
      • loki does a good job here
    • Making the worker list dynamically editable
      • makes it easy to add/remove a user-submitted worker at runtime
      • this is a smaller case of web-based config editor
      • do we really need worker "names"? Can a worker just connect to the master and say "I can do builds"? (Its identity would be in the build history for debugging purposes)