Opened 3 years ago

Last modified 13 months ago

#3302 reopened defect

[nine] Sending several changes with same ssid shall fail

Reported by: joeholley Owned by:
Priority: major Milestone: 0.9.5
Version: master Keywords: changes, changelists


I've encountered an issue with build not (apparently) pulling the correct changelist from the database in the latest 0.9.0b1 (commit 5a5ed84848fd78aab99d1f40804645c423afae27). My primary triggering mechanism is using a PBChangeSource() and sending changes using the 'buildbot sendchange' commandline. I've attached a basic master.cfg that assumes a single linux slave, and a bash script of two 'buildbot sendchange' commands to cause this configuration to exhibit the problem. To reproduce:

  1. Start a buildbot master using the attached master.cfg.
  2. Connect a single linux slave (name: 'slave', pw: 'slavepasswd') that also has buildbot installed.
  3. Run the attached '' shell script from the master. It issues two 'buildbot sendchange' commands: these kick off 'job1' and 'job2' schedulers. Both of these schedulers only do two things:
    1. Print out their properties (for debugging)
    2. Run a 'buildbot sendchange' command kicking off the 'broken' scheduler, with identical revision/category/repository/branches, but different values for the custom property 'myproperty'
  4. 'job1' and 'job2' will both be triggered, and each will both trigger a 'broken' build.
  5. Open the webUI, and navigate to 'broken' builder's builds '1' and '2' (kicked off by 'buildbot sendchange commands from 'job1' and 'job2' respectively)
  6. Navigate to the changes tabs for these two jobs. Note that:
    1. Build 1 of 'broken' shows two changes in its changelist:
      1. The 'buildbot sendchange' from '' that kicked off 'job1', and
      2. The 'buildbot sendchange' command run by 'job1' to kick off 'broken' build 1.
    2. Build 2 of 'broken shows no changelists at all! However, it WAS kicked off, and apparently grabbed the changelist from the 'buildbot sendchange' run by 'job1' to kick off 'broken' build 1 - you can verify this by looking at the only step in 'broken' change 2, which lists all the properties the build got - including the custom property 'myproperty', which has the value set by the 'buildbot sendchange' run by 'job1' to kick off 'broken' build 1.

Expected results (which I was able to achieve with 0.8.12) are each broken build getting the changelist (and only the changelist) that triggered its build, and therefore having the custom properties sent with the triggering 'buildbot sendchange' command available to the build. Let me know if I can provide any more help.

Attachments (2)

master.cfg (4.3 KB) - added by joeholley 3 years ago.
Buildmaster Config (249 bytes) - added by joeholley 3 years ago.
Shell script that sends two 'buildbot sendchange' commands to reproduce the error.

Download all attachments as: .zip

Change History (17)

Changed 3 years ago by joeholley

Buildmaster Config

Changed 3 years ago by joeholley

Shell script that sends two 'buildbot sendchange' commands to reproduce the error.

comment:1 Changed 3 years ago by tardyp

I can reproduce the problem.

Here is the dump of the db.

Interestingly, there are only 2 sourcestamps.

sqlite3 state.sqlite .dump |grep -i insert |grep -v logchu |grep -v ptardy
INSERT INTO "buildsets" VALUES(1,NULL,'The SingleBranchScheduler scheduler named ''job1 scheduler'' triggered this build',1436694539,1,1436694541,0,NULL,NULL);
INSERT INTO "buildsets" VALUES(2,NULL,'The SingleBranchScheduler scheduler named ''broken scheduler'' triggered this build',1436694540,1,1436694540,0,NULL,NULL);
INSERT INTO "buildsets" VALUES(3,NULL,'The SingleBranchScheduler scheduler named ''broken scheduler'' triggered this build',1436694541,1,1436694541,0,NULL,NULL);
INSERT INTO "buildsets" VALUES(4,NULL,'The SingleBranchScheduler scheduler named ''job2 scheduler'' triggered this build',1436694541,1,1436694547,0,NULL,NULL);
INSERT INTO "objects" VALUES(1,'broken scheduler','buildbot.schedulers.basic.SingleBranchScheduler');
INSERT INTO "objects" VALUES(2,'job1 scheduler','buildbot.schedulers.basic.SingleBranchScheduler');
INSERT INTO "objects" VALUES(3,'job2 scheduler','buildbot.schedulers.basic.SingleBranchScheduler');
INSERT INTO "builders" VALUES(1,'broken',NULL,'0b8a1caec23d75d1154b8d9bef9cec6c03697638');
INSERT INTO "builders" VALUES(2,'job2',NULL,'6362af2cb6ba28bb5a8769ac6d3992d8ae83f36d');
INSERT INTO "builders" VALUES(3,'job1',NULL,'803e16c8f2d5e702463cb1fa5270a534eebc8e47');
INSERT INTO "buildslaves" VALUES(1,'slave','{"admin": null, "host": null, "version": "0.9.0b1-35-gdd9e199", "access_uri": null}');
INSERT INTO "changesources" VALUES(1,'PBChangeSource:None','50dc25ec34aca4aa2dda681c938f2f3d900641c8');
INSERT INTO "schedulers" VALUES(1,'broken scheduler','b4e06d038f372c68bd9d2ccfc54609a3cd971424');
INSERT INTO "schedulers" VALUES(2,'job1 scheduler','ca5bd471fc208c6502ab00ad5daeb16e2b373066');
INSERT INTO "schedulers" VALUES(3,'job2 scheduler','ef5e09203a5794bdb77845499caebb5403ee6bff');
INSERT INTO "sourcestamps" VALUES(1,'fffd6c7d63d32dd40d3fe7520b269fb27946cc50','main','48849',NULL,'test','','',1.43669453908565497397e+09);
INSERT INTO "sourcestamps" VALUES(2,'ee49f9cd6dce86b0d89ad01e4791ef49f919c0ab','main','3',NULL,'test','','',1.43669454009563589101e+09);
INSERT INTO "changesource_masters" VALUES(1,1);
INSERT INTO "builder_masters" VALUES(1,1,1);
INSERT INTO "builder_masters" VALUES(2,2,1);
INSERT INTO "builder_masters" VALUES(3,3,1);
INSERT INTO "connected_buildslaves" VALUES(1,1,1);
INSERT INTO "buildset_properties" VALUES(1,'scheduler','["job1 scheduler", "Scheduler"]');
INSERT INTO "buildset_properties" VALUES(2,'scheduler','["broken scheduler", "Scheduler"]');
INSERT INTO "buildset_properties" VALUES(3,'scheduler','["broken scheduler", "Scheduler"]');
INSERT INTO "buildset_properties" VALUES(4,'scheduler','["job2 scheduler", "Scheduler"]');
INSERT INTO "buildrequests" VALUES(1,1,3,0,1,0,1436694539,1436694541,0);
INSERT INTO "buildrequests" VALUES(2,2,1,0,1,0,1436694540,1436694540,0);
INSERT INTO "buildrequests" VALUES(3,3,1,0,1,0,1436694541,1436694541,0);
INSERT INTO "buildrequests" VALUES(4,4,2,0,1,0,1436694541,1436694547,0);
INSERT INTO "changes" VALUES(1,'user','','main','48849','',1436694539,'cat1','test','','',1,NULL);
INSERT INTO "changes" VALUES(2,'user','','main','3','',1436694540,'broken','test','','',2,1);
INSERT INTO "changes" VALUES(3,'user','','main','3','',1436694541,'broken','test','','',2,2);
INSERT INTO "changes" VALUES(4,'user','','main','48849','',1436694541,'cat2','test','','',1,3);
INSERT INTO "builds" VALUES(1,1,3,1,1,1,1436694539.16285,1.43669454137539505953e+09,'finished',0);
INSERT INTO "builds" VALUES(2,1,1,2,1,1,1.43669454020036888117e+09,1.43669454043189692496e+09,'finished',0);
INSERT INTO "builds" VALUES(3,2,1,3,1,1,1.43669454119141292575e+09,1.43669454148307895657e+09,'finished',0);
INSERT INTO "builds" VALUES(4,1,2,4,1,1,1436694542.04719,1.4366945473732960224e+09,'finished',0);
INSERT INTO "configured_buildslaves" VALUES(1,1,1);
INSERT INTO "configured_buildslaves" VALUES(2,2,1);
INSERT INTO "configured_buildslaves" VALUES(3,3,1);
INSERT INTO "buildrequest_claims" VALUES(1,1,1436694539);
INSERT INTO "buildrequest_claims" VALUES(2,1,1436694540);
INSERT INTO "buildrequest_claims" VALUES(3,1,1436694541);
INSERT INTO "buildrequest_claims" VALUES(4,1,1436694542);
INSERT INTO "buildset_sourcestamps" VALUES(1,1,1);
INSERT INTO "buildset_sourcestamps" VALUES(2,2,2);
INSERT INTO "buildset_sourcestamps" VALUES(3,3,2);
INSERT INTO "buildset_sourcestamps" VALUES(4,4,1);
INSERT INTO "build_properties" VALUES(1,'MASTER_PORT_9989_TCP_ADDR','"localhost"','SlaveEnvironment');
INSERT INTO "build_properties" VALUES(1,'MASTER_PORT_9989_TCP_PORT','"19989"','SlaveEnvironment');
INSERT INTO "build_properties" VALUES(1,'branch','"main"','Build');
INSERT INTO "build_properties" VALUES(1,'buildername','"job1"','Builder');
INSERT INTO "build_properties" VALUES(1,'buildnumber','1','Build');
INSERT INTO "build_properties" VALUES(1,'codebase','""','Build');
INSERT INTO "build_properties" VALUES(1,'project','""','Build');
INSERT INTO "build_properties" VALUES(1,'repository','"test"','Build');
INSERT INTO "build_properties" VALUES(1,'revision','"48849"','Build');
INSERT INTO "build_properties" VALUES(1,'scheduler','"job1 scheduler"','Scheduler');
INSERT INTO "build_properties" VALUES(1,'slavename','"slave"','BuildSlave');
INSERT INTO "build_properties" VALUES(2,'branch','"main"','Build');
INSERT INTO "build_properties" VALUES(2,'buildername','"broken"','Builder');
INSERT INTO "build_properties" VALUES(2,'buildnumber','1','Build');
INSERT INTO "build_properties" VALUES(2,'codebase','""','Build');
INSERT INTO "build_properties" VALUES(2,'myproperty','"foo"','Change');
INSERT INTO "build_properties" VALUES(2,'project','""','Build');
INSERT INTO "build_properties" VALUES(2,'repository','"test"','Build');
INSERT INTO "build_properties" VALUES(2,'revision','"3"','Build');
INSERT INTO "build_properties" VALUES(2,'scheduler','"broken scheduler"','Scheduler');
INSERT INTO "build_properties" VALUES(2,'slavename','"slave"','BuildSlave');
INSERT INTO "build_properties" VALUES(3,'branch','"main"','Build');
INSERT INTO "build_properties" VALUES(3,'buildername','"broken"','Builder');
INSERT INTO "build_properties" VALUES(3,'buildnumber','2','Build');
INSERT INTO "build_properties" VALUES(3,'codebase','""','Build');
INSERT INTO "build_properties" VALUES(3,'myproperty','"foo"','Change');
INSERT INTO "build_properties" VALUES(3,'project','""','Build');
INSERT INTO "build_properties" VALUES(3,'repository','"test"','Build');
INSERT INTO "build_properties" VALUES(3,'revision','"3"','Build');
INSERT INTO "build_properties" VALUES(3,'scheduler','"broken scheduler"','Scheduler');
INSERT INTO "build_properties" VALUES(3,'slavename','"slave"','BuildSlave');
INSERT INTO "build_properties" VALUES(4,'MASTER_PORT_9989_TCP_ADDR','"localhost"','SlaveEnvironment');
INSERT INTO "build_properties" VALUES(4,'MASTER_PORT_9989_TCP_PORT','"19989"','SlaveEnvironment');
INSERT INTO "build_properties" VALUES(4,'branch','"main"','Build');
INSERT INTO "build_properties" VALUES(4,'buildername','"job2"','Builder');
INSERT INTO "build_properties" VALUES(4,'buildnumber','1','Build');
INSERT INTO "build_properties" VALUES(4,'codebase','""','Build');
INSERT INTO "build_properties" VALUES(4,'project','""','Build');
INSERT INTO "build_properties" VALUES(4,'repository','"test"','Build');
INSERT INTO "build_properties" VALUES(4,'revision','"48849"','Build');
INSERT INTO "build_properties" VALUES(4,'scheduler','"job2 scheduler"','Scheduler');
INSERT INTO "build_properties" VALUES(4,'slavename','"slave"','BuildSlave');
INSERT INTO "change_properties" VALUES(2,'myproperty','["foo", "Change"]');
INSERT INTO "change_properties" VALUES(3,'myproperty','["bar", "Change"]');
INSERT INTO "steps" VALUES(1,0,'SetPropertiesFromEnv',1,1.43669453918509197228e+09,1.43669453923100090024e+09,'Set',0,'[]',0);
INSERT INTO "steps" VALUES(2,1,'shell',1,1.43669453928981208801e+09,1.43669453935477399821e+09,'''/bin/echo "{u''MASTER_PORT_9989_TCP_ADDR'': ...''',0,'[]',0);
INSERT INTO "steps" VALUES(3,2,'shell_1',1,1.43669453939670610422e+09,1.43669454021646308895e+09,'''buildbot sendchange ...''',0,'[]',0);
INSERT INTO "steps" VALUES(4,0,'shell',2,1.43669454024388599392e+09,1436694540.30937,'''/bin/echo "{u''branch'': ...''',0,'[]',0);
INSERT INTO "steps" VALUES(5,3,'shell_2',1,1.43669454031876301768e+09,1.43669454119284200665e+09,'''buildbot sendchange ...''',0,'[]',0);
INSERT INTO "steps" VALUES(6,0,'shell',3,1.43669454125417399401e+09,1.43669454134607100484e+09,'''/bin/echo "{u''branch'': ...''',0,'[]',0);
INSERT INTO "steps" VALUES(7,0,'SetPropertiesFromEnv',4,1.43669454206995391839e+09,1.43669454211288094517e+09,'Set',0,'[]',0);
INSERT INTO "steps" VALUES(8,1,'shell',4,1.43669454220234489444e+09,1.43669454725236701961e+09,'''sleep 5''',0,'[]',0);
INSERT INTO "steps" VALUES(9,2,'shell_1',4,1.43669454728515100477e+09,1.43669454733373689646e+09,'''/bin/echo "{u''MASTER_PORT_9989_TCP_ADDR'': ...''',0,'[]',0);
INSERT INTO "logs" VALUES(1,'properties','properties',1,1,2,'t');
INSERT INTO "logs" VALUES(2,'stdio','stdio',2,1,44,'s');
INSERT INTO "logs" VALUES(3,'stdio','stdio',3,1,43,'s');
INSERT INTO "logs" VALUES(4,'stdio','stdio',4,1,41,'s');
INSERT INTO "logs" VALUES(5,'stdio','stdio',5,1,43,'s');
INSERT INTO "logs" VALUES(6,'stdio','stdio',6,1,41,'s');
INSERT INTO "logs" VALUES(7,'properties','properties',7,1,2,'t');
INSERT INTO "logs" VALUES(8,'stdio','stdio',8,1,42,'s');
INSERT INTO "logs" VALUES(9,'stdio','stdio',9,1,44,'s');

comment:2 Changed 3 years ago by tardyp

  • Resolution set to invalid
  • Status changed from new to closed

After looking at the problem more thoroughly, the explanation is indeed because you did not different revision for changes.

Even if you can send several changes for the same repository/branch/revision, they will count for only one sourcestamp.

Then the list of changes that are displayed in the UI is distinct changes since the last successfull build (we should arguably be more clear in the UI) those are not the changes that triggerred the build.

I mark the bug closed. But I am sure there is something wrong with you are experiencing that this particular config did not reproduce exactly.

Please don't hesitate to create another bug with more details on your real use case.

comment:3 Changed 2 years ago by riziero

I also managed to repro this on latest Nine. By using the 'sendchange pattern' to kick off other jobs (as in the cfg attached) the properties get mixed up and the changes are messed up like you say.

I then reworked the cfg to use trigger steps to kick off 'broken' scheduler and at least the properties 'foo' and 'bar' get passed properly, still changes beheave weirdly.

@TardyP:any chances this can be re-opened? IMO this could be something to re-consider.

Use case:

  • send a change to kick off a build also consisting of sub jobs like above.
  • if it fails I'd like to be able to reckick it using the same CL, branch, repo. That would either fail or report wrong changes.

comment:4 Changed 2 years ago by dustin

  • Milestone changed from 0.9.0b1 to 0.9.0
  • Resolution invalid deleted
  • Status changed from closed to reopened

comment:5 Changed 2 years ago by tardyp

  • Summary changed from [nine] Changes not populating correctly to [nine] Sending several changes with same ssid shall fail

User shall not send several changes with same ssid. By design, this will not work in nine (because of 0..1:1 change:ssid)

We should report an error in case somebody creates a change with same ssid.

If we want to support this, we need to refactor a lot of the change stuff.

comment:7 Changed 2 years ago by riziero

In the .sh script attached, a different category (cat1, cat2) are being used for each change, which should result in different ssid's. Still they end up overlapping.

Oh ok cat is not part of the hash.

Last edited 2 years ago by riziero (previous) (diff)

comment:8 Changed 2 years ago by ian

You asked about my use case in #3366:

We want to run a number of builds for a given commit. Sometimes we want to rerun just one or two of those builds for a particular commit; for example, if one of the builds spuriously fails (perhaps the worker ran out of disk space) then we want to redo only that build for this commit. The category or properties seemed like the natural place to record what builds we want to happen for the change.

comment:9 Changed 2 years ago by dustin

Would the rebuild functionality be better?

comment:10 Changed 2 years ago by riziero

I think that would not be the same as one may want to re-do the same sourcestamp with different properties.

sure it would be better than nothing ;)

Last edited 2 years ago by riziero (previous) (diff)

comment:11 Changed 2 years ago by dustin

  • Milestone changed from 0.9.0 to 0.9.1

comment:12 Changed 16 months ago by tardyp

  • Milestone changed from 0.9.1 to 0.9.2

Ticket retargeted after milestone closed

comment:13 Changed 15 months ago by tardyp

  • Milestone changed from 0.9.2 to 0.9.3

Ticket retargeted after milestone closed

comment:14 Changed 14 months ago by tardyp

  • Milestone changed from 0.9.3 to 0.9.4

Ticket retargeted after milestone closed

comment:15 Changed 13 months ago by tardyp

  • Milestone changed from 0.9.4 to 0.9.5

Ticket retargeted after milestone closed

Note: See TracTickets for help on using tickets.