Opened 7 years ago

Last modified 4 years ago

#2124 new defect

IRC oversubscribes to buildsets, logs too much

Reported by: dustin Owned by:
Priority: major Milestone: 0.9.+
Version: 0.8.5 Keywords: irc
Cc:

Description (last modified by sa2ajj)

From Gavin:

18:04 <       gmcdonald+> ..
18:04 <       gmcdonald+> why is my twistd.log full of these:
18:04 <       gmcdonald+> buildmaster@aegis:~/master1$ cat twistd.log | grep
'Buildset <buildbot.status.buildset.BuildSetStatus instance at 0x3838560>
added' | wc -l
18:04 <       gmcdonald+> 7593
18:04 <       gmcdonald+> ..
18:05 <       gmcdonald+> all added at the same time I might add
18:09 <       gmcdonald+> ..
18:09 <       gmcdonald+> buildmaster@aegis:~/master1$ cat twistd.log | grep
'Not notifying for a build in the wrong category' | wc -l
18:09 <       gmcdonald+> 1002
18:09 <       gmcdonald+> ..
18:09 <       gmcdonald+> thats new too
18:10 <       gmcdonald+> ..

it seems that for every one thing done logs are repeated thousands of times, or I can believe the status being set on one builder actually is happening over 7000 times in one second?

---

Both of these messages come from words.IRC; the first makes me think that it's being "oversubscribed" to build status, so that buildsetSubmitted is being called many times for each new buildset. The latter could be similar, but may also be a reasonable number of ignored buildsets by IRC bots that are not interested in them. So that message should probably just be removed.

Change History (7)

comment:1 Changed 7 years ago by dustin

There are a few problems here:

  • words.py logs *way* too much. Really.
  • most likely some kind of multiple subscription to the same status object - gavin's seeing runs of 107 log messages, with only ~40 bots
  • every bot subscribes to every builder, even if the categories don't match

comment:2 Changed 7 years ago by dustin

and

  • no tests

comment:3 Changed 6 years ago by Tom Prince

IRC: Remove some of the extraneous logging and subscriptions.

see #2124.

Changeset: 5d3ecc2dd1de0f265227de55aee4ef8bbc9475eb

comment:4 Changed 6 years ago by dustin

I think that this is occurring because the IRC client subscribes to status for each contact. So if you've had 107 people/channels that have talked to the bot, you'll have 107 contacts and 107 subscriptions.

This is kinda dumb, but I'd rather fix this when reconfiguring this to use the MQ architecture, rather than now.

I will, however, add some tests to the file.

comment:5 Changed 6 years ago by dustin

  • Milestone changed from 0.8.6 to 0.8.+

(Some) tests added:

commit 4e6d50880aeb796e8bef21fc8c7e674277c9394b
Author: Dustin J. Mitchell <dustin@mozilla.com>
Date:   Sun Feb 5 16:06:26 2012 -0600

    Add tests for IRC client (with some minor refactors)

comment:6 Changed 4 years ago by dustin

  • Milestone changed from 0.8.+ to 0.9.+

Ticket retargeted after milestone closed

comment:7 Changed 4 years ago by sa2ajj

  • Description modified (diff)
  • Summary changed from IRC oversubsribes to buildsets, logs too much to IRC oversubscribes to buildsets, logs too much
Note: See TracTickets for help on using tickets.