Opened 5 years ago

Closed 4 years ago

#3012 closed defect (duplicate)

SVNPoller gets stuck.

Reported by: Ben Owned by:
Priority: major Milestone: 0.9.0
Version: master Keywords: multimaster
Cc:

Description

That's been the last time I heard from the SVNPoller:

2014-11-03 16:27:37+0100 [-] SVNPoller: polling OpenVAS
2014-11-03 16:27:37+0100 [-] SVNPoller: svnurl=https://wald.intevation.org/svn/openvas, root=https://wald.intevation.org/svn/openvas, so prefix=
2014-11-03 16:27:38+0100 [-] SVNPoller: no changes
2014-11-03 16:27:38+0100 [-] SVNPoller: _process_changes 20765 .. 20765
2014-11-03 16:27:38+0100 [-] SVNPoller: finished polling None
2014-11-03 16:29:37+0100 [-] SVNPoller: polling OpenVAS
2014-11-03 16:29:37+0100 [-] SVNPoller: svnurl=https://wald.intevation.org/svn/openvas, root=https://wald.intevation.org/svn/openvas, so prefix=
2014-11-03 16:29:38+0100 [-] SVNPoller: no changes
2014-11-03 16:29:38+0100 [-] SVNPoller: _process_changes 20765 .. 20765
2014-11-03 16:29:38+0100 [-] SVNPoller: finished polling None
2014-11-03 16:31:37+0100 [-] SVNPoller: polling OpenVAS
2014-11-03 16:31:37+0100 [-] SVNPoller: svnurl=https://wald.intevation.org/svn/openvas, root=https://wald.intevation.org/svn/openvas, so prefix=
2014-11-03 16:31:38+0100 [-] SVNPoller: no changes
2014-11-03 16:31:38+0100 [-] SVNPoller: _process_changes 20765 .. 20765
2014-11-03 16:31:38+0100 [-] SVNPoller: finished polling None
2014-11-03 16:33:37+0100 [-] SVNPoller: polling OpenVAS
2014-11-03 16:35:34+0100 [-] SVNPoller: Error in  while polling
        Traceback (most recent call last):
        Failure: twisted.internet.utils._UnexpectedErrorOutput: got stderr: "svn: OPTIONS of 'https://wald.intevation.org/svn/openvas': SSL handshake failed: Secure connection truncated (https://wald.intevation.org)\n"

2014-11-03 16:35:37+0100 [-] SVNPoller: polling OpenVAS

Since then, complete silence. been restarting the master, stoping, starting it, upgrading, anything (except touched that part of the config), and still nothing.

If I take my config and create a new master with it, it starts polling right away. Something has to be there in my master directory that the Poller doesn't like ...

Change History (12)

comment:1 Changed 5 years ago by dustin

That nothing happens after a *restart* makes me think maybe this is an SVN problem -- the poller doesn't store any state outside of the the repo itself.

What does running 'svn info' in the repodir do?

Even if that segfaults or sometihng, we should still be logging the result properly (and retrying)

comment:2 Changed 5 years ago by sa2ajj

We discussed it on IRC.

It looks like the process does not get to 'svn info' (polling OpenVAS would be in the log all the time).

comment:3 Changed 5 years ago by Ben

@dustin: This is nothing server side, as creating another master with the same config works fine. Or Forcing a build works fine (checkout the last revision without an ounce of a warning ...)

I'm glad I didn't tried to mess-up with my master directory by fear of seeing the bug disappear without proper explanation ...

comment:4 Changed 5 years ago by Ben

I believe it is a PollingChangeSource trouble as my log message added at the end of the SVNPoller.__init__ get printed, but still nothing happens ...

And as sa2ajj said, if it was some kind of SVN trouble, we would at least see an attempt at polling in the log every two minutes ... I guess the polling is not triggered.

Last edited 5 years ago by sa2ajj (previous) (diff)

comment:5 Changed 5 years ago by Ben

Looks like my Poller never gets activated. The doPoll attribute/function/descriptor is created, but never triggered according to my extra logging ...

I could probably link it to that commit: https://github.com/buildbot/buildbot/commit/8bab6038342d562b5fa57c36e8f9c164dfab9509

I'm now trying to understand who should call that activate() method ...

comment:6 Changed 5 years ago by Ben

If I stop my master, and look in the database, the Poller is still active. That's weird !

comment:7 Changed 5 years ago by sa2ajj

  • Milestone changed from undecided to 0.9.0

Well, if a poller was active in the database, it would not behave as expected when started and, I assume, would not mark itself as non-active.

comment:8 Changed 5 years ago by Ben

So it looks like the same trouble as #2959. The master fears that he is in a multi-master setup, and hence doesn't dare overwriting some (wrong) DB values, and hence stays in an inconsistent state ... That smells pretty bad ...

While we can probably fix it for this case, we really need to think of some better guard against those troubles (this and #2959). As it will happen again !

comment:9 Changed 5 years ago by dustin

See #3015

comment:10 Changed 5 years ago by sa2ajj

  • Keywords multimaster added

comment:11 Changed 5 years ago by dustin

Closing in favor of #3015.

comment:12 Changed 4 years ago by dustin

  • Resolution set to duplicate
  • Status changed from new to closed
Note: See TracTickets for help on using tickets.