Opened 4 years ago

Last modified 20 months ago

#3071 new defect

/builds/:id/changes returns error

Reported by: Ben Owned by:
Priority: major Milestone: 0.9.+
Version: master Keywords: data
Cc: delanne

Description (last modified by sa2ajj)

Nothing in twisted log, but when visiting the URL in a browser, I get this:

{"error": "TypeError(\"'NoneType' object has no attribute '__getitem__'\",)"}

I don;t know where to look at ... __getitem__ is pretty generic

Change History (18)

comment:1 Changed 4 years ago by sa2ajj

  • Description modified (diff)

comment:2 Changed 4 years ago by Ben

Before you ask, git bisect was not very helpful there ...

comment:3 Changed 4 years ago by dustin

This seems to work OK for a forced build. I'm working on a build with a real change now.

comment:4 Changed 4 years ago by dustin

Nope, that worked for me, too:

{
  "changes": [
    {
      "author": "change", 
      "branch": "master", 
      "category": null, 
      "changeid": 1, 
      "codebase": "", 
      "comments": "", 
      "files": [], 
      "parent_changeids": [], 
      "project": "", 
      "properties": {}, 
      "repository": "", 
      "revision": "1332", 
      "revlink": "", 
      "sourcestamp": {
        "branch": "master", 
        "codebase": "", 
        "created_at": 1417365722, 
        "patch": null, 
        "project": "", 
        "repository": "", 
        "revision": "1332", 
        "ssid": 2
      }, 
      "when_timestamp": 1417365722
    }
  ], 
  "meta": {
    "total": 1
  }
}

Any idea how to reproduce, then?

Last edited 3 years ago by dustin (previous) (diff)

comment:5 Changed 4 years ago by Ben

Here is the full Traceback:

2014-12-01 13:59:50+0100 [-] while handling API request
	Traceback (most recent call last):
	  File "/home/benoit/buildbot/openvas/venv/local/lib/python2.7/site-packages/twisted/internet/defer.py", line 1097, in _inlineCallbacks
	    result = result.throwExceptionIntoGenerator(g)
	  File "/home/benoit/buildbot/openvas/venv/local/lib/python2.7/site-packages/twisted/python/failure.py", line 389, in throwExceptionIntoGenerator
	    return g.throw(self.type, self.value, self.tb)
	  File "/home/benoit/buildbot/buildbot/master/buildbot/www/rest.py", line 376, in renderRest
	    request.write(data)
	  File "/usr/lib/python2.7/contextlib.py", line 35, in __exit__
	    self.gen.throw(type, value, traceback)
	--- <exception caught here> ---
	  File "/home/benoit/buildbot/buildbot/master/buildbot/www/rest.py", line 111, in handleErrors
	    yield
	  File "/home/benoit/buildbot/buildbot/master/buildbot/www/rest.py", line 310, in renderRest
	    data = yield ep.get(rspec, kwargs)
	  File "/home/benoit/buildbot/openvas/venv/local/lib/python2.7/site-packages/twisted/internet/defer.py", line 1097, in _inlineCallbacks
	    result = result.throwExceptionIntoGenerator(g)
	  File "/home/benoit/buildbot/openvas/venv/local/lib/python2.7/site-packages/twisted/python/failure.py", line 389, in throwExceptionIntoGenerator
	    return g.throw(self.type, self.value, self.tb)
	  File "/home/benoit/buildbot/buildbot/master/buildbot/data/changes.py", line 73, in get
	    changes = yield self.master.db.changes.getChangesForBuild(buildid)
	  File "/home/benoit/buildbot/openvas/venv/local/lib/python2.7/site-packages/twisted/internet/defer.py", line 1099, in _inlineCallbacks
	    result = g.send(result)
	  File "/home/benoit/buildbot/buildbot/master/buildbot/db/changes.py", line 192, in getChangesForBuild
	    while ((toChanges[cb]['changeid'] not in change['parent_changeids']) and
	exceptions.TypeError: 'NoneType' object has no attribute '__getitem__'
	
2014-12-01 13:59:50+0100 [-] REST error: TypeError("'NoneType' object has no attribute '__getitem__'",)

comment:6 Changed 4 years ago by dustin

  • Cc delanne added

Ah, so it looks like this is a problem with fromChanges and toChanges having different codebases. @delanne, any ideas?

comment:7 Changed 4 years ago by Ben

Example of fromChanges and toChanges that breaks it:

fromChanges:
{u'openvas-libraries': None, u'openvas-scanner': {'category': None, 'files': [u'ChangeLog', u'src/openvassd.c'], 'parent_changeids': [287], 'repository': u'https://wald.intevation.org/svn/openvas', 'author': u'jan', 'project': u'OpenVAS', 'comments': u'* src/openvassd.c (main): Updated/improved version info.\n\n', 'changeid': 422, 'codebase': u'openvas-scanner', 'branch': u'trunk', 'when_timestamp': datetime.datetime(2014, 12, 8, 11, 21, 25, tzinfo=tzutc()), 'revlink': u'', 'properties': {}, 'sourcestampid': 461, 'revision': u'21126'}}

toChanges:
{u'openvas-libraries': {'category': None, 'files': [u'ChangeLog', u'misc/plugutils.c'], 'parent_changeids': [], 'repository': u'https://wald.intevation.org/svn/openvas', 'author': u'mime', 'project': u'OpenVAS', 'comments': u'Added newline to output.', 'changeid': 85, 'codebase': u'openvas-libraries', 'branch': u'trunk', 'when_timestamp': datetime.datetime(2014, 10, 28, 11, 57, 50, tzinfo=tzutc()), 'revlink': u'', 'properties': {}, 'sourcestampid': 105, 'revision': u'20710'}, u'openvas-scanner': None}

As I understand it, the from build was generated with a different codebase as the to build.

comment:8 Changed 4 years ago by Ben

Here another one:

fromChanges:
{u'openvas-libraries': None, u'openvas-manager': {'category': None, 'files': [u'src/manage_sqlite3.c', u'ChangeLog'], 'parent_changeids': [421], 'repository': u'https://wald.intevation.org/svn/openvas', 'author': u'timopollmeier', 'project': u'OpenVAS', 'comments': u'\t* src/manage_sqlite3.c (cached_task_severity): In overrides case return\n\toverrides_severity of cache struct after updating it instead of\n\treturning the severity component without overrides.', 'changeid': 425, 'codebase': u'openvas-manager', 'branch': u'trunk', 'when_timestamp': datetime.datetime(2014, 12, 8, 15, 3, 18, tzinfo=tzutc()), 'revlink': u'', 'properties': {}, 'sourcestampid': 464, 'revision': u'21129'}}

toChanges:
{u'openvas-libraries': None, u'openvas-manager': None}

I have no idea why the to is empty.

comment:9 Changed 4 years ago by sa2ajj

Possibly relevant PR: GH:1429

comment:10 Changed 4 years ago by sa2ajj

The PR landed to master.

comment:11 Changed 4 years ago by Ben

That PR was not related to this issue.

comment:12 Changed 4 years ago by Ben

The Traceback changed due to modification of the code, but is still there:

Traceback (most recent call last):
          File "/home/benoit/buildbot/openvas/venv/local/lib/python2.7/site-packages/twisted/internet/defer.py", line 1097, in _inlineCallbacks
            result = result.throwExceptionIntoGenerator(g)
          File "/home/benoit/buildbot/openvas/venv/local/lib/python2.7/site-packages/twisted/python/failure.py", line 389, in throwExceptionIntoGenerator
            return g.throw(self.type, self.value, self.tb)
          File "/home/benoit/buildbot/buildbot/master/buildbot/www/rest.py", line 374, in renderRest
            request.write(data)
          File "/usr/lib/python2.7/contextlib.py", line 35, in __exit__
            self.gen.throw(type, value, traceback)
        --- <exception caught here> ---
          File "/home/benoit/buildbot/buildbot/master/buildbot/www/rest.py", line 111, in handleErrors
            yield
          File "/home/benoit/buildbot/buildbot/master/buildbot/www/rest.py", line 308, in renderRest
            data = yield ep.get(rspec, kwargs)
          File "/home/benoit/buildbot/openvas/venv/local/lib/python2.7/site-packages/twisted/internet/defer.py", line 1097, in _inlineCallbacks
            result = result.throwExceptionIntoGenerator(g)
          File "/home/benoit/buildbot/openvas/venv/local/lib/python2.7/site-packages/twisted/python/failure.py", line 389, in throwExceptionIntoGenerator
            return g.throw(self.type, self.value, self.tb)
          File "/home/benoit/buildbot/buildbot/master/buildbot/data/changes.py", line 73, in get
            changes = yield self.master.db.changes.getChangesForBuild(buildid)
          File "/home/benoit/buildbot/openvas/venv/local/lib/python2.7/site-packages/twisted/internet/defer.py", line 1099, in _inlineCallbacks
            result = g.send(result)
          File "/home/benoit/buildbot/buildbot/master/buildbot/db/changes.py", line 189, in getChangesForBuild
            if change and change['changeid'] != toChanges.get(cb, {}).get('changeid'):
        exceptions.AttributeError: 'NoneType' object has no attribute 'get'

comment:13 Changed 4 years ago by Ben

GH:1455 is for the second case (comment 8)

comment:14 Changed 4 years ago by sa2ajj

Landed.

comment:15 Changed 4 years ago by Ben

Just to make it clear, the issue here is that 'absolute sourcestamps' (linked to a change) are compared to 'relative sourcestamps' (not linked to a change).

Such relative sourcestamps are created in different scenarii:

  • In case of multi-codebase builds, all codebase not triggering the build are made relative.
  • In case of forced build, all sourcestamps are created relative.

comment:16 Changed 4 years ago by dustin

The first step to fixing this will be to produce a failing test case.

Even if the underlying model issue that Ben has identified isn't fixed, the endpoint should return a more sensible error in the failing cases.

comment:17 Changed 3 years ago by dustin

  • Milestone changed from 0.9.0 to 0.9.+
Note: See TracTickets for help on using tickets.