Opened 4 years ago

Last modified 4 years ago

#3287 accepted enhancement

Set up automated testing of buildslave on Python 3

Reported by: dustin Owned by: sa2ajj
Priority: major Milestone: ongoing
Version: master Keywords:
Cc: rutsky, jvlomax

Description

This basically entails running trial buildslave.tests on Py3.3, 3.4, and 3.5.

Let's start by running this in Travis (since that's self-serve), then port to metabuildbot.

Change History (15)

comment:1 Changed 4 years ago by sa2ajj

What I am curious about is: trial is a part of Twisted which is not Python 3 compatible; how can we use trial to run things with Py3.{3,4,5}?

comment:2 Changed 4 years ago by dustin

Some patches landed in April to make 'trial' work a little bit better. I'm willing to believe (but would like to hear for sure from Lurimax) that it works well enough to run Buildslave tests.

comment:3 Changed 4 years ago by jvlomax

I'm very unsure about how ready trial is for py3. It is included in the ported section of twisted, but I'm having some trouble making it run like it should. I'm currently just using

python -m buildslave/tests

to run the tests, which currently do not pass due to twisted currently failing spectacularly under python3.

I think it would be best to wait with setting up the automated testing of buildslave until twisted.spread is ported (working on it now).

comment:4 Changed 4 years ago by dustin

Wow, I don't expect that particular invocation to actually run the tests -- it's probably just importing things. I'm sad that's still failing :(

comment:5 Changed 4 years ago by jvlomax

It will run like that on py3, and from what it looks like it tries to invoke trial to do the testing. You can run something very similar on py2 (pip install unittest2)

python -m unittest2 

you can give it a try. It at the very least claims to have run the tests. If that means just importing things I have no clue :)

It is mainly the twisted.spread package that is blocking us atm. I'm hoping to have it ported by the end of next week. No promises though, twisted development seems to be quite a messy affair

comment:6 Changed 4 years ago by dustin

Oh, that makes more sense!

comment:7 Changed 4 years ago by sa2ajj

Python comes with a script called compileall.py. I think we could use it to make sure that the code is Python3 compatible.

comment:8 Changed 4 years ago by sa2ajj

What bothers me is:

  • FreeBSD does not have Python 3.5 (and the last official release is 3.4.3
  • there seem to be no virtualenv package for Python 3.x

comment:9 Changed 4 years ago by jvlomax

I don't think we need to rush ahead with python3.5. 3.3 and 3.4 are the most common versions on almost all linux distributions. 3.5 only just came out. Besides, there are no reasons 3.5 should break any compatibility with previous versions, so we should support 3.3, 3.4, and 3.5.

There is a package for virtualenv on py3, I'm running it on both windows and Linux.

compileall.py looks like an interesting tools. From what I understand it will only catch syntax errors?

comment:10 Changed 4 years ago by sa2ajj

re compileall.py: yes, it will catch all syntax errors (I just tried it on master's tree).

re virtualenv: our infrastructure is FreeBSD based, so I'd prefer to use OS' package for that. And FreeBSD is not Linux :)

comment:11 Changed 4 years ago by jvlomax

Then my only suggestion would be to use pip. Not sure if it is applicable in your situation though :)

comment:12 Changed 4 years ago by sa2ajj

That we can do :) It's just different from other build slaves.

comment:13 Changed 4 years ago by sa2ajj

  • Owner set to sa2ajj
  • Status changed from new to accepted

comment:14 Changed 4 years ago by sa2ajj

Basically, the idea is to run python<X.Y> -m compileall -q buildslave from slave directory.

comment:15 Changed 4 years ago by rutsky

  • Cc rutsky jvlomax added

Among other errors during running buildslave's tests there is missing twisted.spread module. Here is corresponding bug report in Twisted: https://twistedmatrix.com/trac/ticket/7598

@jvlomax, do you know about twisted.spread status?

According to this twisted.spread.pb, twisted.spread.flavors and twisted.spread.jelly modules are not yet ported.

Note: See TracTickets for help on using tickets.