Opened 9 years ago

Closed 9 years ago

#897 closed defect (fixed)

Allow Twisted dependency to be fulfilled by separate tarballs

Reported by: clepple Owned by:
Priority: major Milestone: 0.8.3
Version: 0.8.0 Keywords: packaging
Cc: clepple@…

Description

In Fink, Twisted is broken up into individual packages based on the separate tarballs at the bottom of this page:

http://twistedmatrix.com/trac/wiki/Downloads

If I run the buildslave (0.8.1rc1) with some of those packages installed, it complains because there is no single package which provides "twisted":

$ buildslave --help Traceback (most recent call last):

File "/sw/bin/buildslave", line 5, in <module>

from pkg_resources import load_entry_point

File "/sw/lib/python2.6/site-packages/pkg_resources.py", line 2603, in <module>

working_set.require(requires)

File "/sw/lib/python2.6/site-packages/pkg_resources.py", line 666, in require

needed = self.resolve(parse_requirements(requirements))

File "/sw/lib/python2.6/site-packages/pkg_resources.py", line 565, in resolve

raise DistributionNotFound?(req) # XXX put more info here

pkg_resources.DistributionNotFound?: twisted>=2.0.0

If I change the requires.txt to say "twisted_core>=2.0.0" it works (although I have all the twisted* packages installed that are needed by the 0.8.0 package, so that probably needs to be augmented with the other slave required twisted-* packages.)

If possible, it would be nice to allow an alternate dependency on "twisted or (twisted_core and twisted_web)", although I am not familiar with the setuptools dependency syntax.

Marking this as "major" since it blocks the Fink package (and potentially other packaging systems which ship a modular Twisted), but it could be worked around with a patch at install time.

Change History (12)

comment:1 Changed 9 years ago by clepple

  • Type changed from undecided to defect

It looks like the "sumo" Twisted tarball provides names like "Twisted Core" and "Twisted Web", so if we use the more specific names, that should work with both distribution styles.

comment:2 follow-up: Changed 9 years ago by dustin

Like this? (Obviously master would require more than just core)

@@ -64,7 +64,7 @@ except ImportError:
     ]
 else:
     setup_args['install_requires'] = [
-        'twisted >= 2.0.0',
+        'Twisted_Core >= 2.0.0',
     ]
     setup_args['entry_points'] = {
         'console_scripts': [}}}

comment:3 in reply to: ↑ 2 Changed 9 years ago by clepple

  • Cc clepple@… added

Replying to dustin:

Like this? (Obviously master would require more than just core)

Exactly. I haven't had a chance to check for the complete list of dependencies for master either, but it is probably at least twisted_core, twisted_web, twisted_words and twisted_mail. (twisted_conch might be used by manhole if it is SSH-based, but I haven't looked at that much.)

comment:4 Changed 9 years ago by dustin

Hmm,

$ easy_install Twisted_Core
# or
$ easy_install 'Twisted_Core >= 2.0.0'

fail badly with setuptools (big 'ol traceback), and less badly with Distribute:

install_dir /home/dustin/tmp/sandbox/lib64/python2.6/site-packages/
Searching for Twisted-Core>=2.0.0
Reading http://pypi.python.org/simple/Twisted_Core/
Reading http://twistedmatrix.com/
No local packages or download links found for Twisted-Core>=2.0.0
error: Could not find suitable distribution for Requirement.parse('Twisted-Core>=2.0.0')

so I'm not sure that this change won't just break other dependencies.

How did Python end up with such a fantastically crappy distribution system? Or, set of distribution systems?

comment:5 Changed 9 years ago by clepple

Hmm, OK. I didn't think to try that from the command line, because with the packages installed, the change to requires.txt (the one mentioned in the original description above) worked when the packages were already installed. I guess they must use different parsers that munge the underscores to hyphens.

Odd, I get nearly the same output as your Distribute quote when I use setuptools 0.6c11 on Python 2.6:

Searching for twisted-core
Reading http://pypi.python.org/simple/twisted_core/
Reading http://twistedmatrix.com/
No local packages or download links found for twisted-core
error: Could not find suitable distribution for Requirement.parse('twisted-core')

Maybe this is a bug with Twisted's package descriptions?

I dunno, I haven't seen any module distribution systems that play well with an OS's existing packaging systems. CPAN is another lovely example.

BTW, should we add a "packaging" keyword to Trac?

comment:6 follow-up: Changed 9 years ago by dustin

As far as I can tell, it works great as a dependency in setup.py, too - but this inconsistency has me worried.

I'd love to add a "packaging" keyword .. would you be willing to be be a maintainer for the mac packages if I do so?

comment:7 in reply to: ↑ 6 Changed 9 years ago by clepple

Replying to dustin:

I'd love to add a "packaging" keyword .. would you be willing to be be a maintainer for the mac packages if I do so?

I am already maintaining the Fink package, but I haven't really done much with MacPorts? since the two systems fulfill the same need in slightly different ways.

comment:8 Changed 9 years ago by dustin

  • Keywords packaging added

OK, here it is: PackagingProject?

comment:9 Changed 9 years ago by dustin

  • Milestone changed from 0.8.1 to 0.8.2

comment:10 Changed 9 years ago by dustin

  • Milestone changed from 0.8.2 to 0.8.3

I'm bumping this to 0.8.3, but it looks like most packaging systems just work *around* the Python dependency calculations.. which is fine, I guess

comment:11 follow-up: Changed 9 years ago by dustin

Charles - I assume you've also worked around this? If so, can we close?

comment:12 in reply to: ↑ 11 Changed 9 years ago by clepple

  • Resolution set to fixed
  • Status changed from new to closed

Replying to dustin:

Charles - I assume you've also worked around this? If so, can we close?

Yes, I'll use the NO_INSTALL_REQS environment variable in the next Fink package.

Note: See TracTickets for help on using tickets.