Evaluating Add-Ons¶
What is the risk of adding a given dependency?
As often happens, a client asked me about adding a given package to their buildout. I just realized that I have a standard response to this so that might be something worth documenting for the wider community of admins and integrators. My criteria are as follows roughly in order of priority:
Good backing:
Find the author or maintainer on the plone.org/products page or on PyPI. Are they a company or individual that is widely known in the community?
Recent versions supported:
Does the add-on support recent versions of its own dependencies? For a Plone package today this means checking if it support Plone 3.
Final release:
Does the package have a final release that supports recent versions of its dependencies?
Released as an egg on a package index:
Is the add-on packaged as an egg and available on an easy_install accessible index such as PyPI?
Test coverage should, of course, be criteria #1. Sadly, we don’t have a good enough test isolation story with ZTC and PTC, test failures in the client’s buildout don’t necessarily mean the add-on is broken. I know there’s been talk about integrating test coverage data into plone.org/products but it’s not here yet. The holy grail would be some sort of integration with distutils/setuptools. It would be great if I could do “python setup.py test –coverage” so that a subsequent “python setup.py register sdist upload” would incorporate the coverage data into the release meta-data such that it would be reported on the PyPI page. Certainly any such solution could be circumvented by the releaser, but I think that would be very rare, that it would more often encourage more maintainers to have good test coverage and this would be a huge win overall.
It’s also interesting to note that I find recent version support more indicative of risk than final release status. This comes from experience. I’ve worked on too many projects of all sizes that had production sites running release candidates, betas, alphas, and even some development versions. One conclusion would be that this means that the software is of poor quality. I don’t think that’s it. I think it’s that we, as a software ecosystem, are bad at releasing. Now eggs are certainly helping with this, but I think we also need to start an aggressive honor/shame (carrot/stick) campaign about releasing. Actually, maybe we should have an honor/shame campaign about add-ons in general. Perhaps a montly post to planet plone and the mailing list with a Hall of Fame and a Hall of Shame? :) Something that considers release status (how long has it been since that beta release? 3 months? Maybe you can release a final version?), test coverage, and meta-data cleanliness, etc.
Updated on 01 December 2008
Imported from Plone on Mar 15, 2021. The date for this update is the last modified date in Plone.
Comments
comments powered by Disqus