Daniel Greenfeld @pydanny The Future of Collaboration by Daniel Greenfeld
Daniel Greenfeld@pydanny
The Future of Collaboration
by Daniel Greenfeld
Daniel Greenfeld@pydanny
Who am I?
Daniel Greenfeld (@pydanny)
Cartwheel Web / Revsys
http://djangopackages.com
https://whitespacejobs.org
Fiancé of Audrey Roy
http://www.flickr.com/photos/pydanny/4442245488
Daniel Greenfeld@pydanny
Mark Pilgrimis
Gone
Daniel Greenfeld@pydanny
Mark Pilgrim
• feedparser
• conbtributed to httplib2
• Dive into Python
• Dive into HTML5
Daniel Greenfeld@pydanny
Mark’s Projects?
• What is the copyright on his material?
• What about his code?
Daniel Greenfeld@pydanny
httplib2
• Mark wasn’t lead, but outage anyway...
• PyPI?
• http://code.google.com/p/httplib2 was down
• Had to find a cached download
• A lot of libraries depend on httplib2
Daniel Greenfeld@pydanny
PyPI Issues?
• Packages are too easily deleted
• Dependency checks for that package
• Request a project hand-off?
• Other projects need to be notified
• RSS feed of these things
• Needs moderation but PyPI busy
Daniel Greenfeld@pydanny
Repeating History?
• http://pypants(.com|.org|.net) is gone
• http://djangolint.com is here today
Daniel Greenfeld@pydanny
Repeating History?
• django-piston
• python.org
• http://opencomparison.org
Daniel Greenfeld@pydanny
A Dark Future
Daniel Greenfeld@pydanny
Critical Packages Breakdown
• A number of critical Python Packages vanish
• Various build scripts fail
• Replace from caches/backups
• Domain Knowledge is gone
Dark Future
Daniel Greenfeld@pydanny
Repercussions
• Annoyance
• Ability for Python to move forward HALTS
• Social Issues
Dark Future
Daniel Greenfeld@pydanny
Not the Future
Daniel Greenfeld@pydanny
• Legacy code tries to install legacy packages
• Various build scripts fail
• Replace from caches/backups MAYBE!
• Domain Knowledge is gone
TODAY
Daniel Greenfeld@pydanny
Library of Alexandria
http://en.wikipedia.org/wiki/File:Ancientlibraryalex.jpg
Daniel Greenfeld@pydanny
Stuff we make today
Is legacy within 5 years
Daniel Greenfeld@pydanny
Trust Issues
Daniel Greenfeld@pydanny
Lack of trust makes collaboration hard
Daniel Greenfeld@pydanny
No collaboration causes the
NIHplague
Daniel Greenfeld@pydanny
http://en.wikipedia.org/wiki/File:Zombie-walk-kids.JPG
Daniel Greenfeld@pydanny
RepercussionsTODAY
• Annoyance
• Ability for Python to move forward HALTS
• Social issues
Daniel Greenfeld@pydanny
Solution?
Sponsorships!
Daniel Greenfeld@pydanny
Sponsorships
• Individual, Corporate, Academic, GSOC
• Advance an open source project forward!
Daniel Greenfeld@pydanny
Sponsorships
• Python core
• Django
• Plone
• Twisted
• Read the Docs (PSF)
• WebOb (PSF)
Tons more I’m missing!
Daniel Greenfeld@pydanny
Problems
• Application process
• Seems focused on short-term development
• Interns often produce unusable code
• Ongoing maintenance?
Daniel Greenfeld@pydanny
Critical Packages Breakdown
• Legacy code tries to install legacy packages
• Various build scripts fail
• Replace from caches/backups MAYBE!
• Domain Knowledge is gone
NOCHANGES
Daniel Greenfeld@pydanny
Repercussions
• Annoyance
• Ability for Python to move forward HALTS
• Social issues
NOCHANGES
Daniel Greenfeld@pydanny
Same as before
Daniel Greenfeld@pydanny
Still haveTrust Issues
Daniel Greenfeld@pydanny
Why should I collaborate with you?
Daniel Greenfeld@pydanny
Future stilllooks dark
Daniel Greenfeld@pydanny
So now what?
Daniel Greenfeld@pydanny
Solution?
Community Managers
Daniel Greenfeld@pydanny
Community Managers
• David Eaves DjangoCon 2011 Keynote
• http://bit.ly/djangocon-keynote-2011-1
• Makes sure packages are maintained!
Daniel Greenfeld@pydanny
Problems
• Need core/senior developers
• They are already busy
• Who assigns authority?
Daniel Greenfeld@pydanny
Solution ReconsideredPSF Paid Community Manager
• Reasonable pay
• Works with PyPI team to do package curation
• Helps project leads if they need to hand off
• Broadcasts so the community knows about it
Daniel Greenfeld@pydanny
Solution ReconsideredPSF Paid Community Manager
• Performance reviews
• Term limits
• Great resume material
• Volunteers have different priorities
Daniel Greenfeld@pydanny
Increaseof
Trust
Daniel Greenfeld@pydanny
This makes collaboration easier
Daniel Greenfeld@pydanny
The Brighter Future
Daniel Greenfeld@pydanny
Critical Packages Breakdown
• A number of critical Python Packages vanish
• Various build scripts fail
• Replace from caches/backups
• Domain Knowledge is gone
Today
Daniel Greenfeld@pydanny
Critical Packages Breakdown
• A number of critical Python Packages vanish
• Various build scripts continue
• Domain Knowledge is at risk
Future
Daniel Greenfeld@pydanny
Repercussions
• Annoyance
• Ability for Python to move forward HALTS
• Social Issues
Today
Daniel Greenfeld@pydanny
Repercussions
• Annoyance
• Ability for Python to move forward slows
• Mitigated Social Issues
Future
Daniel Greenfeld@pydanny
Precedents
• Ubuntu Community Manager
• Isn’t there a Fedora Community Manager?
• Twilio Community Managers
Daniel Greenfeld@pydanny
But...
Daniel Greenfeld@pydanny
I want more
Daniel Greenfeld@pydanny
More reasons to trust
Daniel Greenfeld@pydanny
More incentivesto collaborate
Daniel Greenfeld@pydanny
More reasons to give code back
Daniel Greenfeld@pydanny
Question:
How do we keep the projects that do this operating?
Daniel Greenfeld@pydanny
Call to Action
Daniel Greenfeld@pydanny
Proposal:
PSFProject Incubation
Daniel Greenfeld@pydanny
The Idea Part I
The PSF provides seed money toopen source projects that can provide a
return of investment on these items:
• Tangible benefit to the Python community
• Money via a viable business model
Daniel Greenfeld@pydanny
The Idea Part II
• Choose from participants in coding contests
• Django Dash & other unborn contests
• PSF offers YC-style seed funding
• Marketing: Blog posts, PyCon booths, python.org
• Lets them go to do their thing
What the PSF Board / Community does:
Daniel Greenfeld@pydanny
The Idea Part III
• Deliverable open source code
• Something that makes the Python world better
• PSF is an investor so dividends come back
What the incubatee provides:
Daniel Greenfeld@pydanny
What this isn’t
• Covering < $100 monthly hosting
• Throwing money at things without a self-supporting business model
• For part-timers - It’s all or nothing
Daniel Greenfeld@pydanny
If I were rich I would do this right now
Daniel Greenfeld@pydanny
Example Projects
Real World
Daniel Greenfeld@pydanny
djangolint.com
Daniel Greenfeld@pydanny
http://djangolint.com
• Winner of this year’s Django Dash
• I want this for all of Python
• Will it be maintained?
• Will it go the way of PyPants?
Daniel Greenfeld@pydanny
http://djangolint.com
• Winner of this year’s Django Dash
• I want this for Python
• API/Syndication? (Open Comparison)
• Will it be maintained?
• Will it go the way of PyPants?
Daniel Greenfeld@pydanny
Pay to analyze privately
Business Model
(Great for client projects!)
Daniel Greenfeld@pydanny
Easy linting increases Trust
Daniel Greenfeld@pydanny
Collaborate with highlint rated projects!
Daniel Greenfeld@pydanny
readthedocs.org
Daniel Greenfeld@pydanny
python.rtfd.org
Daniel Greenfeld@pydanny
http://rtfd.org
• Placed in the 2010 Django Dash
• python.rtfd.org
• plone.rtfd.org
• pydanny-event-notes.rtfd.org
Daniel Greenfeld@pydanny
A measurable benchmark!
Daniel Greenfeld@pydanny
Documentationincreases
Trust
Daniel Greenfeld@pydanny
Documentationincreases
TrustTrust
increasesCollaboration
Daniel Greenfeld@pydanny
Can it do more?
• Support other markup languages?
Daniel Greenfeld@pydanny
Business Model
Pay for private doc hosting?
Daniel Greenfeld@pydanny
depot.io
Daniel Greenfeld@pydanny
http://depot.io
• A place to freeze your Python dependencies
• Let’s PyPI focus on what it does best - Indexing
• Provides additional security
• Fetch packages any way you want
• Not yet launched. ☹Developer paying the bills?
Daniel Greenfeld@pydanny
http://depot.io
• PyPI is still the canonical source
• Less need for Paid Community manager
• Will probably archive all legacy packages
Possible Advantages
(Except ones flagged for critical things like security)
• OMG CAN WE HAVE THIS PLEASE?
Daniel Greenfeld@pydanny
Dependabilityincreases
TrustTrust
increasesCollaboration
Daniel Greenfeld@pydanny
http://depot.io• PSF should incubate this project
• Open source it?
• + Lowers the bus factor
• - Easy launch of copy versions of site
• - Does it become harder to earn profit?
• + Once launched, SLA is key
Daniel Greenfeld@pydanny
PyPI
Daniel Greenfeld@pydannyhttp://pypi.python.org/pypi
Daniel Greenfeld@pydanny
PyPI
• Launched before the days of OS X
• The canonical source for Python packages
• Focusing hard on just Indexing
Daniel Greenfeld@pydanny
Business Model
Pay for a PyPI Appliance?
Daniel Greenfeld@pydanny
Dependabilityincreases
TrustTrust
increasesCollaboration
Daniel Greenfeld@pydanny
opencomparison.org
Daniel Greenfeld@pydannyhttp://djangopackages.com/
Daniel Greenfeld@pydannyhttp://pyramid.opencomparison.org/
Daniel Greenfeld@pydanny
opencomparison.org
• Competed in Django Dash 2010
• More sites coming soon!
• Plone
• Flask
• Python!
• A year of expansion and code cleanup
Daniel Greenfeld@pydanny
opencomparison.org
Largest sprint group
at PyCon 2011!
Daniel Greenfeld@pydanny
http://bit.ly/django-reg
Many good people
Most downloads Maintained?
Much recent work
Daniel Greenfeld@pydanny
http://bit.ly/django-reg
Many good people
Most downloads Maintained?
Much recent work
Django Social Authis my current
registration preference
Daniel Greenfeld@pydanny
Metricsincrease
TrustTrust
increasesCollaboration
Daniel Greenfeld@pydanny
• More OAUTH and OpenID Support
• Improved search
• Language neutral to support other tools
• Better caching
• Additional RSS feeds
• Give us more hard metrics please!
I ♥ Open Comparison
Daniel Greenfeld@pydanny
Business Model
???
Daniel Greenfeld@pydanny
Business Model
• Consulting for Open Comparison
• Enterprise Support?
• Ideas anyone?
Daniel Greenfeld@pydanny
Results?
Daniel Greenfeld@pydanny
Self-sustainingPython ecosystem
Daniel Greenfeld@pydanny
Critical Packages Breakdown
• A number of critical Python Packages vanish
• Various build scripts fail
• Replace from caches/backups
• Domain Knowledge is gone
Today
Daniel Greenfeld@pydanny
Critical Packages Breakdown
• A number of critical Python Packages don’t vanish
• Various build scripts continue
• Domain Knowledge remains
Future
Daniel Greenfeld@pydanny
Repercussions
• Annoyance
• Ability for Python to move forward HALTS
• Social Issues
Today
Daniel Greenfeld@pydanny
Repercussions
• Complacency about stability of ecosystem
• Ability for Python to move forward continues
• What Social Issues?
Future
Daniel Greenfeld@pydanny
The Future?
Daniel Greenfeld@pydanny
The Ideas
• Paid Community Manager
• PSF project incubation
Daniel Greenfeld@pydanny
Shoot them down!
Daniel Greenfeld@pydanny
Paid Community Manager
• If the PSF doesn’t do it, there is already PyPI
• Maybe the PSF shouldn’t be involved
• Ubuntu didn’t debate the position
Daniel Greenfeld@pydanny
PSF Project Incubation
This is not cheap!At least $15K per incubation
Daniel Greenfeld@pydanny
PSF Project Incubation
This is not cheap!At least $15K per incubation
Problem
Daniel Greenfeld@pydanny
PSF Project Incubation
• Already exist, just not with the PSF
• How much produced code comes out?
• Imagine the energy of a start-up giving back!
• Is happening already so PSF ought to profit...
Daniel Greenfeld@pydanny
Thanks Github!
http://octodex.github.com/#collabocats
Daniel Greenfeld@pydanny
Questions?