Python Evolution

Python Evolution2.4, 2.5, 2.6, 3.0by Myroslav Opyr, CTO

© Quintagroup 2008

Contents

• Python 2.x featureso 2.4o 2.5o 2.6

• Why Python 3k?• Distribution as competitive benefit• References

Python 2.4 features

• built-in sets (PEP 218)• generator expressions (PEP 289)• new subprocess module (PEP 324)• function/method decorators (PEP 318)

Python 2.5 features

• abstract syntax tree (AST)• 'with' statement (PEP 343)• absolute/relative imports (PEP 328)• distutils gained dependency info (PEP 314)• wsgiref package (PEP 333)• coroutines (PEP 342)

Non-features• PyPI accepts uploads• Continuous Integration with buildbot

Python 2.6 features

• no more string exceptions• The "bytes" Object (PEP 358)• warnings for features to be removed in P3k• multiprocessing (PEP 371)• ast module• setuptools module• 2to3

Non-features• Python bugtracker moved to roundup

Questions?

pause...

Python 3k/3000/3.0

• Incompatible changeso What Python 3.0 won't beo What Python 3.0 will be

• Release Plan• Migration process

o 2to3

Python 3.0. Evolution philosophy

Joel on Software: Things You Should Never Do, Part I: Netscape was waiting so long between releases by making the

single worst strategic mistake that any software company can make:

They decided to rewrite the code from scratch.

http://www.joelonsoftware.com/articles/fog0000000069.html

What Python 3.0 won't be

PEP-3099 highlights:• Python 3000 will not be case-insensitive• Python 3000 will not be a rewrite from scratch• self will not become implicit• lambda will not be renamed• No braces (from __future__ import braces)• Iterating over a dictionary will continue to yield the keys• The interpreter prompt (>>>) will not change. It gives Guido

warm fuzzy feelings

What Python 3.0 will be

PEP-3000 Python 3.0 development guidelines:• PEP Numbering• Timeline• Compatibility and Transition• Implementation Language

PEP-3100 Python 3.0 implementation "reduce feature duplication by removing old ways of doing things.

A general principle of the design will be that one obvious way of doing something is enough."

Python 3.0. Core language

• True division becomes default behavior• exec as a statement is not worth it -- make it a function• Add optional declarations for static typing (annotations)• Support only new-style classes; classic classes will be gone• Replace print by a function• The softspace attribute of files goes away.• Use except E1, E2, E3 as err: if you want the error variable.• None becomes a keyword; also True and False• as becomes a keyword• Comparisons other than == and != between disparate types will

raise an exception unless explicitly supported by the type

Python 3.0. Core language (cont.)

• floats will not be acceptable as arguments in place of ints for operations where floats are inadvertantly accepted

• Remove from ... import * at function scope.• Imports

o Imports will be absolute by default.o Relative imports must be explicitly specified.

• Some expressions will require parentheses that didn't in 2.x: o List comprehensions will require parentheses around the

iterables. This will make list comprehensions more similar to generator comprehensions. [x for x in 1, 2] will need to be: [x for x in (1, 2)]

• Set literals and comprehensions. {x} means set([x]); {x, y} means set([x, y]).

Python 3.0. Core language (cont.)

To be removed:• String exceptions: use instances of an Exception class• raise Exception, "message": use raise Exception("message")• `x`: use repr(x)• The <> operator: use != instead• Drop unbound methods• __getslice__, __setslice__, __delslice__; remove slice opcodes

and use slice objects.• __oct__, __hex__: use __index__ in oct() and hex() instead.• __methods__ and __members__

Python 3.0. Atomic Types

• Remove distinction between int and long types; 'long' built-in type and literals with 'L' or 'l' suffix disappear

• Make all strings be Unicode, and have a separate bytes() type. The new string type will be called 'str'. See PEP 3137.

• Return iterable views instead of lists where appropriate for atomic type methods (e.g. dict.keys(), dict.values(), dict.items(), etc.); iter* methods will be removed.

To be removed:• file.xreadlines() method• dict.has_key() method; use in operator• list.sort() and builtin.sorted() methods: eliminate cmp parameter

Python 3.0. Built-in Namespace

• Make built-ins return an iterator where appropriate (e.g. range(), zip(), map(), filter(), etc.)

• Remove input() and rename raw_input() to input(). If you need the old input(), use eval(input()).

• Introduce trunc(), which would call the __trunc__() method on its argument; suggested use is for objects like float where calling __int__() has data loss, but an integral representation is still desired

• Exception hierarchy changes• Add a bin() function for a binary representation of integers

Python 3.0. Built-in Namespace (cont.)

To be removed:• apply(): use f(*args, **kw) instead• buffer(): must die (use a bytes() type instead)• callable(): just use hasattr(x, '__call__')• compile(): put in sys• coerce(): no longer needed• execfile(), reload(): use exec()• intern(): put in sys• reduce(): put in functools, a loop is more readable most of the

times• xrange(): use range()• StandardError: this is a relic from the original exception hierarchy;

subclass Exception instead.

Python 3.0. Release Plan

PEP-361: Python 2.6 and 3.0 are lock-stepped (the same procedure was used for 1.6 and 2.0):• Feb 29 2008: Python 2.6a1 and 3.0a3 are released• Apr 02 2008: Python 2.6a2 and 3.0a4 are released• May 08 2008: Python 2.6a3 and 3.0a5 are released• Jun 18 2008: Python 2.6b1 and 3.0b1 are released• Jul 02 2008: Python 2.6b2 and 3.0b2 planned• Aug 06 2008: Python 2.6rc1 and 3.0rc1 planned• Aug 20 2008: Python 2.6rc2 and 3.0rc2 planned• Sep 03 2008: Python 2.6 and 3.0 final

Python 3.0. Migration

Python 3.0 will break backwards compatibility with Python 2.x.There is no requirement that Python 2.6 code will run unmodified on Python 3.0. Not even a subset.Python 2.6 will support forward compatibility in the following two ways:• It will support a "Py3k warnings mode" which will warn dynamically (i.e.

at runtime) about features that will stop working in Python 3.0, e.g. assuming that range() returns a list.

• It will contain backported versions of many Py3k features, either enabled through __future__ statements or simply by allowing old and new syntax to be used side-by-side (if the new syntax would be a syntax error in 2.x).

Python 3.0. Migration (cont.)

The recommended development model for a project that needs to support Python 2.6 and 3.0 simultaneously is as follows:1. You should have excellent unit tests with close to full coverage.– Port your project to Python 2.6.– Turn on the Py3k warnings mode.– Test and edit until no warnings remain.– Use the 2to3 tool to convert this source code to 3.0 syntax. Do not manually edit the

output!– Test the converted source code under 3.0.– If problems are found, make corrections to the 2.6 version of the source code and go

back to step 3.– When it's time to release, release separate 2.6 and 3.0 tarballs (or whatever archive

form you use for releases).It is recommended not to edit the 3.0 source code until you are ready to reduce 2.6 support to pure maintenance (i.e. the moment when you would normally move the 2.6 code to a maintenance branch anyway).

Python 3.0. Questions?

pause...

Distributing Python code

• Why distribution is important?• distutils• setuptools• easy_install• virtualenv• Private PyPI

Why distribution is important?

• Simpler installation makes Python software reusable andmore accessible

• More users• More pressure to package developers• Better distributions

o more meta-datao more documentationo more testso more platforms

• "CheeseCake" rating system (better package kwalitee)

distutils

For end-users: python setup.py install

For developers: python setup.py sdist

For packagers: python setup.py bdist

Since Python-2.5, you can host your packages directly on PyPI python setup.py upload

setuptools

• Automatically find/download/install/upgrade dependencies at build time using the EasyInstall tool, which supports downloading via HTTP, FTP, Subversion, and SourceForge, and automatically scans web pages linked from PyPI to find download links. (It's the closest thing to CPAN currently available for Python.)

easy_install grockproject

• Create Python Eggs - a single-file importable distribution format python setup.py bdist_egg

setuptools (cont.)

• Command aliases - create project-specific, per-user, or site-wide shortcut names for commonly used commands and options

• PyPI upload support - upload your source distributions and eggs to PyPI• Deploy your project in "development mode", such that it's available on sys.path, yet

can still be edited directly from its source checkout.• Easily extend the distutils with new commands or setup() arguments, and

distribute/reuse your extensions for multiple projects, without copying code.• Create extensible applications and frameworks that automatically discover

extensions, using simple "entry points" declared in a project's setup script.

easy_install

easy_install SQLObject

easy_install -f http://pythonpaste.org/package_index.html SQLObject

easy_install http://example.com/path/to/MyPackage-1.2.3.tgz

easy_install /my_downloads/OtherPackage-3.2.1-py2.3.egg

easy_install --upgrade PyProtocols

easy_install .

easy_install --editable --build-directory ~/projects SQLObject

virtualenv

virtualenv is a tool to create isolated Python environments. The basic usage is: $ python virtualenv.py ENV

This creates ENV/lib/python2.4/site-packages , where anylibraries you install will go. It also creates ENV/bin/python,which is a Python interpreter that uses this environment.

Allows bootstrap scripts that prepare environment.

Alternatives:• workingenv• virtual-python• zc.buildout

Private PyPI

Folder...with eggs exposed (and password-protected) via Apache. Works with easy_install, requires "manual" uploading to folder, can be as simple as copy to shared folder

haufe.eggservergrok-powered solution. Recently upload support added. No dedicated security model (everyone is trusted and allowed to upload packages).Plone Software Center 1.5Plone product. Support upload, categorization, security. Has buildout.

Questions?

References

• http://www.python.org/download/releases/2.4/highlights/• http://www.python.org/download/releases/2.5/highlights/• http://www.joelonsoftware.com/articles/fog0000000069.html• http://www.python.org/dev/peps/pep-0361/• http://www.python.org/dev/peps/pep-3000/• http://www.python.org/dev/peps/pep-3099/• http://www.python.org/dev/peps/pep-3100/• http://www.python.org/download/releases/3.0/• http://mail.python.org/pipermail/python-dev/2006-April/064145.html• http://www.python.org/doc/current/inst/inst.html• http://docs.python.org/dist/dist.html• http://pypi.python.org/pypi/setuptools/• http://peak.telecommunity.com/DevCenter/setuptools• http://pypi.python.org/pypi/virtualenv• http://pypi.python.org/pypi/haufe.eggserver/• http://tarekziade.wordpress.com/2008/03/20/how-to-run-your-own-private-pypi-cheeseshop-se

rver/

Thanks

http://sites.google.com/a/quintagroup.com/techtalks/Topics/python-evolution