Component Oriented Programming in Python Brian Kelley Bioreason, Inc.

Component Oriented Programming in Python

Brian Kelley

Bioreason, Inc.

What is Component Programming?

Object Oriented programming provides a distinction between a class and the instance of the class.A component in engineering is considered ‘physically realizable’

In a stereo system a speaker is a componentYou can purchase speakers from various companies and interchange them as you see fit.How the speaker attaches to your stereo is just as important as how the speaker works internally (that is produces sound)

What is Component Programming? (cont)

A software component can be thought of as a component in the engineering sense

The component is an independent entity

The component is realizable, that is, the component exists in binary form (the reality of software)

Going back to our speaker analogy: the way the speaker communicates to the stereo is called an ‘interface’ that is communicated through a ‘wire protocol’

The distinction between object, instance and class

Basically a component provides a service with a known interface.

This is very similar to the Python standard library…

A component may deliver an object (in pythonic speak an instance) that may be created by a class.

A component should not have persistent state.

The instances it delivers might.

Summary

Using Clemens Szyperski’s summary:

A software component is a unit of composition with contractually specified interfaces and explicit context dependencies only. A software component can be deployed independently and is subject to composition by third parties. p 34 Component Software

Okay, why python?

Python’s biggest strength (and perhaps sometimes weakness) is its dynamic nature.

Dynamic (run-time) typing

Instances can be modified at run time (new methods may be added, attributes may be added)

Well described query interface for instances. (I.e. hasattr(inst, ‘foo’))

Okay, why Python?(cont)The object model lends itself to component programming

Modules provide interfaces to create objects (ala Modula-3) You could consider a module a component for instance.Objects have queryable and dynamic interfaces (ala SmallTalk). Note this is much different than a static interface but is more powerful.

Corba/COM/XML-RPC interfaces already exist!I can’t overstate this actually. A lot of excellent work by people a lot more talented than myself has made it easy for me to get my work done.

Issues Static Typing versus Dynamic Typing

One of the main complaints from non PythonersMyth #1 Static typing is ‘safer’

Run time typing(through a query interface) is actually much safer. Just harder to find. How many times have I core dumped C++ by typecasting to the wrong or unsupported type?

Myth #2 Static typing forms better interfacesMeta Models and abstraction can save you an awful lot of time. (We will give an example later) Just give your function an argument that walks and talks like a duck and it’s happy. (Almost, some functions like marshal are very picky…)

Killer Feature

In general, with a well thought out design, multiple component solutions can be used interchangeably without affecting the majority of the codebase.

For example a COM solution can be a drop in replacement for a CORBA solution or a Windows solution can be a drop in for a Unix solution.

The penalty for changing the wire protocol can be minimized substantially.

Simple Example File ObjectPython’s file service provides a good API for handling file objects.

fileob = open(filename, mode)

fileob is a file object with a well known interfacefileob.read()fileob.write()…

Many operations can be encapsulated by using the interface of the file object. Suppose you want to read write to a buffer?

buffer = StringIO.StringIO()Buffer behaves like a file object!

More file object like entitiesUnix/windows pipes can be treated like file objects.Sockets can be abstracted to act like file objects.So now we have four different services that behave like a file object. Because python is dynamically typed any method/instance whatever that expects a file object can now handle a pipe or a socket or etc.

There are always exceptions to the rule. The marshal module, for instance, is very picky about file objects.

Application of Component Programming

Talking between a Controller and a Computation or other long running processes.

Normally one might consider a multi-threading approach to solving this problem but as Swaine once wrote “Multithreading will rot your teeth.”More practically if Python is running a gui and expects to multi-thread to a component in the same process there is a substantial performance hit.

Solution: have the computation run in an independent process.

Requirements

Asynchronous I/O between controller and computation

These can also be classified as events

The computation indicates how far along the computation is at various intervals.

The ability to terminate the computation must exist (when the user hits cancel for instance)

This is a job for components!

Interface

For simplicity I’ll describe the interfaces in python. First up is the controller.

class Controller:

def showProgress(self, i):

‘’’(i)->set the progress to i where 0<=i<=100’’’

def cancel(self):

‘’’cancel the computation’’’

Computation InterfaceNotice that showProgress is kind of strange in that the computation will call a method that creates an event.

I’ll also cheat and show code for the ComputationModel to save some time and space

class ComputationModel:

def start(self):

‘’’start the computation’’’

for i in xrange(100000):

if i%1000 == 0:

self.showProgress(i/1000)

def showProgress(self, progress):

‘’’create a progress event with value progress’’’

Composition for testing

These classes can be tested right now!We are using python as the wire protocol here.

class Gui(Controller):

def __init__(self, computation):

self.process = computation(self)

# we need some mechanism to connect the

# controller to the computation/model

self.process.initiaize(self)

self.process.start()

class Computation(ComputationModel):

def connect(self, gui):

self._gui = gui

def showProgress(self, i):

self._gui.showProgress(i)

Note that we aren’t meeting the requirements here. We can’t terminate the computation and any gui we have will lock up completely during the computation (no asynchronous I/O) But it sure beats debugging a multithreaded/distributed app!

Spawning a new process

There are three ways of spawning a new process that I will discuss

Spawning a Unix child process

Spawning a Windows child process

Using CORBA as a server

A little bit of honestyPython is a great environment for component programming and plug-in architectures but…

Some of the underlying code for spawning and communicating with a unix/windows process is hairy stuff. I won’t go into much depth with the details.

What you should get from the rest of the talk is this:Given the proper tools, I.e. CORBA or the simple Unix/Windows process communication infrastructure as long as you design an appropriate interface the wiring protocol is not that important. This simply is not true in a static language.For instance the only way (that I know) how to do this in C++ is with a pure abstract base class.

Brief Explanation of the Child Process Controller

GUIController

ParentProxy ChildProxy

Computation

stdin

stdoutstdin

stdout

Messages from the parent to the child are sent from the parent’s stdout to the child’s stdin. Because both the parent and the child are python class instances we can simplify the protocol.

For example if the child executes print ‘self.showProgress(45)’ the parent’s stdin receives ‘self.showProgress(45)’ Using the python exec statement we can do the following:

exec ‘self.showProgress(45)’ in {‘self’:guiController.__dict__}

Is exactly the same as guiController.showProgress(45)

Unix versus Windows

The Unix version uses pipes while the windows version uses win32 named pipes. Both allow non-blocking I/O which is asynchronous. (One of the requirements)

They use different methods to terminate the child process but in both cases when the computation goes out of scope or is deleted via “del computation” the child process will terminate.

Corba-Style ImplementationCORBA is a well defined standard for component oriented programming. Some of its features are:

A language independent interface description (IDL) Implementations can be written in multiple languages C++/Python/Perl/Java etc…

The computation interface is trivial except for that fact that it needs to send messages to the parent.

Do they support events/listeners? Events are a pain in CORBA and hard to use from Python. Listeners are much easier.Do they work with your GUI’s event loop? (This really only matters if the server is running in a GUI)

CORBA Interface module Computation { // Create the listener on the client side and // send to the server interface ProgressListener() { void showProgress(in short progress); }; interface Computor { void start(); void addProgressListener(in ProgressListener p); };};

Implementation

CORBA allows implementations to be written either on the server or on the client.

Create the computor object by requesting the CORBA server for an object of type Computor

Create the ProgressListener on the client giving the implementation access to the ControllerModel’s showProgress function.

class ComputationServer(Computation__POA.Computor,

ComputationModel):

def __init__(self):

self.showProgress = DoNothing()

def addProgressListener(self, listener):

self.showProgress = listener

class DoNothing:

def __call__(self, progress):

pass

Server Implementation

callback that goes

Nowhere.

Add the client callback

This is a wrapper/mixin for the

ComputationModel. The model isn’t

changed at all!

Client Implementation

class ClientProgressListener(Computation__POA.ProgressListener):

def __init__(self, controller):

self.controller = controller

def showProgress(self, progress):

self.controller.showProgress(progress)

This is an omni ORB example. Computation__POA

Dispatches events to their implementations. These can

Be implemented on the server or the client (good for us!)

Conclusion

You should have seen that for two completely different transport layers, there was NO modification to the model/view/controller code

Okay, we cheated, we didn’t show initialization

Making a wrong decision can be refactored relatively easy (given a good initial design)

Python itself models component oriented programming.