Pattern-Oriented Software Architecture Applying Concurrent & Networked Objects to Develop & Use Distributed Object Computing Middleware Pattern-Oriented Software Architecture Pattern-Oriented Software Architecture Applying Concurrent & Networked Objects to Develop & Use Distributed Object Computing Middleware Dr. Douglas C. Schmidt [email protected]http://www.posa.uci.edu/ Electrical & Computing Engineering Department The Henry Samueli School of Engineering University of California, Irvine Montag, 19. April 2004
136
Embed
Pattern-Oriented Software Architecture · Pattern-Oriented Software Architecture Applying Concurrent & Networked Objects to Develop & Use Distributed Object Computing Middleware Dr.
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Electrical & Computing Engineering DepartmentThe Henry Samueli School of Engineering
University of California, Irvine
Montag, 19. April 2004
2
Distributed Architecture
Middleware Patterns Tutorial Outline
Describe OO techniques & languagefeatures to enhance software quality
Stand-aloneArchitecture
Illustrate how/why it’s hard to build robust, efficient,& extensible concurrent & networked applications• e.g., we must address many complex topics that are lessproblematic for non-concurrent, stand-alone applications
OO techniques & language features include:•Patterns (25+), which embody reusable softwarearchitectures & designs
•Frameworks & components, which embody reusablesoftware implementations
In general, software has notimproved as rapidly or aseffectively as hardware
In general, software has notimproved as rapidly or aseffectively as hardware
Increasing software productivityand QoS depends heavily on COTSIncreasing software productivityand QoS depends heavily on COTS
4
Addressing the COTS “Crisis”
However, this trend presents many vexingR&D challenges for mission-criticalsystems, e.g.,• Inflexibility and lack of QoS• Security & global competition
Distributed systems must increasinglyreuse commercial-off-the-shelf (COTS)hardware & software• i.e., COTS is essential to R&D success
Why we should care:
•Despite IT commoditization, progress inCOTS hardware & software is often notapplicable for mission-criticaldistributed systems
•Recent advances in COTS softwaretechnology can help to fundamentallyreshape distributed system R&D
5
R&D Challenges & Opportunities
High-performance, real-time,fault-tolerant, & secure systems
IOM
IOM
IOM
IOM
IOM
IOM
IOM
IOM
IOM
IOM
IOM
IOM
IOM
IOM
IOM
IOM
IOM
IOM
BSE
BSE
BSE
BSE
BSE
BSE
BSEBSE
BSE
Power-aware ad hoc,mobile, distributed, &embedded systems
Applying COTS to Hot Rolling MillsGoals•Control the processing of moltensteel moving through a hot rollingmill in real-time
System Characteristics•Hard real-time process automationrequirements• i.e., 250 ms real-time cycles
•System acquires valuesrepresenting plant’s current state,tracks material flow, calculates newsettings for the rolls & devices, &submits new settings back to plant
Key Software Solution Characteristics
•Affordable, flexible, & COTS•Product-line architecture•Design guided by patterns & frameworks
•Affordable, flexible, & COTS•Product-line architecture•Design guided by patterns & frameworks
•Windows NT/2000•Real-time CORBA (ACE+TAO)
www.siroll.de
13
Example:
Applying COTS to Real-time Image Processing
Goals•Examine glass bottlesfor defects in real-time
Problem•Distributed application functionalityis subject to change since it isoften reused in unforeseencontexts, e.g.,•Accessed from different clients•Run on different platforms•Configured into different run-timecontexts
A component is anencapsulation unit withone or more interfacesthat provide clients withaccess to its services
A framework is an integratedcollection of classes thatcollaborate to produce areusable architecture for afamily of related applications
A class is a unit ofabstraction &implementation inan OOprogramminglanguage
MIDDLEWARE
Solution•Don‘t structure distributed applications as amonoliths, but instead decompose them intoclasses, frameworks, & components
Solution•Don‘t structure distributed applications as amonoliths, but instead decompose them intoclasses, frameworks, & components
17
A Comparison of Class Libraries,Frameworks, & Components
The Wrapper Facade design patternencapsulates the functions and data provided byexisting non-object-oriented APIs within moreconcise, robust, portable, maintainable, andcohesive object-oriented class interfaces.
The Component Configurator design patternallows an application to link and unlink itscomponent implementations at run-time withouthaving to modify, recompile, or statically relinkthe application. Component Configurator furthersupports the reconfiguration of components intodifferent application processes without having toshut down and re-start running processes.
The Interceptor architectural pattern allowsservices to be added transparently to aframework and triggered automatically whencertain events occur.
The Extension Interface design pattern allowsmultiple interfaces to be exported by acomponent, to prevent bloating of interfaces andbreaking of client code when developers extendor modify the functionality of the component.
Event Handling Patterns
The Reactor architectural pattern allows event-driven applications to demultiplex and dispatchservice requests that are delivered to anapplication from one or more clients.
The Proactor architectural pattern allowsevent-driven applications to efficientlydemultiplex and dispatch service requeststriggered by the completion of asynchronousoperations, to achieve the performancebenefits of concurrency without incurringcertain of its liabilities.
The Asynchronous Completion Token designpattern allows an application to demultiplexand process efficiently the responses ofasynchronous operations it invokes onservices.
The Acceptor-Connector design patterndecouples the connection and initialization ofcooperating peer services in a networkedsystem from the processing performed by thepeer services after they are connected andinitialized.
The Scoped Locking C++ idiomensures that a lock is acquired whencontrol enters a scope and releasedautomatically when control leaves thescope, regardless of the return pathfrom the scope.
The Strategized Locking design patternparameterizes synchronizationmechanisms that protect a component’scritical sections from concurrentaccess.
The Thread-Safe Interface designpattern minimizes locking overhead andensures that intra-component methodcalls do not incur ‘self-deadlock’ bytrying to reacquire a lock that is held bythe component already.
The Double-Checked LockingOptimization design pattern reducescontention and synchronizationoverhead whenever critical sections ofcode must acquire locks in a thread-safe manner just once during programexecution.
Concurrency Patterns
The Active Object design pattern decouples methodexecution from method invocation to enhance concurrencyand simplify synchronized access to objects that reside intheir own threads of control.
The Monitor Object design pattern synchronizes concurrentmethod execution to ensure that only one method at a timeruns within an object. It also allows an object’s methods tocooperatively schedule their execution sequences.
The Half-Sync/Half-Async architectural pattern decouplesasynchronous and synchronous service processing inconcurrent systems, to simplify programming withoutunduly reducing performance. The pattern introduces twointercommunicating layers, one for asynchronous and onefor synchronous service processing.
The Leader/Followers architectural pattern provides anefficient concurrency model where multiple threads taketurns sharing a set of event sources in order to detect,demultiplex, dispatch, and process service requests thatoccur on the event sources.
The Thread-Specific Storage design pattern allows multiplethreads to use one ‘logically global’ access point to retrievean object that is local to a thread, without incurring lockingoverhead on each object access.
23
Example of Applying Patterns & Frameworks:
Real-time CORBA & The ACE ORB (TAO)TAO Features•Open-source•500+ classes &500,000+ lines of C++
•ACE/patterns-based•30+ person-years ofeffort
•Ported to UNIX,Win32, MVS, & manyRT & embedded OSs
• e.g., VxWorks, LynxOS,Chorus, QNX
www.cs.wustl.edu/~schmidt/TAO.html
Protocol Properties Explicit Binding
Thread Pools
Scheduling Service
Standard Synchronizers
Portable Priorities
•Large open-source user community•www.cs.wustl.edu/~schmidt/TAO-users.html
Solution•Apply the Layers architecturalpattern to create a multi-tierarchitecture that separatesconcerns between groups ofsubtasks occurring at distinctlayers in the distributed system
Solution•Apply the Layers architecturalpattern to create a multi-tierarchitecture that separatesconcerns between groups ofsubtasks occurring at distinctlayers in the distributed system
Separating Concerns Between TiersContext• Distributed systems are nowcommon due to the advent of• The global Internet• Ubiquitous mobile & embeddeddevices
Problem• One reason it’s hard to build COTS-based distributed systems is becausea large number of capabilities mustbe provided to meet end-to-endapplication requirements
•Services in the middle-tier participatein various types of tasks, e.g.,•Workflow of integrated “business”processes
•Connect to databases & otherbackend systems for data storage& access
Database Tier• e.g., persistentdata
DBServer
DBServer
Middle Tier• e.g., commonbusiness logic
comp
comp
Application
Server
Presentation Tier• e.g., thin clients
Client Client
28
Applying the Layers Pattern toImage Acquisition
Image servers are middle tier components that:•Provide server-side functionality
•e.g., they are responsible for scalable concurrency & networking•Can run in their own address space•Are integrated into containers that hide low-level system details
Image servers are middle tier components that:•Provide server-side functionality
•e.g., they are responsible for scalable concurrency & networking•Can run in their own address space•Are integrated into containers that hide low-level system details
•Share the same addressspace with their clients• Their clients are containersthat provide all theresources
•Exchange messages withthe middle tier components
29
Pros & Cons of the Layers Pattern
This pattern has four benefits:•Reuse of layers
• If an individual layer embodies a well-defined abstraction & has a well-defined &documented interface, the layer can bereused in multiple contexts
•Support for standardization• Clearly-defined and commonly-acceptedlevels of abstraction enable thedevelopment of standardized tasks &interfaces
•Dependencies are localized• Standardized interfaces between layersusually confine the effect of code changesto the layer that is changed
•Exchangeability• Individual layer implementations can bereplaced by semantically-equivalentimplementations without undue effort
This pattern also has liabilities:•Cascades of changing behavior
• If layer interfaces & semanticsaren’t abstracted properly thenchanges can ripple when behaviorof a layer is modified
•Lower efficiency• A layered architecture can be lessefficient than a monolithicarchitecture
•Unnecessary work• If some services performed by lowerlayers perform excessive orduplicate work not actually requiredby the higher layer, performancecan suffer
•Difficulty of establishing thecorrect granularity of layers• It’s important to avoid too many &too few layers
30
Overview of Distributed Object ComputingCommunication Mechanisms
Solution•DOC middleware provides multiple types of communication mechanisms
Problem•A single communicationmechanism does not fit alluses!
ContextIn multi-tier systems both the tiers & thecomponents within the tiers must beconnected via communication mechanisms
Next, we’ll explorevarious patterns thatapplications can applyto leverage thesecommunicationmechanisms
Next, we’ll explorevarious patterns thatapplications can applyto leverage thesecommunicationmechanisms
31
Improving Type-safety & Performance
Context• The configuration ofcomponents indistributed systems isoften subject to changeas requirements evolve
• A Service implements the object, which is notaccessible directly
• A Proxy represents the Service andensures the correct access to it• Proxy offers same interface as Service
• Clients use the Proxy to access the Service
Proxy
service
Client
Service
service1 1
AbstractService
serviceSolutionApply the Proxy design pattern toprovide an OO surrogate throughwhich clients can access remoteobjects
SolutionApply the Proxy design pattern toprovide an OO surrogate throughwhich clients can access remoteobjects
Problems• Low-level message passing is fraught withaccidental complexity
• Remote components should look like localcomponents from an application perspective
• i.e., clients & servers should be oblivious tocommunication issues
: Service: Proxy: Clientservice
service
pre-processing:marshaling
post-processing:unmarshaling
32
Applying the Proxy Patternto Image Acquisition
Proxy
get_image()
Client
Image Xfer
get_image()1 1
AbstractService
get_image()
Invoke get_image() call
We can apply the Proxy patternto provide a strongly-typedinterface to initiate & coordinatethe downloading of imagesfrom an image database
When proxies are generated automatically by middleware theycan be optimized to be much more efficient than manualmessage passing•e.g., improved memory management, data copying, &compiled marshaling/demarshaling
When proxies are generated automatically by middleware theycan be optimized to be much more efficient than manualmessage passing•e.g., improved memory management, data copying, &compiled marshaling/demarshaling
Image Xfer Service
Image Xfer ServiceXfer ProxyXfer Proxy
ImageDatabase
RadiologyClient
RadiologyClient
33
Pros & Cons of the Proxy Pattern
This pattern provides three benefits:•Decoupling clients from the locationof server components• By putting all location information &addressing functionality into a proxyclients are not affected by migration ofservers or changes in the networkinginfrastructure
•Potential for time & spaceoptimizations• Proxy implementations can be loaded “on-demand” and can also be used to cachevalues to avoid remote calls
• Proxies can also be optimized to improveboth type-safety & performance
•Separation of housekeeping &functionality• A proxy relieves clients from burdens thatdo not inherently belong to the task theclient performs
This pattern has two liabilities:•Potential overkill viasophisticated strategies• If proxies include overlysophisticated functionality they manyintroduce overhead that defeats theirintended purpose
•Less efficiency due toindirection• Proxies introduce an additional layerof indirection that can be excessive ifthe proxy implementation isinefficient
Problem• Many object models assign a single interfaceto each component
• This design makes it hard to evolvecomponents without• Breaking existing client interfaces• Bloating client interfaces
Solution• Apply the ExtensionInterface design pattern toallow multiple interfaces tobe exported by acomponent, to preventbloating of interfaces &breaking of client codewhen developers extendor modify componentfunctionality
Solution• Apply the ExtensionInterface design pattern toallow multiple interfaces tobe exported by acomponent, to preventbloating of interfaces &breaking of client codewhen developers extendor modify componentfunctionality
CreateComponent
Factory
*
*
CreateInstance
CreateComponent
Componentnew
*1
QueryInterface
Root
implements
1+
<<extends>>
Ask for a referenceto an interface
Call anoperation on aninterface
Initialize
unititialize
ServerQueryInterface
service_i
ExtensionInterface i
Call_service
Client
35
Extension Interface Pattern Dynamics
: Client
Start_client
: Factory
CreateInstance(Ext.Intf. 1)new
Ref. To Extension1
create
create
service_1
QueryInterface(Extension Interface 2)
Ref. To Extension2
service2
Note how each extension interface canserve as a “factory” to return objectreference to other extension interfaces
Note how each extension interface canserve as a “factory” to return objectreference to other extension interfaces
: Component : Extension 1 : Extension 2
service_2
36
Pros & Cons of theExtension Interface Pattern
This pattern has five benefits:•Separation of concerns
• Interfaces are strictly decoupled fromimplementations
•Exchangeability of components• Component implementations can evolveindependently from clients that accessthem
•Extensibility through interfaces• Clients only access components via theirinterfaces, which reduces coupling torepresentation & implementation details
•Prevention of interface bloating• Interfaces need not contain all possiblemethods, just the ones associated with aparticular capability
•No subclassing required• Delegation—rather than inheritance—isused to customize components
This pattern also has liabilities:•Overhead due to indirection
• Clients must incur theoverhead of several round-tripsto obtain the appropriate objectreference from a servercomponent
•Complexity & cost fordevelopment & deployment• This pattern off-loads theresponsibility for determiningthe appropriate interface fromthe component designer to theclient application
37
Ensuring Platform-neutral & Network-transparent OO Communication
Problem•We need an architecture that:
•Supports remote method invocation•Provides location transparency•Allows the addition, exchange, orremove of services dynamically
•Hides system details from thedeveloper
Solution•Apply theBroker patternto provide OOplatform-neutralcommunicationbetweennetworked client& servercomponents
Solution•Apply theBroker patternto provide OOplatform-neutralcommunicationbetweennetworked client& servercomponents
Context•Using the Proxy pattern isinsufficient since it doesn‘t addresshow•Remote components are located•Connections are established•Messages are exchanged across anetwork
Pros & Cons of the Broker PatternThis pattern has five benefits:•Portability enhancements
• A broker hides OS & network system detailsfrom clients and servers by using indirection &abstraction layers, such as APIs, proxies,adapters, & bridges
• Interoperability with other brokers• Different brokers may interoperate if theyunderstand a common protocol for exchangingmessages
•Reusability of services• When building new applications, brokersenable application functionality to reuseexisting services
•Location transparency• A broker is responsible for locating servers, soclients need not know where servers arelocated
•Changeability & extensibility ofcomponents• If server implementations change withoutaffecting interfaces clients should not beaffected
This pattern also has liabilities:
•Restricted efficiency• Applications using brokers maybe slower than applicationswritten manually
•Lower fault tolerance• Compared with non-distributedsoftware applications,distributed broker systems mayincur lower fault tolerance
•Testing & debugging may beharder• Testing & debugging ofdistributed systems is tediousbecause of all the componentsinvolved
41
Supporting Async Communication
Solution•Apply the Async Forwarder/Receiver designpattern to allow asynchronous communicationbetween clients & servers
Solution•Apply the Async Forwarder/Receiver designpattern to allow asynchronous communicationbetween clients & servers
Context• Some clients wantto send requests,continue their work,& receive theresults at somelater point in time
Problem•Broker implementations based on conventional RPCsemantics often just support blocking operations• i.e., clients must wait until two-way invocations return
•Unfortunately, this design can reduce scalability &complicate certain use-cases
Introduce intermediary queue(s) between clients &servers:• A queue is used to store messages
• A queue can cooperate with other queues toroute messages
• Messages are sent from sender to receiver• A client sends a message, which is queued &then forwarded to a message processor on aserver that receives & executes them
• A Message API is provided for clients & servers tosend/receive messages
Local Queue
storeforwardremove
Message <<route>>
Client Message API<<send>>
Remote Queue
storeforwardremove<<exec>>
MessageProcessor Message API<<recv>>
42
Async Forwarder/Receiver Pattern Dynamics
: Client
: Messagecreate
storeMessage
forwardMessage
: MessageProcessor
createreceive
receive
Message
Message
exec
send Message
: Message API
: LocalQueue
: RemoteQueue
: MessageAPI
Reply
store
forwardReply
recv Reply
send
Reply
recv
Otherprocessing
43
Applying the Async Forwarder/ReceiverPattern to Image Acquisition
We can apply the Async Forwarder/Receiverpattern to•Queue up image request messages remotelywithout blocking the diagnostic/clinicalworkstation clients
•Execute the requests at a later point & return theresults to the client
Local Queue
storeforwardremove
Message <<route>>
RadiologyClient Message API<<send>>
Remote Queue
storeforwardremove<<exec>>
Message API<<recv>>
Image ServerMessage
Processor
Image Xfer Service
Image Xfer Service
ImageDatabase
RadiologyClient
RadiologyClient
This design also enables other, more advanced capabilities, e.g.,•Multi-hop store & forward persistence•QoS-driven routing, where requests can be delivered to the“best” image database
This design also enables other, more advanced capabilities, e.g.,•Multi-hop store & forward persistence•QoS-driven routing, where requests can be delivered to the“best” image database
44
This pattern provides three benefits:•Enhances concurrency bytransparently leveraging availableparallelism• Messages can be executed remotely onservers while clients perform otherprocessing
•Simplifies synchronized access toa shared object that resides in itsown thread of control• Since messages are processed seriallyby a message processor target objectsoften need not be concerned withsynchronization mechanisms
•Message execution order can differfrom message invocation order• This allows reprioritizing of messages toenhance quality of service
This pattern also has someliabilities:
•Message execution order candiffer from message invocationorder• As a result, clients must be careful notto rely on ordering dependencies
•Lack of type-safety• Clients & servers are responsible forformatting & passing messages
•Complicated debugging• As with all distributed systems,debugging & testing is complex
Pros & Cons of the AsyncForwarder/Receiver Pattern
45
Supporting OO Async Communication
Solution•Apply the Active Object design pattern to decouple method invocation frommethod execution using an object-oriented programming model
Solution•Apply the Active Object design pattern to decouple method invocation frommethod execution using an object-oriented programming model
• A proxy provides an interface that allowsclients to access methods of an object
• A concrete method request is created forevery method invoked on the proxy
• A scheduler receives the method requests& dispatches them on the servant whenthey become runnable
• An activation list maintains pendingmethod requests
• A servant implements the methods• A future allows clients to access theresults of a method call on the proxy
Context• Some clients want to invokeremote operations, continuetheir work, & retrieve theresults at a later point in time
Problem•Using the explicit message-passing API ofthe Async Forwarder/Receiver pattern canreduce type-safety & performance•Similar to motivation for Proxy pattern...
Future
Scheduler
enqueue dispatch
MethodRequest
guardcall
*
Proxy
method_1method_n
ActivationList
enqueuedequeue
Servant
method_1method_n
creates creates maintains
ConcreteMethodRequest1
ConcreteMethodRequest2
46
• A client invokes a method on theproxy
• The proxy returns a future to theclient, & creates a methodrequest, which it passes to thescheduler
• The scheduler enqueues themethod request into theactivation list (not shown here)
• When the method requestbecomes runnable, the schedulerdequeues it from the activationlist (not shown here) & executesit in a different thread than theclient
• The method request executes themethod on the servant & writesresults, if any, to the future
• Clients obtain the method’sresults via the future
Active Object Pattern Dynamics
: Future
method
enqueue
: Proxy : Scheduler : Servant
: MethodRequest
dispatch call method
read
write
: Client
Clients can obtain result from futuresvia blocking, polling, or callbacksClients can obtain result from futuresvia blocking, polling, or callbacks
47
Applying the Active Object Patternto Image Acquisition
•OO developers generally prefermethod-oriented request/responsesemantics to message-orientedsemantics
•The Active Object pattern supportsthis preference via strongly-typedasync method APIs:•Several types of parameters canbe passed:•Requests contain in/inoutarguments
•Results carry out/inoutarguments & results
•Callback object or poller object canbe used to retrieve results
Future
Scheduler
enqueue dispatch
MethodRequestguardcall
*
Proxy
method_1method_n
ActivationList
enqueuedequeue
Servantmethod_1method_n
creates creates maintains
get_image() put_image()
Image Xfer Service
Image Xfer Service
ImageDatabase
RadiologyClient
RadiologyClient
get_image()
future results
48
This pattern provides four benefits:•Enhanced type-safety
• Compared with async message passing•Enhances concurrency & simplifiessynchronized complexity• Concurrency is enhanced by allowing client threads& asynchronous method executions to runsimultaneously
• Synchronization complexity is simplified by using ascheduler that evaluates synchronizationconstraints to guarantee serialized access toservants
•Transparent leveraging of availableparallelism• Multiple active object methods can execute inparallel if supported by the OS/hardware
•Method execution order can differ frommethod invocation order• Methods invoked asynchronous are executedaccording to the synchronization constraintsdefined by their guards & by scheduling policies
This pattern also has someliabilities:
• Performance overhead• Depending on how an activeobject’s scheduler isimplemented, contextswitching, synchronization, &data movement overhead mayoccur when scheduling &executing active objectinvocations
• Complicated debugging• It is hard to debug programsthat use the Active Objectpattern due to the concurrency& non-determinism of thevarious active objectschedulers & the underlyingOS thread scheduler
Pros & Cons of the Active Object Pattern
49
Decoupling Suppliers & Consumers
Decouple suppliers (publishers) &consumers (subscribers) of events:• An Event Channel stores/forwards events• Publishers create events & store them in aqueue maintained by the Event Channel
• Consumers register with event queues,from which they retrieve events
• Events are used to transmit state changeinfo from publishers to consumers
• For event transmission push-models &pull-models are possible
Problem•Having each client call a specific serveris inefficient & non-scalable•A polling strategy leads toperformance bottlenecks
•Work lists could be spread acrossdifferent servers
•More than one client may beinterested in work list content
Context• In large-scale electronic medicalimaging systems, radiologists mayshare “work lists” of patient imagesto balance workloads effectively
Solution•Apply the Publisher/Subscriberpattern to decouple imagesuppliers from image consumers
Solution•Apply the Publisher/Subscriberpattern to decouple imagesuppliers from image consumers
50
Publisher/Subscriber Pattern Dynamics
•The Publisher/Subscriberpattern helps keep thestate of cooperatingcomponents synchronized
•To achieve this it enablesone-way propagation ofchanges: one publishernotifies any number ofsubscribers aboutchanges to its state
attachSubscriber
produce
pushEventevent
eventpushEvent
consume
detachSubscriber
: Event
: Subscriber: Event Channel: Publisher
Key design considerations for the Publisher/Subscriber pattern include:•Push vs. pull interaction models•Control vs. data event notification models•Multicast vs. unicast communication models•Persistence vs. transient queueing models
Key design considerations for the Publisher/Subscriber pattern include:•Push vs. pull interaction models•Control vs. data event notification models•Multicast vs. unicast communication models•Persistence vs. transient queueing models
51
Applying the Publisher/SubscriberPattern to Image Acquisition
•Radiologists can subscribe toan event channel in order toreceive notifications producedwhen modalities publish eventsindicating the arrival of a newimage
•This design enables a group ofdistributed radiologists tocollaborate effectively in anetworked environment
52
Pros & Cons of thePublisher/Subscriber Pattern
This pattern has two benefits:•Decouples consumers &producers of events• All an event channel knows is that ithas a list of consumers, eachconforming to the simple interface ofthe Subscriber class
• Thus, the coupling between thepublishers and subscribers is abstract& minimal
•n:m communication models aresupported• Unlike an ordinary request/responseinteraction, the notification that apublisher sends needn’t designate itsreceiver, which enables a broaderrange of communication topologies,including multicast & broadcast
There is also a liability:•Must be careful with potentialupdate cascades• Since subscribers have noknowledge of each other’s presence,applications can be blind to theultimate cost of publishing eventsthrough an event channel
• Thus, a seemingly innocuousoperation on the subject may causea cascade of updates to observers &their dependent objects
53
Locating & Creating Components Effectively
• An Abstract Home declares an interface foroperations that find and/or create abstractinstances of components
• Concrete Homes implements the abstractHome interface to find specific instances and/orcreate new ones
• Abstract Comp declares interface for a specifictype of component class
• Concrete Comp define instances• A Primary Key is associated with a component
Context• Our electronic medicalimaging system containsmany componentsdistributed in a network
Solution•Apply the Factory/Finder pattern to separate the management of componentlifecycles from their use by client applications
Solution•Apply the Factory/Finder pattern to separate the management of componentlifecycles from their use by client applications
AbstractComp
operation
ConcreteComp
operation
Primary Key
ConcreteHome
findcreate
AbstractHome
findcreate
Problem•How to create new components and/or findexisting ones• Simple solutions appropriate for stand-aloneapplications don’t scale
• “Obvious” solutions for distribution also don’t scale
54
Factory/Finder Pattern Dynamics
• Homes enable the creation &location of components, butwe still need a global namingservice to locate the homes
• Homes enable the creation &location of components, butwe still need a global namingservice to locate the homes
Applying the Factory/Finder Patternto Image Acquisition
AbstractComp
operation
ImageXferComp
operation
Primary Key
ImageXferHome
findcreate
AbstractHome
findcreate
1. Deploy
2. BindFactory
Image Xfer Interface
Image Xfer Interface
4. Intercept & delegate
Factory ProxyFactory Proxy6. Find Image
Factory/FinderFactory/Finder
NamingServiceNamingService
3. Find Factory
ImageDatabase
RadiologyClient
RadiologyClient
5.New
ConfigurationConfiguration
ContainerContainer
•We can apply the Factory/Finderpattern to create/locate imagetransfer components for imagesneeded by radiologists
• If a suitable component alreadyexists the component home willuse it, otherwise, it will create anew component
56
Pros & Cons of the Factory/Finder Pattern
This pattern has three benefits:•Separation of concerns
• Finding/creating individualcomponents is decoupled fromlocating the factories that find/createthese components
• Improved scalability• e.g., general-purpose directorymechanisms need not manage thecreation & location of large amounts offiner-grained components whoselifetimes may be short
•Customized capabilities• The location/creation mechanism canbe specialized to support keycapabilities that are unique for varioustypes of components
This pattern also has some liabilities:•Overhead due to indirection
• Clients must incur the overhead ofseveral round-trips to obtain theappropriate object reference
•Complexity & cost fordevelopment & deployment• There are more steps involved inobtaining object references, which cancomplicate client programming
57
Context•Component developers maynot know a priori where theircomponents will execute
•Thus, containers areintroduced to:• Shield clients & componentsfrom the details of theunderlying middleware,services, network & OS
• Manage the lifecycle ofcomponents & notifycomponents about lifecycleevents• e.g., activation, passivation, &
transaction progress• Provide components uniformaccess to infrastructureservices• e.g., transactions, security, &
persistence• Register & deploy components
DeclarativeProgramming
ServerComponent
TransactionSecurityResources...
ServerComponent
TransactionSecurityResources...
...
ImperativeProgramming
Container
Client Client
Extending Components Transparently
58
Extending Components Transparently (cont‘d)
Problem• Components should be able to specifydeclaratively in configuration files whichexecution environment they require
• Containers then should provide the rightexecution environment•e.g., by creating a new transaction ornew servant when required
• Framework represents the concreteframework to which we attach interceptors
• Concrete Interceptors implement theevent handler for the system-specificevents they have subscribed for
• Context contains information about theevent & allows modification of systembehavior after interceptor completion
• The Dispatcher allows applications toregister & remove interceptors with theframework & to delegate events tointerceptors
Solution•Apply the Interceptorarchitectural pattern to attachinterceptors to a frameworkthat can handle particularevents by invoking associatedinterceptors automatically
Solution•Apply the Interceptorarchitectural pattern to attachinterceptors to a frameworkthat can handle particularevents by invoking associatedinterceptors automatically
59
Interceptor Pattern Dynamics• Interceptor are a “meta-programming mechanism”
Applying the Interceptor Patternto Image Acquisition
• A container provides genericinterfaces to a component that itcan use to access containerfunctionality•e.g., transaction control,persistence, security,etc.
• A container intercepts all incomingrequests from clients•It reads the component’srequirements from a XMLconfiguration file & does somepre-processing before actuallydelegating the request to thecomponent
• A component provides eventinterfaces the container invokesautomatically when particularevents occur•e.g., activation or passivation
Container
Component
XMLconfig
61
Pros & Cons of the Interceptor PatternThis pattern has five benefits:•Extensibility & flexibility
• Interceptors allow an application to evolvewithout breaking existing APIs &components
•Separation of concerns• Interceptors decouple the “functional”path from the “meta” path
•Support for monitoring & control offrameworks• e.g., generic logging mechanisms can beused to unobtrusively track applicationbehavior
•Layer symmetry• Interceptors can perform transformationson a client-side whose inverse areperformed on the server-side
•Reusability• Interceptors can be reused for variousgeneral-purpose behaviors
This pattern also has liabilities:•Complex design issues
• Determining interceptor APIs &semantics is non-trivial
•Malicious or erroneousinterceptors• Mis-behaving interceptors canwreak havoc on applicationstability
•Potential interceptioncascades• Interceptors can result in infiniterecursion
62
Minimizing Resource Utilization
Problem• It may not feasible to have all imageserver implementations running allthe time since this ties up end-system resources unnecessarily
Solution• Apply the Activator pattern to spawn servers on-demand inorder to minimize end-system resource utilization
Solution• Apply the Activator pattern to spawn servers on-demand inorder to minimize end-system resource utilization
Context• Image servers are simply one ofmany services running throughoutan distributed electronic medicalimage system
• When incoming requests arrive, theActivator looks up whether a target object isalready active & if the object is not running itactivates the implementation
• The Activation Table stores associationsbetween services & their physical location
• The Client uses the Activator to get serviceaccess
• A Service implements a specific type offunctionality that it provides to clients
• A container can be used to activate & passivate a component•A component can be activated/passivated by itself, the container,after each method call, after each transaction, etc.
• A container can be used to activate & passivate a component•A component can be activated/passivated by itself, the container,after each method call, after each transaction, etc.
•The Activator pattern is available invarious COTS technologies:•UNIX Inetd “super server”•CORBA Implementation Repository
•We can use the Activator patternto launch image transfer serverson-demand
iiop://ringil:5500/poa_name/object_name
65
Pros & Cons of the Activator PatternThis pattern has three benefits:•Uniformity• By imposing a uniform activationinterface to spawn & control servers
•Modularity, testability, & reusability• Application modularity & reusability isimproved by decoupling serverimplementations from the manner inwhich the servers are activated
•More effective resource utilization• Servers can be spawned “on-demand,”thereby minimizing resource utilizationuntil clients actually require them
This pattern also has liabilities:•Lack of determinism & orderingdependencies• This pattern makes it hard todetermine or analyze the behaviorof an application until itscomponents are activated at run-time
•Reduced security or reliability• An application that uses theActivator pattern may be lesssecure or reliable than anequivalent statically-configuredapplication
• Increased run-time overhead &infrastructure complexity• By adding levels of abstraction &indirection when activating &executing components
66
Enhancing Server (Re)Configurability
This pattern allows an application to link & unlink itscomponent implementations at run-time so new &enhanced services can therefore be added withouthaving to modify, recompile, statically relink, or shutdown & restart a running application
•Certain factors are static, such asthe number of available CPUs &operating system support forasynchronous I/O
•Other factors are dynamic, suchas system workload
ContextThe implementation of certainimage server components dependson a variety of factors:
ProblemPrematurely committing to a particularimage server component configurationis inflexible and inefficient:•No single image server configuration isoptimal for all use cases
•Certain design decisions cannot bemade efficiently until run-time
Solution•Apply the Component Configurator designpattern to enhance server configurability
Solution•Apply the Component Configurator designpattern to enhance server configurability
<<contains>>
components*
ComponentConfigurator
ComponentRepository
ConcreteComponent A
ConcreteComponent B
Component init() fini() suspend() resume() info()
67
Component Configurator Pattern Dynamics
: ComponentConfigurator
init()
: ConcreteComponent A
: ConcreteComponent B
: ComponentRepository
insert()
insert()
init()
Concrete
run_component()
run_component()
fini()
remove()
remove()
fini()
Comp. A
ConcreteComp. B
ConcreteComp. A
ConcreteComp. B
1.Componentinitialization
2.Componentprocessing
3.Componenttermination
68
Applying the Component ConfiguratorPattern to Image Acquisition
<<contains>>
components*
ComponentConfigurator
ComponentRepository
LRUFile Cache
LFUFile Cache
Component init() fini() suspend() resume() info()
•For example, an image server can applythe Component Configurator pattern toconfigure various Cached VirtualFilesystem strategies•e.g., least-recently used (LRU) orleast-frequently used (LFU)
Image servers can use theComponent Configurator pattern todynamically optimize, control, &reconfigure the behavior of itscomponents at installation-time orduring run-time
Concrete components can bepackaged into a suitable unit ofconfiguration, such as adynamically linked library (DLL)
Concrete components can bepackaged into a suitable unit ofconfiguration, such as adynamically linked library (DLL)
Only the componentsthat are currently in useneed to be configuredinto an image server
Only the componentsthat are currently in useneed to be configuredinto an image server
69
Reconfiguring an Image ServerImage serverscan also bereconfigureddynamically tosupport newcomponents &new componentimplementations
IDLE
RUNNING
SUSPENDED
CONFIGUREinit()
RECONFIGUREinit()
fini()
fini()
resume()
suspend()
EXECUTErun_component()
SUSPEND
RESUME
TERMINATE
TERMINATE
Reconfiguration State Chart
LRU FileCache
ImageServer
# Configure an image server.dynamic File_Cache Component *
img_server.dll:make_File_Cache() "-t LRU"
INITIAL CONFIGURATION
AFTER RECONFIGURATION
LFU FileCache
ImageServer
# Reconfigure an image server.Remove File_Cachedynamic File_Cache Component *
img_server.dll:make_File_Cache() "-t LFU"
70
Pros and Cons of theComponent Configurator Pattern
This pattern offers four benefits:•Uniformity
• By imposing a uniform configuration &control interface to manage components
•Centralized administration• By grouping one or more components intoa single administrative unit that simplifiesdevelopment by centralizing commoncomponent initialization & terminationactivities
•Modularity, testability, & reusability• Application modularity & reusability isimproved by decoupling componentimplementations from the manner in whichthe components are configured intoprocesses
•Configuration dynamism & control• By enabling a component to bedynamically reconfigured withoutmodifying, recompiling, statically relinkingexisting code & without restarting thecomponent or other active componentswith which it is collocated
This pattern also incurs liabilities:•Lack of determinism & orderingdependencies• This pattern makes it hard todetermine or analyze the behavior ofan application until its components areconfigured at run-time
•Reduced security or reliability• An application that uses theComponent Configurator pattern maybe less secure or reliable than anequivalent statically-configuredapplication
• Increased run-time overhead &infrastructure complexity• By adding levels of abstraction &indirection when executingcomponents
•Overly narrow common interfaces• The initialization or termination of acomponent may be too complicated ortoo tightly coupled with its context tobe performed in a uniform manner
71
Tutorial Example 2:
High-performance Content Delivery Servers
•Support many content delivery serverdesign alternatives seamlessly• e.g., different concurrency & event models
•Design is guided by patterns to leveragetime-proven solutions
•Support many content delivery serverdesign alternatives seamlessly• e.g., different concurrency & event models
•Design is guided by patterns to leveragetime-proven solutions
Key Solution Characteristics• Implementation is based on ACEframework components to reduceeffort & amortize prior effort
•Open-source to control costs & toleverage technology advances
• Implementation is based on ACEframework components to reduceeffort & amortize prior effort
•Open-source to control costs & toleverage technology advances
Key SystemCharacteristics•Robust implementation
• e.g., stop malicious clients•Extensible to other protocols
request events, receivesHTTP GET requests, &coordinates JAWS’s eventdemultiplexing strategywith its concurrencystrategy.
• As events are processedthey are dispatched to theappropriate ProtocolHandler.
Protocol Handler• Performs parsing & protocol
processing of HTTP requestevents.
• JAWS Protocol Handler designallows multiple Web protocols, suchas HTTP/1.0, HTTP/1.1, & HTTP-NG, to be incorporated into a Webserver.
• To add a new protocol, developersjust write a new Protocol Handlercomponent & configure it into theJAWS framework.
Cached Virtual Filesystem• Improves Web server
performance by reducing theoverhead of file system accesseswhen processing HTTP GETrequests.
• Various caching strategies, such asleast-recently used (LRU) or least-frequently used (LFU), can beselected according to the actual oranticipated workload & configuredstatically or dynamically.
73
Applying Patterns to Resolve KeyJAWS Design Challenges
Patterns help resolve the following common design challenges:•Efficiently demuxing asynchronousoperations & completions
•Transparently parameterizingsynchronization into components
•Encapsulating low-level OS APIs•Decoupling event demultiplexing &connection management fromprotocol processing
•Scaling up performance via threading•Implementing a synchronized requestqueue
•Minimizing server threading overhead•Using asynchronous I/O effectively
•Encapsulating low-level OS APIs•Decoupling event demultiplexing &connection management fromprotocol processing
•Scaling up performance via threading•Implementing a synchronized requestqueue
•Minimizing server threading overhead•Using asynchronous I/O effectively
Double-checkedLocking
Optimization
Thread-specific Storage
74
Encapsulating Low-level OS APIsProblem• The diversity of hardware & operatingsystems makes it hard to build portable& robust Web server software byprogramming directly to low-leveloperating system APIs, which aretedious, error-prone, & non-portable
The Wrapper Facade patternencapsulates data & functionsprovided by existing non-OO APIswithin more concise, robust,portable, maintainable, & cohesiveOO class interfaces
: Application
method()
: WrapperFacade
: APIFunctionA
functionA()
: APIFunctionB
functionB()
Applicationcalls methods
callsAPI FunctionA()
callsAPI FunctionB()
calls API FunctionC()
void methodN(){functionA();
}
void method1(){functionA();
}functionB();
Wrapper Facade
data
method1()…methodN()Solution
• Apply the Wrapper Facade designpattern to avoid accessing low-leveloperating system APIs directly
Solution• Apply the Wrapper Facade designpattern to avoid accessing low-leveloperating system APIs directly
Context• A Web server must manage a variety ofOS services, including processes,threads, Socket connections, virtualmemory, & files. Most operating systemsprovide low-level APIs written in C toaccess these services
75
Other ACE wrapper facades used inJAWS encapsulate Sockets, process &thread management, memory-mappedfiles, explicit dynamic linking, & timeoperations
Other ACE wrapper facades used inJAWS encapsulate Sockets, process &thread management, memory-mappedfiles, explicit dynamic linking, & timeoperations
Applying the Wrapper Façade Pattern in JAWS
JAWS uses the wrapper facades defined by ACE to ensure its frameworkcomponents can run on many operating systems, including Windows, UNIX,& many real-time operating systems
For example, JAWS usesthe Thread_Mutexwrapper facade in ACEto provide a portableinterface to operatingsystem mutual exclusionmechanisms
Thread_Mutex
mutex
acquire()tryacquire()release()
void acquire(){
calls methods
callsmutex_lock()
callsmutex_trylock()
calls mutex_unlock()
void release(){mutex_unlock(mutex);
}mutex_lock(mutex);
}
JAWS
The Thread_Mutex wrapper in the diagramis implemented using the Solaris thread APIThe Thread_Mutex wrapper in the diagramis implemented using the Solaris thread API
www.cs.wustl.edu/~schmidt/ACE/
The ACE Thread_Mutex wrapper facade isalso available for other threading APIs, e.g.,pSoS, VxWorks, Win32 threads or POSIXPthreads
The ACE Thread_Mutex wrapper facade isalso available for other threading APIs, e.g.,pSoS, VxWorks, Win32 threads or POSIXPthreads
76
Pros and Cons of the Wrapper Façade PatternThis pattern provides three benefits:•Concise, cohesive, & robust higher-level object-oriented programminginterfaces• These interfaces reduce the tedium &increase the type-safety of developingapplications, which descreases certaintypes of programming errors
•Portability & maintainability• Wrapper facades can shield applicationdevelopers from non-portable aspects oflower-level APIs
•Modularity, reusability &configurability• This pattern creates cohesive & reusableclass components that can be ‘plugged’into other components in a wholesalefashion, using object-oriented languagefeatures like inheritance & parameterizedtypes
This pattern can incur liabilities:•Loss of functionality
• Whenever an abstraction is layeredon top of an existing abstraction it ispossible to lose functionality
•Performance degradation• This pattern can degradeperformance if several forwardingfunction calls are made per method
•Programming language &compiler limitations• It may be hard to define wrapperfacades for certain languages dueto a lack of language support orlimitations with compilers
77
Problem•Developers often coupleevent-demuxing &connection code withprotocol-handling code
Decoupling Event Demuxing & ConnectionManagement from Protocol Processing
Context
•Web servers can be accessedsimultaneously by multipleclients
Client
Client
Client
HTTP GETrequest
Connectrequest
HTTP GETrequest
Web Server
Socket Handles
•They must demux & processmultiple types of indicationevents arriving from clientsconcurrently
SolutionApply the Reactor architectural pattern & the Acceptor-Connector designpattern to separate the generic event-demultiplexing & connection-management code from the web server’s protocol code
SolutionApply the Reactor architectural pattern & the Acceptor-Connector designpattern to separate the generic event-demultiplexing & connection-management code from the web server’s protocol code
Event Dispatcher
Sockets
select()
•A common way to demux eventsin a server is to use select()
•Thus, changes to event-demuxing & connection codeaffects the server protocolcode directly & may yieldsubtle bugs• e.g., porting it to use TLI orWaitForMultipleObjects()
78
The Reactor PatternThe Reactor architecturalpattern allows event-drivenapplications to demultiplex& dispatch service requeststhat are delivered to anapplication from one ormore clients.
The Acceptor-Connector PatternThe Acceptor-Connector design pattern decouples the connection &initialization of cooperating peer services in a networked system from theprocessing performed by the peer services after being connected & initialized.
• If client is initializing manypeers that can be connectedin an arbitrary order
• If client is establishingconnections over highlatency links
• If client is asingle-threadedapplications
83
Applying the Reactor and Acceptor-Connector Patterns in JAWS
handle_event ()get_handle()
handle_event ()get_handle()
Handleowns
dispatches*
notifies**
handle set
Reactor
handle_events()register_handler()remove_handler()
Event Handler
handle_event ()get_handle()
HTTPAcceptor
HTTPHandlerSynchronous
Event Demuxer
select ()
<<uses>>
The Reactor architecturalpattern decouples:1.JAWS generic
synchronous eventdemultiplexing &dispatching logic from
2.The HTTP protocolprocessing it performsin response to events
1.The connection & initialization of peer client & server HTTP servicesfrom
2.The processing activities performed by these peer services oncethey are connected & initialized
The Acceptor-Connector design pattern can use a Reactor as itsDispatcher in order to help decouple:
84
Reactive Connection Management& Data Transfer in JAWS
85
Pros and Cons of the Reactor PatternThis pattern offers four benefits:•Separation of concerns
• This pattern decouples application-independent demuxing & dispatchingmechanisms from application-specific hookmethod functionality
•Modularity, reusability, &configurability• This pattern separates event-drivenapplication functionality into severalcomponents, which enables the configurationof event handler components that are looselyintegrated via a reactor
•Portability• By decoupling the reactor’s interface fromthe lower-level OS synchronous eventdemuxing functions used in itsimplementation, the Reactor patternimproves portability
•Coarse-grained concurrency control• This pattern serializes the invocation of eventhandlers at the level of event demuxing &dispatching within an application process orthread
This pattern can incur liabilities:•Restricted applicability
• This pattern can be appliedefficiently only if the OS supportssynchronous event demuxing onhandle sets
•Non-pre-emptive• In a single-threaded application,concrete event handlers thatborrow the thread of their reactorcan run to completion & prevent thereactor from dispatching otherevent handlers
•Complexity of debugging &testing• It is hard to debug applicationsstructured using this pattern due toits inverted flow of control, whichoscillates between the frameworkinfrastructure & the method call-backs on application-specific eventhandlers
86
Pros and Cons of the Acceptor-Connector Pattern
This pattern provides three benefits:•Reusability, portability, & extensibility
• This pattern decouples mechanisms forconnecting & initializing service handlers fromthe service processing performed after servicehandlers are connected & initialized
•Robustness• This pattern strongly decouples the servicehandler from the acceptor, which ensures that apassive-mode transport endpoint can’t be usedto read or write data accidentally
•Efficiency• This pattern can establish connections activelywith many hosts asynchronously & efficientlyover long-latency wide area networks
• Asynchrony is important in this situationbecause a large networked system may havehundreds or thousands of host that must beconnected
This pattern also has liabilities:•Additional indirection
• The Acceptor-Connector patterncan incur additional indirectioncompared to using the underlyingnetwork programming interfacesdirectly
•Additional complexity• The Acceptor-Connector patternmay add unnecessary complexityfor simple client applications thatconnect with only one server &perform one service using asingle network programminginterface
87
Scaling Up Performance via Threading
Solution•Apply the Half-Sync/Half-Asyncarchitectural pattern to scale upserver performance by processingdifferent HTTP requestsconcurrently in multiple threads
Solution•Apply the Half-Sync/Half-Asyncarchitectural pattern to scale upserver performance by processingdifferent HTTP requestsconcurrently in multiple threads
Context•HTTP runs over TCP, which usesflow control to ensure that sendersdo not produce data more rapidlythan slow receivers or congestednetworks can buffer and process
•Since achieving efficient end-to-endquality of service (QoS) is importantto handle heavy Web traffic loads, aWeb server must scale upefficiently as its number of clientsincreases
Problem•Processing all HTTP GET requestsreactively within a single-threadedprocess does not scale up, becauseeach server CPU time-slice spendsmuch of its time blocked waiting for I/Ooperations to complete
•Similarly, to improve QoS for all itsconnected clients, an entire Web serverprocess must not block while waiting forconnection flow control to abate so itcan finish sending a file to a client
This solution yields two benefits:1. Threads can be mapped to separate CPUs
to scale up server performance via multi-processing
2. Each thread blocks independently, whichprevents a flow-controlled connection fromdegrading the QoS other clients receive
• This pattern defines two serviceprocessing layers—one async & onesync—along with a queueing layerthat allows services to exchangemessages between the two layers
• The pattern allows sync services,such as HTTP protocol processing,to run concurrently, relative both toeach other & to async services, suchas event demultiplexing
: External EventSource
: Async Service : Queue
notification
read()
enqueue()
message
: Sync Service
work()
message
read()
message
work()
notification
89
Applying the Half-Sync/Half-AsyncPattern in JAWS
<<get>><<get>>
<<get>>
<<put>>
<<ready to read>>
SynchronousService Layer
AsynchronousService Layer
QueueingLayer
Worker Thread 1 Worker Thread 3
ReactorSocket
Event Sources
Request Queue
HTTP AcceptorHTTP Handlers,
Worker Thread 2
•JAWS uses the Half-Sync/Half-Asyncpattern to processHTTP GET requestssynchronously frommultiple clients, butconcurrently inseparate threads
•JAWS uses the Half-Sync/Half-Asyncpattern to processHTTP GET requestssynchronously frommultiple clients, butconcurrently inseparate threads
•The worker threadthat removes therequestsynchronouslyperforms HTTPprotocol processing &then transfers the fileback to the client
•The worker threadthat removes therequestsynchronouslyperforms HTTPprotocol processing &then transfers the fileback to the client
• If flow control occurson its client connectionthis thread can blockwithout degrading theQoS experienced byclients serviced byother worker threads inthe pool
• If flow control occurson its client connectionthis thread can blockwithout degrading theQoS experienced byclients serviced byother worker threads inthe pool
90
Pros & Cons of theHalf-Sync/Half-Async Pattern
This pattern has three benefits:•Simplification & performance
• The programming of higher-levelsynchronous processing services aresimplified without degrading theperformance of lower-level systemservices
•Separation of concerns• Synchronization policies in eachlayer are decoupled so that eachlayer need not use the sameconcurrency control strategies
•Centralization of inter-layercommunication• Inter-layer communication iscentralized at a single access point,because all interaction is mediatedby the queueing layer
This pattern also incurs liabilities:•A boundary-crossing penalty maybe incurred• This overhead arises from contextswitching, synchronization, & datacopying overhead when data istransferred between the sync & asyncservice layers via the queueing layer
•Higher-level application servicesmay not benefit from the efficiencyof async I/O• Depending on the design of operatingsystem or application frameworkinterfaces, it may not be possible forhigher-level services to use low-levelasync I/O devices effectively
•Complexity of debugging & testing• Applications written with this pattern canbe hard to debug due its concurrentexecution
91
Context•The Half-Sync/Half-Asyncpattern contains a queue
•The JAWS Reactor thread is a‘producer’ that inserts HTTPGET requests into the queue
•Worker pool threads are‘consumers’ that remove &process queued requests
<<get>><<get>>
<<get>>
<<put>>
Worker Thread 1
Worker Thread 3
Reactor
Request Queue
HTTP AcceptorHTTP Handlers,
Worker Thread 2
Implementing a Synchronized Request Queue
Problem•A naive implementation of a request queue will incur raceconditions or ‘busy waiting’ when multiple threads insert & removerequests•e.g., multiple concurrent producer & consumer threads cancorrupt the queue’s internal state if it is not synchronized properly
•Similarly, these threads will ‘busy wait’ when the queue is emptyor full, which wastes CPU cycles unnecessarily
92
The Monitor Object Pattern
•This pattern synchronizesconcurrent method executionto ensure that only onemethod at a time runs withinan object
• It also allows an object’smethods to cooperativelyschedule their executionsequences
2..*
usesuses *
Monitor Object
sync_method1()sync_methodN()
Monitor Lock
acquire()release()
Client
Monitor Condition
wait()notify()notify_all()
Solution•Apply the Monitor Object design pattern to synchronize the queue efficiently& conveniently
Solution•Apply the Monitor Object design pattern to synchronize the queue efficiently& conveniently
• It’s instructive to compare Monitor Object pattern solutions with Active Objectpattern solutions•The key tradeoff is efficiency vs. flexibility
93
Monitor Object Pattern Dynamics: Monitor
Object: Monitor
Lock: MonitorCondition
sync_method1()
wait()
dowork()
: ClientThread1
: ClientThread2
acquire()
dowork()
acquire()sync_method2()
release()
notify()
dowork()
release()
the OS thread schedulerautomatically suspendsthe client thread
the OS threadschedulerautomatically resumes the clientthread and thesynchronizedmethod
the OS thread scheduleratomically reacquiresthe monitor lock
the OS thread scheduleratomically releasesthe monitor lock
1. Synchronizedmethodinvocation &serialization
2. Synchronizedmethod threadsuspension
3. Monitorconditionnotification
4. Synchronizedmethod threadresumption
94
Applying the Monitor Object Pattern in JAWS
The JAWS synchronizedrequest queueimplements the queue’snot-empty and not-fullmonitor conditions via apair of ACE wrapperfacades for POSIX-stylecondition variables
usesuses 2
Request Queue
put()get()
Thread_Mutex
acquire()release()
HTTPHandler
Thread Condition
wait()notify()notify_all()
WorkerThread
<<put>> <<get>>
•When a worker thread attempts to dequeue an HTTP GET requestfrom an empty queue, the request queue’s get() methodatomically releases the monitor lock & the worker thread suspendsitself on the not-empty monitor condition
•The thread remains suspended until the queue is no longer empty,which happens when an HTTP_Handler running in the Reactorthread inserts a request into the queue
•When a worker thread attempts to dequeue an HTTP GET requestfrom an empty queue, the request queue’s get() methodatomically releases the monitor lock & the worker thread suspendsitself on the not-empty monitor condition
•The thread remains suspended until the queue is no longer empty,which happens when an HTTP_Handler running in the Reactorthread inserts a request into the queue
95
Pros & Cons of the Monitor Object Pattern
This pattern provides two benefits:•Simplification of concurrencycontrol• The Monitor Object pattern presentsa concise programming model forsharing an object amongcooperating threads where objectsynchronization corresponds tomethod invocations
•Simplification of schedulingmethod execution• Synchronized methods use theirmonitor conditions to determine thecircumstances under which theyshould suspend or resume theirexecution & that of collaboratingmonitor objects
This pattern can also incur liabilities:•The use of a single monitor lock canlimit scalability due to increasedcontention when multiple threadsserialize on a monitor object
•Complicated extensibilitysemantics• These result from the coupling betweena monitor object’s functionality & itssynchronization mechanisms
•It is also hard to inherit from a monitorobject transparently, due to theinheritance anomaly problem
•Nested monitor lockout• This problem is similar to the precedingliability & can occur when a monitorobject is nested within another monitorobject
96
accept()
Minimizing Server Threading Overhead
•When a connection request arrives, theoperating system’s transport layer creates a newconnected transport endpoint, encapsulates thisnew endpoint with a data-mode socket handle &passes the handle as the return value fromaccept()
Context•Socket implementations in certain multi-threadedoperating systems provide a concurrent accept()optimization to accept client connection requests &improve the performance of Web servers thatimplement the HTTP 1.0 protocol as follows:
passive-modesocket handle
accept()
accept()
accept()
•The OS allows a pool of threads in a Web serverto call accept() on the same passive-modesocket handle
•The OS then schedules one of the threads inthe pool to receive this data-mode handle,which it uses to communicate with itsconnected client
accept()
accept()
97
Drawbacks with the Half-Sync/Half-Async Architecture
Solution•Apply the Leader/Followersarchitectural pattern tominimize server threadingoverhead
Solution•Apply the Leader/Followersarchitectural pattern tominimize server threadingoverhead
Problem•Although Half-Sync/Half-Asyncthreading model is morescalable than the purely reactivemodel, it is not necessarily themost efficient design
•CPU cache updates
<<get>><<get>>
<<get>>
<<put>>
Worker Thread 1
Worker Thread 3
Reactor
Request Queue
HTTP AcceptorHTTP Handlers,
Worker Thread 2
•e.g., passing a requestbetween the Reactor thread& a worker thread incurs:
•This overhead makes JAWS’ latencyunnecessarily high, particularly onoperating systems that support theconcurrent accept() optimization
•Dynamic memory (de)allocation,
•A context switch, &•Synchronization operations,
98
The Leader/Followers Pattern
This pattern eliminates the need for—&the overhead of—a separate Reactorthread & synchronized request queueused in the Half-Sync/Half-Async pattern
This pattern eliminates the need for—&the overhead of—a separate Reactorthread & synchronized request queueused in the Half-Sync/Half-Async pattern
The Leader/Followers architecturalpattern provides an efficientconcurrency model where multiplethreads take turns sharing eventsources to detect, demux, dispatch, &process service requests that occur onthe event sources
optimization then the Leader/Followerspattern can be implemented by the OS
2.Otherwise, this pattern can beimplemented as a reusable framework •The Half-Sync/Half-Async
design can reorder &prioritize client requestsmore flexibly, because ithas a synchronized requestqueue implemented usingthe Monitor Object pattern
• It may be more scalable,because it queues requestsin Web server virtualmemory, rather than the OSkernel
Although Leader/Followers threadpool design is highly efficient theHalf-Sync/Half-Async design may bemore appropriate for certain types ofservers, e.g.:
101
Pros and Cons of theLeader/Followers Pattern
This pattern provides two benefits:•Performance enhancements
• This can improve performance as follows:• It enhances CPU cache affinity andeliminates the need for dynamic memoryallocation & data buffer sharing betweenthreads
• It minimizes locking overhead by notexchanging data between threads, therebyreducing thread synchronization
• It can minimize priority inversion becauseno extra queueing is introduced in theserver
• It doesn’t require a context switch tohandle each event, reducing dispatchinglatency
•Programming simplicity• The Leader/Follower pattern simplifies theprogramming of concurrency models wheremultiple threads can receive requests,process responses, & demultiplexconnections using a shared handle set
This pattern also incur liabilities:
• Implementation complexity• The advanced variants of theLeader/ Followers pattern arehard to implement
•Lack of flexibility• In the Leader/ Followersmodel it is hard to discard orreorder events because thereis no explicit queue
•Network I/O bottlenecks• The Leader/Followers patternserializes processing byallowing only a single threadat a time to wait on the handleset, which could become abottleneck because only onethread at a time candemultiplex I/O events
102
Using Asynchronous I/O EffectivelyContext•Synchronous multi-threading may not be themost scalable way to implement a Web serveron OS platforms that support async I/O moreefficiently than synchronous multi-threading
passive-modesocket handle
AcceptEx()AcceptEx()AcceptEx()
I/O CompletionPort
GetQueuedCompletionStatus()
GetQueuedCompletionStatus()
GetQueuedCompletionStatus()
•When these async operations complete, WinNT1.Delivers the associated completion events
containing their results to the Web server2.Processes these events & performs the appropriate
actions before returning to its event loop
•For example, highly-efficient Web servers canbe implemented on Windows NT by invokingasync Win32 operations that perform thefollowing activities:•Processing indication events, such as TCPCONNECT and HTTP GET requests, viaAcceptEx() & ReadFile(), respectively
•Transmitting requested files to clientsasynchronously via WriteFile() orTransmitFile()
103
The Proactor PatternProblem•Developing software that achievesthe potential efficiency & scalabilityof async I/O is hard due to theseparation in time & space of asyncoperation invocations & theirsubsequent completion events
Solution•Apply the Proactor architectural patternto make efficient use of async I/O
Solution•Apply the Proactor architectural patternto make efficient use of async I/O
Handle
<<executes>>
*
<<uses>>is associated with
<<enqueues>>
<<dequeues>>
<<uses>> <<uses>>Initiator
<<demultiplexes & dispatches>>
<<invokes>>
Event QueueCompletion
AsynchronousOperation Processor
execute_async_op()
AsynchronousOperation
async_op()
AsynchronousEvent Demuxer
get_completion_event()
Proactor
handle_events()
CompletionHandler
handle_event()
ConcreteCompletion
Handler
This pattern allows event-drivenapplications to efficiently demultiplex &dispatch service requests triggered by thecompletion of async operations, therebyachieving the performance benefits of
concurrencywithout incurringits many liabilities
104
Dynamics in the Proactor Pattern
Result
CompletionHandler
Completion
: AsynchronousOperation
: Proactor CompletionHandler
exec_async_
handle_
Result
service()
: AsynchronousOperationProcessor
: Initiator
async_operation()
Result
handle_events()
event
event
Ev. Queue
operation ()
: Completion
Event Queue
Result
event()
1. Initiateoperation
2. Processoperation
3. Run eventloop
4. Generate& queuecompletionevent
5. Dequeuecompletionevent &performcompletionprocessing Note similarities & differences with the Reactor pattern, e.g.:
•Both process events via callbacks•However, it’s generally easier to multi-thread a proactor
Note similarities & differences with the Reactor pattern, e.g.:•Both process events via callbacks•However, it’s generally easier to multi-thread a proactor
105
Applying the Proactor Pattern in JAWSThe Proactor patternstructures the JAWSconcurrent server toreceive & processrequests from multipleclients asynchronously
Handle
<<executes>>
*
<<uses>>is associated with
<<enqueues>>
<<dequeues>>
<<uses>> <<uses>>Web Server
<<demultiplexes & dispatches>>
<<invokes>>
I/O Completion Port
Windows NTOperating System
execute_async_op()
AsynchronousOperation
AcceptEx()ReadFile()WriteFile()
AsynchronousEvent Demuxer
GetQueuedCompletionStatus()
Proactor
handle_events()
CompletionHandler
handle_event()
HTTPAcceptor
HTTPHandler
JAWS HTTP components are split into two parts:1. Operations that execute asynchronously
•e.g., to accept connections & receive client HTTP GETrequests
2. The corresponding completion handlers that process theasync operation results•e.g., to transmit a file back to a client after an asyncconnection operation completes
106
Proactive Connection Management& Data Transfer in JAWS
107
Pros and Cons of the Proactor PatternThis pattern offers five benefits:•Separation of concerns
• Decouples application-independent asyncmechanisms from application-specificfunctionality
•Portability• Improves application portability by allowing itsinterfaces to be reused independently of the OSevent demuxing calls
•Decoupling of threading fromconcurrency• The async operation processor executes long-duration operations on behalf of initiators soapplications can spawn fewer threads
•Performance• Avoids context switching costs by activatingonly those logical threads of control that haveevents to process
•Simplification of applicationsynchronization• If concrete completion handlers spawn nothreads, application logic can be written withlittle or no concern for synchronization issues
This pattern incurs some liabilities:•Restricted applicability
• This pattern can be applied mostefficiently if the OS supportsasynchronous operationsnatively
•Complexity of programming,debugging, & testing• It is hard to program applications& higher-level system servicesusing asynchrony mechanisms,due to the separation in time &space between operationinvocation and completion
•Scheduling, controlling, &canceling asynchronouslyrunning operations• Initiators may be unable tocontrol the scheduling order inwhich asynchronous operationsare executed by anasynchronous operationprocessor
Context• In a proactive Webserver async I/Ooperations will yieldI/O completion eventresponses that mustbe processedefficiently
Problem•As little overhead as possible should be incurred todetermine how the completion handler will demux &process completion events after async operationsfinish executing
•When a response arrives, the application shouldspend as little time as possible demultiplexing thecompletion event to the handler that will process theasync operation’s response
•Together with each async operationthat a client initiator invokes on aservice, transmit information thatidentifies how the initiator shouldprocess the service’s response
Solution•Apply the Asynchronous Completion Token design pattern to demux &process the responses of asynchronous operations efficiently
Solution•Apply the Asynchronous Completion Token design pattern to demux &process the responses of asynchronous operations efficiently
•Return this information to the initiatorwhen the operation finishes, so that itcan be used to demux the responseefficiently, allowing the initiator toprocess it accordingly
109
The Asynchronous Completion Token Pattern
Structure and Participants
Dynamic Interactions
handle_event()
110
Applying the AsynchronousCompletion Token Pattern in JAWS
DetailedInteractions
(HTTP_Acceptoris both initiator &
completion handler)
111
Pros and Cons of the AsynchronousCompletion Token Pattern
This pattern has some liabilities:•Memory leaks
• Memory leaks can result if initiators useACTs as pointers to dynamicallyallocated memory & services fail toreturn the ACTs, for example if theservice crashes
•Authentication• When an ACT is returned to an initiatoron completion of an asynchronousevent, the initiator may need toauthenticate the ACT before using it
•Application re-mapping• If ACTs are used as direct pointers tomemory, errors can occur if part of theapplication is re-mapped in virtualmemory
This pattern has four benefits:•Simplified initiator data structures
• Initiators need not maintain complexdata structures to associate serviceresponses with completion handlers
•Efficient state acquisition• ACTs are time efficient because theyneed not require complex parsing ofdata returned with the service response
•Space efficiency• ACTs can consume minimal data spaceyet can still provide applications withsufficient information to associate largeamounts of state to processasynchronous operation completionactions
•Flexibility• User-defined ACTs are not forced toinherit from an interface to use theservice’s ACTs
112
Transparently ParameterizingSynchronization into Components
Context•The various concurrencypatterns described earlier impactcomponent synchronizationstrategies in various ways
•e.g.,ranging from no locks toreaders/writer locks
•In general, components must runefficiently in a variety ofconcurrency models
Problem•It should be possible to customize JAWScomponent synchronization mechanismsaccording to the requirements of particularapplication use cases & configurations
•Maintaining multiple versions ofcomponents manually is not scalable
Solution•Apply the Strategized Locking design pattern to parameterize JAWScomponent synchronization strategies by making them ‘pluggable’ types
Solution•Apply the Strategized Locking design pattern to parameterize JAWScomponent synchronization strategies by making them ‘pluggable’ types
•Each type objectifies aparticular synchronizationstrategy, such as a mutex,readers/writer lock,semaphore, or ‘null’ lock
•Instances of these pluggable types can bedefined as objects contained within acomponent, which then uses these objects tosynchronize its method implementationsefficiently
Content_Cache;•Multi-threaded file cache using a thread mutex.typedef File_Cache<Thread_Mutex>
Content_Cache;•Multi-threaded file cache using a readers/writerlock.typedef File_Cache<RW_Lock>
Content_Cache;
Note that the variouslocks need not inheritfrom a common baseclass or use virtual
methods!
Note that the variouslocks need not inheritfrom a common baseclass or use virtual
methods!
115
Pros and Cons of theStrategized Locking Pattern
This pattern provides three benefits:•Enhanced flexibility & customization
• It is straightforward to configure &customize a component for certainconcurrency models because thesynchronization aspects of components arestrategized
•Decreased maintenance effort forcomponents• It is straightforward to add enhancements &bug fixes to a component because there isonly one implementation, rather than aseparate implementation for eachconcurrency model
• Improved reuse• Components implemented using this patternare more reusable, because their lockingstrategies can be configured orthogonally totheir behavior
This pattern also incurs liabilities:•Obtrusive locking
• If templates are used toparameterize locking aspects thiswill expose the locking strategies toapplication code
•Over-engineering• Externalizing a locking mechanismby placing it in a component’sinterface may actually provide toomuch flexibility in certain situations•e.g., inexperienced developersmay try to parameterize acomponent with the wrong typeof lock, resulting in impropercompile- or run-time behavior
116
Ensuring Locks are Released ProperlyContext•Concurrentapplications,such as JAWS,contain sharedresources thatare manipulatedby multiplethreadsconcurrently
Problem•Code that shouldn’t execute concurrently must beprotected by some type of lock that is acquired & releasedwhen control enters & leaves a critical section, respectively
•If programmers must acquire & release locks explicitly, it ishard to ensure that the locks are released in all pathsthrough the code
Solution•In C++, apply the Scoped Lockingidiom to define a guard class whoseconstructor automatically acquires alock when control enters a scope &whose destructor automaticallyreleases the lock when control leavesthe scope
Solution•In C++, apply the Scoped Lockingidiom to define a guard class whoseconstructor automatically acquires alock when control enters a scope &whose destructor automaticallyreleases the lock when control leavesthe scope
// A method.const void *lookup (const string &path) const { lock_.acquire (); // The <lookup> method // implementation may return // prematurely… lock_.release ();}
•e.g., in C++ control can leave a scope due to a return,break, continue, or goto statement, as well as from anunhandled exception being propagated out of the scope
117
Applying the Scoped LockingIdiom in JAWS
template <class LOCK>class Guard {public: // Store a pointer to the lock and acquire the lock. Guard (LOCK &lock) : lock_ (&lock) { lock_->acquire (); }
// Release the lock when the guard goes out of scope, ~Guard () { lock_->release (); }private: // Pointer to the lock we’re managing. LOCK *lock_;};
Generic Guard Wrapper Facade
template <class LOCK>class File_Cache {public: // A method. const void *lookup (const string &path) const { // Use Scoped Locking idiom to acquire // & release the <lock_>automatically. Guard<LOCK> guard (*lock); // Implement the <lookup> method. // lock_ released automatically… }
Applying the GuardInstances of the guardclass can be allocatedon the run-time stack toacquire & release locksin method or blockscopes that definecritical sections
Instances of the guardclass can be allocatedon the run-time stack toacquire & release locksin method or blockscopes that definecritical sections
118
Pros and Cons of theScoped Locking Idiom
This idiom has one benefit:• Increased robustness
• This idiom increases therobustness of concurrentapplications by eliminatingcommon programming errorsrelated to synchronization &multi-threading
• By applying the ScopedLocking idiom, locks areacquired & releasedautomatically when controlenters and leaves criticalsections defined by C++method & block scopes
This idiom also has liabilities:•Potential for deadlock when usedrecursively• If a method that uses the Scoped Locking idiomcalls itself recursively, ‘self-deadlock’ will occur ifthe lock is not a ‘recursive’ mutex
•Limitations with language-specificsemantics• The Scoped Locking idiom is based on a C++language feature & therefore will not be integratedwith operating system-specific system calls• Thus, locks may not be released automaticallywhen threads or processes abort or exit inside aguarded critical section
• Likewise, they will not be released properly ifthe standard C longjmp() function is calledbecause this function does not call thedestructors of C++ objects as the run-time stackunwinds
119
Minimizing Unnecessary Locking
Context•Components in multi-threadedapplications that contain intra-component method calls
•Components that have applied theStrategized Locking pattern
Problem•Thread-safe components should bedesigned to avoid unnecessarylocking
•Thread-safe components should bedesigned to avoid “self-deadlock”
Solution•Apply the Thread-safe Interface design pattern to minimize locking overhead& ensure that intra-component method calls do not incur ‘self-deadlock’ bytrying to reacquire a lock that is held by the component already
Solution•Apply the Thread-safe Interface design pattern to minimize locking overhead& ensure that intra-component method calls do not incur ‘self-deadlock’ bytrying to reacquire a lock that is held by the component already
• Interface methods check•All interface methods, such as C++public methods, should onlyacquire/release component lock(s),thereby performing synchronizationchecks at the ‘border’ of the component.
This pattern structures all components that process intra-component methodinvocations according two design conventions:
• Implementation methods trust• Implementation methods, such asC++ private and protectedmethods, should only performwork when called by interfacemethods.
120
Motivating the Need for theThread-safe Interface Pattern
template <class LOCK>class File_Cache {public: // Return a pointer to the memory-mapped file associated with // <path> name, adding it to the cache if it doesn’t exist. const void *lookup (const string &path) const { // Use the Scoped Locking idiom to acquire // & release the <lock_> automatically. Guard<LOCK> guard (lock_); const void *file_pointer = check_cache (path); if (file_pointer == 0) { // Insert the <path> name into the cache. // Note the intra-class <insert> call. insert (path); file_pointer = check_cache (path); } return file_pointer; } // Add <path> name to the cache. void insert (const string &path) { // Use the Scoped Locking idiom to acquire // and release the <lock_> automatically. Guard<LOCK> guard (lock_); // ... insert <path> into the cache... }private: mutable LOCK lock_; const void *check_cache (const string &) const; // ... other private methods and data omitted...};
Since File_Cacheis a template wedon’t know the
type of lock usedto parameterize it.
121
Applying the Thread-safe InterfacePattern in JAWS
template <class LOCK>class File_Cache {public: // Return a pointer to the memory-mapped file associated with // <path> name, adding it to the cache if it doesn’t exist. const void *lookup (const string &path) const { // Use the Scoped Locking idiom to acquire // and release the <lock_> automatically. Guard<LOCK> guard (lock_); return lookup_i (path); }private: mutable LOCK lock_; // The strategized locking object.
// This implementation method doesn’t acquire or release // <lock_> and does its work without calling interface methods. const void *lookup_i (const string &path) const { const void *file_pointer = check_cache_i (path); if (file_pointer == 0) {
// If <path> name isn’t in the cache then// insert it and look it up again.insert_i (path);file_pointer = check_cache_i (path);// The calls to implementation methods <insert_i> and// <check_cache_i> assume that the lock is held & do work.
}return file_pointer;
Note fewer constraintson the type of LOCK…
122
Pros and Cons of the Thread-safeInterface Pattern
This pattern has some liabilities:•Additional indirection and extra methods
• Each interface method requires at least oneimplementation method, which increases thefootprint of the component & may also add anextra level of method-call indirection for eachinvocation
•Potential for misuse• OO languages, such as C++ and Java, supportclass-level rather than object-level accesscontrol
• As a result, an object can bypass the publicinterface to call a private method on anotherobject of the same class, thus bypassing thatobject’s lock
•Potential overhead• This pattern prevents multiple components fromsharing the same lock & prevents locking at afiner granularity than the component, which canincrease lock contention
This pattern has three benefits:• Increased robustness
• This pattern ensures that self-deadlock does not occur due tointra-component method calls
•Enhanced performance• This pattern ensures that locksare not acquired or releasedunnecessarily
•Simplification of software• Separating the locking andfunctionality concerns can helpto simplify both aspects
123
Synchronizing Singletons Correctly
Context•JAWS uses various singletons to implement components where only oneinstance is required•e.g., the ACE Reactor, the request queue, etc.
Problem•Singletons can be problematic in multi-threaded programs
class Singleton {public: static Singleton *instance () { if (instance_ == 0) { // Enter critical section. instance_ = new Singleton; // Leave critical section. } return instance_; } void method_1 (); // Other methods omitted.private: static Singleton *instance_; // Initialized to 0 by linker.};
Either too little locking…class Singleton {public: static Singleton *instance () { Guard<Thread_Mutex> g (lock_); if (instance_ == 0) { // Enter critical section. instance_ = new Singleton; // Leave critical section. } return instance_; }private: static Singleton *instance_; // Initialized to 0 by linker. static Thread_Mutex lock_;};
… or too much
124
The Double-checked LockingOptimization Pattern
Solution•Apply the Double-Checked Locking Optimization design pattern to reducecontention & synchronization overhead whenever critical sections of codemust acquire locks in a thread-safe manner just once during programexecution
Solution•Apply the Double-Checked Locking Optimization design pattern to reducecontention & synchronization overhead whenever critical sections of codemust acquire locks in a thread-safe manner just once during programexecution
// Perform first-check to// evaluate ‘hint’.if (first_time_in is FALSE){ acquire the mutex // Perform double-check to // avoid race condition. if (first_time_in is FALSE) { execute the critical section set first_time_in to TRUE } release the mutex}
class Singleton {public: static Singleton *instance () { // First check if (instance_ == 0) { Guard<Thread_Mutex> g(lock_); // Double check. if (instance_ == 0) instance_ = new Singleton; } return instance_; }private: static Singleton *instance_; static Thread_Mutex lock_;};
125
Applying the Double-Checked LockingOptimization Pattern in ACE
template <class TYPE>class ACE_Singleton {public: static TYPE *instance () { // First check if (instance_ == 0) { // Scoped Locking acquires and release lock. Guard<Thread_Mutex> guard (lock_); // Double check instance_. if (instance_ == 0) instance_ = new TYPE; } return instance_; }private: static TYPE *instance_; static Thread_Mutex lock_;};
ACE defines a“singleton adapter”template to automatethe double-checkedlocking optimization
Pros and Cons of the Double-CheckedLocking Optimization Pattern
This pattern has two benefits:•Minimized locking overhead
• By performing two first-time-inflag checks, this patternminimizes overhead for thecommon case
• After the flag is set the firstcheck ensures that subsequentaccesses require no furtherlocking
•Prevents race conditions• The second check of the first-time-in flag ensures that thecritical section is executed justonce
This pattern has some liabilities:•Non-atomic pointer or integralassignment semantics• If an instance_ pointer is used as the flag ina singleton implementation, all bits of thesingleton instance_ pointer must be read &written atomically in a single operation
• If the write to memory after the call to new isnot atomic, other threads may try to read aninvalid pointer
•Multi-processor cache coherency• Certain multi-processor platforms, such as theCOMPAQ Alpha & Intel Itanium, performaggressive memory caching optimizations inwhich read & write operations can execute ‘outof order’ across multiple CPU caches, suchthat the CPU cache lines will not be flushedproperly if shared data is accessed withoutlocks held
127
Logging Access Statistics Efficiently
Context•Web servers often needto log certain information•e.g., count number oftimes web pages areaccessed
Problem•Having a central logging object in a multi-threaded server process can become abottleneck•e.g., due to synchronization required toserialize access by multiple threads
Solution•Apply the Thread-Specific Storagepattern to allow multiple threads touse one ‘logically global’ accesspoint to retrieve an object that islocal to a thread, without incurringlocking overhead on each objectaccess
Solution•Apply the Thread-Specific Storagepattern to allow multiple threads touse one ‘logically global’ accesspoint to retrieve an object that islocal to a thread, without incurringlocking overhead on each objectaccess
Application <<uses>>callsThread
n m
n x mmaintains
m Thread-SpecificObject Setget(key)set(key, object)
Thread-SpecificObject Proxy
keymethod1()…methodN()
Thread-SpecificObjectmethod1()…methodN()
Key Factory
create_key()
128
key 1
key n
thread m
Thread-SpecificObject
Thread-SpecificObject Proxy
Thread-SpecificObject Set
accesses
manages
The application thread identifier, thread-specificobject set, & proxy cooperate to obtain thecorrect thread-specific object
[k,t]
thread 1
Thread-Specific Storage Pattern Dynamics
: Thread-SpecificObject Proxy
method()
: ApplicationThread
: Thread-SpecificObject Set
: Thread-SpecificObject
key
set()
create_key()
: KeyFactory
keyTSObject
129
class Error_Logger {public: int last_error (); void log (const char *format, ...);};
Applying the Thread-SpecificStorage Pattern to JAWS
• It’s possible to implement this patternso that no locking is needed toaccess thread-specific data
•Ease of use• When encapsulated with wrapperfacades, thread-specific storage iseasy for application developers touse
•Reusability• By combining this pattern with theWrapper Façade pattern it’s possibleto shield developers from non-portable OS platform characteristics
•Portability• It’s possible to implement portablethread-specific storage mechanismson most multi-threaded operatingsystems
This pattern also has liabilities:• It encourages use of thread-specific global objects• Many applications do not requiremultiple threads to access thread-specific data via a common access point
• In this case, data should be stored sothat only the thread owning the data canaccess it
• It obscures the structure of thesystem• The use of thread-specific storagepotentially makes an application harderto understand, by obscuring therelationships between its components
• It restricts implementation options• Not all languages supportparameterized types or smart pointers,which are useful for simplifying theaccess to thread-specific data
131
UML models ofa softwarearchitecturecan illustratehow a systemis designed, butnot why thesystem isdesigned in aparticular way
Tutorial Example 3:Applying Patterns to Real-time CORBA
Patterns are used throughout The ACE ORB (TAO) Real-time CORBA implementation to codify expert knowledge &to generate the ORB’s software architecture by capturingrecurring structures & dynamics & resolving commondesign forces
Patterns are used throughout The ACE ORB (TAO) Real-time CORBA implementation to codify expert knowledge &to generate the ORB’s software architecture by capturingrecurring structures & dynamics & resolving commondesign forces
http://www.posa.uci.edu
132
The Evolution of TAOTAO ORB• Compliant with CORBA2.4 & some CORBA 3.0• AMI• INS• Portable Interceptors
• Pattern-centric design• Key capabilities
• QoS-enabled• Configurable• Pluggable protocols
• IIOP• UIOP• Shared memory• SSL• VME
• Open-source• Commercially supported
• www.theaceorb.com• Available now
• ZEN (RT Java/RT CORBA)
• www.zen.uci.edu
133
The Evolution of TAO
A/V STREAMING
DYNAMIC/STATICSCHEDULING
A/V Streaming Service• QoS mapping• QoS monitoring• QoS adaptationACE QoS API (AQoSA)• GQoS + RAPI• Integration with A/VStreaming ETA Winter2001