Experience on Developing Adaptive Middleware Based Systems :
Cost, benefit and design approach
Research Seminar
Outline
• About Me• Background and Motivation• Research Challenges and Business Demands• Experience in 5 Aspects• Collaboration Opportunities• Research Vision
About Me
• Researcher, 2004 - present. ESE, NICTA – Software architectures and middleware systems
• PhD, 2001-2004. University of Sydney– Supervisors: Prof. Alan Fekete and Prof. Ian Gordon– Thesis:
• A framework of performance prediction of component-based applications
• Reviewers: Dr. Len Bass, Prof. John Grundy and Dr. Piyush Maheshwari
PhD Thesis : Performance Prediction Method
client broker account stockitem stockholding stocktx Transaction Manager
BuyStock (visits=7%)
check account credit (ratio=1) get stock price (ratio=1)
update (ratio=1)
start transaction
transaction rollback if there is not enough credit (ratio=1)
commit transaction
insert transaction record (ratio=1)
R/W
FindByPK
CS
LC A/P
Y [hr] N [1-hr]
R [p] W [1-p]
Cached
LC A/P
Y [h] N [1-h]
Cached
LD
R or W
SD
W [1-p]R [p]
0200400600800
100012001400
50 100 200 300 400 500Res
pons
e tim
e (R
) in
ms,
read
-onl
y in
tens
ive
CMP Predicted
CMP Measured
RM Predicted
RM Measured
OCC Predicted
OCC Measured
... ... ...
C lien ts
R eq u es t q u eu e C o n ta in er q u eu e J M S S e rv e r
m 1 m '
M D B q u eu e
m 2
D ataS o u r c e q u eu e
C lo sed Q u eu e O pen Q u eu e
Performance Prediction
Client SessionBean EntityHome EntityBean
setValuefindByPrimaryKey
getValue
setValue
getValueSet findByNonPrimaryKey
getAllValues
*[ fo r ea c h e n t ity ]
Architecture model(calibrating)
Application model(design modeling)
Performance Model(populating)
Performance Profile(benchmarking)Performance Modeling
How I get here
ChameleonDashboard
MDAperf,MDAbench
Adaptive Server
Framework
CAmkESprototype
Middleware & SA
Queueingtheory
PerformanceAnalysis
Component Architecture for
Embedded Systems
Adaptive Middleware
Automated CapacityPlanning
DSTOAirborne Mission
System Evaluation
Medicare AustraliaeTax MTS performance
assessment
PhD research
Stage 1: Extend research capability
Stage 2: Industry trial of technologies developed from research
Stage 3: Establish long term research goals Align with ArMature Strategy
Tools and Platforms forPerformance and Dependability Management
for Large Scale Distributed Systems
Research Mission
Devising analysis models, architectures and frameworks to improve the design and runtime performance and dependability of distributed software systems.
Background and Motivation
What is Middleware?
• Middleware is connectivity software that consists of a set of enabling services that allow multiple processes running on one or more machines to interact across a network. – Cited from SEI definition of middleware,
ww.sei.cmu.edu/str/descriptions/middleware.html
Directory Directory and Security and Security
ServicesServicesExistingExisting
ApplicationsApplicationsand Dataand Data
BusinessBusinessDataData
DataDataServerServerWebWeb
ApplicationApplicationServerServer
Storage AreaStorage AreaNetworkNetwork
BPs andBPs andExternalExternalServicesServices
WebWebServerServer
DNSDNSServerServer
DataData
Dozens of systems and applications
Hundreds of components
Thousands of tuning
parameters
Middleware Behind the Scene of Complexity
diagram from Jeff Kaphart’s talk on Software Agents and the Information Economy
What is Middleware?
What is Middleware?
• Middleware Characteristics– Many kinds and widely used – Pervasive and ambient to the end users– Failure and malfunction of middleware can directly affect
user experience• Capability of current middleware systems
– Stack of services to facilitate distributed computing– Already built in with ‘smartness’
• Challenges that middleware-based systems are facing today– Increasing complexity– Heterogeneity– Mapping from business goals to middleware QoS
management
Adaptive Middleware Value Proposition
• Provide a sense and respond solution that supports the response to stimulus – threats and opportunities to the enterprise – in a timely and safe fashion
Adaptive Middleware Characteristics
RespondInvoke actions in
real-time
SenseDetect stimuli across theEnvironment in real-time
AnalyseAggregate stimuli across
multiple sources; Analyse the reality
PlanCompare reality and
expectationMake optimal plans
of actions
Research Challenges and Business Demands
Business Demands and Open ProblemsResearch Challenges
1. Extensible software architectures
2. Mechanisms to enable dynamic monitoring and adaptation
3. Accurate and reliable predictive models
4. Mapping high level business goals to low level QoS management
Business Demands
1. Sense and Respond
2. Asynchrony
3. Global situational awareness
4. Accuracy and efficiency
5. Lower cost and higher quality than human administration
Software Engineering
Solutions
1. Architectural styles (patterns) and frameworks
2. Separation of concerns
3. Non-instrument probes
4. Process orchestration
5. Model-based analysis
6. Empirical evaluation
Developing Adaptive Systems : ROI
• Enterprises are already middleware based– Large scale, loose coupled, heterogeneous environment
• Enterprises have key elements for adaptation– Databases, rules engines, service buses, data-mining
capabilities….• ROI: Incremental cost versus incremental benefits.
– Costs:• Additional components in software stack • Additional development time
– Benefits• Applications are more efficient • QoS assurance
Experience in 5 Aspects
Take-away Points
1. Requirement : adaptation is specialized for domains
2. Design principle: separation of concerns3. Development method: composition rather than
green field development4. Evaluation criteria: preserving safe changes5. Research adoption: unrealistic assumptions setup
barriers
Research Statement
• Integrating analysis models with software architectures to provide specialized solutions for domain specific applications to self-manage the performance and dependability and to preserve safe changes.
Point 1 : adaptation is specialized for domains
Enterprise Systems
Defence systemsRoad and traffic management
systems
Is there anything generic and reusable?
• Initial attempt : adaptive message compression/decompression of Web services
• A little bit more complex : adaptive image processing of Web applications
• There IS something generic and reusable– Architecture styles and patterns– Common components for constructing sense-analyse-
plan-respond loop• Development of Adaptive Server Framework (ASF)
Aims for ASF
• Factor out common services into infrastructure• Reduce effort to build adaptive components• Transparently enhance adaptive components with
advanced features• Make it faster, easier to build and enhance
adaptive capability
Adaptive Server Framework Architecture
ASF Supported Adaptive Application Servers
Middleware (.Net Framework/JAVA EE/ Enterprise Service Bus)Middleware (.Net Framework/JAVA EE/ Enterprise Service Bus)
Standard-based Management LayerStandard-based Management Layer
Operating SystemOperating System
Engine
ExecutorEffector
Sensor
Effector
Sensor Monitor Analyzer Repository
PolicyManager
Event Correlation
Security Manager
Web Services Wrapper
ASF service ASF service layerlayer
Tested andand Optimized
Sample ASF-based Implementation
p u b lic v o id re lo ad In f o ( ) ;p u b lic v o id en ab le ( ) ;p u b lic v o id d is ab le ( ) ;p u b lic b o o lean is E n ab led ( ) ;
< < AS F In ter f ac e> >Lifec y c le
p u b lic S tr in g g e tT y p e( ) ;p u b lic S tr in g g e tN am e( ) ;p u b lic E lem en tI n f o g e tI n fo ( ) ;p u b lic v o id s e tIn f o ( Elem en tIn fo in f o ) ;
< < A S F In ter fac e> >S ta te m an ag em en t
p u b lic v o id h an d leM es s ag e( M es s ag e m s g ) ;
< < AS F In te r f ac e> >M es s ag e C o n s u m er
< < AS F In te r fac e> >M es s ag e P r o d u c e r
p u b lic v o id ad d M es s ag eC o n s u m er (S tr in g m es s ag eN am e, M es s ag eC o n s u m er c o n s u m er ) ;p u b lic v o id ad d M es s ag eC o n s u m er s (S t r in g m es s ag eN am e , Ar ray L is t c o n s u m ers ) ;p u b lic v o id r em o v eM es s ag eC o n s u m er ( S tr in g m es s ag eN am e ,
M es s ag eC o n s u m er c o n s u m er ) ;p u b lic Ar ray L is t g e tAllM es s ag eC o n s u m ers ( ) ;p u b lic v o id r em o v eAllM es s ag eC o n s u m ers ( ) ;p u b lic Ar ray L is t g e tM es s ag eC o n s u m erBy M es s ag eT y p e ( S tr in g m es s ag eN am e) ;p u b lic Ar ray L is t g e tAllM es s ag eT y p es ( ) ;
p u b lic v o id d is p a tc h M es s ag e( M es s ag e m es s ag e , M es s ag eC o n s u m er c o n s u m er ) ;
< < AS F C o m p o n en t> >S en s o r
< < AS F C o m p o n en t> >M o n ito r
< < Ad ap tiv e C o m p o n en t> >Ban d w id th S en s o r
< < Ad ap tiv e C o m p o n en t> >Im ag eP ro c es s in g M o in to r
p u b lic F u tu r e s u b m it( R u n n ab le tas k ) ;p u b lic F u tu r e s u b m it( C a llab le tas k ) ;p u b lic v o id ex ec u te (R u n n ab le ta s k ) ;
< < AS F In ter fac e> >C o n c u r ren c y< < AS F S er v ic e> >
T h r ead Q u eu ed Ex ec u to rp u b lic v o id b in d T o E x ec u to r ( Ex ec u to r e ) ;p u b lic v o id u n b in d T o E x ec u to r (E x ec u to r e ) ;
< < AS F In ter fac e> >E x ec u to r C lien t
p u b lic v o id h an d leN o tif ic a tio n ( N o tif ic a tio n n o tif ic a tio n ) ;
< < A S F In ter f ac e> >Bo o tS tr ap< < AS F C o m p o n en t> >
En g in e< < AS F C o m p o n en t> >
Im ag eP ro c es s in g E n g in e
d is p atc h tas k to
initi
aliz
e an
d en
able
inst
ance
d is p atc h tas k to
initi
aliz
e and
enab
le in
stanc
e
< < AS F C o m p o n en t> >C o n f ig u ra tio n E f f ec to r
p u b lic v o id ad d P o lic y T o S c o p e (P o lic y p ) ;p u b lic D ec is io n req u es tG u id an c e (P o lic y p ) ;p u b lic P o lic y re s o lv eC o n f lic t io n (P o lic y p ,
Ar ray L is t en ab led P o lic y L is t ) ;
p r iv a te S tr in g p o lic y N am e;p r iv a te S tr in g p o lic y S c o p e ;p r iv a te S tr in g p o lic y D es c r ip tio n ;p r iv a te b o o lean p o lic y E n ab led ;p r iv a te Ar ray L is t C o n d it io n s ;p r iv a te C o n d it io n p o lic y C o n d it io n ;p r iv a te D ec is io n p o lic y D ec is io n ;p r iv a te in t p o lic y P r io r ity ;
p u b lic Ar ray L is t g e tEn ab led P o lic ies ( ) ;p u b lic Ar ray L is t g e tD is ab led P o lic ies ( ) ;p u b lic Ar ray L is t g e tAllP o lic ie s ( ) ;p u b lic v o id at tac h P o lic y ( P o lic y p ) ;p u b lic v o id d e ttac h P o lic y (P o lic y p ) ;
< < AS F In te r f ac e> >P o lic y Ac tu ato r
initi
aliz
e and
enab
le in
stan
ce
< < AS F S erv ic e> >P o lic y M an ag er
in it ia lize an den ab le in s tan c e
req u es t g u id an c e f r o m
< < AS F C o m p o n en t> >P o lic y
< < AS F C o m p o n en t> >P o lic y Utility
< < Ad ap tiv e C o m p o en n t>M es s ag in g T as k < < AS F C o m p o n en t> >
T as k s
Customized,Application specificAdaptive component
ASF pre-defined component with default implementation
What is the software architecture design principle?
• Typical Stovepipe Solution Architecture
• Business logic and adaptation are ‘crosscutting’ • “One-off” solutions, hard to weave in customized
adaptation
J S P J S F
C o n t r o l View
P res en ta t io nLay er
P r o c es sLay er
D ata Bin d in g
I n te r f ac e ( W S D L )
S er v ic e L ay er Bu s in es s Lay er
B u s in es sC o m p o n en t
B u s in es sC o m p o n en t
E J BJav a
C las s es
P er s is ten t L ay er
Point 2 : Separation of Concerns
J S P J S F
C o n t r o l View
P res en ta tio nL ay er
P ro c es sL ay er
D ata Bin d in g
In te r f ac e ( W S D L )
S erv ic e L ay e r Bu s in es s L ay er
B u s in es sC o m p o n en t
B u s in es sC o m p o n en t
E J BJ av a
C las ses
P er s is ten t L ay er
• Separate business logic components and adaptive components • Plug-and-play deployment model• Adaptive middleware provides programming models and
services• Easy to reconfigure and incorporate new adaptation
What is the development method?
You (ABC) Loan
Broker
Loan quote please?
Quote please?
Quote please?
Quote please?
6.25%
6%
5.4%
5.4% from NAB
Credit histo
ry?
He’s alrig
ht
Let’s get a loan!
Overloadcontrol
Failovercontrol
Scenario 1 : Overload Control
http://www.eaipatterns.com/SmartProxy.html
Scenario 2 : Automatic Failover
http://www.eaipatterns.com/SmartProxy.html
What is the development method?
Deployment boudnary
Bank 1
Bank 2
Bank 3
Bank 4
J2EE Server
Web Service Container
Enterprise Service Bus
Point 3 : Composition, Not From Scratch
• Legacy is the reality…• Enterprises have key elements for composing
adaptive systems– existing middleware, software frameworks, rules engines,
service buses, data-mining capabilities…• Composing adaptive systems saves time and
money !• But how ?
– Extending the architecture– Adding in new components– Adding in new mechanisms
Composing Overload and Failover Control• Application specific
scenarios driven what to be composed
• ASF helps with how to compose them
Rest of System
1
Token Bucket1
Smart proxy
m
n
Deployment boudnary
Bank 1
Bank 2
Bank 3
Bank 4
J2EE Server
Web Service Container
Enterprise Service Bus
SmartProxy
MonitorTester
Planer
Manager
Esper
Complex event processing
Quartz
Job scheduling
ReinforcementLearning
OS Monitors
jBPMBusiness process engine
General SOA ArchitecturesWith Services Connected Through
Endpoints
Current Work : Developing a Process Driven Monitoring and Actuation Framework
Increasing Sophistication of ASF
J S P J S F
C o n t r o l View
P res en ta tio nLay er
P ro c es sLay er
D ata Bin d in g
In te r f ac e ( W S D L )
S erv ic e L ay er Bu s in es s Lay er
B u s in es sC o m p o n en t
B u s in es sC o m p o n en t
E J BJ av a
C las ses
P er s is ten t Lay er
ProcessOrchestration Chameleon
Dashboard
ASF Machine Learning
Algorithms
Model drivendevelopment
Chameleon Dashboard
Point 4 : Preserve Safe Changes
• Changes are made to the configuration, the behaviour, and the structure of the system.
• Demands on – Efficient architecture designs Architecture evaluation
methods– Accuracy of analysis models Simulation and empirical
evidence– Optimization of learning process Heuristic hints– Detecting and recovering from unsafe changes OS
mechanisms– Tools for mapping between business goals and infrastructure
QoS management Business process and policy management– Tools for development and maintenance IDE and model
driven development
Current Work : Enabling Design Level Prediction
Design models analysis/simulation models
Analysis/simulation models Prediction
Point 5 : Avoid Unrealistic Assumptions
• Single server
• Hard coded control logic
• Empirical evaluation of accuracy
• Distributed as Web services
• Hard coded control logic
• Empirical evaluation of accuracy
• Distributed as any objects, components and services
• Control logic modelled as processes
• Built-in algorithms
• Design level prediction
ASF Version 1
ASF Extension with
services ASF
Version 2
Summary
• Adaptation is specialized for domains• Good separation of concerns is the design
principle• Composing adaptive systems saves money
and time• Preserving safe changes is the key to be
successful• Unrealistic assumptions blocks the way of
research adoption
Collaboration Opportunities
Research Collaboration
• A platform to demonstrate integrated NICTA research capabilities– AMP is under ArMature umbrella, leveraging process orchestration to
control QoS management
• Opportunities for interdisciplinary research collaboration– Software engineering methodologies and tools : testing and formal
analysis– Machine learning algorithms : mining rules, problem diagnosis– Computing theories: queueing theory, random algorithms– Operational research : linear programming and optimization– Networking QoS control mechanisms : token bucket, QoS
differentiation
• Impact– A technology demonstrator with the differentiator, the capability of
preserving safe changes
Industry Collaboration
• Todo
Research Vision
Sense and Respond: Beyond Middleware • "Sense and respond systems" are systems that
detect critical conditions in an extended distributed environment and respond proactively.
– Prof. K. Mani Chandy , California institute of technology
WP3 Linkages
Process QoS
SystemQoS
COST=F(v1, v2, v3,…, vn)
ExecutionEngine
ExecutionEngine
WP3
WP2.2
PerformanceAnalysis
WP2.3
Design Level Prediction Run Time Adaptation
From imagination to impact
Backup Slides
PhD Thesis : Performance Prediction Method
client broker account stockitem stockholding stocktx Transaction Manager
BuyStock (visits=7%)
check account credit (ratio=1) get stock price (ratio=1)
update (ratio=1)
start transaction
transaction rollback if there is not enough credit (ratio=1)
commit transaction
insert transaction record (ratio=1)
R/W
FindByPK
CS
LC A/P
Y [hr] N [1-hr]
R [p] W [1-p]
Cached
LC A/P
Y [h] N [1-h]
Cached
LD
R or W
SD
W [1-p]R [p]
0200400600800
100012001400
50 100 200 300 400 500Res
pons
e tim
e (R
) in
ms,
read
-onl
y in
tens
ive
CMP Predicted
CMP Measured
RM Predicted
RM Measured
OCC Predicted
OCC Measured
... ... ...
C lien ts
R eq u es t q u eu e C o n ta in er q u eu e J M S S e rv e r
m 1 m '
M D B q u eu e
m 2
D ataS o u r c e q u eu e
C lo sed Q u eu e O pen Q u eu e
Performance Prediction
Client SessionBean EntityHome Ent ityBean
setValuefindByPrimaryKey
getValue
setValue
getValueSet findByNonPrimaryKey
getAllValues
* [ fo r eac h e nt ity ]
Architecture model(calibrating)
Application model(design modeling)
Performance Model(populating)
Performance Profile(benchmarking)Performance Modeling
Sense and Respond: Beyond Middleware • Emerging platforms with much simpler
programming models, larger scope and more unpredictable usage behavior.
• Design and develop adaptware to provide internet facing applications with cost-effective adaptation capability of managing performance and dependability
• Adaptware consists of a stack of adaptive components and a lightweight runtime environment deployable to the platforms
