ADAPT IST-2001-37126 Composite Services Gustavo Alonso Swiss Federal Institute of Technology (ETHZ) Zürich, Switzerland
Dec 20, 2015
ADAPT IST-2001-37126
Composite Services
Gustavo Alonso
Swiss Federal Institute of Technology (ETHZ)
Zürich, Switzerland
ADAPTADAPT IST-2001-37126
Outline
• Our vision and grand goals
• State of the art and influence on ADAPT
• Progress so far
• Work on year two
ADAPTADAPT IST-2001-37126
Our vision
Automating the integration of IT infrastructures through Web service composition, business protocols, and conversation
specifications
Servicespecification
Automatic extraction of service data
Automatedsupport for composition
Automated validation of the compositionEnactment
ADAPTADAPT IST-2001-37126
Our grand goals
• Through automation and leveraging Web services we hope to:– reduce the development cost of B2B applications– make Web service technology available to SMEs– create the basics for plug&play Web service technology– complement standardization in areas that are currently not
being well covered by industry efforts– hide the complexity and changing nature of Web service
technology from the end user – contribute to standardization efforts in composite services– open source platform
ADAPTADAPT IST-2001-37126
State of the art
• Many important developments in the last year:– BPEL specification– BPEL implementations (Collaxa, BPELWS4j)– Many additional specifications relevant to composition (WS-
CAF, WSIF, WS-Coordination, WS-Transactions, Grid, etc.)
• Many changes and not a clear direction
• Competing standards without a clear winner (ebXML, xCBL)
• Luckily, this situation does not negatively influence our plans within ADAPT (so far, we will keep a close watch)– our goals still beyond the scope of industry efforts– we profit from all the tools that have become available
ADAPTADAPT IST-2001-37126
Influences on ADAPT
CONCEPTUAL DESIGN
• Use of Web service specifications is ill defined and changing
• The range of design options is widening and may require us to focus on a particular application type
• The focus is also shifting in industry (from simple services to conversations)
ACTUAL ARCHITECTURE
• Increasing number of tools available (to enhance the work on ADAPT, or to give us a link to products)
• May have to postpone some design decisions until a later stage than planned
• Otherwise, the initial goals of the ADAPT platform for composition remain unchanged
ADAPTADAPT IST-2001-37126
Progress so far
• Two prototype composition engines (based on previous work) capable of automatically importing information about Web services (WSDL description), embedding the operations within a workflow process, and invoking the services using SOAP– Centralized engine (Java)– Distributed engine (Java)– Bottom up composition
• Graphical tool for composition
• Consensus formed on model behind composition (Pi calculus) and properties of composition that we will support (transactions, choreography).
ADAPTADAPT IST-2001-37126
Evaluation of progress so far
• All the background work has been done:– thorough understanding of available technology– flexible design to keep our options open (centralized,
distributed, bottom up composition, top down composition, conversation based composition, etc.)
• Existing prototypes provide excellent platform for experimentation and exploring interaction of heterogeneous systems
• Consensus emerging on what are the limits to what we can do in ADAPT, what we can take from others (e.g., TAPAS), and where the strength of the open source platform will be
ADAPTADAPT IST-2001-37126
Planned work for year two
• Consolidation of the prototypes– possible decision to go with only one engine (not necessary)
• Closer look at top down composition (as an extension or alternative)
• Tying together of graphical tool, engines and composition model in a single unified framework– careful attention to standards (BPEL)
• Work on adaptive composition and automated analysis– transactions– choreography and business protocols
• First version of the ADAPT platform
ADAPT IST-2001-37126
http://www.iks.ethz.ch/jopera
ADAPTADAPT IST-2001-37126
JOpera Architecture
Kernel
Navigator DispatcherProcessTemplate
Plugin
ProcessTemplate
PluginU
NIX
SO
AP
JAV
A
....
StateInformation
Storage
Event Queues
TaskExecutionScheduler
ADAPTADAPT IST-2001-37126
JOpera Distributed Kernel
Navigator
Navigator
Dispatcher
Dispatcher
Dispatcher
ProcessTemplate
Plugin
ProcessTemplate
Plugin
ProcessTemplate
Plugin
ProcessTemplate
PluginU
NIX
UN
IXU
NIX
SO
AP
SO
AP
SO
AP
JAV
AJA
VA
JAV
A
........
....
StateInformation
Storage
Event QueuesTask
ExecutionScheduler
Kernel
ADAPTADAPT IST-2001-37126
JOpera Storage Architecture
State InformationStorageLocalMain
Memory
State InformationStorage
RemoteTupleSpace
State InformationStorage
Main MemoryWrite-through
Cache
RemoteTupleSpace
ADAPTADAPT IST-2001-37126
Cost of persistent storage
0
50
100
150
200
250
300
350
400
1 64 128 256 512 1024
Number of concurrent processes
Ave
rag
eW
all
Tim
e(s
eco
nd
s)
Volatile
Storage Configuration:
Process Size: tasksControl Flow:
10Parallel
Persistent (with Cache)
Persistent
0
100
200
300
400
500
600
700
1 64 128 256 512 1024
Number of concurrent processes
Ave
rag
eW
all
Tim
e(s
eco
nd
s)
Volatile
Storage Configuration:
Process Size: tasksControl Flow:
10Sequential
Persistent (with Cache)
Persistent
0
500
1000
1500
2000
1 64 128 256 512 1024
Number of concurrent processes
Ave
rag
eW
all
Tim
e(s
eco
nd
s)
0
1000
2000
3000
4000
5000
1 64 128 256 512 1024
Number of concurrent processes
Ave
rag
eW
all
Tim
e(s
eco
nd
s)
VolatileVolatile
Storage Configuration:Storage Configuration:
Process Size: tasksControl Flow:
100Matrix
Process Size: tasksControl Flow:
100Parallel
Persistent (with Cache)Persistent (with Cache)
PersistentPersistent
ADAPTADAPT IST-2001-37126
Process instantiation time
0.1
1
10
100
1000
0 5 10 15 20 25
Number of Navigators
Pro
cess
inst
anti
atio
nti
me
(mil
lise
con
ds)
State Information Storage
Centralized, Persistent
Distributed, Volatile
Process Size1 task10 tasks100 tasks
ADAPTADAPT IST-2001-37126
Throughput degradationThroughput of a monolythic kernel
0 5 10 15 20 25 30
Matrix(100)
Parallel(100)
Parallel (10)
Sequential(10)
Sequential(1)
Pro
cessto
po
log
yan
dsize
Throughput (tasks/sec)
64
128
256
512
1024
Number ofconcurrentprocesses
ADAPTADAPT IST-2001-37126
Scalability (Response time)Distributed Tuple Space - Sequential (1 sec)
10
100
1000
10000
0 5 10 15 20 25
641
1
164
64
64
128
128
128
128
256
256
256
256
512
512
512
512
1024
1024
2048
10242048
1024
2048
2048
Number of Navigators
BB
aa
tt
cc
hh
EE
xx
ee
cc
uu
tt
ii
oo
nn
TT
ii
mm
ee
((
ss
ee
cc
oo
nn
dd
ss
))
Distributed Tuple Space - Sequential (10 sec)
10
100
1000
10000
0 5 10 15 20 25
Number of Navigators
Bat
chE
xecu
tio
nT
ime
(sec
on
ds)
Tuple Space (Local) - Sequential (1 sec)
10
100
1000
10000
0 5 10 15 20 25
Number of Navigators
Tuple Space (Local) - Sequential (10 sec)
10
100
1000
10000
Configuration: storageTask duration: seconds
Persistent10
Configuration: storageTask duration: second
Persistent1
Configuration: storageTask duration: seconds
Volatile10
Configuration: storageTask duration: second
Volatile1
0 5 10 15 20 25
Number of Navigators
Bat
chE
xecu
tio
nT
ime
(sec
on
ds)
1
ADAPTADAPT IST-2001-37126
Scalability (throughput)
1o Parallel tasks - Persistent storage
1o Parallel tasks - Volatile storage
0
20
40
60
80
100
120
140
160
0 5 10 15 20 25Number of Navigators
Th
rou
gh
pu
t(t
asks
/sec
on
d)
Number ofconcurrentprocesses
6412825651210242048
0
50
100
150
200
250
300
350
400
0 5 10 15 20 25Number of Navigators
Th
rou
gh
pu
t(t
asks
/sec
on
d)
Number ofconcurrentprocesses
6412825651210242048
ADAPTADAPT IST-2001-37126
Split/Merge Options
ADAPTADAPT IST-2001-37126
Reliable WS Call
Static
Dynamic
output
input
SYS.progService
ServiceCall,ServiceCall_Alternate,ServiceCall_Backup,ServiceCall_Backup2
serviceNames
matchName
getServiceList
ServiceCall*
output
input
SYS.progService
ADAPTADAPT IST-2001-37126
Visual XML Transformation<Adresse>
<name> </name><nachname> </nachname><strasse> </strasse><plz> </plz><ort> </ort><kanton> </kanton>
</Adresse>
CesarePautassoHirschengraben 84
8072Zurich
ZH
<Address><name> </name><street> </street><number> </number><town> </town><state> </state>
<country> </country></Address>
Cesare PautassoHirschengraben84
ZurichZH
Switzerland<zip> </zip>8072
name street number ziptown state country
pack_Address
name nachname strasse plz ort kanton
unpack_Adresse
ab
a b
Concatenate
s n
a
Split Switzerland
ADAPTADAPT IST-2001-37126
WS Demo Process
symbol country
StockQuoteConvert Input
quote
StockQuoteConvert Output
symbol
Result
Quote
country1country2
Result
Convert
usa
a b
result
Multiply
StockQuoteConvert - DataFlow
This process is included as an example of WS composition with the current JOpera release