S-CUBE LP: Chemical Modeling: Workflow Enactment based on the Chemical Metaphor

www.s-cube-network.eu

S-Cube Learning Package

Chemical Modeling: Workflow Enactment based on the Chemical Metaphor

INRIA, CNR, SZTAKI

Zsolt Németh, SZTAKI

Learning Package Categorization

S-Cube

Service Infrastructure

Multi-level and self-adaptation

Supporting adaptation of

service-based applications

Learning Package Overview

Workflow, workflow management

Problem and requirement analysis

– enactment in large-scale heterogeneous environments

A chemical metaphor for workflow enactment

A coordination model for workflow enactment formalized in

the -calculus

Further challenges

This talk is about workflow but…

Workflow is just an example

– It is a common programming model for grids

– Features many coordination problems

Focus: coordination

– Large-scale distributed, dynamic, error prone environment and

applications

– Some degree of autonomy, adaptability, self-* must be provided

Workflow

“The computerized facilitation or automation of a

business process in whole or in part” – The Workflow Management

Coalition

a collection of activities that are processed in some order

and where both data-flow and control-flow relationships

may be present

The workflow Management Reference Model (Workflow Management Coalition)

Process Analysis, Modeling

& Definition Tools

Process Definition

Workflow Enactment Service

Applications &

Tools User (human)

Process Design

& Definition

Build time

Run time

Process Instantiation

& Control

Interaction with

Users & Application Tools

Scope of enactment

Abstract workflow

standalone application components

the order in which they are executed

names, references, etc. are logical

Represented by a graph

typically: DAG

Problem

Abstract workflow

Concrete workflow

Physical environment

Scope of enactment

Concrete workflow

standalone application components

the order in which they are executed

selected resources, services

additional activities (e.g. file transfer, staging, etc.)

all names, references, etc. are physical

Problem

Abstract workflow

Concrete workflow

Physical environment







the -calculus

Further challenges

A common scenario (e.g. grid workflow)

Abstract workflow

Resources

Workflow engine


Abstract workflow

Resources

Information system

Workflow engine


Abstract workflow

Resources

Information system

Workflow engine


Abstract workflow

Resources

Information system

Workflow engine

Concrete workflow


Abstract workflow

Resources

Information system

Workflow engine

Concrete workflow

Problem analysis (current approaches)

The abstract workflow is static

(Typically) no advanced control structures

A priori mapping/ partly a priori mapping

Mapping based on stored information

A priori simulation/ a priori test/ a priori optimisation

Centralised engine

Human interaction

Limited autonomy, limited ability for adaptation

Overall lack of any high level model

Requirement analysis

Workflow enactment in large-scale heterogeneous environments

– should provide a higher level of autonomy

– should be able to adapt to changing conditions

– should be distributed

– should be able to make decisions on partial and actual information

– should support arbitrarily complex control structures

Often a complex problem can be solved in a more simple way using nature inspired models







the -calculus

Further challenges

This talk is about a chemical metaphor but…

It is just an example

– Chemical reactions are reasonably similar to workflow enactment

– It has a well established formalism

Generally: nature inspiration for solving complex problems

– Simple, primitive parts behave “intelligently” as a whole

– Ants, termites, cells, molecules, etc.

A chemical metaphor

Reactions are

– autonomous

– distributed

– concurrent

– depending on local conditions

– depending on actual conditions

– not following any a priori pattern

– evolving in time

A chemical metaphor

Reactions are

autonomous

distributed

concurrent

depending on local conditions

depending on actual conditions

not following any a priori pattern

evolving in time

Workflow enactment should

provide a higher level of autonomy

be able to adapt to changing conditions

be distributed

be able to make decisions on partial and actual information

support arbitrarily complex control structures

A vision of chemical enactment

Resources


Resources

Activities


Resources

Activities

Control


React






A coordination model for workflow enactment formalized

in the -calculus

Further challenges

From a vision to a model

Materialize information: resource quantums

Define an abstract chemical coordination model

– independent from actual technical realizations

– provide a high-level abstract framework for refinement

– formalism: -calculus

Advance, refine: find chemical examples for

– Complex control

– Fault tolerance

– Resource management

– Optimization

– etc

Resource quantums

The usual solutions

P1 P2

P3

P4

P5

P6

• stored information may be time sensitive

P6

Resource quantums

The usual solutions

P1

P2

P3

P4

P5

Resource quantums (« materialization of information »)

resources are represented as tickets

P1 P2

P3

P4

P5

P6

• tickets represent a guaranteed service

Resource quantums (« materialization of information »)

P2

P5

P1

P3

P4

P6

The -calculus

a declarative, functional formalism

inherently concurrent model of computation

basic data structure: multiset (chemical solution)

– passive molecules: booleans, integers, tuples, naming molecules

– active molecules: -abstraction

reaction: active molecules capture other molecules

– x.M, N → M[x:=N]

execution: perform reactions until a stable (inert) chemical

solution is resulted

The -calculus: -abstraction

Active molecules: -abstraction: PC.M

– P is a pattern that selects elements for the reaction

– C is a condition; the reaction takes place if C is true

– M is the action

Example: i:x, j:y i≤j, x>y. (j+1):x, j:y

Semantics: capture i:x and j:y and replace them by (j+1):x,

j:y if i≤j, x>y

-abstraction is one-shot

Universal matching symbol:

The -calculus

-terms are

– Commutative: M1,M2≡M2,M1

– Associative: (M1,M2),M3≡M1,(M2,M3)

– Realize Brownian motion

Reactions

– Locality: if M1→M2, then M,M1→M,M2

– Solution: if M1→M2, then <M1>→<M2>

Conditional reactions

– xC.M

Atomic capture – x1,x2,…xn.M

Resources

a resource quantum is modeled as a sub-solution

– recall: chemical solutions can be nested

elements in the solution are inert attribute:value pairs

there are mandatory attributes otherwise, the format is felxible

<id:R1, type:comp, proc:16, OS:Linux, …>

<id:N1, type:net, bandwidth:23, …>

<id:R3, type:comp, proc:1, installed:equsolver, network:N1 …>

Elementary activities

the chemical model does not execute an activity, just enacts it – execution is external

exit from the chemical world:

– execute activity on resource using parameter

– a symbolic notation that can be realised in many ways

return to the chemical world

– the execution produces some result or error put back in form of a solution (control molecule)

– <ActivityID:result>

– <ActivityID:result, error:errorcode, …>

– <ActivityID:result, executed:R1, …>

execute A on R→ <A:result…>,<id:R,…>

Resource dependency

activity A needs a resource

<id:r, type:comp, proc:1, >. execute A on r

<id:r, type:comp, proc:1, >.

execute A on r

< id:R1, type:comp,

proc:16, OS:Linux, …>

< id:R2, type:comp, proc:1,…>

< id:R3, type:comp,

proc:1, OS:SunOS, …>

< id:R4, type:comp,

proc:1, memory:23, …> < id:R5, type:comp,

proc:1, disk:12, …>

Resource dependency

<id:r, type:comp, proc:1, >. execute A on r

Captured a matching resource

< id:R1, type:comp,



< id:R4, type:comp,



execute A on R3

Resource dependency

Both the resource and the activity are replaced by a control

molecule

< id:R1, type:comp,



< id:R4, type:comp,



<A:12, error:0, executed:R3>

< id:R3, type:comp,

proc:1, OS:SunOS, …>

Data and control dependency

activity A waits a result from activity B

<B:x, >. execute A using x

<B:x, >. execute A using x <B:18>

<C:0, error:5>

<D:42>

<H:apple>


<B:x, >. execute A using x

Captured a corresponding control molecule

execute A using 18

<C:0, error:5>

<D:42>

<H:apple>


Replaced by the result

<A:2, error:0,...>

<C:0, error:5>

<D:42>

<H:apple>

Complex dependencies

An activity needs both input from another activity and

resources

– <id:r, 1>, <B:x, 2>. execute A on r using x

Dependencies can be arbitrarily combined

– find a resource r1 for activity A and a resource r2 for B so that r1 and r2

have the same operating system

– <id:r1, OS:x, 1>, <id:r2, OS:x, 2> .

execute A on r1, execute B on r2

Conditions can be added as well

– <id:r, disk:x, memory:y, > x>30, y>12.

execute A on r

Workflow patterns

Sequence

Conditional

Split

Synchronizing merge

P-split

– p-out-of-n activities follow A

P-merge

– p-out-of-n results trigger activity A

Loop







the -calculus

Further challenges

Challenges

What we have so far is a framework

– Principles of a chemical coordination model

– It is just the beginning!

Let’s explore and advance

– Solve further aspects of workflow enactment: fault tolerance,

optimization, resource control, complex workflow structures,

constraints, co-allocation, compensation, etc.

– Find further nature analogon within the chemical model:

temperature, weight, magnetic properties, size, gravity, catalysts,

membranes, etc.

Challenges (examples)

Resource control by chemical agents

– withdraw, modify, block, accept, reject, group, etc.

Assert a neutralization agent: <id:r1, >.

Replace a molecule:

<id:r1, proc:1, OS:x, >.<id:r1, proc:1, OS:Linux, >

Combine two molecules:

(<id:r1, proc:1, >, <id:r1, proc:1, >).

<id:r1, proc:2, >


Fault-tolerance by chemical agents

– error molecules, retry, rollback, redundancy, compensation, etc.

Assert an error handling molecule that reactivates in case of

error

(<Ai:x, >, <Aj:y, >, <id:r, R, >).

(execute A on r using x y,

(<A:errork, >,<id:r’, R, >).execute A on r’ using x y))


Complex resource allocation and co-llocation scenarios by

molecules

– e.g. reserve r1 and r2 so that there is a network link between them

– (<id:r1, proc:1, network:n, >,

<id:r2, proc:4, network:n, >,

<id:n, type:net, >).

(execute A on r1, (<A:x, >.execute B on r2 using x))

Conclusion

The chemical metaphor: autonomous, adapting, distributed,

actual

The nature inspired coordination model

– activities, resources and control are modeled using the same

formalism

– -calculus is not just description but defines execution semantics as

well

– workflow structure, resources and control can be

added/withdrawn/modified in a fully dynamic and adaptive way

Connections to other teaching units

Foundations

– The Chemical Computing model and HOCL Programming

Applications

– Dynamic Adaptation with the Chemical Model

© S-Cube – 50/<Max>

References

Zsolt Németh, Christian Pérez, Thierry Priol: Distributed workflow coordination: molecules and reactions.

IPDPS 2006

Zsolt Németh, Christian Pérez, Thierry Priol: Workflow Enactment Based on a Chemical Metaphor. SEFM

2005: 127-136

Manuel Caeiro, Zsolt Németh, Thierry Priol: A Chemical Model for Dynamic Workflow Coordination. PDP 2011:

215-222

http://portal.acm.org/citation.cfm?id=1719370




Acknowledgements

The research leading to these results has

received funding from the European

Community’s Seventh Framework

Programme [FP7/2007-2013] under grant

agreement 215483 (S-Cube).

S-CUBE LP: Chemical Modeling: Workflow Enactment based on the Chemical Metaphor

Technology