Support for Human Users in Scientific Workflows

Dimitrios DentsasInstitute of Architecture of Application Systems

d.dentsas@googlemail.com

Support for Human Users

in Scientific Workflows

Diploma Thesis

Contents

Motivation

Properties of Scientific Workflows

Automation boundaries

Roles of WS-HT and BPEL4People

Integration of Humans

Basics

Message Structures & Communication Flows

Outline Solutions

Research Design

SW4H – Scientific Workflows for Humans

Abstract Architecture

Using transactions

Message-based Routing

Demo – Editing a task

Motivation

Numerical simulations have become the third pillar of

science

Analyzing and visualizing data-centric simulation results

is a highly repetitive process but nevertheless necessary

Automation and orchestration of work units in scientific

simulations are needed

Workflow-Management-Systems (WfMS) are an

established tool in the business domain for such

challenges

Scientific Workflow Management Systems (SWfMS)4

Motivation – Automation boundaries

Simulation steps cannot be fully automated if:

The requirements for automation are too difficult to meet

The „decision making process“ can only be executed by

„cognitive entities“

e.g. Quality assurance of simulation results

Smart allocation of resources

Controlling and manipulating critical parameters

Scenario: Simulation executions over the weekend

Motivation – Current needs

There is demand for a smooth integration of human users

in SWfMS

Network-based communication channels are hardly ever

Efficient management of simulation-related tasks

Web service standards like WS-HumanTask and

BPEL4People are used in the business domain.

As specifications they are part of the WS-*

They determine how to integrate human users into business

processes

The WS-BPEL language has to be extended

Motivation

Integration of Humans

Basics

Cloud-based integration of applications

Special requirements for integrating human users have to

be considered and managed

Human Communication Flows are introduced in order to

reach these objectives

In this paradigm all participants hold one of the following

roles:

Communication Initiator

Human Communication Manager

Communication Services

Communication Devices

Communication participants

Message structure

We distinguish between two types of communication

parameters:

Communication Parameters:

Communication type (Notification, Response required/optional)

List of all message receivers

Communication channels (Skype, Chat, etc.)

Further options (response schema, attachments, default responses)

Communication Message Properties:

subject

Importance level

Message structure

Messages consist of:

Communication Request Message

Communication Message

Channel-Specific Message

Communication Flows – User Notification

Communication Flows – Response Required

Communication Flows – Response Optional

SW4H - Message structure

Structure of all input and output XML messages of the

WSDL operations

Structure of all input and output XML messages of the WSDL

operations

Outline

Solutions

Approach – The communication flow

A multitude of different technical realization possibilities.

Two very popular ones are:

1. BPEL-Process

2. Message-oriented Middleware (MOM) EAI through Enterprise Integration Patterns

BPEL MOM

Pros • uniform modeling

environment

• Easier integration with

process fragments

• Large open source

community

Cons • Tight coupling with

workflows

• Introduction of new

dependencies by

using EAI-

Frameworks

Approach – The communication flow

Requirements for EAI-Frameworks:

Fault tolerance

Guaranteed deliveries

Scalability

Support for popular protocols of the ISO/OSI application layer

Apache Camel

Apache ActiveMQ[4]

Approach– Data model

Don‘t reinvent the wheel!

What similar solutions are there for the data model?

Project Bangkok (Uni Stuttgart )

Tempo (Intalio)

Apache HISE (Incubator Project)

Project Bangkok and Apache HISE are open source

implementations of the WS-Human Task specification.

Tempo implements BPEL4People (REST APIs).

Project Bangkok

An embedded Human Task Manager for Java

APIs for client applications

Support for:

HumanTaskModel vs HumanTaskInstance

People queries (logical people groups, literals)

Task presentation

User management

Scientific-Workflows-for-Humans (SW4H)

Human Communication Manager

What it is based on:

Project Bangkok

Apache Camel (Routing Engine)

Apache ActiveMQ (Embedded Message Broker)

What it offers:

Web-frontend with a task worklist

Easily integratable into the Eclipse platform

A multitude of integrated Messaging Endpoints

Exchangable Back-End systems for persistency (currently

MySQL)

SW4H – Abstract Architecture

Integration of new services

Using a uniform Service Endpoint Interface (SEI)

Service Integration API

Human Interaction Activity

Implemented as a process

fragment

Asynchronous communication

One <invoke> activity

One <pick> activity

Conditional constructs with a

<if> activity

Invoking the Communication

Inititation API

(sendMessage, onMessage)29

SW4H – Communication Manager

Task-loop – A command interpreter

The message body of coupled

devices is analyzed:

#<command>( [-h] -p1 foo –p2 bar)

forward

resend

useChannel

getUsers

release

Task-loop – Parsing of XSD documents

XML-Schema for describing tasks is analyzed with a XSD-Parser

A proper subset of the XML-Schema spezification

Simple types :

Explicit and anonymous type definitions

Primitive data types:

string, decimal, integer, boolean, date, time

Restrictions and facets:

enumeration, maxinclusive, mininclusive, maxexclusive, length, maxlength, minl

ength, pattern, totaldigits

Complex types:

Explicit and anonymous type definitions

Recursive complex types

Child elements:

sequence, choice, all

Attributes:

maxoccurs, minoccurs, required, optional

SW4H – Messaging

Implementation of the Communication Flows

EIP-Diagram – The constituents are:

Implementation of the Communication Flows

User Notification: The constituents are : Message: Header and body

Recipient List: Content-based Routing

Persistent Queues

Endpoints: User-Devices

Transactional behavior

Local transactions of the messaging layer

Cooperation of the Spring JmsTransaction Manager with

the JMS Broker (ActiveMQ)

Transactional behavior

Persistent Queues act as „Safe Havens“

Reaching atomic behavior between queues

SW4H – Presentation Layer

SW4H Frontend – Use cases

SW4H Frontend – Login window

SW4H Frontend – Task window

SW4H Frontend – Invitation dialog

SW4H Frontend – Registration invitation

SW4H Frontend – My Devices

SW4H Frontend – My Services

SW4H Frontend – Registration form

SW4H Frontend – Eclipse integration

Editing a task

Communication Request Message

XML-Schema of

the task

static user list

Communication

Parameters

Demo Run – Contacting the HCM

Using XMPP (Google GTALK)

Demo Run – Editing

Demo Run – The resulting response message

The XSD-Parser processes all inputs and produces a

valid XML document

Callback – The task is finished successfully and sends a

response message back to the workflow

Thank you for your attention

Questions?…

http://code.google.com/p/sw4h-2012/

List of references

(1) M.Sonntag,D.Karastoyanova: (2010) Next Generation Interactive Scientific

Experimenting Based On The Workflow Technology. 21st IASTED

International Conference on Modelling and Simulation.

(2) Schumm, David; Karastoyanova, Dimka: Integrating Humans in Scientific

Workflows: Integrate, Register & Communicate. In: The 4th SimTech Status

Seminar, 2011

(3) Schumm, David; Fehling, Christoph; Karastoyanova, Dimka;

Leymann, Frank; Rütschlin, Jochen: Processes for Human Integration in

Automated Cloud Application Management, Universität

Stuttgart, Technischer Bericht Nr. 2012/02, 2012.

(4) http://camel.apache.org/

(5) http://activemq.apache.org/

(6) Ian J. Taylor, Ewa Deelman, Dennis B. Gannon, and Matthew Shields.

2006. Workflows for E-Science: Scientific Workflows for Grids. Springer-

Verlag New York, Inc., Secaucus, NJ, USA.

End Of Document

Support for Human Users in Scientific Workflows

scientific simulations

scientific workflows

response requiredoptionallist

simulation executions

business domain

options response schema

systems wfms

repetitive process

Technology

Building Scientific Workflows with Spotify's Luigi

BPEL for Scientific Workflows

Automating Environmental Computing Applications with...

Provenance in Scientific Workflows on SEEK

Optimizing Complex Scientific Workflows Using ... -...

Balanced Task Clustering in Scientific Workflows

Reproducibile scientific workflows - Acting on Change 2016

Scientific Workflows in e-Science

Scientific workflows for computational reproducibility in...

PhD Thesis: Mining abstractions in scientific workflows

From Scientific Workflows to Research Objects: Publication.....

Towards user-definable, semi- automated workflows for...

An Introduction to Scientific Workflows

WorkflowSim: A Toolkit for Simulating Scientific Workflows.....

Flexible Scientific Workflows Using Dynamic Embedding

Data Management, Workflows, & Scientific Information...