Performance_and_Cost_Evaluation

ResearchResearch

University of StuttgartUniversitätsstr. 3870569 StuttgartGermany

Phone +49-711-685 88337 Fax +49-711-685 88472

Santiago Gómez Sáez, Vasilios Andrikopoulos, Michael Hahn, Dimka Karastoyanova, Frank Leymann, Marigianna Skouradaki, Karolina Vukojevic-Haupt

Institute of Architecture of Application [email protected]

Performance and Cost Evaluation for the Migration of a Scientific

Workflow Infrastructure to the Cloud

CLOSER 2015

22© Santiago Gómez Sáez

Research

Agenda

Motivation The OPAL Simulation Environment Experiments

Methodology & Setup Evaluation Results

Conclusion & Future Work

33© Santiago Gómez Sáez

Research

Motivation – Simulation Workflows

Workflows comprise set of tasks by means of defining their control-flow data-flow dependencies

Automated & flexible execution of simulation-based experiments Long-running and irregular executions Often comprise data provisioning tasks & complex calculations Wide amount of resources during execution

(1) SimTech Cluster of Excellence: http://www.iaas.uni-stuttgart.de/forschung/projects/simtech/index.php

44© Santiago Gómez Sáez

Research

Motivation (2) – VM Selection Alternatives

OPAL_SImulation: Sim_Workflow

Apache_Tomcat:Servlet_Container

Simulation_Service:

Web_Service

MySQL: Engine_and_Auditing_DB

AWS_EC2_m3.large: AWS_EC2

Ubuntu13.0:Virt_Linux_OS

interacts-with

(2) Andrikopoulos et al.: Optimal Distribution of Applications in the Cloud. In: Proceedings of CAiSE’14

Apache_ODE:WF_Engine

interacts-with

AWS_EC2_t2.micro: AWS_EC2

AWS_EC2_c3.large: AWS_EC2

AWS_EC2_r3.large:

AWS_EC2

Auditing_Service:

Web_Service

interacts-with ActiveMQ:Message_Broker

alt_hosted_on application specific node application non-specific node alternative nodehosted_on

interacts-with

55© Santiago Gómez Sáez

Research

VMi

$

performance

reqi

Motivation (3) – Multi-dimensional Analysis

66© Santiago Gómez Sáez

Research

OPAL Simulation Environment

KMC-Simulation for Solid Bodies Thermal aging of copper-alloyed steel on an atomistic scale Simulation workflow orchestrates Fortran-based OPAL simulation

services

(3) Sonntag et al.: Workflow-based Distributed Environment for Legacy Simulation Applications. In: Proceedings of ICSOFT’11

77© Santiago Gómez Sáez

Research

Experiments – Methodology & Setup

Performance & monetary cost trade-off Impact on outsourcing the OPAL Simulation

Environment to the Cloud (IaaS) Evaluate different IaaS VM instances types

Micro General Purpose Compute Optimized Memory Optimized

Impact when scaling the load concurrent scientists (users) equal simulation requirements

88© Santiago Gómez Sáez

Research

Experiments – Methodology & Setup

Complete simulation stack hosted in one VM On-premise

in-house virtualized environment vs. Off-premise scenarios

Amazon EC2 Windows Azure Rackspace

10 concurrent users sending 10 random & uniformly distributed requests

JMeter 2.9 as load driver Measured latency (ms.) perceived by the end user (scientist) Extrapolated to 1K experiments for monetary cost analysis

99© Santiago Gómez Sáez

Research

Experiments – VM Instances Type & Prices Jan. 2015

European region (on-premise, AWS & Azure) US region (Rackspace)

1010© Santiago Gómez Sáez

Research

Experiments – VM Instances Type & Prices Jan. 2015

1111© Santiago Gómez Sáez

Research

Experimental Results (1) – AVG Latency/user

-12%+6%

+6.5%

1212© Santiago Gómez Sáez

Research

Experimental Results (2) – Cost for 1K experiments

-27% -6%-35%

-3% +9%+8%+60% +57%

+83%-22% -7%-31%

1313© Santiago Gómez Sáez

Research

Findings

Average latency reduced when using compute optimized instances

Increase monetary cost of 9% and 61% in average when using compute optimized and memory optimized VM instances

The monetary cost tends to increase when using Microsoft Azure optimized VM instances

Due to low costs of Rackspace IaaS services and the enhanced performance w.r.t. other scenarios, the total monetary cost is nearly 40% less in average

1414© Santiago Gómez Sáez

Research

Conclusion & Future Work

Report a performance & price analysis for migrating the OPAL Simulation Infrastructure to the Cloud using different IaaS providers and different VM types

Analysis of PaaS & DBaaS offerings Multi-cloud environment Use experimental results to assess application

developers in the (re-)distribution of their application components in Cloud environments

Santiago Gómez SáezE-mail: [email protected] of Architecture of Applications Systems (IAAS)University of Stuttgart (Germany)

15

Research

Thanks for your attention!!

16

Research

Backup slides

1717© Santiago Gómez Sáez

Research

Experiments – On-premise Pricing Calculation (1)

: acquisition cost: maintenance costY: number of years of the server clusterk: cost of the invested capitalTC:

TCPU: total number of CPU cores in the clusterH: expected number of operational hours: expected utilization

(4) Walker: The Real Cost of a CPU Hour. In: IEEE Computer, 42:35-41

1818© Santiago Gómez Sáez

Research

Experiments – On-premise Pricing Calculation Micro Instance

: 8500 U$: 7500 $/year, including personnel, cooling, power, etc.Y: 2.5 years oldk: assumed a cost of 5% on the invested capitalTC:

96153 CPU hoursTCPU: 16H: 6 days/week; 960K CPU hours/year: 80%

= 0.133 U$/h

(4) Walker: The Real Cost of a CPU Hour. In: IEEE Computer, 42:35-41