A Reference Platform for Software Performance Engineering in ...

A Reference Platform for

Software Performance

Engineering in DevOps

Teerat Pitakrat Jonas Heinisch

“DevOps is a set of practices intended to reduce the time between committing a change to a system and the change being placed into normal production,[…]”

Bass, Zhu, and Weber (2015)

2

Deployments at Amazon (2011)

• Mean time between deployments (weekday) 11.6s

• Max deployments per hour 1,079 • Mean number of hosts simultaneously

receiving a deployment 10,000 • Max number of hosts simultaneously

receiving a deployment 30,000 3 DevOps, Continuous Integration and Deployment on AWS , AWS Summit, 2015

Infrastructure Technology Requirement Architecture Development Build Testing Deployment Execution

4

Software Performance Engineering Approaches

Requirement Architecture Development Build Testing Deployment Execution

Performance Regression

Testing

Capacity Planning

Online Resource

Management

Performance Monitoring

Design-time Performance Prediction

Design-Space Exploration

(Optimization)

Performance Model

Extraction

Performance Problem Diagnosis

Profiling

Load Testing

Performance Unit Testing

SLA Negotiation

5

Software Performance Engineering Approaches

Requirement Architecture Development Build Testing Deployment Execution

Performance Regression

Testing

Capacity Planning

Online Resource

Management

Performance Monitoring

Design-time Performance Prediction

Design-Space Exploration

(Optimization)

Performance Model

Extraction

Performance Problem Diagnosis

Profiling

Load Testing

Performance Unit Testing

SLA Negotiation

ContiPerf

6

Acknowledgement

This work is a part of the SPEC RG DevOps working group’s blueprint for performance-aware DevOps http://research.spec.org/devopswg/

7

Objective

• Showcase/benchmarking environment for software performance engineering in DevOps – Ready-to-use – Community-driven – Extensible – Reproducible – Representative for real-world scenarios – Based on open standards/technologies – Provides performance-aware CI/CD pipeline – Provides sample applications

8

Infrastructure Technology Requirement Architecture Development Build Testing Deployment Execution

9

Fabric8 • Open source integrated developer platform by Red Hat • Provides continuous delivery pipeline by:

– Combining existing tools

– Providing an integration layer and UI • Based on

– Docker

– Kubernetes

10

Docker • Open source container technology • Runs on any Linux machines (kernel 2.6.32+) • Lightweight • Fast startup time

Example (Redis server): • Create Dockerfile

• Build • Run

$ docker run –d redis

FROM ubuntu:14.04

RUN apt-get update && apt-get install -y redis-server

EXPOSE 6379

ENTRYPOINT ["/usr/bin/redis-server"]

$ docker build –t redis .

11

12

Kubernetes

• Open source container cluster manager • Provides

– Load balancing – Service discovery – Fault tolerance – Horizontal scaling – Rolling update

13

Fabric8 Technology Stack Requirement Architecture Development Build Testing Deployment Execution

14

Fabric8 Demo

15

Reference Platform for SPE

Physical / virtual machines

Operating system

RSS reader JPetStore MediaStore DVDStore CoCoME SPECjEnterprise

Requirement Architecture Development Build Testing Deployment Execution

16

How to Add Performance-aware Services

1. Package service as a Docker image

2. Create yaml file to describe the deployment

3. Deploy service to Kubernetes

apiVersion: extensions/v1beta1

kind: Deployment

metadata:

name: kieker-monitoring-server

spec:

replicas: 1

template:

metadata:

labels:

app: kieker

spec:

containers:

- name: kieker

image: kieker:1.12

ports:

- containerPort: 80

kieker-monitoring-server.yaml 17

How to Add Services into Pipeline

• Extend Fabric8 (Jenkins) pipeline

18

Example Use Cases

19

Modeling Testing Online Performance Management

Monitoring

Applications

OPEN.xtrace

Problem Diagnosis Model Extraction Online Failure Prediction

Summary Goals • Reference platform for software performance engineering in

DevOps • Extensible performance-aware CI/CD pipeline • Reproducible environment for research

Current state • Migrating SPE tools/services to Docker/Kubernetes • Extending Fabric8 pipeline • Migrating sample applications to Fabric8

Source code • https://github.com/spec-rgdevops/

20