Devoxx 2016 Using Jenkins, Gerrit and Spark for Continuous Delivery Analytics

@gitenterprise#DevoxxPL

Platinum Sponsors:

Speed-up your Continuous Delivery

with BigData

Luca Milanesio

GerritForge

@gitenterprise#DevoxxPL

• Founded in the UK

• HQ in London, Offices in

the UK and Italy

• Committed to

OpenSource

About GerritForge

@gitenterprise#DevoxxPL

• Co-founder and

Director of GerritForge

• over 20 years in Agile

Development and ALM

• Contributor to many

OpenSource projects

• Mind and hands of

@gitenterprise and

@gerritreview

About Luca Milanesio

@gitenterprise#DevoxxPL

NOT a lecture on Continuous Delivery

NOT a lecture on Code Review

NOT a lecture on BigData

What NOT to expect?

@gitenterprise#DevoxxPL

SO WHAT?#!@$!

@gitenterprise#DevoxxPL

SHOWa scenario of Continuous Delivery

@gitenterprise#DevoxxPL

LEARN together about the problems

@gitenterprise#DevoxxPL

EXPERIMENT my approach leveraging BigData

@gitenterprise#DevoxxPL

DISCUSS and evolve the approach together

@gitenterprise#DevoxxPL

Continuous Delivery Pipeline

Push Build

Unit Tests

Push Build

Unit Tests

Automated

Tests

Push Build

Unit Tests

Automated

Tests

Beta Users

Tests

User's feedback Publish to all Users

Failed

Failed

@gitenterprise#DevoxxPL

Code Review & CD Validation

A1

refs/for/master

master

C2 C2*

A1 C2(local)

masterC2*

push

Verified: -1

(build failed)

push

Verified: +1

(build OK)

Reviewed: +2

CI Build: OK

C2*

CI Build: OK

amend

@gitenterprise#DevoxxPL

Git/Gerrit(Code Review)

Jenkins (Continuous

Integration & Delivery)

Docker(Release and Service

Packaging)

Mesos &

Marathon(Delivery and Resource

Manangement)

Components

@gitenterprise#DevoxxPL

DEMO

See the pipeline in action

@gitenterprise#DevoxxPL

We use micro-services

• 100+ Components

• 200+ Repositories

• 300+ People and 100,000+ of review comments

• Cross-components [implicit] dependencies

• Lots of build chains

• 10,000+ of builds

Now feel the pain :-(

@gitenterprise#DevoxxPL

"It was working until 5

mins ago … then

someone changed

something"

Rings any

bell?

Problem #1 : Ripple effect

@gitenterprise#DevoxxPL

In order to deliver Feature ABC

I NEED

Ver. N of Service-A with

Ver. M of Service-B and

Ver. K of Service C

Problem #2 : Dependencies

@gitenterprise#DevoxxPL

"Where can I find an environment

to

run ALL my Micro-Services

integration tests?"

Problem #3 : Resources

@gitenterprise#DevoxxPL

Release small changes is GOOD

Release often every change is GOOD

BUTHow long it takes the "Big Picture" and End-to-End?

Where is my bottleneck?

Problem #4 : Cycle time

@gitenterprise#DevoxxPL

Micro-Services are

distributed

Builds are distributed

Deployments are

distributed

Tackling THIS

have you ever felt

like drowning in

your logs?

Problem #5 : Lots of logs

@gitenterprise#DevoxxPL

Problems #1 + #2 + #3 + #4 + #5 =

lead to your daily

Micro-Services mental meltdown

Problem #6 : all together

@gitenterprise#DevoxxPL

• Collect all review events

• Collect all logs

• Channel them to a

central store

• Crunch and Crunch

continuously

• Never delete

• Process, inspect and

learn

BigData to the rescue

@gitenterprise#DevoxxPL

DEMO

Continuous Delivery Analytics

@gitenterprise#DevoxxPL

•Problem #1 – Ripple effect?

•Problem #2 – Dependencies?

•Problem #3 – Resources?

•Problem #4 – Cycle Time?

•Problem #5 – Logs aggregation?

•Problem #6 – Avoid mental meltdown?

Did we solve the problems?

@gitenterprise#DevoxxPL

• Collect more data (System, JVM, Review

scores)

• Real-time statistics

• Events correlation

• Feedback loop to Code-Review

• Production data harvest

• More?

Future directions

@gitenterprise#DevoxxPL

• My blog

www.gitenterprise.me

• Gerrit Code Review

www.gerritcodereview.co

m

• Jenkins

www.jenkins.io

• RabbitMQ

rabbitmq.com

• Apache Flume

flume.apache.org

• Hadoop HDFS

hadoop.apache.org

• Apache Spark

spark.apache.org

• Kibana

github.com/elastic/kibana

Resources