Road 2 devops at Stater

Road to DevOps @ Stater

Pascal Dufour(Agile test consultant)

www.stater.nl

2

Marketshare

3

• Market share at the start of 2014: 30%

Marketshare

4

• Market share at the start of 2014: 30%• Market share Q1 2015: 40%

Marketshare

5

• Market share at the start of 2014: 30%• Market share Q1 2015: 40%• Goal: 70 % market share by the end of 2018

Innovation

66

E-Consument

Financieel akkoordSnel zekerheid voor de

consument

Accello

EXECUTION ONLY

Bouwdepot

Mijn gegevens

Bouwdepot App

E-Notaris

3 pilars for growth

7

70% market share

New customers

Expanding BPO offering(mid office) Innovation

(digitization)

Our IT landscape

8

Our challenges

•Unsatisfied customers

•Unstable testing environments

•Painful releases

•System outage

• Incidents

Our plan

10

CD pipelineScrumDevops

Delivery was “old school”

The answer: Continuous Delivery

OntwikkelOmgeving

TestOmgeving

AcceptatieOmgeving

ProductieOmgeving

Automated provisioning

Agile softwaredevelopment

Continuous integration Automated testing

Automated deployment

13

CD Pipeline

14

XL testview

Automated flows for releases

16

Paradigm shift

17

• Release more frequently

• Use tools that best fit the job

• Fewer code bases

• Automate repetitive tasks

Agile software development

Where to start with Agile:

. You are here

19

It’s a project:

20

Scrum

21http://blog.manageagile.com/2012/06/finding-right-team.html

26 agile teams

22

TAAS- Test framework- Coaching- Test automation

Added Operation engineers

23

TAAS- Test framework- Coaching- Test automation

TAAS24

Taas environment

Ontw.

Test.

Acc.

Prod.

Ontw.

Ontw.

costumer

XLtest

NEW

business

Maturity matrix

Maturity

25

LEVEL          

5 Agile Cultuur Zorgen dat agile mindset onderdeel van cultuur blijft door een levendige omgeving te creëren.

4 Adaptief Reageren op wijzigingen door meerdere niveaus van feedback

3 Effectief Ontwikkelen van software op een effectieve manier

2 Evolutionair Opleveren van software eerder en meer frequent

1 Samenwerking Communicatie en samenwerking vergroten

translate

Maturity matrix

26

Maturity matrix => actions

27

Go live test

Exploratory test

Performancetest

batches

Performancetest

online

Regressiontest

(auto)

Regressiontest

(handmatig)

Acceptancetest

(manual)

Acceptancetest (auto)

Integratietest

Unittest

Intaketest

Product

SubsystemSyst

em

Component

files

Team D

Team B

Team C

Smoketest

Maturity matrix => actions

Product

SubsystemSyst

em

Component

files

Team DTeam CTeam B

28

Go live test

Exploratory test

Performancetest

batches

Performancetest

online

Regressiontest

(auto)

Regressiontest

(handmatig)

Acceptancetest

(manual)

Acceptancetest (auto)

Integratietest

Unittest

IntaketestSmoketest

Testing example

The User Acceptance Test in a DayAutomation • Workshops to transition from manual testing to Living Documentation.• Use of Persona• External help to automate• Run in CI• Run in CD

29

Stater - 3 minuten demo30

Manual testing

Regression testing on the acceptance environment

AutomatischeChecks

REGRESSIE TESTEN OP ACCEPTATIE OMGEVING

HandmatigeTesten

OperationsAt the driver seat

Automation of UAT

Stater - 3 minuten demo31

Dev Test Acceptance Production

ManualTestingAutomatic

Checks

ManualTesting

AutomaticChecks

ManualTesting

Automation current state

Succes

32

First test in the continuous delivery

pipelineTook us

10 weeks

Setup. more setup. more setup more, more setup, almost there, more more setuptest Lening in beheer

Stater - 3 minuten demo33

AutomatischeChecks

Regression on Test environment

ManualTesting

STAP 1

Flakiness

0

What are we testing?

34

Visualise!

Build monitors for real time insights

35

Build monitors for real time insights

36

Build monitors for real time insights

37

Bereikt de afgelopen 2 sprints

Stater - 3 minuten demo38

Test Automation ‘praat[laat’

Sprint 0 Sprint 1 Sprint 2 Sprint 3

KickoffPilot

TA Uitleg

Refinement + RetroPilot

RetroPilot

Refinement

Development

Review en Fitnesse scenario

support

Sprint n

Pilo

t te

amTA

AS

SMDev

Team + business

AgileIntake

insp

anni

ngin

span

ning Regelt samen

met Agile coach intake

TAcoach

presentatie plan

TAcoachUitleg

DevTeam /

Dev / test

DevTeam +

Business +

initi

ator

Agile coach

SM / POSMSM / TA

coach

SM / TA

coach

Bouw US+ Automatisering

scenario

TAcoach

SelectUser story

Support op aanvraagTAcoach

TAcoach

DevTeam +

Business +

TAcoach

DevTeam

DevTeam +

Business +

SM SM

DevTeam +

Business +

TAcoach

TAcoach

SM SM

Refinement +

Review en Fitnesse scenario

User story+ Automatisering

scenario

TA adoptie

DevTeam + business

Speed…

41

Deliver test environment from 1-2 weeks to 1 hour

Provisioning of servers from 1-2 days to 30 minutes

Deploying software from 1-2 weeks to 30 minutes

Integration from 2 months to 30 minutes

Lessons learned for Stater

42

• Everyone training• Wait until someone wants to start• One coach• Community of practice with “junior”• Bottom up approach• Ownership of Test automation in one team• If time permits the team can pick it up• Wait to be asked to try automation• Automation is not easy• Design patterns also apply for test code• “Real“ developer needed• Agile mindset is not easy

What works for Stater

43

• Pilot teams “3 sprints”• Open community• Walk in sessions• Training while being coached• Discuss guidelines• Sonar• Development skills needed• Place improvement on the backlog (20%)• Start with a picture• Don’t tell them how to automate, explain what automation can do for you.• Test automation is very useable to accelerate your manual testing• Business/OPS at the driver seat• Checks for the CD pipeline• Support by management

VISIBILITY

The challenges…

• Extending the test coverage of automated test• More teams using test automation.• More ATDD • Not all testers are developers.• Collaborative code ownership• OPS in the agile teams

Next steps…

Question?

45