What i've learned about test automation and DevOps

1

WHAT I'VE LEARNED ABOUT TESTAUTOMATION AND DEVOPS

2

[~]$ whoamiFilipe Bartolomeu

Integration and Enterprise Architect

Program Manager

2

3 . 1

A SHORT STORY

HOW AUTOMATION WENT WRONG!

3 . 2

My Boss requested me to implement a Test AutomationCompetence to reduce Production Incidents...

3 . 3

AUTOMATE! AUTOMATE! AUTOMATE!

3 . 4

3 . 5

LATER... THAT WEEK

3 . 6

3 . 7

3 . 8

AND I THOUGHT IKNEW TEST

AUTOMATION!!!

3 . 9

JUST FIX IT!

4 . 1

ACT 2

HOW WE MANAGED TO TURN THINGSAROUND

And learned from it!

4 . 2

AUTOMATION IS NO SILVER BULLET!It won't start delivering at day zer0It's no guarantee of qualityIt won't replace testers

4 . 3

WHAT WAS THEPROBLEM?

4 . 4

4 . 5

4 . 6

But.. Unit Tests aren’t the Holy Grail...

Unit tests are not related to user/business requirementsDevelopers just want to prove that the code works!

4 . 7

WE HAD TO CLIMB THELADDER!

4 . 8

4 . 9

With the help from the OPS team we devised a flexible testenvironment

Routing & Virtualization

4 . 10

4 . 11

ARE WE DONE YET?

4 . 12

THE PROBLEM IS THE NOT THE TECHNICALSTUFF!

Bussiness Logic? Specification? Test Conditions? Undestandthe Requirements! Environment...

4 . 13

CAN YOU GUESS THENEXT TEAM WE TALKED

TO?

4 . 14

4 . 15

All the applications that require integrated data for testingwill profit.

#1 Create a test data generation process

4 . 16

#2 Review and automate the manual regresion Test Set.

Shi the saved time for new features testing.

4 . 17

#3 THE AUTOMATION FRAMEWORK SHOULDALLOW THE QUALITY TEAM:

(1) Allow testers to automate tests without having to code(Keyword-driven FW)

(2) Performing data-driven tests (using the same test withdiverse input data)

(3) Easy mantainance by using an object repository (tostore object properties)

4 . 18

Test Framework Design Approaches

KEYWORD DRIVEN+DATA DRIVEN

Readability; Reuse; Flexibility; Loosely Coupled

4 . 18

4 . 19

EVEN FURTHER...

4 . 20

PAIRWISE TESTING

Exaustive Testing: 2^9 * 5 = 2.560 tests!

Pairwize test generation: 12 tests

4 . 21

4 . 22

TEAM

4 . 23

4 . 24

SOFTWARE ENGINEER VS TESTER

5 . 1

TEST AUTOMATION

NEXT STEPS

5 . 2

SHIFT LEFT SHOULD BE BY NOW A REALITY

Begin Testing Activities as soon as the dirst dra arrives!

5 . 3

BUT A NEW 'SHIFT-RIGHT' IS EMERGING

Shi -Right: Test in Production

A/B Testing, Feature Flags, Actionable Analytics, Bug FindingChallenge

5 . 4

TRYING TO BOOK MY HOLIDAYS...

5 . 5

SPOT THE DIFFERENCE

SAME PAGE, DIFFERENT BROWSER

5 . 6

VISUAL VALIDATION TESTING

Test Scope: WebBrowsers, Devices, OSs,Screen resolutions...

6

QUESTIONS?

7

THE END

Filipe Bartolomeu

[email protected]

http://filipebartolomeu.weebly.com/