1
WHAT I'VE LEARNED ABOUT TESTAUTOMATION AND DEVOPS
2
[~]$ whoamiFilipe Bartolomeu
Integration and Enterprise Architect
Program Manager
2
3 . 1
A SHORT STORY
HOW AUTOMATION WENT WRONG!
3 . 2
My Boss requested me to implement a Test AutomationCompetence to reduce Production Incidents...
3 . 3
AUTOMATE! AUTOMATE! AUTOMATE!
3 . 4
3 . 5
LATER... THAT WEEK
3 . 6
3 . 7
3 . 8
AND I THOUGHT IKNEW TEST
AUTOMATION!!!
3 . 9
JUST FIX IT!
4 . 1
ACT 2
HOW WE MANAGED TO TURN THINGSAROUND
And learned from it!
4 . 2
AUTOMATION IS NO SILVER BULLET!It won't start delivering at day zer0It's no guarantee of qualityIt won't replace testers
4 . 3
WHAT WAS THEPROBLEM?
4 . 4
4 . 5
4 . 6
But.. Unit Tests aren’t the Holy Grail...
Unit tests are not related to user/business requirementsDevelopers just want to prove that the code works!
4 . 7
WE HAD TO CLIMB THELADDER!
4 . 8
4 . 9
With the help from the OPS team we devised a flexible testenvironment
Routing & Virtualization
4 . 10
4 . 11
ARE WE DONE YET?
4 . 12
THE PROBLEM IS THE NOT THE TECHNICALSTUFF!
Bussiness Logic? Specification? Test Conditions? Undestandthe Requirements! Environment...
4 . 13
CAN YOU GUESS THENEXT TEAM WE TALKED
TO?
4 . 14
4 . 15
All the applications that require integrated data for testingwill profit.
#1 Create a test data generation process
4 . 16
#2 Review and automate the manual regresion Test Set.
Shi the saved time for new features testing.
4 . 17
#3 THE AUTOMATION FRAMEWORK SHOULDALLOW THE QUALITY TEAM:
(1) Allow testers to automate tests without having to code(Keyword-driven FW)
(2) Performing data-driven tests (using the same test withdiverse input data)
(3) Easy mantainance by using an object repository (tostore object properties)
4 . 18
Test Framework Design Approaches
KEYWORD DRIVEN+DATA DRIVEN
Readability; Reuse; Flexibility; Loosely Coupled
4 . 18
4 . 19
EVEN FURTHER...
4 . 20
PAIRWISE TESTING
Exaustive Testing: 2^9 * 5 = 2.560 tests!
Pairwize test generation: 12 tests
4 . 21
4 . 22
TEAM
4 . 23
4 . 24
SOFTWARE ENGINEER VS TESTER
5 . 1
TEST AUTOMATION
NEXT STEPS
5 . 2
SHIFT LEFT SHOULD BE BY NOW A REALITY
Begin Testing Activities as soon as the dirst dra arrives!
5 . 3
BUT A NEW 'SHIFT-RIGHT' IS EMERGING
Shi -Right: Test in Production
A/B Testing, Feature Flags, Actionable Analytics, Bug FindingChallenge
5 . 4
TRYING TO BOOK MY HOLIDAYS...
5 . 5
SPOT THE DIFFERENCE
SAME PAGE, DIFFERENT BROWSER
5 . 6
VISUAL VALIDATION TESTING
Test Scope: WebBrowsers, Devices, OSs,Screen resolutions...
6
QUESTIONS?
