A Test Pyramid Heresy - a fresh look at test automation strategies
Post on 28-Jan-2018
213 Views
Preview:
Transcript
@wakaleo 1
A fresh look at test automation strategiesThe Testing Pyramid Heresy
John Ferguson Smart http://johnfergusonsmart.com
@wakaleo
John Ferguson
Smart
John Ferguson Smart
“I help teams of smart people learn to work together more efficiently,
to deliver better software sooner”
2
https://johnfergusonsmart.com
https://johnfergusonsmart.com
“Test Automation is like any other tool It’s either a benefit or a hazard”
3
@wakaleo
https://johnfergusonsmart.com
How much are your tests worth?
4
Weekly cost vs
weekly savings$0
$20,000
$40,000
$60,000
$80,000
$100,000
$120,000
$140,000
$160,000
$180,000
1/09/2015 1/10/2015 1/11/2015 1/12/2015
Test0suite0cost0vs0savings0(per0week)
Cost0saved0par0week Test0suite0cost0per0week
Team cost is constant
Time saved increases with the number of executed tests
@wakaleo
https://johnfergusonsmart.com
How much are your tests worth?
5
$0
$200,000
$400,000
$600,000
$800,000
$1,000,000
$1,200,000
1/09/2015 1/10/2015 1/11/2015 1/12/2015
What0is0your0test0suite0worth?
Test0suite0ROI Test0suite0cost
Cumulated cost vs
cumulated savings
Cumulated value should increase faster than cumulated cost
@wakaleo
https://johnfergusonsmart.com
Now factor in high maintenance
6
20-40%Typical maintenance overhead for poorly-written test suites
@wakaleo
https://johnfergusonsmart.com
Now factor in high maintenance
7
Number of tests
Main
tena
nce
over
head
@wakaleo
https://johnfergusonsmart.com
$0$50,000
$100,000$150,000$200,000$250,000$300,000$350,000$400,000$450,000$500,000
1/09/2015 1/10/2015 1/11/2015 1/12/2015 1/01/2016
What0is0your0test0suite0worth?
Test0suite0cost Test0suite0ROI
Now factor in high maintenance
8
High-maintenance costs reduce the number of tests a team can write
High maintenance costs can cause your test suite to lose value
@wakaleo
https://johnfergusonsmart.com
Enter the Testing Pyramid
9
@wakaleo
https://johnfergusonsmart.com
…or the Testing Ice-cream cone
10
@wakaleo
https://johnfergusonsmart.com
In other words…
11
Don’t use a web test where a service or unit test could do the same job.
12
13
“All models are wrong, but some are useful.”
- George Box
14
15
The Testing Pyramid is about how a test is implemented.
But is this the best way to classify our tests?
@wakaleo
Developer TestsQA TestsBusiness Tests
Tests come from different sources
16
@wakaleo
Developer TestsQA TestsBusiness Tests
Tests come from different sources
17
@wakaleo
Developer TestsQA TestsBusiness Tests
Tests come from different sources
18
@wakaleo
Developer TestsQA TestsBusiness Tests
Tests come from different sources
19
@wakaleo
DemonstrateDescribeDiscover
Tests have different roles
20
@wakaleo
DemonstrateDescribeDiscover
Tests have different roles
21
@wakaleo
https://johnfergusonsmart.com
Test (activities) that discover
22
Example Mapping
Impact Mapping
Feature Mapping
@wakaleo
DemonstrateDescribeDiscover
Tests have different roles
23
@wakaleo
https://johnfergusonsmart.com
Test (activities) that describe
24
@wakaleo
https://johnfergusonsmart.com
Test (activities) that describe
25
@wakaleo
DemonstrateDescribeDiscover
Tests have different roles
26
@wakaleo
https://johnfergusonsmart.com
Example and Feature Mapping
Tests have different roles
27
Dis
cove
r
Describe
Demonstrate
Impact Mapping
Executable specifications
Performance testsFunctional tests
Unit tests
Regression tests
28
A new testing quadrant
29
A new testing quadrant
30
A new testing quadrant
31
A new testing quadrant
32
A new testing quadrant
33
Goldilocks Test Suites
top related