Agile Defect ManagementFocus On Prevention
David JellisonDirector, Quality Engineering
[email protected]: davidjellison
Twitter: davidjellison
You Will Learn Categorizing defects drives defect context
management Testing is part of delivering a story or feature Detecting and resolving defects as the work is
delivered prevents backlog Attaching an in-progress defect to a current
story encapsulates the defect in the current work
Escaping defects become the exception rather than the norm
2
Twitter: davidjellison
The One Finding The Most Bugs Wins!
Lord of the Rings: Legolas & Gimli counting kills
Downstream Testing Approach
Twitter: davidjellison
Preventing Bugs is Craftsmanship
upstream Testing Approach
Twitter: davidjellison
Quest for Zero Defects You can only manage what is known
Crosby’s “Zero Defect” standard manages to requirements (http://en.wikipedia.org/wiki/Zero_Defects)
Agile changes the game, managing business value to the customer frequently Need a pattern to manage defects that we
inevitably discover during and after release
Never really reach 0 Defects
5
Defect Discovery
Moving Defect Discovery Earlier
6
Twitter: davidjellison
Traditional Defect Discovery Trends
7
Twitter: davidjellison
Late Defect Discovery Costs Opportunity cost
Not working on other things Developer Context switching
Time to get acclimated and then back again Rebuild and test time
More cycles to deliver the fix Damage control
Support time and reputation Other hidden costs…
8
Twitter: davidjellison
Traditional Defect Discovery Trends
Defect debt accumulation
Delay in resolution
Defects resolved
Dev/QE aligned
9
Twitter: davidjellison
Defect Discovery Improvement Plan Stop defect debt accumulation
Move defect discovery up in the development cycle Embed QE into teams with Dev/QE alignment Prevent defects through design review & test
automation Write failing regression tests as defects are discovered
Test in sandbox Don’t wait for final integration environment to test Continuous build and continuous integration patterns
Whole-team responsibility for quality Clear acceptance criteria for all new work Everyone tests Automated test code reviews with developer
10
Defect Prevention
Automating Along The Way
11
Twitter: davidjellison
Defect Discovery Late
Downstream test teams focus on defect discovery
12
Twitter: davidjellison
Defect Discovery Early
Embedded test teams focus on defect prevention
Regression tests assure no system failures
13
Twitter: davidjellison
Agile Practices Smaller
cycles Collaboration Design
reviews Test early and
often Test Driven
Development Continuous
Integration Automated
Regression
Scrum
Kanban
14
Twitter: davidjellison
Automated Regression Tests
UnattendedTest Runs
AvailableTest Time
15
Defect Categorization
Introducing WIP Defects
16
Twitter: davidjellison
Categorizing Defects Escaping Defects
Discovered defects allowed into the field Undiscovered defects found in the field
17
Twitter: davidjellison
Categorizing Defects WIP (work in progress) Defects
Defects discovered in new code not yet delivered to field
Becomes additional acceptance criteria for delivery
Treat as subtasks of feature work Caged by the parent work in progress
18
Twitter: davidjellison
Managing Defect Discovery Its OK to not write
defects if there are failing automated tests to manage
Promote failing tests to WIP defects if need to manage separately
Promote WIP defects that are still failing at time of delivery to escaping defects and added to the backlog
19
Focus on Prevention Develop failing
regression tests when discovered manually
Continuous feedback through failing test run reports
Associate WIP defects with feature work in progress
20
Managing Escaping Defects Delivering business
value over no defects
No unknown failures in regression tests
Coordinate planned escaping defects with customer support
Reduce escaping defects to less than you can count on two hands21
Twitter: davidjellison
Managing Escaping Defects
“Jellison Jazz Hands”
Twitter: davidjellison
Driving escaping defects down
23 Number s for illustration only
Twitter: davidjellison
Escaping Defect Reduction Plan Set expectation in the teams that fixing WIP
defects is part of the acceptance criteria Add escaping defects in the field to the
backlog and rank along with other work Plan to fix some number of escaping defects
from the backlog in each development cycle Treat escaping defects as the exception rather
than the norm
24
Examples
Using WIP Defects
25
Twitter: davidjellison
Example: WIP Defects/Escaping Defects Shifting the focus to WIP defects results in far
fewer escaping defects in the field, even with increased discovery
26
WIP Defects [blue]Escaping Defects [red]
Twitter: davidjellison
Example: WIP Defects/Escaping Defects Each team may be in a different place with
business risk and can manage escaping defects differently
WIP Defects [ red & blue]Escaping Defects [green]
27
Develop a theme strategy
Twitter: davidjellison
What We Covered Categorizing defects drives defect context
management Testing is part of delivering a story or feature Detecting and resolving defects as the work is
delivered prevents backlog Attaching an in-progress defect to a current
story encapsulates the defect in the current work
Escaping defects become the exception rather than the norm
28
Twitter: davidjellison
Contact: David JellisonDir, QEConstant Contact, Inc.1601 Trapelo RoadWaltham, MA [email protected]: 339-222-5712
Blog: http://davidjellison.wordpress.com Twitter: davidjellison http://ConstantContact.com/Careers
29