A practical guide to implementing Agile QA process on Agile Projects Agile QA Syed Rayhan Co-founder, Code71, Inc. Contact: [email protected] Blog: http://blog.syedrayhan.com Company: http://www.code71.com Product: http://www.scrumpad.com
Nov 28, 2014
A practical guide to implementing Agile QA process on Agile Projects
Agile QA
Syed RayhanCo-founder, Code71, Inc.Contact: [email protected]: http://blog.syedrayhan.comCompany: http://www.code71.comProduct: http://www.scrumpad.com
www.Code71.com www.ScrumPad.comCopyright 2010, Code71, Inc.
2
My Background
Expertise
Career
Iterative incremental development
Technology planning and architecture
On-shore/Off-shore software development using Agile/Scrum
Interests
Co-founder, Code71, Inc. 14+ years of total experience Co-author of “Enterprise Java with UML”
Cultural aspect of self-organizing team Scrum for projects delivered remotely Agile engineering practices
www.Code71.com www.ScrumPad.comCopyright 2010, Code71, Inc.
3
Agenda
Recap
Individual Practices
Section 2
Section 4
Section 3
A Case Study
Holistic View of QA
Section 5
IntroductionSection 1
Q&ASection 6
www.Code71.com www.ScrumPad.comCopyright 2010, Code71, Inc.
4
What to Expect
Focus
Context
Build common base of understanding
Develop a set of guidelines- process, roles, and team composition
Address typical questions asked
K ey Takeaways
How to perform QA on an Agile/Scrum project
Agile/QA best practices
Teams and organizations are adopting Agile/Scrum
Teams struggle with making the transition from waterfall to
Agile/Scrum
www.Code71.com www.ScrumPad.comCopyright 2010, Code71, Inc.
5
Agenda
Recap
Individual Practices
Section 2
Section 4
Section 3
A Case Study
Holistic View of QA
Section 5
IntroductionSection 1
Q&ASection 6
www.Code71.com www.ScrumPad.comCopyright 2010, Code71, Inc.
6
The challenges?
Is QA part of the development team?
Can we fit QA in the same iteration as development?
How can we scale Agile QA?
Who does QA?
Does QA costs more in Agile as product seems to change from sprint to sprint?
Do we need “test plan”?
Are story acceptance tests enough? When do we know testing is done?
Who defines test cases?
Do we need to track bugs?
www.Code71.com www.ScrumPad.comCopyright 2010, Code71, Inc.
7
What is QA (Quality Assurance)?
To ensure Software is working right
How?
Test, Test, Test
We will primarily focus on single-team model for our discussion
www.Code71.com www.ScrumPad.comCopyright 2010, Code71, Inc.
8
Types of Testing?
Unit Testing
Integration Testing
Functional Testing
System Testing
Regression Testing
Acceptance Testing
Load Testing
Smoke Testing
Black Box
White Box
www.Code71.com www.ScrumPad.comCopyright 2010, Code71, Inc.
9
Who Performs What?
Unit Testing
Integration Testing
System Testing
Regression Testing
Acceptance Testing
Load Testing
Smoke Testing
What? Who? When? Automation?
Developer
Developer
Tester
Developer/Tester
Client/Users
Tester/Support Engineer
Performance Engineer
Coding
Coding
Test
Build/Test
Deployment/Delivery
Deployment
Deployment
Always
Always
Possible
Possible
Possible
Possible
Always
www.Code71.com www.ScrumPad.comCopyright 2010, Code71, Inc.
10
Right tools for right tests?
Unit Testing
Integration Testing
System Testing
Regression Testing
Acceptance Testing
Load Testing
Smoke Testing
Test Tool NUnit, JUnit, Mock, DBUnit
Unit test tools, HttpUnit, SoapUI, RESTClient
JMeter, Httperf
FIT, FitNesse
Selenium, Fit, WET, Watir, WatiN
Unit test tools, System test tools
Regression test tools
www.Code71.com www.ScrumPad.comCopyright 2010, Code71, Inc.
11
What is missing?
Assumptions
1. Right Spec
2. Right Design
Reality?
Measures
Spec review
Design review
Spec
Design
Code reviewCode
3. Right amount of Tests
Test CoverageTest
Assumptions are farther from truth
4. Right Tests
www.Code71.com www.ScrumPad.comCopyright 2010, Code71, Inc.
12
Team Composition?
1Dev
Test
2 3
1 2 3
iterationregion
Prod 1 2 …
1Dev
Test
2 3
1 2 3
iterationregion
Prod 1 2 …
Out-of-Cycle
In-Cycle
2
1
Integrated Team
Separate Team
Developer to tester ratio?
www.Code71.com www.ScrumPad.comCopyright 2010, Code71, Inc.
13
Quality Funnel
QG#4QG#3QG#2QG#1 QG#5
bugs
bugs
BacklogReview
DesignReview
Unit & Int.Test,
CI
CodeReview
SystemTest
www.Code71.com www.ScrumPad.comCopyright 2010, Code71, Inc.
14
Agenda
Recap
Individual Practices
Section 2
Section 4
Section 3
A Case Study
Holistic View of QA
Section 5
IntroductionSection 1
Q&ASection 6
www.Code71.com www.ScrumPad.comCopyright 2010, Code71, Inc.
15
Test Coverage
Tests coverage metrics can tell you what code is not tested
“A measure of the proportion of a program exercised by a test suite, usually expressed as a percentage.”
• Function coverage
• Path coverage
• Statement coverage
Types of coverage
Definition
Measure Usually expressed as a percentage
www.Code71.com www.ScrumPad.comCopyright 2010, Code71, Inc.
16
Continuous Integration as the Glue
Monitor Source Code
AutomatedBuild
RegressionTest
Test Coverage
Report
Continuous Integration is Continuous QA
www.Code71.com www.ScrumPad.comCopyright 2010, Code71, Inc.
17
Test Case Prioritization
high
high
low
low
1
2
Frequency of use
Risk of having bugs
3
4
hot
warm
normal
cold
www.Code71.com www.ScrumPad.comCopyright 2010, Code71, Inc.
18
“Inspect and Adapt” through QA Lens
Five “whys” of root cause analysis
Prioritize bugs over stories
Log bugs found by testersDiscover
Triage & fix
Prevent
www.Code71.com www.ScrumPad.comCopyright 2010, Code71, Inc.
19
Definition of “Done”
Is QA part of your definition of “Done?”
www.Code71.com www.ScrumPad.comCopyright 2010, Code71, Inc.
20
Tracking Quality
Quality Metrics
Defect Rate Bug count per iteration
Defect Density Bug count per module Bug count per function point
What else should we track?
www.Code71.com www.ScrumPad.comCopyright 2010, Code71, Inc.
21
Technical Debt & QA
“Aged” open bugs can contribute to increasing “technical debt”
“Reoccurring” bugs may indicate hidden “technical debt”
www.Code71.com www.ScrumPad.comCopyright 2010, Code71, Inc.
22
Agenda
Recap
Individual Practices
Section 2
Section 4
Section 3
A Case Study
Holistic View of QA
Section 5
IntroductionSection 1
Q&ASection 6
www.Code71.com www.ScrumPad.comCopyright 2010, Code71, Inc.
23
A Case Study
ProjectA large enterprise system that includes technologies like ASP.Net, BizTalk, Workflow, Scanning, SQL Server, Data Warehouse, and Mainframe
Team2 product owners, 1 scrum master, 1 architect, 5 developers, 1 QA tester
Sprint 2 weeks
QA Process Refine scope, acceptance test cases
Identify UI elements, test dataQA schedule
Write test scriptstest data
Day 1, 2 Day 3, 4 Day 5-8 Day 9 Day 10
FinalTest&Fix
Demo&Acceptance Test
www.Code71.com www.ScrumPad.comCopyright 2010, Code71, Inc.
24
Agenda
Recap
Individual Practices
Section 2
Section 4
Section 3
A Case Study
Holistic View of QA
Section 5
IntroductionSection 1
Q&ASection 6
www.Code71.com www.ScrumPad.comCopyright 2010, Code71, Inc.
25
Recap “In-cycle QA” or “Integrated Team” is critical to the success of a project
System testing is not the only “quality gate,” it includes all types of testing and
reviews
Test automation is critical to “in-cycle” QA
Target at least 90% test coverage
“Continuous Integration” is “Continuous QA”
Prioritize test cases based on risk and frequency of usage
www.Code71.com www.ScrumPad.comCopyright 2010, Code71, Inc.
26
Recap contd.
All known bugs should be fixed first
Right size story with well-thought out acceptance tests improves quality
Include all “Quality Gates” as part of definition of “Done”
Analyze each bug to understand where (Quality Gate) it should have been
caught and improve (Inspect and adapt)
QA is not a designated person’s responsibility, it is a team’s responsibility
(self-organizing team)
www.Code71.com www.ScrumPad.comCopyright 2010, Code71, Inc.
27
Q&A
“QA is not an act, but a habit”
“QA is making sure right software works right”
Please contact for on-site training or Webinar:
Contact: [email protected]: http://blog.syedrayhan.comCompany: http://www.code71.comProduct: http://www.scrumpad.com