A Practical Guide to implementing Agile QA process on Scrum Projects
Agile QA
Syed RayhanCo-founder, Code71, Inc.Contact: [email protected]: http://blog.syedrayhan.comCompany: http://www.code71.comProduct: http://www.scrumpad.com
2www.Code71.com www.ScrumPad.com
Copyright 2004-2008 Code71, Inc.
My Background
Expertise
Career
Iterative incremental development
Technology planning and architecture
On-shore/Off-shore software development using Agile/Scrum
Interes ts
Co-founder, Code71, Inc. 13+ years of total experience Co-author of “Enterprise Java with UML”
Cultural aspect of self-organizing team Scrum for projects delivered remotely Agile engineering practices
3www.Code71.com www.ScrumPad.com
Copyright 2004-2008 Code71, Inc.
Agenda
Recap
Individual Practices
Section 2
Section 4
Section 3
A Case Study
Holistic View of QA
Section 5
IntroductionSection 1
Q&ASection 6
4www.Code71.com www.ScrumPad.com
Copyright 2004-2008 Code71, Inc.
What to Expect
Focus
Context
Build common base of understanding
Develop a set of gudielines- process, roles, and team composition
Address typical questions asked
Key Takeaways
How to perform QA on an Agile/Scrum project
Agile/QA best practices
Teams and organizations are adopting Agile/Scrum
Teams struggle with making the transition from waterfall to
Agile/Scrum
5www.Code71.com www.ScrumPad.com
Copyright 2004-2008 Code71, Inc.
Agenda
Recap
Individual Practices
Section 2
Section 4
Section 3
A Case Study
Holistic View of QA
Section 5
IntroductionSection 1
Q&ASection 6
6www.Code71.com www.ScrumPad.com
Copyright 2004-2008 Code71, Inc.
The challenges?
Is QA part of the development team?
Can we fit QA in the same iteration as development?
How can we scale Agile QA?
Who does QA?
Does QA costs more in Agile as product seems to change from sprint to sprint?
Do we need “test plan”?
Are story acceptance tests enough? When do we know testing is done?
Who defines test cases?
Do we need to track bugs?
7www.Code71.com www.ScrumPad.com
Copyright 2004-2008 Code71, Inc.
What is QA (Quality Assurance)?
To ensure Software is working right
How?
Test, Test, Test
8www.Code71.com www.ScrumPad.com
Copyright 2004-2008 Code71, Inc.
Types of Testing?
Unit Testing
Integration Testing
Functional Testing
System Testing
Regression Testing
Acceptance Testing
Load Testing
Smoke Testing
Black Box
White Box
9www.Code71.com www.ScrumPad.com
Copyright 2004-2008 Code71, Inc.
Who Performs What?
Unit Testing
Integration Testing
System Testing
Regression Testing
Acceptance Testing
Load Testing
Smoke Testing
What? Who? When? Automation?
Developer
Developer
Tester
Developer/Tester
Client/Users
Tester/Support Engineer
Performance Engineer
Coding
Coding
Test
Build/Test
Deployment/Delivery
Deployment
Deployment
Always
Always
Possible
Possible
Possible
Possible
Always
10www.Code71.com www.ScrumPad.com
Copyright 2004-2008 Code71, Inc.
Right tools for right tests?
Unit Testing
Integration Testing
System Testing
Regression Testing
Acceptance Testing
Load Testing
Smoke Testing
Test Tool NUnit, JUnit, Mock, DBUnit
Unit test tools, HttpUnit, SoapUI, RESTClient
JMeter, Httperf
FIT, FitNesse
Selenium, Fit, WET, Watir, WatiN
Unit test tools, System test tools
Regression test tools
11www.Code71.com www.ScrumPad.com
Copyright 2004-2008 Code71, Inc.
What is missing?
Assumptions
1. Right Spec
2. Right Design
Reality?
Measures
Spec review
Design review
Spec
Design
Code reviewCode
3. Right amount of Tests
Test CoverageTest
Assumptions are farther from truth
4. Right Tests
12www.Code71.com www.ScrumPad.com
Copyright 2004-2008 Code71, Inc.
Team Composition?
We will primarily focus on single-team model for our discussion
1Dev
Test
2 3
1 2 3
iterationregion
Prod 1 2 …
1Dev
Test
2 3
1 2 3
iterationregion
Prod 1 2 …
Out-of-Cycle
In-Cycle
2
1
13www.Code71.com www.ScrumPad.com
Copyright 2004-2008 Code71, Inc.
Team Dynamics
Product owners
Developers
backlog
Acceptance tests
Additional tests
Team reviewTesters
14www.Code71.com www.ScrumPad.com
Copyright 2004-2008 Code71, Inc.
QG#4QG#3QG#2
Quality Funnel
QG#1 QG#5
bugs
bugs
BacklogReview
DesignReview
Unit & Int.Test,
CI
CodeReview
SystemTest
15www.Code71.com www.ScrumPad.com
Copyright 2004-2008 Code71, Inc.
Agenda
Recap
Individual Practices
Section 2
Section 4
Section 3
A Case Study
Holistic View of QA
Section 5
IntroductionSection 1
Q&ASection 6
16www.Code71.com www.ScrumPad.com
Copyright 2004-2008 Code71, Inc.
Test Coverage
Tests coverage metrics can tell you what code is not tested
“A measure of the proportion of a program exercised by a test suite, usually expressed as a percentage.”
• Function coverage
• Path coverage
• Statement coverage
Types of coverage
Definition
Measure Usually expressed as a percentage
17www.Code71.com www.ScrumPad.com
Copyright 2004-2008 Code71, Inc.
Continuous Integration as the Glue
Monitor Source Code Build Regression
TestTest
Coverage
Report
Continuous Integration is Continuous QA
18www.Code71.com www.ScrumPad.com
Copyright 2004-2008 Code71, Inc.
Test Case Prioritization
high
high
low
low
1
2
Frequency of use
Risk of having bugs
3
4
hot
warm
normal
cold
19www.Code71.com www.ScrumPad.com
Copyright 2004-2008 Code71, Inc.
Inspect and Adapt through QA Lens
Five whys of root cause analysis
Prioritize bugs over stories
Log bugs found by testers
Quality Metrics Defect Rate Bug count per iteration
Defect Density Bug count per module Bug count per function point
20www.Code71.com www.ScrumPad.com
Copyright 2004-2008 Code71, Inc.
How to Scale?
Presentation
Business
Data
Service#1 Service#2 Service#3
Application#1 Application#2
SOA
21www.Code71.com www.ScrumPad.com
Copyright 2004-2008 Code71, Inc.
Agenda
Recap
Individual Practices
Section 2
Section 4
Section 3
A Case Study
Holistic View of QA
Section 5
IntroductionSection 1
Q&ASection 6
22www.Code71.com www.ScrumPad.com
Copyright 2004-2008 Code71, Inc.
A Case Study
ProjectA large enterprise system that includes technologies like ASP.Net, BizTalk, Workflow, Scanning, SQL Server, Data Warehouse, and Mainframe
Team2 product owners, 1 scrum master, 1 architect, 5 developers, 1 QA tester
Sprint 2 weeks
QA Process Refine scope, acceptance test cases
Identify UI elements, test dataQA schedule
Write test scriptstest data
Day 1, 2 Day 3, 4 Day 5-8 Day 9 Day 10
FinalTest&Fix
Demo&Acceptance Test
23www.Code71.com www.ScrumPad.com
Copyright 2004-2008 Code71, Inc.
Agenda
Recap
Individual Practices
Section 2
Section 4
Section 3
A Case Study
Holistic View of QA
Section 5
IntroductionSection 1
Q&ASection 6
24www.Code71.com www.ScrumPad.com
Copyright 2004-2008 Code71, Inc.
Recap “In-cycle” QA is critical to the success of a project
System testing is not the only “quality gate,” it includes all types of testing
and reviews
Test automation is critical to “in-cycle” QA
Target at least 90% test coverage
“Continuous Integration” is “Continuous QA”
Prioritize test cases based on risk and frequency of usage
25www.Code71.com www.ScrumPad.com
Copyright 2004-2008 Code71, Inc.
Recap contd.
All known bugs should be fixed first
Right size story with well-thought out acceptance tests improves quality
Include all “Quality Gates” as part of definition of “Done”
Analyze each bug to understand where (Quality Gate) it should have been
caught and improve (Inspect and adapt)
QA is not a designated person’s responsibility, it is a team’s responsibility
(self-organizing team)
26www.Code71.com www.ScrumPad.com
Copyright 2004-2008 Code71, Inc.
Q&A
“QA is not an act, but a habit”
Contact: [email protected]: http://blog.syedrayhan.comCompany: http://www.code71.comProduct: http://www.scrumpad.com
“QA is making sure right software works right”