pyright © 2009 Intel Corporation. No part of this presentation may be reproduced in any way without written permission of Intel Corporation. Building a Scrum Based Test Strategy Ray Arell Sr. Engineering Manager, Intel Corporation [email protected]
Dec 18, 2015
Copyright © 2009 Intel Corporation. No part of this presentation may be reproduced in anyway without written permission of Intel Corporation.
Building a Scrum Based Test Strategy
Ray ArellSr. Engineering Manager, Intel Corporation
Copyright © 2009 Intel Corporation. No part of this presentation may be reproduced in anyway without written permission of Intel Corporation.
Course Outline
Key ConceptsPersonas Scrum Lifecycle Strategy Final Thoughts
Copyright © 2009 Intel Corporation. No part of this presentation may be reproduced in anyway without written permission of Intel Corporation.
Part 1: Key ConceptsLearning Objectives:
Establish a base understanding of ScrumUnderstanding of what makes up a general test strategy
Copyright © 2009 Intel Corporation. No part of this presentation may be reproduced in anyway without written permission of Intel Corporation.
Manifesto for Agile Development
Individuals and interactions over processes and toolsWorking software over contract negotiationResponding to change over following the plan
4
We are uncovering better ways of developing software by doing it and
helping others do it. Though this work we have come to value:
Copyright © 2009 Intel Corporation. No part of this presentation may be reproduced in anyway without written permission of Intel Corporation.
Common Agile Methods
Copyright © 2009 Intel Corporation. No part of this presentation may be reproduced in anyway without written permission of Intel Corporation.
More Methods
Automated testingBarely sufficient documentationBottleneck managementCoding standardsCollective code ownership Co-locationContinuous team integration and CMCustomer focus group reviewCustomer onsiteDaily standupDeferred Commitment
RefactoringRetrospectivesRisk managementSelf-taskingSimple, robust designSmall releasesSustainable paceTest-driven developmentTest firstUnit testingUnity statementUse casesUser stories
• Design metaphor• Exploratory spikes• Feature-based
planning• Group design• Information radiators• Inspections• “Intentional” design• Issue tracking• Last Responsible
Moment• Monitor and adjust• Pair programming• Project velocity
Copyright © 2009 Intel Corporation. No part of this presentation may be reproduced in anyway without written permission of Intel Corporation.
Scrum Project Framework
7
Each sprint is a
fixed duration
Team works from a prioritized
product backlog
Short daily team
meetings
Must deliver
working and fully
tested code
Copyright © 2009 Intel Corporation. No part of this presentation may be reproduced in anyway without written permission of Intel Corporation.
Product BacklogOwned by the product ownerFormal list of product requirements written as user stories
Contains the acceptance criteria Estimated by the development team on the effort to completePriority and customer value of the item
Backlog is reprioritized at the start of each iterationDefects are also tracked on the backlog
8
Copyright © 2009 Intel Corporation. No part of this presentation may be reproduced in anyway without written permission of Intel Corporation.
Scrum Roles
Product Owner: Owns the backlog and stakeholder relationshipVoice of the stakeholder to the development teamMakes decisions and priority adjustmentsRemoves obstacles
Scrum Master: Drives the daily processTracks progress and removes obstaclesFacilitates, mediates and protects team
The Team:Programmers, test professionals, and other key peopleSelf managed/breaks down user stories down to tasksReports daily on status in standup meetings (progress, plan, issues)Responsible for demo to stakeholders
Copyright © 2009 Intel Corporation. No part of this presentation may be reproduced in anyway without written permission of Intel Corporation.
User Center Design and Standup Area
10
Copyright © 2009 Intel Corporation. No part of this presentation may be reproduced in anyway without written permission of Intel Corporation.
Example Stand-up
11
Agenda:1. What have you done since yesterday's meeting?2. What are you going to get done today?3. What obstacles do you need removed?
adsfsadfdsafsadfasdfSasdfAdsfa
asdfdasfdsafSdkjfsfsaAsfadf
adsfsadfdsafsadfasdfSasdfAdsfa
asdfdasfdsafSdkjfsfsaAsfadf
adsfsadfdsafsadfasdfSasdfAdsfa
asdfdasfdsafSdkjfsfsaAsfadf
adsfsadfdsafsadfasdfSasdfAdsfa
asdfdasfdsafSdkjfsfsaAsfadf
adsfsadfdsafsadfasdfSasdfAdsfa
asdfdasfdsafSdkjfsfsaAsfadf
adsfsadfdsafsadfasdfSasdfAdsfa
asdfdasfdsafSdkjfsfsaAsfadf
adsfsadfdsafsadfasdfSasdfAdsfa
asdfdasfdsafSdkjfsfsaAsfadf
adsfsadfdsafsadfasdfSasdfAdsfa
asdfdasfdsafSdkjfsfsaAsfadf adsfsadfdsafsadfasdf
SasdfAdsfa
asdfdasfdsafSdkjfsfsaAsfadf
Copyright © 2009 Intel Corporation. No part of this presentation may be reproduced in anyway without written permission of Intel Corporation.
Review Release Plans(Developers)
Distribution, review and adjustment of product requirements
(Developers)
Sprint
Develop Features
Test
Internal Review
Adjust
CustomerReview
Review software
Compare backlog with developed software
Editing of backlog requirements
Add new backlog items to teams
Assign backlog items to teams
Plan next review
(Developers)
(Project Stakeholders)
All Project Requirements Complete(Quality, Features, Cost)
Adapted from : Wikipedia Scrum (development) ; Many Authors
Copyright © 2009 Intel Corporation. No part of this presentation may be reproduced in anyway without written permission of Intel Corporation.
Key Points on ScrumSprints are a fixed delivery date
Stories that are not done are pushed to product backlog
Stories and defects live in the backlog togetherSoftware repairs are prioritized with the customer
Scrum does not dictate what processes are used within the 2-4 week sprint
This is left up to the development team
Validation is a part of the development teamValidation and test are not visitors--they are a part of the team
Copyright © 2009 Intel Corporation. No part of this presentation may be reproduced in anyway without written permission of Intel Corporation.
Key Points on Scrum (Continued)Product backlog: is the repository of all the high-level features for a product
Documented user stories
Sprint backlog: are the stories being worked on by the team
They are decomposed into workable tasks
Requirements: are documented in storiesStories are expected to evolve and changeDecomposition of stories into tasks will drive clarificationTasks are manageable pieces of work
Copyright © 2009 Intel Corporation. No part of this presentation may be reproduced in anyway without written permission of Intel Corporation.
User Stories and Acceptance Criteria User stories
Is a software system requirement that is written in everyday languageIt is typically one to two sentencesIt needs to be descriptive and should provide enough information to define the acceptance criteria
Acceptance criteriaList of measurements that will be used to verify “done”Can define both functional and non-functional test requirementsDefined with the customer input
Further Reading: http://en.wikipedia.org/wiki/User_stories
Copyright © 2009 Intel Corporation. No part of this presentation may be reproduced in anyway without written permission of Intel Corporation.
ExamplesUser Story:
Application Start: As a user I expect that when the application is invoked it will display the software license agreement for a default of 5 seconds. Display time can be updated to a max of 100 seconds
Validation Criteria:1. Visually verify that the license agreement is displayed for
the default time period2. Set display time to n seconds and verify that it displayed
for that period of time3. Verify that default time cannot be set less than 0 and no
greater than 100 seconds
Copyright © 2009 Intel Corporation. No part of this presentation may be reproduced in anyway without written permission of Intel Corporation.
What is a Test Strategy?
It is the high-level plan on how testing is going to be done. It defines the type of testing that is
going to be used, when it will be used, resources needed and an articulation of key
constraints and assumptions.
Copyright © 2009 Intel Corporation. No part of this presentation may be reproduced in anyway without written permission of Intel Corporation.
Purpose of Test Strategy
Provides a framework for the entire test process Manages expectations throughout the product development cycleCommunicates strategic choices with the customer and the development team
Copyright © 2009 Intel Corporation. No part of this presentation may be reproduced in anyway without written permission of Intel Corporation.
Primary Objectives
Provide maximum return on investment byFinding the defects that have the highest customer impact and exposure to your company as fast as possibleMaking efficient use of resourcesProviding the most important and accurate information to enable informed decision makingEnabling faster repair
Copyright © 2009 Intel Corporation. No part of this presentation may be reproduced in anyway without written permission of Intel Corporation.
Why it is important to define?
Determine the test costStrategy is translated immediately into time required for testing, and time is converted into cost of testing
Adjust your testing effort to available timeIf testing time is fixed, apply test estimation on test strategy to determine what can be achieved within the given time
Communicate early with the customersTest strategy makes it clear which parts cannot be tested, or cannot be tested fully, and what risks will therefore be incurred
Copyright © 2009 Intel Corporation. No part of this presentation may be reproduced in anyway without written permission of Intel Corporation.
Typical Test Strategy Content
1. Test Levels2. Roles and Responsibilities3. Environment Requirements4. Testing Tools5. Risks and Mitigation6. Test Schedule7. Regression Test Approach8. Test Groups9. Test Priorities10. Test Status Collections and Reporting11. Test Records Maintenance12. Requirements Traceability Matrix13. Test Summary
21
http://en.wikipedia.org/wiki/Test_strategy
Copyright © 2009 Intel Corporation. No part of this presentation may be reproduced in anyway without written permission of Intel Corporation.
Defect Tolerance and Test Reduction
22
Annoyance Loss of Revenue Loss of life
Copyright © 2009 Intel Corporation. No part of this presentation may be reproduced in anyway without written permission of Intel Corporation.
Technical Debt
“Shipping first time code is like going into debt. A little debt speeds development so long as it is paid back promptly with a rewrite.... The danger occurs when the debt is not repaid. Every minute spent on not-quite-right code counts as interest on that debt. Entire engineering organizations can be brought to a stand-still under the debt load of an unconsolidated implementation, object-oriented or otherwise.”-- Ward Cunningham
23
The current version of the plugin is 0.2 and uses the following formula to calculate the debt :
Copyright © 2009 Intel Corporation. No part of this presentation may be reproduced in anyway without written permission of Intel Corporation.
Summary of Key Topics Scrum
Development periods have a fixed cadence (2-4 weeks)Teams are self-managedTeams work from a product backlog of user storiesUser stories define the high-level requirements It is readable It is testable
Test StrategyDefines the high-level plan on how testing is going to be doneServes as a communication vehicle with your customer and the development teamIt is focused on finding defects as fast as possibleHelps to identify and remove the risk of shipping a low-quality product
Copyright © 2009 Intel Corporation. No part of this presentation may be reproduced in anyway without written permission of Intel Corporation.
Discussion
25
Copyright © 2009 Intel Corporation. No part of this presentation may be reproduced in anyway without written permission of Intel Corporation.
Part 2: Key ConceptsLearning Objectives:
Understanding of customer personas Create several personas
Copyright © 2009 Intel Corporation. No part of this presentation may be reproduced in anyway without written permission of Intel Corporation.
Different User Representations
Amount of Data
Amount of Detail
User Archetype
User Profile-Single User
MarketData
ActorAnd
Agent
User Role
User Class/Category
Personas
Copyright © 2009 Intel Corporation. No part of this presentation may be reproduced in anyway without written permission of Intel Corporation.
Usability Research
Usability and Customer Research is key to ScrumExpert reviewsOn-line surveys Field research with customersPersona building
Usability Research
Copyright © 2009 Intel Corporation. No part of this presentation may be reproduced in anyway without written permission of Intel Corporation.
What is a Persona?Personas are representations of the group of users of your productIt is an amalgam that documents the behavior—goals, skills, attitudes, and environment—of the end-userIt is based on research and knowledge of the end-usersIt is documented and used in both the development and testing of your product
Copyright © 2009 Intel Corporation. No part of this presentation may be reproduced in anyway without written permission of Intel Corporation.
EdwardApplication Engineer
“There is no ‘One Size Fits All’ with ISVs. Every ISV has a different environment, architecture and
customer needs.”
Ed has been working in this role for 4 years. He was a SW Engineer before that for 10 years. His work focuses on the implementation of
AMT capabilities. Ed’s primary role is assisting ISV engineers in implementing specific features by customizing a solution for their
given environment. Much of his day is spent troubleshooting issues and writing new code to test. Usually, Ed travels to the ISV and
spends time face to face working with their implementation team. Given the economic climate, he has made changes to the way he interacts with his customers. Most of his interaction with ISVs is
done over the phone and via email.
Goals:• Simplify integration for ISV
partners• Solve issues fast• Demonstrate AMT value
Values:• Good customer relationships• Flexible architecture• Good documentation
Obstacles:• Each ISV requires custom solution• Troubleshooting issues remotely• Translating code to meet ISV needs
Design Implications:• Design should demonstrate how a
feature could be integrated--represent believable user experience
• Vanilla design = palatable for all potential customers
• Design should avoid appearing as a competing product
Copyright © 2009 Intel Corporation. No part of this presentation may be reproduced in anyway without written permission of Intel Corporation.
Example Persona: SeanSenior Information Security Analyst
“I am a security nerd.”
Sean is a self proclaimed security nerd. This assessment seems justified considering his 15 years in the IT field—9 of which he has specialized in security. Currently Sean manages the patching pipeline for 110,000 systems and 35,000 servers. His priority is the health of these systems and the data they contain. He must ensure the security of the environment while minimizing interference to the workforce’s productivity. Another part of Sean’s job is to research and evaluate new technology—however, carving out time to do this can be challenging. His team is responsible for deciding what technologies will be implemented into the IT environment. All new technology is thoroughly vetted before releasing into the company.
Goals:• Maintain a secure environment• Stay ahead of the next threat• Minimal interference to workforce
productivity
Values:• Data: securing user/corporate
data; security intelligence; performance metrics
• Speed & efficiency• Control & access
Obstacles:• Keeping confidential user info
protected• Differing platforms (& performance)
across the company• Rogue HDDs with unsecured data • Difficult to manage remote/mobile
systems due to an infrequent network connection
• End users introducing unapproved technology into environment. Design Implications:
• Easy to learn, intuitive interface• Instant gratification• Eliminate error-prone conditions—
stakes are too high (data)• Visualize data – represent client system
status • Reduce/control exposure to confidential
info
Copyright © 2009 Intel Corporation. No part of this presentation may be reproduced in anyway without written permission of Intel Corporation.
Applying the PersonaCustomer satisfaction based testing
Using the goals, motivations, and experience of the customer to set your path in session-based testingHelps to bring a higher priority to defects based on possible bad customer experiences
Scrubbing the acceptance criteria of your user stories
“What would <Persona> care about?”
Focusing the test environmentThe persona lets you know where it is going to be used
32
Copyright © 2009 Intel Corporation. No part of this presentation may be reproduced in anyway without written permission of Intel Corporation.
Why are they useful to product teams?
Provides a detailed and consistent understanding of various audience groups. Addresses how teams supports the needs of a marketFeatures and testing can be prioritized based on how well they address the needs of one or more personas.Provide a human "face" to the customer
33
Copyright © 2009 Intel Corporation. No part of this presentation may be reproduced in anyway without written permission of Intel Corporation.
Personas do not stand aloneAlthough personas have many benefits, they alone will not ensure the success of the product. Business goals and technology constraints also are important to consider.
Business
UserTechnology
Nirvana of SW products
Copyright © 2009 Intel Corporation. No part of this presentation may be reproduced in anyway without written permission of Intel Corporation.
Strategy: Using Customer Personas
Agile is about building high customer value with each release…. but what customer?
Customer personas create an amalgam of your target customersUtilizing this, you have a greater capability of satisfying your end-user marketIn theory, if you satisfy the persona, then you satisfy your market
35
Copyright © 2009 Intel Corporation. No part of this presentation may be reproduced in anyway without written permission of Intel Corporation.
Summary of Key Topics Personas can help you focus your testThey require extensive user researchCustomer satisfaction based testing is a useful addition to your test methodologyRemember that they don’t stand alone
Copyright © 2009 Intel Corporation. No part of this presentation may be reproduced in anyway without written permission of Intel Corporation.
Discussion
37
Copyright © 2009 Intel Corporation. No part of this presentation may be reproduced in anyway without written permission of Intel Corporation.
Part 3: Scrum Lifecycle Strategy Learning Objectives:
Discover how the lifecycle and methods integrate togetherReview example criteria for measuring when a story is complete and when your product is ready to ship
Copyright © 2009 Intel Corporation. No part of this presentation may be reproduced in anyway without written permission of Intel Corporation.
Scrum ConstraintsScrum project framework puts three major constraints on testing
Products are delivered on a fixed cadence and cannot be pushed out All features need to be working and meet the acceptance criteriaNo features are shipped to the customer if it is not tested, repaired, and retested
User stories/features by design are expected to evolveDetails and acceptance criteria in the backlog will evolve overtimeMay be deferred until the maximum amount of information is availableDevelopment of the product itself may fill in the gaps
Customers may shift prioritiesThey are the customer after all!
39
As nerve-racking as it seems, test planning needs to be distributed within each development sprint.
Copyright © 2009 Intel Corporation. No part of this presentation may be reproduced in anyway without written permission of Intel Corporation.
1. Nightly build testing (Smoke Test)2. Weekly regression 3. Story validation
1. Customer acceptance testing
1. Full Testing of new features2. Regression of prior functionality3. Distribution criteria completed
1. Requirements inspection2. Establish story validation criteria3. Customer reviews
High-level Scrum Test Strategy
ProjectBacklogProduct
Backlog
Product Iteration
Develop
Ready For Test
In TestDone
Blocked
Clarify
SprintBacklog
Daily Standup
Defects
Copyright © 2009 Intel Corporation. No part of this presentation may be reproduced in anyway without written permission of Intel Corporation.
Sprinting Over the Entire Project
Slide 41
Project Concept
RequirementsGathering
Adapted from : Agile Project Management, Jim Highsmith
Sprint 1 Theme
Architecture1.1
Feature 3Development
Feature 2Development
Customer delivery, review, update
requirements
Sprint 2 Theme
Architecture1.2
Feature 1Development
Feature 5Development
Sprint n Theme
Architecture1.n
Feature(s) nDevelopment
Sprint 0 Sprint 1 Sprint 2 Sprint n
Customer delivery, review, update
requirements
Customer delivery,
review, done
Hard/Complex/Must Haves Less Complex
Copyright © 2009 Intel Corporation. No part of this presentation may be reproduced in anyway without written permission of Intel Corporation. 42
Copyright © 2009 Intel Corporation. No part of this presentation may be reproduced in anyway without written permission of Intel Corporation. 43
Sprint 1 Sprint 2 Sprint 3 Sprint 4 Sprint 5 Sprint 6 Sprint 7 Sprint n
Code GrowthBurn down
Application Scaffolding Application Build-out Cleanup Retrospective and
Customer Survey
Requirement Inspections and Customer Reviews
Regression And Formal Customer Testing
Core Sprint Testing Methods
Customer Engagement and Usability Studies
Story Boarding and Prototypes
Test Strategy
Copyright © 2009 Intel Corporation. No part of this presentation may be reproduced in anyway without written permission of Intel Corporation. 44
Strategy: Test Scheduling
Wed Thur Fri Mon Tue Wed Thur Fri Mon Tue
Continuous Integration
Release Testing
Atomic / Session-Based Testing
Inspections
Tuesday is a better release
day than Friday
ImportantMake sure the story
is testable!
Copyright © 2009 Intel Corporation. No part of this presentation may be reproduced in anyway without written permission of Intel Corporation.
Do Check
User Story Inspection
ProductProposal and Description
User StoryRev x.x
Develop or Update User Story
Story Testable?
Develop or Update Validation Criteria
Review Criteria with Product
Owner/Customer
Passed Review
Product Design
Document
User StoryRev x.x -1
User StoryRev x.x +1
Copyright © 2009 Intel Corporation. No part of this presentation may be reproduced in anyway without written permission of Intel Corporation.
Example Entry Criteria for “In Development”
User Story is reviewed for testablityIf is not testable, then it is moved to “Blocked”
Copyright © 2009 Intel Corporation. No part of this presentation may be reproduced in anyway without written permission of Intel Corporation.
Example Entry Criteria for “Ready For Test”
Story is implemented per user story description with no blocked issuesCode has been reviewed by the authorIntegrated and works on engineering baselineAuthor has run unit level testing on story and has high confidence of being doneUsage documentation and help files updatedArchitecture document updatedBuild files updated
Copyright © 2009 Intel Corporation. No part of this presentation may be reproduced in anyway without written permission of Intel Corporation.
Exploratory Testing
48
The quality of the testing is dependent on
the tester's skill of inventing test cases and
finding defects. The more the tester knows about the product and different test methods, the better the testing
will be.
Copyright © 2009 Intel Corporation. No part of this presentation may be reproduced in anyway without written permission of Intel Corporation.
Session-based TestingA software test method that combines accountability and exploratory testingDeveloped by Jonathan and James BachTest sessions document:
CharterSessionSession ReportDebriefParsing Results
Planning and debrief sessions
49
Copyright © 2009 Intel Corporation. No part of this presentation may be reproduced in anyway without written permission of Intel Corporation.
Example Entry Criteria for “Done”
Passes peer inspectionPasses integrated build testingCode coverage targets have been achievedMeets the validation criteria established/defined with the customer
Copyright © 2009 Intel Corporation. No part of this presentation may be reproduced in anyway without written permission of Intel Corporation.
Example Criteria: Story CompletedA story is complete when:
Story is implemented per user story descriptionMeets the validation criteria established and defined with the customerPasses the code inspection and peer reviewPasses unit level testingIntegrated and checked into the engineering baselinePasses regression testingUsage documentation and help files updatedEtc…
Copyright © 2009 Intel Corporation. No part of this presentation may be reproduced in anyway without written permission of Intel Corporation.
Test Development and Execution Process Example
Change-Based Test Management (CBTM): Software validation methodology centered on monitoring change to maximize the effectiveness of the validation cycle
Change-Based Test Management is about:Establishing a link between your test cases and regions within your product Tests built to target a region of code Test code coverage analysis Monitoring the changes as your product is being developed Source control delta reportsPriority sorting of tests Critical and changed regions first Unchanged regions last
Copyright © 2009 Intel Corporation. No part of this presentation may be reproduced in anyway without written permission of Intel Corporation.
CBTM
Test Suite Test Suite
Test List Prioritized List
Test 1
Test 2
Test 3
Test n
Test 4 Test n
Test 1
Test 4
Test 2
Test 3
TestCoverage
ProductDelta Report
Test Development and Execution Process
First
LastRemember that the longest test dictates the test process time
Copyright © 2009 Intel Corporation. No part of this presentation may be reproduced in anyway without written permission of Intel Corporation.
Example Distribution CriteriaMinimum Criteria:
Obtain any required external standards compliance certificationComplete smoke test and regression of final build for integrationRemove debugging and testing code from softwareDocument customer-visible defects for release notesInstall/uninstall of product on clean system; ensure instructions are complete and accuratePerform virus scan of all release files, image and mediaVerify the presence/accuracy of copyright noticesVerify that no unintended disclosure of restricted information is presentVerify that all trademarks are correct
Copyright © 2009 Intel Corporation. No part of this presentation may be reproduced in anyway without written permission of Intel Corporation.
Staying “Done”Regression is key to making sure that the prior sprints completed stories stay “done”It is also important that customers don’t see things consistently breakingPick a regression strategy that fits the complexity of the product
Copyright © 2009 Intel Corporation. No part of this presentation may be reproduced in anyway without written permission of Intel Corporation.
Example: A Simple Regression StrategyMaster Done List
Top Customer Features
Defects Repaired
Other
Weekly:• Verify a random pick of 10%Release:• Regress all new features • Passes global product
requirements • Verify a random pick of 30%• Anything you may have a hunch
about
Weekly:• Verify a random pick of 5%Release:• Verify a random pick of 10%
Weekly:• Verify a random pick of 5%Release:• Regress all prior critical defects for
3 sprint cycles
Past Sprints
Copyright © 2009 Intel Corporation. No part of this presentation may be reproduced in anyway without written permission of Intel Corporation.
Prioritize Test Areas
Prioritizing CriteriaHas higher likelihood of having defectsIf broken, will have a higher impact on the customerFrequency of use by customerIf broken, will cause more failures in other requirementsTrust your Spidy-sense
Copyright © 2009 Intel Corporation. No part of this presentation may be reproduced in anyway without written permission of Intel Corporation.
Defect Likelihood ChecklistNew Things
New features, concepts, technology, tools
High ComplexityFunctions with many interfaces (functions that require interfacing with external systems) or high complexity
Frequently changing thingsFunctions that have a lot of updates or bug fixes (frequently changing code)Functions with poor requirements and designAmbiguous specs can lead to incorrect or conflicting implementations
DependenciesIf broken, will cause more failures in other requirements
Copyright © 2009 Intel Corporation. No part of this presentation may be reproduced in anyway without written permission of Intel Corporation.
Defect Likelihood Checklist - 2Rushed work
Some tasks or projects are chronically under-funded and all aspects of work quality sufferFunctions developed under extreme time pressureLate changes: Rushed decisions, rushed or demoralized staff lead to mistakes
Tired ProgrammersLong overtime over several weeks or months yields inefficiencies and errors
Learning CurveFunctions owned by new developer
Copyright © 2009 Intel Corporation. No part of this presentation may be reproduced in anyway without written permission of Intel Corporation.
Features (Not) to be Tested Some of the reasons why a feature may be out of the testing scope
Low priority as a result of risk analysis Tested before and is considered stableWill not be included in the current releaseWill be tested by third partyWill be released but not tested or documented as a functional part of the current version
Copyright © 2009 Intel Corporation. No part of this presentation may be reproduced in anyway without written permission of Intel Corporation.
Summary of Key Topics It is important to establish clear entry criteria to each phase of the scrum development cycle “Done” needs to measured more than onceRegression is key to having continued user satisfaction
Copyright © 2009 Intel Corporation. No part of this presentation may be reproduced in anyway without written permission of Intel Corporation.
Points to RememberAlways review your strategy with your management, customers, subcontractors, and development team members
Test strategies for Scrum need to focus on ensuring high customer satisfaction
Invest in user research and use tools like personas to focus your testing effort and priority
Invest in growing your people. None of this can be done without the talented people who know how to break things
Copyright © 2009 Intel Corporation. No part of this presentation may be reproduced in anyway without written permission of Intel Corporation.
Thank You!
63
Copyright © 2009 Intel Corporation. No part of this presentation may be reproduced in anyway without written permission of Intel Corporation.
References
• Various Authors, Exploratory Testing, Wikipedia • Various Authors, Test Strategy, Wikipedia • Various Authors, Scrum (development), Wikipedia • Various Authors, Session-based testing, Wikipedia • The Scrum Alliance, http://www.Scrumalliance.org/ • Ray Arell, Change-Based Test Management, (ISBN: 0971786127) • James Bach, Heuristic Risk-Based Testing, STQE 11/99• James Bach, Risk and Requirements-Based Testing, Computer, June 1999• Ingrid Ottevanger, A Risk-Based Test Strategy, StarEast 2000• Bret Pettichord, The role of information in Risk Based testing, StarEast 2001• James Bach, Risk-Based Testing Troubleshooter, Paper Draft• Erik Petersen, Smarter Testing with the 80:20 Rule, StarWest 2002• Anne Campbell, Using Risk Analysis in Testing, StarEast 2000• Paul Gerrard, Risk-Based E-Business Testing, System Evolutif• Gregory T Daich, Defining a Software Testing Strategy • Jim Highsmith, Agile Project Management• Ruku Tekchandani, Building a Effective Test Strategy• John Pruitt and Tamara Adlin, The Persona Lifecycle• Pettichord, Kaner, Bach, Lessons Learned in Software Testing, on-line • Jonathan Bach, Session-Based Test Management , http://www.satisfice.com/articles/sbtm.pdf