Contracting an outsourced agile test team
Post on 15-Jan-2015
314 Views
Preview:
DESCRIPTION
Transcript
Clareice Chaney, Clyneice Chaney &
XBOSoft
Best Way to Contract an Outsourced Agile Test
Team Webinar
Best Way to Contract an Outsourced Agile Test Team
Clareice Chaney CPCM, PMPClyneice Chaney PMP, CQMgr.
XBOSoft info
• Founded in 2006• Dedicated to software quality
• Software QA consulting• Software testing services
• Offices in San Francisco, Beijing, Oslo and Amsterdam
On the XBO side:
Amy Jan
About the SpeakersClareice Chaney has over 30 years of experience in commercial and government contracting with an emphasis in information technology contracting and specific expertise in performance-based contracting techniques.
Clyneice Chaney brings over 20 years of testing, quality assurance, and process improvement experience. Clyneice holds certifications from the American Society for Quality as a Certified Quality Manager, QAI Global Institute's Certified Quality Analyst, and Project Management Institute's Professional Project Manager.
Housekeeping• Everyone except the speakers is muted
• Questions via the gotowebinar control on the right side of your screen
• Questions can be asked throughout the webinar, we’ll try to fit them in when appropriate
• General Q & A at the end of the webinar
• You will receive info on recording after the webinar
Next Webinar -July 30
• Mobile Testing Tutorial• JeanAnn Harrison• BYOD• Registration page in your
chat
Agenda
Outsourced Testing Concepts
Contracts & Testing
Procuring Agile Projects
Monitoring Agile Testing
Outsourced Testing: Typical Categories
Consulting Services Quality Assurance Quality Management
Staff augmentationFunctional testingIntegration TestingPerformance Testing Testability Assessment
Test managementTCOERisk –based testingTest AutomationAgile TestingTest Data Management
Managed quality servicesQuality center of excellenceOn-site advisory/ assessmentShared outcome QA
Outsourced Testing Challenges
• No single best testing practice
• Complexity of system under test
• Test automation
– Misaligned baselines– Operational
inefficiencies– Poor risk and
performance management
– Collaboration failures
Is There a Difference Outsourcing Agile?
Requirement
Design
Build
Test
Deploy ?
Contract
Traditional
Agile Foils controlling systemsDefines deliverable differently
Agile & Contracting: Understanding a Basic Contradiction
• You need an agreement to define a fair and professional relationship.
• Contracts typically don’t allow for quick or easy changes
• Poorly constructed contracts have the potential to derail business objectives
Impact of Agile on Contracts
• Co-location• Decision making authority• Customer participation• Pricing concerns• Cross-functional; not separate
teams– Test often before delivery and part
of development process– More automated test & analysis for
early buy identification
What’s the Fix?
• Modular contracting to provide flexibility. Several smaller acquisitions that:– Are easier to manage individually– Address complex IT objectives– Provide for delivery, implementation & testing of workable
systems solutions in increments– Allows subsequent increments to take advantage of changes in
technology or requirements
And…. Fix # 2
First phase is a typical service/
consulting contract
2nd phase is a series of fixed price contract where you have defined:• What to build • Time frame • How much it will
cost
Results: A framing contract that is really
about the collaboration and the
end goal. Defines: # of iterations and the T&Cs for both
phases
And Fix # 3: Focus On What You know
Contract Project Objectives rather than the solution
• Target Cost• Target Schedule• Business objectives with a high level
overview of how the SW will help• Time and resources budgets
Well defined planning and acceptance process for each iteration
Shared Benefit Contracts
• Goal is to provide financial incentives for meeting project objectives and penalties if they aren’t met
• Not technical objectives… business objectives
Road to Good Outsourced Test Performance
Supplier Performance
# 1 Identify outcomes
# 2 Determine outcome measures
# 3 Define Service Levels
# 4 Establish Monitoring Methods & Monitor Outcomes
Terminology
Work
OutputsBenchmarks Indicator Accomplishments
What you Need to Know?
Quality of Service Quality of Product
Defect Density
Quality Attributes
Status Quality Target
Number of Defects
Schedule
Test Effectiveness
Test Planning Effectiveness
Defect Detection
Adapted from James Bach, Rapid Software Testing.
Measuring the Suppliers OutcomesNumber of Tests Attempted by Given Time
Number of Tests Passed by a Given Time
Number of Bugs Found by Given Time
Test Effectiveness Ratio
Defect Detection Capability
Duration of Testing Processes
Defects by Category/ Severity
Product Defect Density
Backlog of Open Bugs
Product Quality Over Time
Defect Detection Capability
Defect Severity Index
Agile Testing Metrics
Copyright © 2011 RGCG, LLC 41
21
Customer Acceptance Tests Defect fixes & Story Changes
Burn Down Burn Up
http://xprogramming.com/xpmag/BigVisibleCharts
5P Performance Measurement Framework
Adapted from: Kalyana: Performance Measurement Framework for Outsourced Testing Projects (April 3, 2009)
Project: for each task or project monitor meeting it’s requirements, service levels
Process: monitor suppliers processes, particularly ones they agreed to perform
Product: monitor excellence of deliverables
People: monitor supplier staff- average experience; availability, capability
Price: $ amount saved, price variance
Setting Testing Service Quality Levels
• Timeliness of service• Accessibility and convenience • Accuracy• Staff Availability• Service Delivery: Courteous?• Adequacy of information
disseminated• Customer satisfaction• Condition and safety of facilities used
Summary
Questions & Answers
Follow us: @XBOSoftQuestions at: services@xbosoft.com
Thanks and hope to see you soon!
top related