QA Center Of Excellence (TCoE) for JITC Should be led from top down and implemented from bottom up.
Jan 18, 2015
QA Center Of Excellence (TCoE)
for JITC
Should be led from top down and
implemented from bottom up.
Agenda
• What is the QA Center of Excellence (TCoE)
• Why implementing TCoE
• Benefits of TCoE
• Key Strategies/ Tactics/ Approach
• Challenges and Opportunities
• Business case
• Q & A
• Name – Moti Demri
• Email – [email protected]
• Phone - 904- 386 -6620
2
What is QA Center of Excellence (TCoE)?
• TCoE is a centralized solution that brings into a shared services
function, people, processes, tools and infrastructure. Standardizing processes
and optimizing resource utilization is to bring in maturity.
• The Test Center of Excellence (TCoE) solution is targeted towards customer
who are challenged with decentralized and under-performing testing functions,
under utilization of testing people, inconsistent test processes and are looking
to move up the maturity ladder towards a standardized testing function, and at
the same time improve efficiency, optimize people and tool utilization and
lower testing costs
• A TCoE may define a best or commonly accepted practice and encourage
adoption of that practice
• A TCoE may govern or enforce standards
3
This is NOT a one person job…it is the entire organization’s effort
4
Why implementing a TCoE?
5
• Reduce incidence of live issues
• Avoid “customer being our best testers”
• Avoid Emergency releases Reduce Reputational Risk
• Reduce number of defects occurring in later stages of testing
• Increase the independence of testing
• Apply rigor to test process
• Focused collaboration between business and test
• Promote the understanding of test requirements across the organization
Improve Quality
• Reduce the need for long test phases
• Increase the usage of automation in Dev and UAT
• Enable earlier detection of defects
• Enable faster more reliable test cycles
• Ensure correct resources (number and skills are being utilized
Cost efficiency
TCoE – Evolution (A gradual approach– from sharing processes to Services Utility)
6
Phase 1– introduction to standards, governance and processes
• Begin establishing fundamental policies around application testing
• Decide on measurable quality standards
• Define basic reporting procedures
• Implement initial governance process
Phase 2 - TCoE functions as a shared service
• Unified testing manual and automation tool across the company
• Training other groups using tools , scripts writing
• Resources Pool
• Share Regression tests and scripts
Phase 3 - Service utility
• TcoE acts as a central source of services and expertise for the entire origination
• included test management office, domain-specific expertise team, shared services teams for specialized testing activities
Phase 4 - Innovation authority and flexible resources pool
• TcoE acts as a central source of services and expertise for the entire origination
• included test management office, domain-specific expertise team, shared services teams for specialized testing activities
Why do we need TCoE (Benefits)
TCoE can accrue many benefits to an organization
• Define and promote testing standards
• Improve quality
• Define testing boundaries and provide specialized testing services
• Faster time to market
• Reduce testing costs
• Bring together different testing organizations to align their test processes and technical disciplines
• Increase the testing value to external groups
• Insure best practices are in place
• Better ways to control training and knowledge transfer
• Provide a practical approach to testing and QA functions
• Consolidation and test process improvement
• TCoE are becoming increasingly common within large and medium scale businesses
• TCoE is a centralized solution that brings together people, processes, tools and infrastructure into a
shared services function
• Standardize processes and optimizing resource utilization
• Everyone on the same page!!!!
7
Why do we need TCoE (Benefits)
• Optimized use of existing resources:
• QA budgets, tools, environments and people
•Faster time-to-market:
• Test times reduce manual by 30%
• and increase test automation levels of 50-70%
•Cost efficiency:
• Typical resource cost reduction is 35% over a 2-year time frame
•Optimized application quality and performance:
• Reduce risk of failure and leakage of high-severity defects to under 2% and improved
customer experience
•Tighter alignment:
• Align quality effort more tightly with business needs by defining and measuring KPIs (Key
performance indicator)
8
9
Co
st
of
Err
or
Co
rrecti
on
Time
$1
$2000
Quality
Process
Quality Process Reduce Cost of Error
Key Strategies and Tactics
• Senior management involvement
• Find a key champions that care about quality and process
• Chose doer NOT talker
• Start working as team not as individual
• Develop very clear vision
• Define clear and achievable goals
• Define tactics and strategies
• Full integration between all departments and business
• Work and think on the enterprise level, not local
• Scalability, usability and
• Semi- integrated model / approach
• Pilot it first on a smaller scale project
• Establishing a TCoE is a phased approach
• Market the TCoE concepts
• Training across the company
• Define measurements and matrixes
10
TCoE - Why others failed to implement?
• No senior management support , need to gain the buy-in
• Need commitment from management and individuals
• No clear vision or goals
• Implementation going too slowly
• No implementation plan
• No agreeable timeframe
• No upfront planning
• Big steps approach implementation
• No dedicated staff
• Not sufficient tools
11
TCoE - Stability and Reliability (Vision)
12
Governance and Collaboration Aspect (between IT and Business)
13
Companywide
IT Governance
Linking Mechanism
Project Management
Co
ord
ina
tio
n
Alignment
Business IT
Company strategy
And
Operation
Business unit
strategy and
Operation
Project Plan
Enterprise
Architecture
Business unit
architecture
Project IT
Architecture
Company
Level
Business
Unit level
Project team
level
Testing department assessment Framework
14
TCoE Maturity Levels
15
16
The standard definition of test metrics allows an organization to provide visibility into test effectiveness, demonstrate delivery excellence, track key goals, and enable efficient management and control through business analytics and predictive management.
Value Delivered Indicates whether
strategy and execution
are contributing to the
bottom line and meeting
financial expectations
Service
Excellence Indicates how well the
organization meets client
and customer needs and
their expectations
Process
Excellence Indicates whether the
organization has the
process capability,
maturity and quality to
deliver business value
Organizational
Excellence Indicates how well the
organization meets
employee needs and
their expectations
Benefit Type Best Practice Metrics Calculation
Effort / Cost Test Effort Rate (Cost of Quality) (Total Test Effort / Total Effort) * 100
Effort / Cost Cost of Poor Quality (Total Rework Effort / Total Effort) * 100
Quality Defect Rate (Total # of Development Defects / Effort Total)
Quality Test Effectiveness (Total # of Development Defects / (Total # of Development Defects
+ Total # of Delivery Defects)) * 100
Quality Defect Ratio (Total # of Development Defects CLOSED / Total # of Development
Defects) * 100
Quality Pass Rate (# of test scripts PASSED / total # of test scripts)
Quality Defect Arrival Rate # of defects OPENED / day
Productivity Defect Closure Rate # of defects CLOSED / day
Productivity Retest Backlog # of defects FIXED (ready for retest) / day
Productivity Defect Turnaround Time # of total CLOSED (fixed + retested) defects / Total Rework Effort
Productivity
Cycle Time Test Script Development Rate # of scripts APPROVED / day
Productivity
Cycle Time Test Script Execution Rate # of scripts EXECUTED / day
Cycle Time Test Duration # of days / cycle
TCoE - Metrics program is essential to measure QA effectiveness