Where are We Going and Who’s Driving? - CQAA 2017... · • Environment synch with production continues to be mostly manual process • Environment challenges with multiple independent

Where are We Going and Who’s Driving? Developing and Designing a Comprehensive QA Roadmap

Elizabeth Wisdom

Sr. QA Manager

Agenda

• QA Strategy vs QA Roadmap

• Anatomy of an Excellent QA Roadmap

• Developing and Selling the Vision

• The Ulta Beauty Example

• Q&A

Objectives for Today

Strategic planning

Leading through change

Developing and implementing a Quality Assurance Roadmap

QA Strategy vs QA Roadmap

QA Strategy: contains the outcome which you are aiming for (e.g. deliverables such as test plans, test results, defects) and the steps you currently take to get there (process, tools, metrics, governance).

QA Roadmap: a time-based plan that defines where you are, where you want to go, and how to get there.

START

END

Challenges

What challenges have you experienced with developing or implementing a QA Roadmap?

1.

2.

3.

4.

Anatomy of an Excellent Roadmap

Logical structure; Priorities aligned with goals

Senior level vision and commitment

Priorities and timeframes

Implementation and action plans

Digestible information

The Journey

Determine the “steering” group

Establish the shared vision and goals

Establish Priorities and Timelines

Prepare the Roadmap Communicate and Socialize the Roadmap

Implement Roadmap via Partnerships

Monitor Progress, Update as Needed

The Example

Centralized: • Conceived and led by QA • Roadmap targeted a few priority needs • Action plans developed for top priorities • Projects developed, funds gathered/allocated for

projects • Projects initiated, managed, and monitored by QA

CONFIDENTIAL & PROPRIETARY 2016

IT QA Roadmap August 2016 FINAL

10

Roadmap Approach

• Define the required capabilities – People, Process, Tools

o Assess current-state

o Determine capabilities needed to execute the IT vision and strategy (IT 5 year plan)

• Design the end-state

o Create the structure to deliver the required capabilities

o Make critical decisions on process governance, technology spending, data

• Develop the “should cost” financial model

o Analyze current costs to identify improvement opportunities

o Develop the cost model to define the future state (ROI, cost savings/year)

• Create the go-forward roadmap

o Outline transition priorities and timing

o Highlight important milestones, dependencies, and challenges

• Monitor the implementation with specific KPIs

o Professional services KPIs

o Technical solution KPIs

Source: A.T.Kearney analysis

11

The Journey – Where the QA Roadmap Input Came from

Input solicited and gathered from various sources: • Face to Face meetings with owners (directors and managers) to collect feedback and wish

list

o Guest Loyalty & Customer Service

o Point of Sale

o Ecommerce and Ecommerce Ops

o Warehouse Management

o Merchandise Planning & Operations

o ERP

o Mobility

o Enterprise Architecture

o Project Lessons learned

• Research & Development (POCs) on automation tools and cloud platforms, ALM tools

• IT PMO

• QA and Performance Test teams

• Vendor partners

12

Challenges & Opportunities

Functional Testing (manual and automated) • Cost of QA - high % of manual testing, data prep, execution rate

• Existing automation not optimized (coverage, execution)

• Limited test capability in production

QA Environment Management • Data not shared across applications or teams (side effect of individual projects vs

common releases)

• Environment configuration errors

• Environment synch with production continues to be mostly manual process

• Environment challenges with multiple independent projects happening simultaneously with cross functional dependencies

• Functional test labs/environments/equipment maintenance overhead

13

Challenges & Opportunities, cont.

QA Tools & Metrics • No insight into code coverage during execution

• Tool usage – mix of ALM tools; lack of consistent linkage between requirements/user stories to code changes to builds to test cases to defects

• Need to measure/assess QA effectiveness

• Need to measure build quality trends and fix-on-fix rate

14

Challenges & Opportunities, cont.

Code Quality from Development • Inconsistent build quality across applications

• Defect trending reports and application performance trend reports generated, but no action/reaction as a result

• Date focused; resources not allocated sufficient time for projects – as a result, quality suffers

Resources and Team Structure • Test execution relies on consultant staff and knowledge not retained with QA FTE;

need for SMEs that are FTEs in key areas

• Regression test execution executed per project; potential for centralized regression team if dependent application teams can align on a common release schedule

15

The Roadmap: 5 Key Areas of Concentration

1. QA Test Automation

2. QA Environment Management

3. QA Tools & Metrics

4. Application Code Quality Improvement

5. QA Resources and Team Structure

Improve Efficiency

Standardize Provide Transparency

The Example: Final Planning & Review

Review with application owners and IT leadership High Level Cost Estimate – Determine ROI Prioritization of roadmap initiatives (as part of budget

process and detailed plan) Solution design discussions with app owners Establish KPIs for monitoring any professional services and

technical solution performance employed during implementation

The Example: Lessons Learned

Don’t worry about getting it perfect; get it “going” Show the ROI - a compelling value proposition is essential Clarify expectations for implementation Combine short-term returns (quick hits) with long-term

commitment Champions and leaders make the difference – keep them

looped in

Success Factors Include the right people at each stage Get senior-level buy-in early Define a clear scope Provide value proposition (e.g. ROI) Coordinate with your organization’s funding cycles Outline a realistic implementation strategy Engage the right people during implementation Identify champions Commit to action Communicate and Socialize the Roadmap Refactor if necessary

Q&A