How to make the step from testing to QA

How to make the step from testing to quality assurance

Jan Jaap CannegieterVice President SYSQA B.V. (The Netherlands)

jcannegieter @sysqa.nl or @jjcannegieter

© S

YS

QA

Alm

ere

Questions that will be answered

• What kind of QA fits a tester best?• Why is everybody else not asking it by themselves?• How to convince your manager to implement QA?• How can I show upfront and afterwards that QA pays?• How much does it pay?• How to deal with agile and outsourcing?

© S

YS

QA

Alm

ere

What is quality assurance?

3

The planned and systematic activities implemented in a quality system so that quality requirements for a product or service will be fulfilled

Doc.

Doc.

Code

Data

Code

Doc. Process

Process

Process

Result

© S

YS

QA

Alm

ere

Product QA

4

Doc.

Doc.

Code

Data

Code

Doc. Process

Process

Process

Result

© S

YS

QA

Alm

ere

Process QA

5

Doc.

Doc.

Code

Data

Code

Doc. Process

Process

Process

Result

© S

YS

QA

Alm

ere

Project QA

6

Doc.

Doc.

Code

Data

Code

Doc. Process

Process

Process

Result

© S

YS

QA

Alm

ere

7

Essence of product QA

Inspections/reviews/audits

Development Testing#

defects

Quality

© S

YS

QA

Alm

ere

9

MANAGEMENT:YEAH, NICE.

BUT WHEN IS TESTING READY?

AND WHAT DOES IT COST?

Problem 1: How do I convince management

© S

YS

QA

Alm

ere

What are important arguments for a manager

• Improve quality• Improve certainty • Improve continuity• Save time• Save money• Ease / comfort• Safety• Status

✔✔✔✖✔✔✖✖

Line

mng

r

✖✖✖✔✔✖✖✔

Proj.

mngr

✔✔✖✖✖✔✔✖

Engi

neer

Problem 2: Where do I find undisputed,

organizational specific data that shows upfront that QA saves

money and time?

© S

YS

QA

Alm

ere

We have the key!

13

Defects

© S

YS

QA

Alm

ere

Calculating the advantage of product QA

Barry Boehm

Capers Jones

SYSQA

14

=

Savings

32

16

8

4

2

1

-

Phase

Requirements

Functional design

Technical design

Code

Unit test

Functional test

Production

© S

YS

QA

Alm

ere

The case

• Big telecom organization• Testdepartment with approximately 200 testers, test

managers etc.• Projects are exceeding time and budget• Quality not good (enough)

15

© S

YS

QA

Alm

ere

16

Phase # defects Factor Pot. savings

Requirements 6 32 192 hours

Functional design 5 16 80 hours

Technical design - 8

Build 7 4 28 hours

Unit test - 2

Totaal 18 300 hours

Calculating costs of poor quality - I

Average € 80,= per hour: ‘avoidable failure costs’: € 24.000,-With review effectiveness of 50% actual savings: € 12.000,-

© S

YS

QA

Alm

ere

17

Phase # defects Factor Pot. savings

Requirements 65 32 2080 hours

Functional design 225 16 3600 hours

Technical design 60 8 480 hours

Build 105 4 420 hours

Unit test - 2 -

Totaal 457   6580 hours

Calculating costs of poor quality - II

Average € 80,= per hour: ‘avoidable failure costs’: € 526.400 ,-With review effectiveness of 50% actual savings: € 263.200,-

© S

YS

QA

Alm

ere

18

YOU HAVE A GO

MANAGEMENTTEST DEPARTMENT:

Problem 3: How do you measure the benefits?

© S

YS

QA

Alm

ere

Benefits per defect found in reviews

22

Savings

32

16

8

4

2

1

-

Phase

Requirements

Functional design

Technical design

Code

Unit test

Functional test

Production

© S

YS

QA

Alm

ere

Benefits

23

First six mounths

# Inspections 124

Solved majors 1385

Time spend on inspections in hours 2476

Saved hours 25354

Return on investment 10.24

Saved money in 6 mouths 1.601.460,-€ 3.2 mln per year

© S

YS

QA

Alm

ere

Different cases

Case 1: shared service center• ROI: 7,1• Saved: € 377.600 / 16%

Case 2: system dev. outsourced• ROI: 11,9• Saved: € 51.200

Case 3: insurance company• ROI: 6,8• Saved: € 1.824.000

Problem 4: How do you expand your focus?

© S

YS

QA

Alm

ere

Optimize and expand QA

Quality of a phase =

26

# defects removed in phase X

(# defects phase X-1) + (# defects injected phase X)Requirements is

your problem

© S

YS

QA

Alm

ere

So, how do I make the step from testing to QA?

• Use the arguments that fits the personal interest of the decision maker

• Analyse your defects: determine moment of injection• Quantify the cost of poor quality / avoidable failure costs

“We could have saved this, would you like to save it in the future.”

• Measure and communicate the benefits• Keep improvingAnd• Develop you political skills• Develop your persuasion skills / sales skills• Develop your change management skills

27

Congratulations,

you’re A QA-officer now

© S

YS

QA

Alm

ere

Everybody happy?

28

This is not the end of the story …

“We are going to implement scrum so we don’t need QA

anymore”

© S

YS

QA

Alm

ere

30

Product backlog

User story 4User story 5

User requirements

User story 1User story 2User story 3

Systemreq., design,

build, test

Systemreq., design,

build, test

Sprints

€

€

Quality Assurance

Quality Assurance in agile projecten

Productvision

Business requirements

“We are going to outsource everything, so we don’t need the QA anymore”

© S

YS

QA

Alm

ere

Outsourcing and quality assurance

Outsourced process• Design• Development • System test

Pro

ject

def

init

ion

•R

equi

rem

ents

•S

uppl

ier

sele

ctio

n•

Con

trac

ts

Acc

epta

nce

© S

YS

QA

Alm

ere

Outsourcing and quality assurance

Outsourced process• Design• Development • System test

Pro

ject

def

init

ion

•R

equi

rem

ents

•S

uppl

ier

sele

ctio

n•

Con

trac

ts

Acc

epta

nce

Quality Assurance

© S

YS

QA

Alm

ere

The answers

• What kind of QA fits a tester best?Product QA

• Why is everybody else not asking it by themselves?Because I used the wrong arguments

• How to convince your manager to implement QA?By showing that QA saves time and money

• How can I show upfront and afterwards that QA pays?Determining the moment of injection of defects and using the calculation model

• How much does it pay?An awful lot of money

• How to deal with agile and outsourcing?In agile and with outsourcing you also need QA

© S

YS

QA

Alm

ere

In case you have got any questions later:[email protected] or @jjcannegieter