J.Gras Six SigmaTSIC 2010

Six Sigma based quality control- Local and national perspectives -

Jérémie Gras Médecin Biologiste

Laboratoire - Clinique St-Luc Bouge

Turning Science Into Caring Conference

Living Tomorrow – Vilvoorde

7th October 2010

- PLAN -

1) Six Sigma basics

2) Internal Quality Control applications- the local perspective

3) External Quality control applications- the national perspective

4) Tendencies and controversies in the use of Six Sigma in clinical laboratories

Jérémie Gras - Six Sigma based quality control, local and national perspectives – TSIC conference 2010

Clinique St-Luc Bouge

1. Six Sigma Basics

• Six Sigma is a global management strategy introduced by Motorola in the 80’s

• First application of Six Sigma occured in the production process

• Most of Fortune 500 companies, especially manufacturing companies, have implemented Six Sigma with tremendous success in terms of customer satisfation and global profitability



1. Six Sigma Basics

But what is Six Sigma exactly ?



1. Six Sigma Basics

A few definitions of Six Sigma:

• Six Sigma is a problem solving methodology

• Six Sigma performance is a statistical term for a process that produces fewer than 3.4 defects per million opportunities (3.4 DPMO or 3.4 ppm)

• A Six Sigma organisation uses Six Sigma to improve performance: continuously lower costs, grow revenue, improve customer satisfaction, reduce complexity, lower cycle times, minimize errors



1. Six Sigma Basics

The Six Sigma methodology

• Define, Measure, Analyse, Improve, Control

• Breaktrough methodology: DMAIC

1) Define= define problems2) Measure= measure performance and determine error rate3) Analyse= analyse data to find the cause of errors4) Improve= improve the process, reduce errors5) Control= control the improved process



1. Six Sigma Basics

The Six Sigma methodology: DMAIC

• Seems simple, but terribly effective

• Measure is the key step: careful documentation with numerical data helps a lot to make decisions !

• Six Sigma introduces a new vision of quality: quantitative

quality that is clearly measurable !



1. Six Sigma Basics

But how can we measure quality ?

• Six Sigma is a management methodology

• But it is also a scale- the Sigma scale

• That scale allows to quantitate quality for most human activities on an industrial scale



1. Six Sigma Basics

The Sigma scale (1/3)

• Six Sigma principles were firstly introduced in the industrial production area, aiming at creating products with a minimal number of defects

• Six Sigma as a value corresponds to 3,4 defects per million opportunities

• 3,4 DPMO or 3,4 ppm represents world class quality



1. Six Sigma Basics



Sigma Errors (%) Errors (ppm)

1 69 % 691462

2 31 % 308538

3 6.7 % 66807

4 0.62 % 6210

5 0.023 % 233

6 0.00034 % 3.4

7 0.0000019 % 0.019


1. Six Sigma Basics




• a Sigma value indicates how much errors are occurring: a high Sigma level correlates with a process with a low number of defects

• a Six Sigma process has so little variation, that even a variation of 6 standard deviations will fit in the tolerance limit for the process

That is World Class Quality

That is where the term “Six Sigma” comes from

1. Six Sigma Basics



Six Sigma applications in clinical laboratories

• In 2010, we can state that Six Sigma has been implemented and deployed in many sectors of In Vitro Diagnostics:

Firstly with IVD manufacturing companies

– Many constructors have implemented Six Sigma in manufacturing and other areas (pricing, etc): Abbott, Beckman Coulter,...

– Some provide special consulting services to improve lab organisation: Valumetrix, also Abbott,…

1. Six Sigma Basics




• In 2010, we can state that Six Sigma has been implemented and deployed in many sectors of In Vitro Diagnostics:

Secondly in clinical laboratories, with two applications:

1) Application of the Six Sigma management methodology (DMAIC) to improve global laboratory performance

2) Quantification of local analytical performance onthe Sigma Scale and QC rules selection

1. Six Sigma Basics




• Successfull exemples of DMAIC applications in clinical laboratories*:

• To reduce errors in the data entering department (Riebling 2004)

• To reduce TAT by taking action on the pneumatic tube system (Simmons 2002)

• To reduce post-analytic errors (Riebling 2005)

* J.M.Gras, M. Philippe. CCLM 2007;45(6):789-796.

1. Six Sigma Basics




• Belgian labs have also implemented Six Sigma for various projects:

• KUL: DMAIC application to increase TAT for samples sent from external laboratories (Wouters et al, presented at Quality in the Spotlight Conference in March 2008)

• UCL: Lean and Six Sigma application to increase TAT in an university corelab (Fillée et al, presented at Siemens Academy in September 2010)

1. Six Sigma Basics




• Other applications concern the application of Six Sigma as a tool for Internal QC management

- PLAN -

1) Six Sigma basics






2. Internal Quality Control Applications

Westgard started it all…

• Applications of Six Sigma in internal QC were suggested by J.O.Westgard in 2005*

• Keys to understand these applications are:

1. It is possible to apply Six Sigma as a quality indicator for laboratory medicine;

2. There is a link between Sigma levels and power curves;

3. The Sigma level equation is based on the process capability concept;

4. This application allows to adapt QC rules for every test.



* J.O. Westgard. Six Sigma Quality and Design. Westgard QC Inc. 2005.


The Sigma level equation:



J.O. Westgard. Six Sigma Quality and Design. Westgard QC Inc. 2005.

Sigma = [(TEa – biasmeas) /CV] *




Different Sigma levels mean different QC strategies

Free T4:

Sigma level= 2.81

Suggested QC Rule: 13s/22s/R4s/41S/8X (N=8)

TSH:

Sigma level= 6.30

Suggested QC Rule: 13s (N=2)


Meaning of the Sigma level equation:



1) In one equation, you find the quality objective, the bias and the CV

2) It is a measure of the adequation between the quality demand (TEa) and the test variability (CV)

3) Outstanding quality indicator (cfr ISO 15189)

4) It is recommanded to calculate the Sigma level at the clinical decision treshold; in practice, we calculate one sigma level per QC material


Sigma levels - practical questions – (1/4)



• Which TEa to use ?

• Various possibilities: • Ricos norms (“desirable precision”)• “Optimum” or “minimum” biological

variation• CLIA• Belgium: d from IPH• RiliBäk• Tonk’s rule• ...

• In practice, try with Ricos norms first

Sigma = [(TEa – biasmeas) /CV]





Ricos TEas available on Westgard website

http://www.westgard.com/biodatabase1/html





• Which CV to use ?

• Various possibilities: • Validation period CV• Cumulated CV (1 week, 1 month, 3 months,...)• ...

• In practice, use a CV that is representative of the lab’s real life (that includes maintenance and reagent changes

• A two months cumulative CV is great






• Which Bias to use ?

• Various possibilities: • Bias as compared to reference method• Peer-related bias • Method related bias

• Currently, the peer-related bias is considered as the most representative...and the easiest to obtain



Sigma levels - practical example



How to perform Six-Sigma based QC for one test ?

1) Select one test

2) Select a QC material

3) Define TEa

4) Evaluate method: CV, bias and Sigma calculation

5) Calculate OPSpecs and Power Function Graph

6) Calculate Ped and Pfr, choose Ped >90 % and Pfr <5%

7) Put it in practice

8) Evaluate the results





Several steps

1) Select one test

• By example

Luteinizing Hormone (LH)

2) Select one QC material

• By example

Bio-Rad LyphoCheck Immunoassay plus

3) Choose TEa

• Can we use biological variation ?





• Can we use biological variation based TEa ?

• Look at analytical CV (CVa) and compare to CVi

• Analytical CV=

• Level 1: 1,91 %

• Level 3: 1,4 %

• CVi (Ricos)= 14,5 %

• Comparison CVa/CVi: ratio CVa/CVi =

• Level 1: 0,131 (<0,25)

• Level 2: 0,096 (<0,25)

BV based TEa is applicable





4) Evaluate method:

a) CVa, bias

• CVa:

• Level 1: 1,91 %

• Level 3: 1,4 %

• Bias (comparison to peer group):

• Level 1: 0 %

• Level 3: 2,44 %





4) Method evaluation:

b) Sigma levels

• Level 1:

• BV desirable = (19,8-0)/1,91= 10,37

• BV optimum = 5,18

• Level 3:

• BV desirable = 12,40

• BV optimum = 5,33



Sigma levels - practical example5) Calculate OPSpecs and PowerFunction graphs



LH

Level 1





LH

Level 1





LH

Level 3





LH

Level 3

2. Internal Quality Control ApplicationsSigma levels - practical example

6) Choose QC rule

7) Put that rule into practice

8) Evaluate results:

1. Follow evolution of Sigma results

2. Number of QC results rejected

3. Peer- group comparison

4. Complains from customers ?

5. External QC performance ?



13s (N=2)

2. Internal Quality Control ApplicationsProtocols in different labs

University lab

Regional lab




Examples of performance- university lab


Test CVi

(%)

CVa

(%)

Precision Bias (%)

TEa (%)

Ricos

Sigma Ricos

TEa (%) tailored

Sigma

tailored

Rule

Albu 3.1 1.83 Min -0.06 3.9 2.1 7.37 3.99 L Multi (n=4)

ALP 6.4 3.79 Des -3.83 11.7 2.08 11.7 2.08 Multi

(n=4)

ALT 24.3 3.56 Opt -1.01 32.1 8.73 20.2 5.39 S 13s

(n=2)

Amy 8.7 1.95 Opt -0.11 14.6 7.43 9.44 5.78 L 13.5s

(n=2)

AST 11.9 2.95 Opt -1.59 15.2 4.61 9.62 2.72 S Multi

(n=4)

UCL data, March 2008

Presented at QITS meeting, Antwerpen, March 2008


Examples of performance- regional lab


CSL Arlon data, 2008-2009

Presented at AACC 2009, Chicago, USA

Clinical Chemistry 2009; 55(S6):A32 (Abstract A-97).

Testing Control material

Frequency TEa TEa type Sigma

level

QC rule applied

Lactate Multiqual 2 times/ day

15% <BV desirable 10 1-3s

Uric Acid Multiqual 1 time/ day 14.8 % BV desirable 12.1 1-4s

Albumin Multiqual 1 time/ day 7,5 % User defined 4.3 1-3s

Amylase Multiqual 1 time/ day 14.6 % BV desirable 11.8 1-4s

ASLO Multiqual 1 time/ day 15 % User defined 4.6 1-3s

Conj. Bilirubin Multiqual 1 time/ day 44.5 % BV desirable 22.3 1-4s

Total Bilirubin Multiqual 1 time/ day 31.1 % BV desirable 10.4 1-3s

Calcium Multiqual 2 times/ day

5 % User defined 5.9 1-3s/2-2s/R4s

Total Chol Multiqual 1 time/ day 9 % BV desirable 4.6 1-3s/2-2s

Chol- HDL Multiqual 1 time/ day 11.1% BV desirable 4.6 1-3s/2-2s

2. Internal Quality Control ApplicationsiQC applications summary:

1. Six Sigma can be applied as a quality indicator of analytical performance

2. Sigma values will provide a summary of test performance that is a reflexion of quality demand and test variability

3. Different sigma values mean different QC rules

4. Application of Six Sigma to design QC procedures has been performed in various lab types



- PLAN -

1) Six Sigma basics






3. External Quality Control Applications

Six Sigma based indicators for national QC programs

• Definition of 3 estimates of quality based on Sigma metrics :

1) National Test Quality (NTQ), where NTQ= TEa / CVgroup

2) National Method Quality (NMQ), where NMQ= [TEa-Biasms] / CVms

3) Local Method Quality (LMQ), where LMQ= TEa /

CVmethodsubgroup



Westgard JO and Westgard SA. Am J Clin Pathol 2006; 125:343-354

3. External Quality Control Applications

Six Sigma based indicators for national QC programs

• Interest to develop such indicators in Belgium ?

• Use of IPH external quality assessment data

• Use of CLIA TEa, comparison with Ricos tables




Belgian indicator : Glucose (GM :14.54 mmol/L – 262 mg/dL)

TEa CLIA = 10 % TEa Ricos: 6.9 %

Using IPH data from 4th Trimester 2007

C/7930 CV (%) LMQ CLIA LMQ Ricos NTQ

1. Hexokinase (N=151) 2.7 3.7 Sigma 2.55 Sigma

2. Glucose deshydrogenase/

NAD (N=1)/ / /

3. Glucose oxydase PAP (N=6) 2 5 Sigma 3.45 Sigma

4. Glucose oxydase O2 elect.

(N=18)

3.4 2.94 Sigma 2.02 Sigma

5. Reflectance photometry

(N =35)2.3 4.34 Sigma 3 Sigma

TOTAL RESULTS (N=211) 2.7 CLIA: 3.7

RIcos: 2.5



Belgian indicator : Amylase TEa CLIA = 30 % TEa Ricos= 14.6 %

C/7930 CV (%) LMQ CLIA LMQ Ricos

1. Kinetic method-VIS photom (N =1)

/ / /

2. Kinetic methods-VIS photometry (chloro PNP maltotrioside) (N =23)


3. Kinetic methods-UV photometry (maltotetraose)

(N =16)


4. Reflectance photometry (amylopectin) (N =35)


5. Kin. meth-VIS photom. (PNP maltoheptaosideethylidene)

(N =131)




Belgian indicator : Calcium (GM :2.29 mmol/L – 9.5 mg/dL)

TEa CLIA = 10.5 % TEa Ricos: 2.4 %

C/7930 CV (%) LMQ CLIA LMQ Ricos NTQ

1. VIS photometry (o- cresolphtalein) (N=127)


2. Reflectance photometry (arsenazo III) (N =34)


3. VIS photometry (arsenazo III) (N =28)

2.4 4.4 Sigma 1 Sigma

4. Indirect potentiometry

(N = 17)1.3 8.07 Sigma 1.84 Sigma

TOTAL RESULTS (N=206) 2.4 CLIA:4.37

RIcos: 1

2. External Quality Control ApplicationseQC applications summary:

1. Six Sigma can be applied as a quality indicator of analytical performance on a national level

2. Methods giving close CVs have in fact a different Sigma performance- differences in methods are amplified by a Sigma level calculation

3. Use of Ricos TEas for national QC indicators need conscious interpretation...



- PLAN -

1) Six Sigma basics






4. Tendencies and ControversiesTendency 1:

Application of Sigma levels as ISO 15189 quality indicators (1/2):

• Quality control procedures are the subject of a QSE (Quality Systems Essential) in the ISO 15189 norm

• It is the QSE 5.6 (« Assuring Quality of Examination Procedures »)



4. Tendencies and ControversiesTendency 1:

Application of Sigma levels as ISO 15189 quality indicators (2/2):

• Some important points of QSE 5.6:• QC results must be recorded• QC design and analytical objectives must also be recorded

• Comparison with peer group is useful and wished

• Many of these points are included in a Sigma value; moreover, Sigma levels for each test are interesting to monitor test performance over time



4. Tendencies and Controversies

Tendency 2:

Groups of tests with similar Sigma performances are monitored using similar QC rules (1/2):

• Many labs find similar groups of tests having similar Sigma levels and that can be monitored by similar QC rules

• These groups allow an intelligent adaptation of QC rules and the establishment of feasible QC protocols




Tendency 2:

Groups of tests with similar Sigma performances are monitored using similar QC rules (2/2):



Group 1

Sigma > 6

13s

QC once a day

Group 2

Sigma > 4.5

13s

QC twice a day

Group 3

Sigma > 3

13s/22s/R4s/41s/10x

QC once a day

Group 4

Sigma < 3

13s/22s/R4s/41s/10x

QC twice a day

Or

Combination with other QC

strategies

QC rules may vary according labs’ QC philosophy


Tendency 3:

Combination of simple QC rules with less known rules:

• 7T QC rule: reject a run when 7 results are going the same way (up or down): allows to easily detect a drift when using “large” general QC rules such as 13s or 15s

• Application of classical rules accross two QC levels (INTER-level application >< Inra-level application: may be useful to react faster and take preemptive corrective actions




Controversy 1:

Biological Variation based TEas are too stringent and not applicable (1/2):

• BV based TEas may look too stringent at first glance

• However, it is possible to design a routine QC protocol based on Six Sigma and using: • BV based TEa (desirable precision)• A two months cumulative CV• Peer related bias




Controversy 1:

Biological Variation based TEas are too stringent and not applicable (2/2):


TEa used for Clinical Chemistry tests

72%

5% 5%

18%

Biological Variationdesirable precision

User defined, morestringent than BVdesirable precision

CLIA

User defined

TEa used for Immunoassays

82%

6%

12%Biological Variationdesirable precision

User defined

Not Applicable (no TEafound)

29%

27%

12%

20%

8%4%

Biological Variationdesirable precision

User defined, morestringent than BVdesirable precisionCLIA

More stringent than CLIA

User defined

Not Applicable

TEa used on the back up automate

CSL Arlon data, 2008-2009

Presented at AACC 2009, Chicago, USA

Clinical Chemistry 2009; 55(S6):A32 (Abstract A-97).


Controversy 2:

There are practical problems encountered with the individuation of QC rules

• Adaptation of QC rules to every test may result in complex QC protocols that are not applicable in practice

• The solution may be to divide tests in various groups according to their performance on the Sigma Scale

• The aim is to create a QC protocol that is reflective of each test Biological Variation characteristics and analytical performance but that is feasible in routine




Controversy 3:

There are problems encountered with rules oversimplification (1/2):

• For some tests with excellent Sigma performance, “loose” QC rules (15s, by example) are sometimes proposed by QC softwares

• These rules are clearly insufficient to detect small changes in QC results




Controversy 3:

There are problems encountered with rules oversimplification (2/2):

• Solutions:

1. Combination of simple rules with innovative rules

2. Do not use QC rules that are too loose

3. Combine classical statistical QC with other concepts, by example, patient mean monitoring




Controversy 4:

There are unanswered questions with Six Sigma QC:

• Implication of the different types of BV based TEa in a Six Sigma QC (optimum-desirable-minimum) ?

• How to perform QC for tests that are run on different analysers ?

• What is the return on investment on such a QC strategy ?

• Is statistical QC enough ?



Challenges when creating a Six Sigma QC protocol



• Review QC procedures for every test (>100 in a corelab)

• Create an advanced QC protocol that is feasible

• Gain acceptance by MLTs

• Evaluate the ROI

• Combine with other QC approaches (using patient data, combination with risk management)

Conclusions



• Six Sigma quality management initiatives have been succesfully launched in laboratories

• QC based on Six Sigma and biological variation based analytical objectives is feasible

• Sigma levels for lab tests are excellent quality indicators• Six Sigma based QC is certainly useful in the context of

ISO 15189 accreditation• Sigma indicators could add information to a national QC

program• Future deployment of Six Sigma based QC can answer

important questions, notably regarding ROI



Many thanks for your attention !

[email protected]

Six Sigma based quality control- Local and national perspectives -

Jérémie Gras Médecin Biologiste

Laboratoire - Clinique St-Luc Bouge

Turning Science Into Caring Conference

Living Tomorrow – Vilvoorde

7th October 2010

J.Gras Six SigmaTSIC 2010

Documents

sigma process

sigma methodology

sigma applications

sigma value

sigma performanceis

sigma principles

sigma inmanufacturing

ahigh sigma levelcorrelates