Tony Badrick - Royal College of Pathologists of Australasia Quality Assurance Programs

The Role of External Quality Assurance in Improving

Patient Safety

Tony Badrick

EQA – the more you look the more you see!

Date of download: 10/11/2016Copyright © 2016 American Medical

Association. All rights reserved.

From: Diagnostic Error in MedicineAnalysis of 583 Physician-Reported Errors

Arch Intern Med. 2009;169(20):1881-1887. doi:10.1001/archinternmed.2009.333

Classification of diagnostic errors in 583 physician-reported cases using the Diagnostic Error Evaluation and Research project tool

to localize where in the diagnostic process error occurred.

Figure Legend:

Benefits of EQA participation

• Characterise test bias and imprecision across multiple methods

• Correlate specific method variables with bias and imprecision

• Identify interfering substances and quantify their effects across multiple methods

Benefits of EQA participation

• Provide clinical laboratories with reliable information for replacing unsatisfactory methods

• Identify clinical laboratories that are at risk for poor performance

• Satisfy accreditation and regulatory requirements

Benefits of EQA participation

• For most laboratories, EQA and accreditation represent a miniscule cost compared to total budget, rarely exceeding 0.1%.

• Contribute directly to opportunities for continual improvement and continuing education.

14

Benefits of EQA participation

• Provide an opportunity that integrates with preventive action, and with detection of occurrences that lead to appropriate remedial and corrective actions.

• Provide the opportunity to earn certificates that provide the public with visual thus providing a reasonable basis for public and client confidence.

Position of EQA in the Laboratory

Laboratory Quality Management Systemeg ISO 15189

Quality Policy

Quality Assurance

Quality Control

Internal Quality

Control

External Quality

Assurance

Other aspects of good

laboratory practice

All measures taken to assure quality

•Sturgeon C. Best Pract & Res Clin Endo & Metab 2013; 27: 802-822

Laboratory Method Standardisation

Refe

rence m

ate

rials

Refe

rence L

abora

torie

s

Re

fere

nce

Me

tho

ds

Tra

ceable

refe

rence In

terv

als

Exte

rnal Q

uality

Assura

nce

Key Requirements of an EQA

• Specimen distribution frequency and reporting requirements

• Specimen material distributed

• Definition of target values

• Assessment of overall performance

• Assessment of individual method performance

Key Requirements of an EQA

• Assessment of method robustness to clinically relevant interference

• Assessment of individual laboratory performance

• Communication with participating laboratories

• Audit of wider aspects of analytical performance and educational activities

EQA driving Improvement

Improvements in performance in medical diagnostics

tests documented by inter-laboratory comparison

programsTholen DW (2002) Accreditation and Quality Assurance Vol 7, Issue 4:146-152

A survey of clinical genetic laboratories in the United

States indicated that increased participation in PT

correlated directly with fewer PT failures and fewer

incorrect patient test reportsHudson KL, Murphy JA, Kaufman DJ et al (2006) Oversight of US genetic testing laboratories

Nat Biotechnol. 24(9):1083–1090.

There is evidence that EQA failure rates decrease with

increased experience performing EQA.

EQA driving Improvement

Failures in the EQA program over the last five years have helped us to

significantly improve our transfusion service in terms of performance

evaluation, patient care and safety issues, and the overall quality of

laboratory practices.Chaudary R, Das SS. Ojha S, Khetan D, and Sonker A (2010) The external quality assessment

scheme: Five years experience as a participating laboratory. Asian J Transfus Sci. 4(1): 28–30.

External quality assessment (EQA) has been shown to improve laboratory

performance and diagnosis in haemostasis. We report here findings from

the World Federation of Haemophilia EQA programme during the

period 2004–2007.Jennings I, Kitchen DP, Woods TAL, et al (2009) Haemophilia Volume 15, Issue 2:571–577

External quality assessment schemes raise standards: evidence from the

UKNEQAS parasitology subschemesKettlehut MM, Chiodini PL, Edwards H and Moody A (2003) J Clin Pathol 2003;56:927-932

doi:10.1136/jcp.56.12.927

EQA driving Improvement

Poor PT/EQA performance because of interpretation

errors has decreased between 1997 and 2006. This may

be derived from continued participation in PT/EQA.Ramsden SC, Deans Z, Robinson DO, et al (2006) Monitoring standards for

molecular genetic testing in the United Kingdom, The Netherlands and Ireland.

Genetic Test 10(3):147–156.

90% of the EQA problems were resolved after the first round of EQA and 99% were resolved by the third round, suggesting that the laboratories had successfully corrected mistakes identified by EQA performance.Hoeltge FA, Phillips MG, Styer PE, Mockridge P (2005) Detection and correction of systematic laboratory problems by analysis of clustered proficiency testing failures. Arch Pathol Lab Med.129(2):186–189.

25

RCPAQAP Anatomical Pathology

26 A

N

A

Haematoxylin and Eosin stain exercise (Technical Module)

Influenza Point of Care Testing

| 29

2014 2015Influenza

antigen

Positive

Influenza

antigen

Negative

Influenza

antigen

Positive

Influenza

antigen

Negative

Influenza

antigen

Reactive

171 7

Influenza

antigen

Reactive

218 5

Influenza

antigen Non-

reactive

78 546

Influenza

antigen Non-

reactive

32 551

Sensitivity 68.7% Sensitivity 87.2%

Specificity 98.7% Specificity 99.1%

RCPAQAP has offered an EQA program for Influenza PoCT since 2012.

The specimens are inactivated virus from human strains of Influenza A and B,

suspended in 500μL of buffer representing an eluted swab.

Overall performance of RCPAQAP participants has improved in the last two

years with sensitivity increasing from 68.7% in 2014 to 87.2% in 2015 and

specificity increasing from 98.7% to 99.1% over the same period.

Kalra clin biochem 37;1052-62: 2004

32

33

PITUS- Australia

A national project for the Standardisation of Pathology Units and Terminology (PUTS) was initiated by the RCPA in July 2011 with support from Government. Around 80 pathologists, scientists, informaticians, and other clinicians worked in 8 working groups to establish guidelines for the use of terminology and standardised units covering each of the pathology disciplines.

34

Reference limits and flaggingabnormal results

Date of Birth

15%

48%

5%

32%

0% 10% 20% 30% 40% 50% 60%

dd/mm/yy

dd/mm/yyyy

dd/month/yy

dd/month/yyyy01 April 1980

01 April 80

01/04/1980

01/04/80

Flagging abnormal results

42%

12%

2%

5%

5%

12%

2%

5%

2%

5%

9%

0% 5% 10% 15% 20% 25% 30% 35% 40% 45%

Bold Result, space, Bold H/L

Bold result, no space, Bold H/L

Bold result, * before result

Bold result, space * after result

Bold result, space, H/L (not bold)

Bold result

H/L before result, space, no bold

Bold result, no space, Bold +/-

Bold result, no space, Bold H/L, Underlined

H/L not bold

* before analyte name, Bold result

Corrected Calcium

6%

31%

3%

3%

3%

17%

3%

9%

3%

3%

3%

3%

3%

6%

6%

0% 5% 10% 15% 20% 25% 30% 35%

Ca (Corr)

Corrected Calcium

Adj. Ca.

CaCorr

CorrectedCa

Ca (corrected)

Calcium (Alb. Corr.)

Calcium Corr

Calcium (corr. Alb.)

CorrCalcium

CorCalcium

Adj. Calcium

Corr. Calcium

Calcium Corrected

Ca Alb Corr

11%

28%

6%

6%

6%

6%

6%

17%

17%

0% 5% 10% 15% 20% 25% 30%

TP, Alb, Est Glob, Bili T, Bili C, AST, ALT, GGT, ALP

TP, Alb, Bili T, Bili C, ALP, AST, ALT, GGT

TP, Alb, Glob, Bili T, Bili C, ALP, GGT, ALT, AST

TP, Alb, Glob, Bili T, Bili C, Bili unconj, ALP, ALT, AST, GGT, Amylase, Lipase, LDH

Bili T, Bili C, ALT, AST, LDH, ALKP, GGT, TP, Alb, Glob

Bili, ALP, ALT, GGT, TP, Alb, Glob

Bili T, Bili C, TP, Alb, GGT, ALT, AST, ALP

Bili T, ALP, GGT, ALT, Alb, TP

TP, Alb, Glob, Bili, Bili C, ALP, ALT, GGT

Variations on the LFT

4%

4%

13%

13%

13%

4%

4%

4%

4%

4%

4%

4%

4%

4%

4%

4%

4%

0% 2% 4% 6% 8% 10% 12% 14%

Chol, Trig, LDL-Chol, HDL-Chol, Total/HDL Ratio

Chol, Trig, HDL Chol, nonHDL chol, LDL Chol, Cholesterol/HDL Ratio

Chol, Trig, HDL Chol, LDL Chol

Chol, Trig, LDL Chol, HDL Chol, Chol/HDL Ratio

Chol, Trig, HDL-C, LDL-C, Ratio,

LDL, Chol/HDL

Chol, Trig, LDL Chol (calc), HDL Chol

Chol, HDL, Trig, LDL, Chol/HDL Ratio, CR Index

Chol, Trig, HDL Chol, LDL (calc)

HDL Chol, Chol, Trig, LDL Chol, Non-HDL Chol

Chol, Trig, HDL Chol, LDL Chol (calc), Non-HDL Chol, Chol/HDL Ratio

LDL Chol, Non HDL Chol, Trig

Chol HDL, Chol LDL, Trig

T Chol , LDL Chol, HDL Chol, Trig

Chol, Trig, HDL, Chol/HDL Ratio, LDL, Non HDLC

Chol, Trig, HDLC, LDL Chol, TC/HDLC, LDL/HDL

Chol, Trig, HDL Chol, Chol/HDL, Non HDL-Chol, LDL Chol

Variations on the lipids

Interpretative Comments

• Commenting on Results of EQA

• Often occurs as part of an normal EQA

– Anatomical Pathology

– Transfusion

– Haematology

• Model for all Reported Comments?

41

Interpretative Comments

The ideal interpretive comment:

(i) describes the abnormalities in the technical

data,

(ii) interprets that information including the

clinical implications such as for diagnosis, and

(iii) provides knowledge for follow-up

including further testing or specialist referral.

Informatics EQA

• EQA is usually an artificial process

– No normal request

– Sample added

– Report not generated

• Should test the entire cycle

– Request

– Result

– Report

44

IT EQA

45

EQA for Referring Doctors

Quality is here to stay

• We need to ensure that EQA is fit for the future

• A systematic approach should be taken to educating, training and developing the skills of the pathology workforce in quality management systems and quality improvement methodology