C24-A2

C24-A2Vol. 19 No. 5

Replaces C24-AFebruary 1999 Vol. 11 No. 6

Statistical Quality Control for Quantitative Measurements:Principles and Definitions; Approved Guideline—Second Edition

This guideline provides definitions of analytical intervals, planning of quality control procedures, andguidance for quality control applications.

ABC

NCCLS...Serving the World's Medical Science Community Through Voluntary Consensus

NCCLS is an international, interdisciplinary, nonprofit, proposed standard or guideline. The document shouldstandards-developing and educational organization that receive a wide and thorough technical review, including anpromotes the development and use of voluntary consensus overall review of its scope, approach, and utility, and a line-standards and guidelines within the healthcare community. by-line review of its technical and editorial content.It is recognized wordwide for the application of its uniqueconsensus process in the development of standards and Tentative A tentative standard or guideline is madeguidelines for patient testing and related healthcare issues. available for review and comment only when aNCCLS is based on the principle that consensus is an recommended method has a well-defined need for a fieldeffective and cost-effective way to improve patient testing evaluation or when a recommended protocol requires thatand healthcare services. specific data be collected. It should be reviewed to ensure

In addition to developing and promoting the use ofvoluntary consensus standards and guidelines, NCCLS Approved An approved standard or guideline has achievedprovides an open and unbiased forum to address critical consensus within the healthcare community. It should beissues affecting the quality of patient testing and health reviewed to assess the utility of the final document, tocare. ensure attainment of consensus (i.e., that comments on

PUBLICATIONS

An NCCLS document is published as a standard, guideline, opinion on good practices and reflect the substantialor committee report. agreement by materially affected, competent, and

Standard A document developed through the consensus established consensus procedures. Provisions in NCCLSprocess that clearly identifies specific, essential standards and guidelines may be more or less stringentrequirements for materials, methods, or practices for use in than applicable regulations. Consequently, conformance toan unmodified form. A standard may, in addition, contain this voluntary consensus document does not relieve thediscretionary elements, which are clearly identified. user of responsibility for compliance with applicable

Guideline A document developed through the consensusprocess describing criteria for a general operating practice,procedure, or material for voluntary use. A guideline may COMMENTSbe used as written or modified by the user to fit specificneeds. The comments of users are essential to the consensus

Report A document that has not been subjected to con- comments are addressed, according to the consensussensus review and is released by the Board of Directors. process, by the NCCLS committee that wrote the

CONSENSUS PROCESS consensus level and those that do not result in a change,

The NCCLS voluntary consensus process is a protocol document. Readers are strongly encouraged to commentestablishing formal criteria for: in any form and at any time on any NCCLS document.

! The authorization of a project West Valley Road, Suite 1400, Wayne, PA 19087, USA.

! The development and open review of documents

! The revision of documents in response to comments byusers Healthcare professionals in all specialities are urged to

! The acceptance of a document as a consensus contact the NCCLS Executive Offices for additionalstandard or guideline. information on committee participation.

Most NCCLS documents are subject to two levels ofconsensus–"proposed" and "approved." Depending on theneed for field evaluation or data collection, documents mayalso be made available for review at an intermediate (i.e.,"tentative") consensus level.

Proposed An NCCLS consensus document undergoes thefirst stage of review by the healthcare community as a

its utility.

earlier versions have been satisfactorily addressed), and toidentify the need for additional consensus documents.

NCCLS standards and guidelines represent a consensus

interested parties obtained by following NCCLS’s

regulations.

process. Anyone may submit a comment, and all

document. All comments, including those that result in achange to the document when published at the next

are responded to by the committee in an appendix to the

Address comments to the NCCLS Executive Offices, 940

VOLUNTEER PARTICIPATION

volunteer for participation in NCCLS projects. Please

Vol. 19 No. 5 C24-A2

i

THE NCCLS consensus process, which is the mechanism for moving a document through two ormore levels of review by the healthcare community, is an ongoing process. Users should expectrevised editions of any given document. Because rapid changes in technology may affect theprocedures, methods, and protocols in a standard or guideline, users should replace outdatededitions with the current editions of NCCLS documents. Current editions are listed in the NCCLSCatalog, which is distributed to member organizations, and to nonmembers on request. If yourorganization is not a member and would like to become one, and to request a copy of the NCCLSCatalog, contact the NCCLS Executive Offices. Telephone: 610.688.0100; Fax: 610.688.0700;E-Mail: [email protected].

Abstract

Statistical Quality Control for Quantitative Measurements: Principles and Definitions; ApprovedGuideline—Second Edition (NCCLS document C24-A2) addresses the principles of statistical qualitycontrol (QC), with particular attention to the planning of a QC strategy, the definition of an analyticalrun, the selection of control materials, and the application of statistical QC in a healthcare laboratory.This guideline is a revision of an earlier guideline. The original definitions are maintained for themanufacturer recommendation run length (MRRL) and the user defined run length (UDRL). Changesinclude a strong emphasis on defining quality up front to guide the selection of control rules and thenumber of control measurements, recognition that methodology should be developed to establish runlengths on a scientific basis, and a recommendation that the best response to an out-of-control situationis to identify the sources of the problem and eliminate the cause, rather than routinely repeating controlmeasurements.

(NCCLS. Statistical Quality Control for Quantitative Measurements: Principles and Definitions; ApprovedGuideline—Second Edition. NCCLS document C24-A2 [ISBN 1-56238-371-X]. NCCLS, 940 WestValley Road, Suite 1400, Wayne, Pennsylvania, 19087, USA, 1999.)

February 1999 NCCLS

ii

C24-A2ISBN 1-56238-371-X

February 1999 ISSN 0273-3099

Statistical Quality Control for Quantitative Measurements:Principles and Definitions; Approved Guideline—Second Edition

Volume 19 Number 5

James O. Westgard, Ph.D., ChairholderRobert W. Burnett, Ph.D.W. Gregory CooperGary A. Graham, Ph.D., DABCCThomas L. Hearn, Ph.D.Curtis A. Parvin, Ph.D.Donald R. Parker, Ph.D.

ABC

February 1999 NCCLS

iv

This publication is protected by copyright. No part of it may be reproduced, stored in a retrieval system,or transmitted in any form or by any means (electronic, mechanical, photocopying, recording, orotherwise) without written permission from NCCLS, except as stated below.

NCCLS hereby grants permission to reproduce limited portions of this publication for use in laboratoryprocedure manuals at a single site, for interlibrary loan, or for use in educational programs provided thatmultiple copies of such reproduction shall include the following notice, be distributed without charge,and, in no event, contain more than 20% of the document's text.

Reproduced with permission, from NCCLS publication C24-A2—Statistical Quality Control forQuantitative Measurements: Principles and Definitions; Approved Guideline—Second Edition.Copies of the current edition may be obtained from NCCLS, 940 West Valley Road, Suite1400, Wayne, Pennsylvania 19087 USA.

Permission to reproduce or otherwise use the text of this document to an extent that exceeds theexemptions granted here or under the Copyright Law must be obtained from NCCLS by written request.To request such permission, address inquiries to the Executive Director, NCCLS, 940 West Valley Road,Suite 1400, Wayne, Pennsylvania 19087 USA.

Copyright ©1999. The National Committee for Clinical Laboratory Standards.

Suggested Citation

NCCLS. Statistical Quality Control for Quantitative Measurements: Principles and Definitions; ApprovedGuideline—Second Edition. NCCLS Document C24-A2 (ISBN 1-56238-371-X). NCCLS, 940 WestValley Road, Suite 1400, Wayne, Pennsylvania, 19087 USA, 1999.

Proposed GuidelineMarch 1985

Tentative GuidelineSeptember 1986

Approved GuidelineMay 1991

Approved Guideline—Second EditionFebruary 1999

ISBN: 1-56238-371-XISSN: 0273-3099

Vol. 19 No. 5 C24-A2

v

Committee Membership

Area Committee on Clinical Chemistry and Toxicology

Basil T. Doumas, Ph.D. Medical College of WisconsinChairholder Milwaukee, Wisconsin

W. Gregory Miller, Ph.D. Virginia Commonwealth UniversityChairholder Richmond, Virginia

Paul D’Orazio, Ph.D. Chiron Diagnostics CorporationMedfield, Massachusetts

John H. Eckfeldt, M.D., Ph.D. Fairview-University Medical CenterMinneapolis, Minnesota

Susan A. Evans, Ph.D. Dade International Inc.Deerfield, Illinois

Thomas P. Moyer, Ph.D. Mayo ClinicRochester, Minnesota

Gary L. Myers, Ph.D. Centers for Disease Control and PreventionAtlanta, Georgia

Patrick J. Parsons, Ph.D. Wadsworth Ctr. For Labs. and ResearchAlbany, New York

Noel V. Stanton, M.S. University of WisconsinMadison, Wisconsin

Working Group on Internal Quality Control Testing

James O. Westgard, Ph.D. University of WisconsinChairholder Madison, Wisconsin

Robert W. Burnett, Ph.D. Hartford HospitalHartford, Connecticut

W. Gregory Cooper Bio-Rad LaboratoriesIrvine, California

Gary A. Graham, Ph.D., DABCC Ortho Clinical DiagnosticsRochester, New York

Thomas L. Hearn, Ph.D. Centers for Disease Control and PreventionAtlanta, Georgia

Curtis A. Parvin, Ph.D. Washington Univ. School of MedicineSt. Louis, Missouri

February 1999 NCCLS

vi

Advisors

Dr. M.L. Castillo de Sánchez Confed. Latinoamericana de Bioquim. Clin.Mexico

Carrie Ebenhardt MDS Autolab SystemsOntario, Canada

Kevin D. Fallon, Ph.D. Instrumentation LaboratoryLexington, Massachusetts

Carl C. Garber, Ph.D. Quest Diagnostics, IncorporatedTeterboro, New Jersey

Veronica C. Malmberg NH Division of Public Health Svcs.Concord, New Hampshire

Donald R. Parker, Ph.D. Bayer CorporationElkhart, Indiana

Donald M. Powers, Ph.D. Ortho Clinical DiagnosticsRochester, New York

Dr. Huberto Marques Tibúrcio Sociedade Brasileira de Análises ClínicasRio de Janeiro

Sharon S. Ehrmeyer, Ph.D. University of WisconsinBoard Liaison Madison, Wisconsin

Beth Ann Wise, M.T.(ASCP), M.S.Ed. NCCLSStaff Liaison Wayne, Pennsylvania

Patrice E. Polgar NCCLSEditor Wayne, Pennsylvania

Vol. 19 No. 5 C24-A2

vii

ACTIVE MEMBERSHIP (as of 1 JANUARY 1999)

Sustaining Members

Abbott LaboratoriesAmerican Association for Clinical ChemistryBayer CorporationBeckman Coulter, Inc.Becton Dickinson and CompanybioMérieux Vitek, Inc.College of American PathologistsDade Behring Inc.Ortho-Clinical Diagnostics, Inc.Pfizer IncRoche Diagnostics, Inc.

Professional Members

American Academy of Family PhysiciansAmerican Association of BioanalystsAmerican Association of Blood BanksAmerican Association for Clinical ChemistryAmerican Association for Respiratory CareAmerican Chemical SocietyAmerican Medical TechnologistsAmerican Public Health AssociationAmerican Society for Clinical Laboratory ScienceAmerican Society of HematologyAmerican Society for MicrobiologyAmerican Society of Parasitologists, Inc.American Type Culture Collection, Inc.Asociacion Espanola Primera de SocorrosAsociacion Mexicana de Bioquimica Clinica A.C.Assn. of Public Health LaboratoriesAssoc. Micro. Clinici Italiani- A.M.C.L.I.Australasian Association of Clinical BiochemistsBritish Society for Antimicrobial ChemotherapyCanadian Society for Medical Laboratory Science—Société Canadienne de Science de Laboratoire MédicalCanadian Society of Clinical Chemists

Clinical Laboratory Management Department of Veterans Affairs Association Deutsches Institut für NormungCollege of American (DIN) Pathologists FDA Center for Devices andCollege of Medical Laboratory Radiological Health Technologists of Ontario FDA Division of Anti-InfectiveCollege of Physicians and Drug Products Surgeons of Saskatchewan Federacion Bioquimica de la Commission on Office Provincia (Argentina) Laboratory Accreditation Health Care FinancingInstitut für Stand. und Dok. im Administration Med. Lab. (INSTAND) Instituto Scientifico HS. International Council for Raffaele (Italy) Standardization in Iowa State Hygienic Laboratory Haematology Manitoba HealthInternational Federation of Massachusetts Department of Clinical Chemistry Public Health LaboratoriesInternational Society for Michigan Department of Public Analytical Cytology HealthItalian Society of Clinical National Association of Testing Biochemistry Authorities - AustraliaJapan Society of Clinical National Institute of Standards Chemistry and TechnologyJapanese Committee for Clinical National Institutes of Health Laboratory Standards Ohio Department of HealthJoint Commission on Oklahoma State Department of Accreditation of Healthcare Health Organizations Ontario Ministry of Health National Academy of Clinical Saskatchewan Health- Biochemistry Provincial Laboratory National Society for South African Institute for Histotechnology, Inc. Medical ResearchOntario Medical Association Swedish Institute for Infectious Laboratory Proficiency Testing Disease Control ProgramOrdre professionnel des technologistes médicaux du QuébecRCPA Quality Assurance Programs PTY LimitedSociedade Brasileira de Analises Clinicas Sociedade Brasileira de Patologia ClinicaSociedad Espanola de Quimica ClinicaVKCN (The Netherlands)

Government Members

Armed Forces Institute of NY Pathology Bayer Corporation - West Haven,Association of Public Health CT Laboratory Directors Bayer-Sankyo Co., Ltd.BC Centre for Disease Control Beckman Coulter, Inc.Centers for Disease Control and Beckman Coulter, Inc. - Palo Prevention Alto, CAChinese Committee for Clinical Beckman Instruments (Japan) Laboratory Standards Ltd.Commonwealth of Pennsylvania Becton Dickinson and Company Bureau of Laboratories

Industry Members

AB BiodiskAbbott LaboratoriesAccuMed International, Inc.Accumetrics, Inc.Amersham Pharmacia BiotechAmmirati Regulatory ConsultingAsséssorAvecor Cardiovascular, Inc.Avocet Medical, Inc.Bayer Corporation - Elkhart, INBayer Corporation - Middletown, VABayer Corporation - Tarrytown,

February 1999 NCCLS

viii

Becton Dickinson Consumer Labtest Sistemas Diagnosticos SmithKline Beecham, S.A. Products Ltda. Streck Laboratories, Inc.Becton Dickinson LifeScan, Inc. (a Johnson & Sysmex Corporation (Japan) Immunocytometry Systems Johnson Company) Sysmex Corporation Becton Dickinson Italia S.P.A. LifeSign, LLC (Long Grove, IL)Becton Dickinson Microbiology Lilly Research Laboratories Vetoquinol S.A. Systems Medical Device Consultants, Vysis, Inc.Becton Dickinson VACUTAINER Inc. Wallac Oy Systems Medical Laboratory Automation Warner-Lambert CompanybioMérieux Vitek, Inc. Inc. Wyeth-AyerstBiometrology Consultants MediSense, Inc. Xyletech Systems, Inc.Bio-Rad Laboratories, Inc. Merck & Company, Inc. YD ConsultantBiotest AG Neometrics Inc. ZenecaBristol-Myers Squibb Company Nichols Institute Diagnostics (Div.Canadian Reference Laboratory of Quest Diagnostics, Inc.) Ltd. Nissui Pharmaceutical Co., Ltd.CASCO•NERL Diagnostics Nippon Becton Dickinson Co., Checkpoint Development Inc. Ltd.Chiron Diagnostics Corporation - Norfolk Associates, Inc. International Operations North American Biologicals, Inc.Chiron Diagnostics Corporation - OBC Associates Reagent Systems Olympus CorporationClinical Lab Engineering Optical Sensors, Inc.COBE Laboratories, Inc. Organon Teknika CorporationCombact Diagnostic Systems Ortho-Clinical Diagnostics, Inc. Ltd. (England)Control Lab (Brazil) Ortho-Clinical Diagnostics, Inc. Cosmetic Ingredient Review (Raritan, NJ)Cubist Pharmaceuticals Ortho-Clinical Diagnostics, Inc. Cytometrics, Inc. (Rochester, NY) Dade Behring Inc. - Deerfield, IL Oxoid Inc.Dade Behring Inc. - Glasgow, Oxoid LTD (U.K.) DE Pfizer IncDade Behring Inc. - Marburg, Pharmacia & Upjohn Germany Procter & GambleDade Behring Inc. - Miami, FL Pharmaceuticals, Inc.Dade Behring Inc. - The Product Development Group Sacramento, CA Radiometer America, Inc.Dade Behring Inc. - San Jose, Radiometer Medical A/S CA David G. Rhoads Associates, DAKO A/S Inc.Diagnostic Products Corporation Rhône-Poulenc RorerDiaSorin Roche Diagnostics GmbHEiken Chemical Company, Ltd. Roche Diagnostics, Inc.Enterprise Analysis Corporation Roche Diagnostic SystemsFort Dodge Animal Health (Div. Hoffmann-La Roche Fujisawa Pharmaceutical Co. Inc.) Ltd. Roche Laboratories (Div.Gen-Probe Hoffmann-La Roche Inc.)Glaxo-Wellcome, Inc. The R.W. JohnsonGreiner Meditech, Inc. Pharmaceutical ResearchHealth Systems Concepts, Inc. InstituteHelena Laboratories Sarstedt, Inc.Hoechst Marion Roussel, Inc. Schering CorporationHybritech, Incorporated Schleicher & Schuell, Inc.Hycor Biomedical Inc. Second OpinionI-STAT Corporation SenDx Medical, Inc.Instrumentation Laboratory Showa Yakuhin Kako Company,Integ, Inc. Ltd.International Technidyne SmithKline Beecham Corporation CorporationKendall Sherwood-Davis & Geck SmithKline Beecham (NZ) Ltd.

Trade Associations

Association of Medical Diagnostic ManufacturersHealth Industry Manufacturers AssociationJapan Association Clinical Reagents Ind. (Tokyo, Japan)Medical Industry Association of Australia

Associate Active Members

20th Medical Group (Shaw AFB, SC)67th CSH Wuerzburg, GE (NY)121st General Hosptial (CA)Acadiana Medical Laboratories, LTD (LA)Advocate Laboratories (IL)The Aga Khan University Medical Center (Pakistan)Alabama Reference LaboratoryAllegheny General Hospital (PA)Allegheny University of the Health Sciences (PA)Allina Laboratories (MN)Alton Ochsner Medical Foundation (LA)Anzac House (Australia)Associated Regional & University Pathologists (UT)Baptist St. Anthony’s Health Network (TX)Baystate Medical Center (MA)Brazileiro De Promocao (Brazil)Bristol Regional Medical Center (TN)Brookdale Hospital Medical Center (NY)Brooke Army Medical Center (TX)Brooks Air Force Base (TX)Broward General Medical Center (FL)Calgary Laboratory Services (Calgary, AB, Canada)

Vol. 19 No. 5 C24-A2

ix

Cardinal Glennon Children’s Hunter Area Pathology Service Nebraska Health System Hospital (MO) (Australia) New Britain General Hospital Central Kansas Medical Center International Health (CT)Champlain Valley Physicians Management Associates, New England Medical Center Hospital (NY) Inc. (IL) Hospital (MA)Children’s Hospital (LA) Intermountain Health Care The New York Blood Center Children's Hospital Medical Laboratory Services (UT) New York State Department of Center (Akron, OH) Jacobi Medical Center (NY) HealthClendo Lab (Puerto Rico) John Randolph Hospital (VA) New York University Medical Colorado Mental Health Institute Johns Hopkins Medical Center at Pueblo Institutions (MD) NorDx (ME)Columbia Tulsa Regional Kaiser Permanente (CA) North Carolina Laboratory of Medical Center (OK) Kenora-Rainy River Regional Public HealthCommonwealth of Kentucky Laboratory Program (Dryden, North Coast Clinical Community Medical Center (NJ) Ontario, Canada) Laboratory, Inc. (OH)CompuNet Clinical Laboratories Klinicni Center (Slovenia) North Shore University (OH) La Rabida Children’s Hospital Hospital (NY)Consolidated Laboratory (IL) Northwestern Memorial Services (CA) LabCorp (NC) Hospital (IL)Covance CLS (IN) Laboratoire de Santé Publique Ohio State University HospitalsDanville Regional Medical Center du Quebec (Canada) Olin E. Teague Medical Center (VA) Lancaster General Hospital (PA) (TX)Detroit Health Department (MI) Langley Air Force Base (VA) Our Lady of Lourdes HospitalDuke University Medical Center Loma Linda University Medical (NJ) (NC) Center (CA) Our Lady of the ResurrectionDuzen Laboratories (Turkey) Los Angeles County and USC Medical Center (IL)E.A. Conway Medical Center Medical Center (CA) Permanente Medical Group (LA) Louisiana State University (CA)East Texas Medical Center Medical Center Providence Health System (OR)Elmhurst Memorial Hospital (IL) Lutheran Hospital (WI) Providence Medical Center Emory University Hospital (GA) Main Line Clinical Laboratories, (WA)Fairview-University Medical Inc. (PA) Queen Elizabeth Hospital Center (MN) Massachusetts General Hospital (Prince Edward Island, Florida Hospital Alta Monte MDS Metro Laboratory Services Canada)Florida Hospital East Orlando (Burnaby, BC, Canada) Queensland Health Pathology Foothills Hospital (Calgary, AB, MDS-Sciex (Concord, ON, Services (Australia) Canada) Canada) Quest Diagnostics (PA)Fox Chase Cancer Center (PA) Med-Chem Laboratories Ltd. Quest Diagnostics Incorporated Fresenius Medical Care/Life (Scarborough, ON, Canada) (NJ) Chem (NJ) Medical Center Hospital (TX) Quintiles Laboratories, Ltd. Grady Memorial Hospital (GA) Memorial Medical Center (LA) (GA)Guthrie Clinic Laboratories (PA) Memorial Medical Center (IL) Regions HospitalHacettepe Medical Center Mercy Health System (PA) Research Medical Center (MO) (Turkey) Mercy Hospital (NC) Riyadh Armed Forces Hospital Harris Methodist Fort Worth Methodist Hospital (TX) (Saudi Arabia) (TX) Methodist Hospital Indiana Saint Mary’s Regional Medical Harris Methodist Northwest Methodist Hospitals of Memphis Center (NV) (TX) (TN) St. Alexius Medical Center Hartford Hospital (CT) Milton S. Hershey Medical (ND)Health Alliance Laboratory (OH) Center (PA) St. Anthony Hospital (CO)Health Network Lab (PA) Mississippi Baptist Medical St. Boniface General HospitalHealth Sciences Centre Center (Winnipeg, Canada) (Winnipeg, MB, Canada) Monte Tabor-Centro Italo- St. Francis Medical Center Hoag Memorial Hospital Brazileiro De Promocao (Brazil) (CA) Presbyterian (CA) Montefiore Medical Center (NY) St. John Hospital and Medical Holmes Regional Medical Center Montreal Children’s Hospital Center (MI) (FL) (Canada) St. John Regional Hospital (St. Holzer Medical Center (OH) Mount Sinai Hospital (NY) John, NB, Canada)Hopital de Chicoutimi Mount Sinai Hospital (Toronto, St. Joseph Hospital (NE) (Chicoutimi, PQ, Canada) Ontario, Canada) St. Joseph’s Hospital - Hopital Saint Pierre (Belgium) National Genetics Institute (CA) Marshfield Clinic (WI)

Naval Hospital Cherry Point (NC) St. Luke’s Hospital (PA)

February 1999 NCCLS

x

St. Luke’s Regional Medical Tulane Medical Center Hospital UZ-KUL Medical Center Center (IA) & Clinic (LA) (Belgium)St. Luke’s-Roosevelt Hospital Twin Lake Regional Medical VA (Albuquerque) Medical Center (NY) Center Center (NM)St. Mary Hospital (NJ) UCSF Medical Center (CA) VA (Dayton) Medical Center St. Mary Medical Center (CA) UNC Hospitals (NC) (OH)St. Mary of the Plains Hospital Unilab Clinical Laboratories VA (Denver) Medical Center (TX) (CA) (CO)St. Vincent’s Hospital United Clinical Laboratories VA (Indianapolis) Medical Center (Australia) (IA) (IN)San Francisco General Hospital University of Alabama - VA (Kansas City) Medical (CA) Birmingham Hospital Center (MO)Seoul Nat’l University Hospital University of Alberta Hospitals VA Outpatient Clinic (OH) (Korea) (Canada) VA (Tuskegee) Medical Center Shanghai Center for the University of Chicago Hospitals (AL) Clinical Laboratory (China) (IL) Viridae Clinical Sciences, Inc. Shands Healthcare (FL) University Hospital (IN) (Vancouver, BC, Canada)SmithKline Beecham Clinical University Hospital (Gent) ViroLogic, Inc. (CA) Laboratories (GA) (Belgium) ViroMed Laboratories, Inc. South Bend Medical University Hospital (London, (MN) Foundation (IN) Ontario, Canada) Waikato Hospital (New Zealand)South Western Area Pathology University Hospital of Walter Reed Army Institute of Service (Austraila) Cleveland (OH) Research (MD)Speciality Laboratories, Inc. The University Hospitals (OK) Warde Medical Laboratory (MI) (CA) University of Medicine & Warren Hospital (NJ)Stanford Health Services (CA) Dentistry, NJ University Washoe Medical Center (NV) Stormont-Vail Regional Medical Hospital Watson Clinic (FL) Center (KS) University of Michigan William Beaumont Hospital (MI)Sun Health-Boswell Hospital University of the Ryukyus Williamsburg Community (AZ) (Japan) Hospital (VA)Sunrise Hospital and Medical University of Texas Medical Wilford Hall Medical Center Center (NV) School at Houston (TX)Sutter Health (CA) University of Virginia Medical Wilson Memorial Hospital (NY)Timmins & District Hospital Center Winchester Hospital (MA) (Timmons, ON, Canada) University of Washington Winn Army Community Hospital The Toledo Hospital (OH) UPMC Bedford Memorial (PA) (GA)Tri-City Medical Center (CA) USAF Medical Center (OH) Yonsei University College of Tripler Army Medical Center Medicine (Korea) (HI) York Hospital (PA)Trumbull Memorial Hospital Zale Lipshy University Hospital (OH) (TX)

Vol. 19 No. 5 C24-A2

xi

OFFICERS BOARD OF DIRECTORS

William F. Koch, Ph.D., Carl A. Burtis, Ph.D. Kenneth D. McClatchey, M.D., President Oak Ridge National Laboratory D.D.S.National Institute of Standards Loyola University Medical and Technology Sharon S. Ehrmeyer, Ph.D. Center

F. Alan Andersen, Ph.D., Donald M. Powers, Ph.D. President Elect Elizabeth D. Jacobson, Ph.D. Ortho-Clinical Diagnostics, Inc.Cosmetic Ingredient Review FDA Center for Devices and

Robert F. Moran, Ph.D., Centers for Disease Control FCCM, FAIC Carolyn D. Jones, J.D., M.P.H. and Prevention Secretary Health Industry Manufacturers mvi Sciences Association Marianne C. Watters,

Donna M. Meyer, Ph.D., Hartmut Jung, Ph.D. Parkland Health and Hospital Treasurer Roche Diagnostics GmbH SystemSisters of Charity Health Care System Tadashi Kawai, M.D., Ph.D. Ann M. Willey, Ph.D.

A. Samuel Koenig, III, M.D., Center Health Past PresidentFamily Medical Care

John V. Bergen, Ph.D., Executive Director

University of Wisconsin

Radiological Health Eric J. Sampson, Ph.D.

International Clinical Pathology New York State Department of

M.T.(ASCP)

February 1999 NCCLS

xii

Vol. 19 No. 5 C24-A2

xiii

Contents

Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . i

Committee Membership . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . v

Active Membership . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii

Foreword . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xv

1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

2 Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

3 Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

4 Purpose of Statistical Quality Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

5 Planning a Statistical Quality Control Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

5.1 Define the Quality Requirement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35.2 Determine Method Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35.3 Identify Candidate Statistical QC Strategies . . . . . . . . . . . . . . . . . . . . . . . . . 45.4 Predict QC Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45.5 Set Goals for QC Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45.6 Select Appropriate QC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45.7 Example QC Planning Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

6 Analytical Intervals Defined . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

6.1 Analytical Run . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46.2 Length of Analytical Run . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56.3 Manufacturer's Recommended Run Length (MRRL) . . . . . . . . . . . . . . . . . . . . 56.4 User's Defined Run Length (UDRL) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56.5 Periodic Reassessment of Run Length . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56.6 Alternative Approaches for Establishing Run Lengths . . . . . . . . . . . . . . . . . . . 5

7 Control Materials . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

7.1 Application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57.2 Characteristics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57.3 Relation to Calibrators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67.4 Concentrations of Analytes in Control Materials . . . . . . . . . . . . . . . . . . . . . . 6

8 QC Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

8.1 Statement of QC Strategy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68.2 Frequency of Control Measurements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68.3 Location of Control Samples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68.4 Decision Criteria or Control Rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78.5 Control Charts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78.6 Setting Control Limits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88.7 Out-of-Control Situations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

February 1999 NCCLS

xiv

Contents (Continued)

9 Interlaboratory QC Programs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

Summary of Comments and Committee Responses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

Summary of Delegate Comments and Responses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

Related NCCLS Publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

Vol. 19 No. 5 C24-A2

xv

Foreword

This document is a revision of an earlier document that has been in use by the laboratory communityfor over ten years. When that earlier document was developed, laboratories were experiencing changesin measurement technology and instrument systems that made many of the conventional quality controlpractices difficult to apply. In response to those needs, the earlier document clarified the fundamentalprinciples and definitions of quality control that should be considered when managing any laboratorymeasurement process.

This revision continues that tradition to appraise, clarify, and define concepts, approaches, and practicesthat should be generally useful in developing a specific quality control strategy for testing withquantitative measurements. It maintains a focus on statistical quality control because of the generalcapability of this technique for monitoring the effects of many instrument, reagent, environment, andoperator variables on the outcome of a testing process.

An example of an important concept is the "analytical run," which in the past often corresponded tothe batch of specimens being analyzed. With many modern analytical systems, the definition of a runis not nearly as clear. A run is better understood in terms of the time or number of analyses for whichthe measurement process is stable.

An example of an important approach is the planning of a quality control procedure. A new sectiondescribes how to develop a specific quality control strategy that takes into account the quality requiredby the test, the performance available from a method, the performance expected from different QCstrategies, and the goals set by the laboratory for QC performance.

An example of an important practice is the way a laboratory responds to an out-of-control situation.Following guidelines on statistical quality control proposed by a European working group of the ExternalQuality Assessment Organizers (EQA-Organizers), there is a strong emphasis on trouble-shooting themeasurement process. This response is appropriate when the quality control procedure is carefullyplanned and control rules are selected to minimize false alarms or false rejections.

This document does not attempt to define specific quality-control strategies that are appropriate for anindividual device or technology, nor does it attempt to describe alternatives to statistical process control.(Currently the NCCLS Subcommittee on Unit Use Testing is dealing with these issues.) It should alsobe noted that there are other types of errors that may affect individual samples, rather than a wholegroup of samples, and those errors will not be detected by this type of statistical QC procedure. Sucherrors may be due to the specific design of an analytical system (e.g., effect of sample viscosity,carryover from previous sample) or possible operator errors that affect individual samples. Special QCprocedures are needed to monitor special vulnerabilities that relate to system design.

Nor does this document consider specific legal requirements that may impose different philosophies orprocedures on quality control practices; e.g., a specific approach for defining quality requirements,specific values for quality requirements, a specific procedure for determining target values for the meansof control materials. NCCLS is interested in feedback from users, worldwide, on how to provide a moreglobal approach for quality control guidelines.

The concepts, approaches, and practices discussed here are interdependent and all must be carefullystudied and considered when developing the specifics for any test, system, or laboratory. In an agewhen the quality of laboratory tests is often taken for granted, this document serves as a reminder thatthere are technical issues that still require a careful scientific approach if laboratories are to achieve thequality needed by the physicians and patients they serve.

February 1999 NCCLS

xvi

Foreword (Continued)

The committee wishes to thank all who commented on the first-edition guideline. All comments werecarefully considered, but not all views could be accommodated. Comments are summarized in theAppendix with responses from the committee.

Standard Precautions

Because it is often impossible to know which might be infectious, all patient blood specimens are tobe treated with standard precautions. For specific precautions for preventing the laboratory transmissionof blood-borne infection from laboratory instruments and materials; and recommendations for themanagement of blood-borne exposure, refer to NCCLS document M29—Protection of LaboratoryWorkers from Instrument Biohazards and Infectious Disease Transmitted by Blood, Body Fluids, andTissue.

Key Words

Quality control, calibration, analytical run, quality control rules.

Vol. 19 No. 5 C24-A2

1

Statistical Quality Control for Quantitative Measurements: Principlesand Definitions; Approved Guideline—Second Edition

1 Introduction

There is abundant literature addressing thetheoretical and practical bases for initiating andmaintaining statistical quality control (QC)procedures in clinical chemistry. However,1-6

there still are many difficulties in the routinepractice of statistical quality control andimprovements depend on a better under-standing of how to:

(1) Plan QC on the basis of the quality requiredfor a test

(2) Select appropriate control rules andnumbers of control samples

(3) Define the analytical run(4) Select appropriate control materials (5) Apply QC and respond to out-of-control

situations Accepted reference value, n - A value that

The emergence of automated clinical chemistry comparison and which is derived as ainstruments using widely different instrumental theoretical or established value based onprinciples has complicated the terminology scientific principles; an assigned value based onassociated with and the procedural steps experimental work of some national ornecessary for statistical control testing. On the international organization; or a consensus valueother hand, these highly automated systems based on collaborative experimental work undercan often perform specific electronic checks the auspices of a scientific or engineeringthat help identify potential problems and alert group.the operator to instrument malfunction. Theadvantage of statistical quality control is to Analyte, n - A substance or constituent formonitor the outcome of the many variables and which the laboratory conducts testing. NOTE:steps in the whole analytical process. This includes any element, ion, compound,

2 Scope

This guideline addresses the purpose ofstatistical quality control for quantitativemeasurements; describes an approach forplanning quality control for a specific test andmethod of measurement; defines variousanalytical intervals; and addresses the use ofquality control material and quality control data,including the use of the data in qualityassurance and interpretation. Therecommendations are applicable to quantitativelaboratory tests in all fields. The documentdoes not contain step-by-step procedures forsetting up and maintaining a statistical qualitycontrol program, or for other aspects of qualitycontrol such as instrument function checks anduse of patient data for quality control purposes.

The committee intends this guideline to apply toa broad spectrum of laboratories, from the lowtest volume to the high test volume. Theanalytical performance and quality controlneeded for a testing process must satisfy themedical applications of the particular test,which relate to inherent clinical aspects of thelaboratory's patient population and not to thelaboratory's size, location, or complexity. In thelow-volume environment, special selectivityshould be exercised in deciding whether or notto implement specific test procedures. Onceimplemented, however, quality control isneeded to assure that the test results willsatisfy the medical needs.

3 Definitionsa

serves as an agreed-upon reference for

substance, factor, infectious agent, cell,organelle, activity (enzymatic, hormonal, orimmunological), or property, the presence orabsence, concentration, activity, intensity, orother characteristics of which are to bedetermined. See also Measurand.

Bias, n - The systematic, {signed} deviation ofthe test results from the accepted referencevalue. NOTE: a) Defined in NCCLS documentNRSCL8-A as “the difference between theexpectation of the test results and an acceptedreference value”; b) In general, the deviation/difference is based on replicate measurementusing an accepted (definitive, reference, or

Please see the most currrent edition of NCCLS documenta

NRSCL8—Terminology and Definitions For Use in NCCLSDocuments.

February 1999 NCCLS

2

designated comparison) method and the method observations or measurement results. Abeing tested, and expressed in the units of the measure of variability/dispersion that is themeasurement or as a percentage. positive square root of the population variance.

Imprecision, n - The random dispersion of a set Statistical quality control, n - A procedure inof replicate measurements and/or values which stable samples are measured and theexpressed quantitatively by a statistic, such as observed results compared with limits thatstandard deviation or coefficient of variation. describe the variation expected when theNOTE: The words "imprecision" and "precision" measurement method is working properly. Theare often inappropriately interchanged (Cf. expected variation is determined by analyzing aEP10). stable control material many times, calculating

Matrix, n - All components of a material system, measurements, then calculating control limits asexcept the analyte. the mean plus and minus certain multiples of

Measurand, n - A particular quantity subject to the observed result on the y-axis versus time onmeasurement. NOTE: This term and definition the x-axis. The control limits are drawn on theencompass all quantities, while the commonly chart. New control results are plotted andused term “analyte” refers to a tangible entity compared with the control limits to assesssubject to measurement. For example, whether the method is “in-control” (points“substance“ concentration is a quantity that within control limits) or “out-of-control” (pointsmay be related to a particular analyte. See also outside of control limits.)Analyte.

Quality control, n - The operational techniques result from an infinite number of measurementsand activities that are used to fulfill of the same measurand carried out underrequirements for quality. repeatability conditions, minus a true value of

Quality control strategy, n - The number of equal to error minus random error; b) Like thecontrol materials, the number of measurements true value, systematic error and its causesto be made on those materials, the location of cannot be completely known. those control materials in an analytical run, andthe statistical control rules or decision criteria tobe used for interpreting the control data anddetermining whether or not to accept or rejectan analytical run.

Random error, n - The result of a measurementminus the mean that would result from aninfinite number of measurements of the samemeasurand carried out under repeatabilityconditions.

Repeatability conditions, n - Conditions whereindependent test results are obtained with thesame method on identical test material in thesame laboratory by the same operator using thesame equipment within a short interval of time.

Reportable range, n - The range of test valuesover which the relationship between theinstrument, kit, or system’s measurementresponse is shown to be valid.

Standard deviation, n - The statisticalmeasurement of imprecision among

the mean and standard deviation (SD) of those

the SD. A control chart is prepared to display

Systematic error, n - The mean that would

the measurand. NOTES: a) Systematic error is

4 Purpose of Statistical QualityControl

Statistical quality control procedures areintended to monitor the analytical performanceof a method and alert analysts to problems thatmight limit the usefulness of a test result for itsintended medical purpose. Quality controlshould assure that the analytical performancecharacteristics of the test are appropriate for themedical decisions that need to be made.

Quality control is generally performed byanalyzing stable specimens (or specimens frompatient populations having a stable charac-teristic) and statistically analyzing the data todescribe analytical performance. The statisticsare used to make judgements about the qualityof analytical results, whether system correctionis necessary, whether patient data should beaccepted or rejected, and for estimating perfor-mance parameters which can be compared tothe analytical and medical goals.

Vol. 19 No. 5 C24-A2

3

Statistical quality control testing is different quality required to assure that test results arefrom external quality control testing. In the medically useful. latter, specimens whose values are unknownare submitted to a laboratory from an outsidesource. External quality control testingmeasures a laboratory's ability to obtain thecorrect result on an unknown specimen. Thespecimens are obtained through qualityassurance programs of private, professional, orpublic organizations or through variousgovernmental agencies responsible forlaboratory licensure. External quality controltesting is useful both for quality controlpurposes and for accreditation and licensure.

5 Planning a Statistical Quality ControlProcedure

For statistical quality control procedures to bemost effective, careful planning is necessary.Quality control planning involves several steps,including the following: (1) Defining the qualityrequirements for the test; (2) Determining thestable (in control) performance characteristics ofthe measurement procedure or analyticalmethod; (3) Identifying candidate quality controlstrategies; (4) Predicting the performancecharacteristics of the candidate quality controlstrategies; (5) Specifying desirable goals for theQC performance characteristics; (6) Selecting aquality control strategy whose predictedperformance meets or exceeds the qualitycontrol performance goals.

A quality requirement may be defined in termsof an allowable total analytical error, such asoften provided by proficiency testing criteria foracceptable performance. The allowable total

exceeded would cause a test result to be ofunacceptable quality. It encompasses both7

random and systematic errors, i.e., both methodimprecision and bias. There also arerecommendations for medically importantchanges in test results that similarly include8

both method imprecision and bias, as well aspreanalytical variables such as the within-subject biological variation. Biologic variationitself provides another basis for defining theallowable imprecision and the allowable bias fora test. Clinical treatment models can also be a

5.2 Determine Method Performance

The performance characteristics of an analyticalprocess that are critical for the proper planningof QC procedures are imprecision and bias.Estimates of these parameters should representthe stable performance of an analytical process.In addition to imprecision and bias, it would beuseful to have information about unstableperformance, such as the expected type,magnitude, and frequency of analytical errors,but this information is not generally available.

5.2.1 Imprecision

Imprecision is estimated by repeatedmeasurements on stable control materials. It isgenerally accepted that a minimum of at least20 different bottles should be assayed onseparate days. NCCLS document EP5 calls forperforming two runs a day for at least 20 days.

5.2.2 Bias

Bias should be evaluated in the context of the

the "truth" or accuracy that is being managedby the laboratory. In many cases, the interest

as a method validation study, a clinicalvalidation study, or a calibration event, in whichcases the bias term is often assumed to be zeroand the objective in QC is to monitor changesfrom that baseline period. Other practicalapproaches for estimating bias include thefollowing:

! Comparison with the certified values byanalysis of certified reference materials withthe same matrix and demonstratedcommutability with test samples.

Comparison with assigned values oncommercial assayed control materials ifspecific for the method being evaluated.

! Comparison with the peer group mean inproficiency testing surveys. An accuracybased comparative method target value maybe used when proficiency testing specimenshave demonstrated commuta-bility withpatient specimens.

10

quality requirement defined in Section 5.1 and

is the stable performance since an event, such

5.1 Define the Quality Requirement

error is the magnitude of analytical error that if

sourcesource of information about the analyof information about the analytticicaall

February 1999 NCCLS

4

! Comparison of results obtained on patientspecimens which are analyzed by the testmethod and another routine laboratorymethod (see NCCLS document EP9—Method Comparison and Bias EstimationUsing Patient Samples).

! Comparison of results obtained on patientspecimens which are analyzed by the testmethod and a reference method (seeNCCLS document EP9—Method Compari-son and Bias Estimation Using PatientSamples).

5.3 Identify Candidate Statistical QCStrategies

A quality control strategy is defined by whatcontrol materials are used, how many control When more than one quality control strategysamples are analyzed, where these control meets the quality control performance goals,samples are located, what quality control rules other characteristics such as cost and ease ofare applied to the control sample implementation can be used to select the bestmeasurements, and when the quality control approach.rules are evaluated. The appropriateness of theQC strategy depends on the quality required,as well as the expected instability of theanalytical method (e.g., type, magnitude, andfrequency of errors). Several QC strategiesmay be defined and evaluated.

5.4 Predict QC Performance

The performance of a quality control strategy selecting appropriate QC strategies.can be predicted from probability calculationsor from computer simulation studies. The mostdirect indicator of the performance of a qualitycontrol procedure is the expected number ofunacceptable patient test results that areproduced (or reported) when an out-of-controlerror condition exists. This will depend on11

the type and magnitude of the out-of-controlerror condition, when the error conditionoccurs and how long it lasts, which in turndepends on how frequently quality controltesting occurs and the probability that thequality control rules detect the error condition.These predictions generally assume the shapeof the error distribution is gaussian, which maynot account for some periodic and irregulareffects observed with real laboratory systems,therefore, the complexity of the predictionmodel needs to match the complexity of thepotential error sources of the method andsystem.

5.5 Set Goals for QC Performance

Quality control performance goals set desirabletargets for quality control performance. Thegoal will depend on the chosen quality controlperformance measure. Thus, one goal couldbe specified as a maximum allowable numberof unacceptable results due to anout-of-control error condition, or a maximumallowable probability of reporting unacceptableresults (maximum defect rate), or a minimumacceptable probability of detecting anout-of-control error condition. Another goalcould specify a maximum acceptableprobability of false rejections.

5.6 Select Appropriate QC

5.7 Example QC Planning Applications

Practical approaches for selecting appropriateQC procedures have been described based onpower function graphs, critical-error graphs,and charts of operating specifications.12

Illustrative applications of QC planning areavailable in the literature to provide guidance in

13,14

6 Analytical Intervals Defined

6.1 Analytical Run

For purposes of quality control, an analyticalrun is an interval (i.e., a period of time orseries of measurements) within which theaccuracy and precision of the measuringsystem is expected to be stable. In laboratoryoperations, control samples are analyzedduring each analytical run to evaluate methodperformance, therefore the analytical rundefines the interval (period of time or numberof specimens) between evaluations of controlresults. Between quality control evaluations,events may occur causing the measurementprocess to be susceptible to variations that areimportant to detect.

Vol. 19 No. 5 C24-A2

5

6.2 Length of Analytical Run 6.6 Alternative Approaches for

The length of an analytical run must be definedappropriately for the specific analytical systemand specific laboratory application. Themanufacturer should recommend run length forthe analytical system (MRRL) (see Section 6.3)and the user should define run length for thespecific application (UDRL) (see Section 6.4).

6.3 Manufacturer's Recommended RunLength (MRRL)

The manufacturer should recommend theperiod of time or series of measurementswithin which the accuracy and precision of themeasuring system, including instruments andreagents, are expected to be stable. Themanufacturer should identify events that maycause the measurement process to besusceptible to variations which are importantto detect.

6.4 User's Defined Run Length (UDRL)

The user should define the period of time orseries of measurements within whichvalidation of the measurement process isimportant based on patient sample stability,number of patient samples being analyzed,cost of reanalysis, work flow patterns,operator characteristics, or similar nonanalyticconsiderations that are in addition to theexpected stability of the accuracy andprecision of the measuring system. The UDRLshould not exceed the MRRL unless the userhas sufficient scientific data to document themodifications.b

6.5 Periodic Reassessment of RunLength

The UDRL should be reassessed at regularintervals over the lifetime of an analyticalmethod or instrument system to account forpossible changes due to instrument wear,reformulated reagents, software upgrades, andother factors that may affect analyticalperformance.

Establishing Run Lengths

There currently are no well-acceptedmethodologies for establishing run lengths in amore scientific manner. It is recognized thatlong run lengths are advantageous formaintaining low cost and high productivity, butthese advantages may be offset by potentialfailure-costs if quality deteriorates, errors goundetected, and test results are misinterpreteddue to these errors. One approach forstudying the cost versus quality issue is toapply industrial models for the economicdesign of control procedures. With further15

investigation and development of thismethodology, or with the development andevaluation of other methodologies, alternativeapproaches can be expected that will allow runlengths to be established by carefullydocumented studies.

7 Control Materials

7.1 Application

Control samples must be analyzed for eachanalyte during the user's defined analytical runlength (UDRL).

7.2 Characteristics

The control material should havecharacteristics which enable it to provideinformation about what is going on with thetesting process. A material whose compositionis similar to or identical with the patient samplematrix being analyzed is generally best. Suchmatrix control materials should be used, whenavailable, and should mimic, insofar aspossible, the unknown specimen. A labora-tory should obtain enough homogeneous andstable material to last for at least one year.Vial to vial variability should be much less thanthe variation expected for the system beingmonitored, and the materials should maintainthe analyte being quantified in a stablestate. If commercial control materials are16,17

not available, the laboratory may prepare it’sown patient pools. If there is no appropriateQC material available, the analysis cannot bethe subject of the type of QC discussed in thisdocument.

For U.S. laboratories, federal and state regulations set theb

maximum UDRL as 24 hours.

February 1999 NCCLS

6

7.3 Relation to Calibrators

Control materials need to be different from thecalibrator materials to ensure that the QCprocedure provides an independent assess-ment of system performance.

7.4 Concentrations of Analytes inControl Materials

The number and concentration of matrixquality control materials should be sufficient todetermine proper operation over the range ofinterest.

7.4.1 Clinical Decision Levels

For most analyte-method combinations, aminimum of two levels (concentrations) ofcontrol materials is recommended. Wherepossible, analyte concentrations should be atclinically relevant levels to reflect valuesencountered in patient specimens.5,18

Concurrently using matrix control samples atdifferent levels allows application of additionalquality control rules which improve inter-pretation of analytical error (i.e., proportionalvs. constant, random vs. systematic).

To ascertain the acceptability of patient data,additional control materials may be added atclinical decision levels appropriate for the testand analytical system. Laboratories shouldplan their quality control strategies to includethese important decision levels unlessperformance can be monitored with fewerlevels (e.g., two materials have levels thatbracket a third clinical decision level, the 2ndand 3rd clinical decision levels are closeenough to be adequately monitored by onecontrol material at mid-concentration of thesedecision levels).

7.4.2 Confirmation of Reportable Range

Control materials may be selected to cover thereportable range. Routine testing of thesecontrol materials may also be helpful forconfirming the expected reportable range.

8 QC Applications

8.1 Statement of QC Strategy

The laboratory should define the controlmaterials that are to be analyzed, the numberof measurements to be made on each material,the location of each material in the analyticalsequence, the decision criteria or control rulesthat are to be applied to decide whether or notanalytical performance is acceptable, and theactions to be followed in response to thedecision on acceptability.

8.2 Frequency of Control Measurements

Quality control samples must be analyzed atleast once during each user-defined analyticalrun length (UDRL). Manufacturers of analyticalsystems or reagents should recommend thenumber of quality control specimens and theirlocation within the run. However,manufacturer recommendations should be usedas guidelines. The frequency and location ofcontrol samples should reflect actual testsystem performance at the site of testing. Theuser may need additional control specimensand a different location in order to meetdifferent laboratory circumstances.

8.3 Location of Control Samples

The user should determine the location ofcontrol samples within a run, keeping in mindthe principle that quality control results shouldbe evaluated before reporting patient resultsfrom the run. The location of control samplesshould consider the type of analytical process,the kinds of errors that might occur, and theprotocol for reporting patient results. Forexample, if the UDRL corresponds to a discretebatch of samples, the controls might belocated at the end of the run to detect shifts,might be spaced evenly throughout the batchto monitor drift, or distributed randomly amongthe patient samples to detect random errors.In any case, the QC results would be evaluatedbefore patient results were reported. For ahigh-volume analyzer that continuouslyproduces test results, an appropriate UDRLmight be defined as a certain interval of time,then QC samples would be analyzed andevaluated at the beginning of a run and if thesystem is determined to be in-control, patientresults could be reported for the remainder of

Vol. 19 No. 5 C24-A2

7

the UDRL. If a quality control fault is detected, taken when the difference between the highresults reported since the previous quality and low measurements is greater than fourcontrol event need to be reviewed. Note that times the standard deviation. Quality controlroutine placement immediately after calibration rules should be designed to detect bothmaterials may give falsely low estimates of random and systematic error. Generallyanalytical imprecision and will not provide any random error will be detected by using 1 andestimate of shift or drift during the run. R ; whereas systematic error will be detected

8.4 Decision Criteria or Control Rules

Control data must be evaluated beforereporting patient data. Decisions are made byinspecting a written or graphic record ofcontrol results or by computer review ofresults. Many different decision criteria orcontrol rules have been used, most of themassuming a Gaussian distribution of therandom errors of the measurement system andsetting control limits from the mean andstandard deviation calculated for the errordistribution observed in each individuallaboratory. Control limits are customarilybased on multiples of the observed standarddeviation on both sides of the observed meanvalue, e.g., the observed mean plus and minus3 times the observed standard deviations.Control limits are usually based on the totalstandard deviation that includes all the sourcesof variation in the stable measurement system.

8.4.1 Representation of Quality Control Rules

Quality control rules can be represented byabbreviations of the form A , where "A"L

represents the number of control observationsand "L" is a control limit derived from Gaussianstatistics. For example, 1 refers to a3s

control rule wherein action is taken when asingle control result is beyond three standarddeviations from the mean. The 2 rule refers2s

to a control rule wherein results from twoconcurrent control samples on the same runare beyond two standard deviations from themean in the same direction, or results fromcontrol samples across runs are beyond twostandard deviations from the mean in the samedirection. Commonly used rejection rules are1 and the 2 , but many others are described.3s 2s3,4

Quality control rules for ranges can berepresented in the form R , where "R" is theL

absolute difference between two controlresults in the same run and "L" is a limitderived from Gaussian statistics. For example,R refers to a control rule where action is4s

3s

4s

by the 2 rule, or procedures noting four2s

consecutive observations exceeding the meanplus 1s, or the mean minus 1s, or seven totwelve consecutive observations on the sameside of the mean. Very large systematic erroris detectable by the 1 rule. Specific rules3s

chosen should be based on the analytical andclinical goals of the particular assay and thisclearly may be different for different analytesand clinical needs.

8.4.2 Error Detection

Quality control procedures should be capableof detecting analytical errors at an appropr-iately high rate accompanied by anappropriately low false rejection rate, based onthe characteristics of the particular analyticalprocedure being monitored, and the relevantmedical requirements for assay quality.20

Using multiple control rules improves errordetection with a low probability of falserejection. The performance of control rulescan be assessed by determining theprobabilities for rejecting analytical runs with

presentations of the probability of rejectionversus size of errors are available.21

8.4.3 False Rejection

Using the 1 rule can warn that the system2s

may be approaching an out-of-controlsituation. However, using the rule as arejection signal may cause an inappropriatelyhigh incidence of false run rejections and is notgenerally recommended when the number ofcontrol measurements is greater than 1.

8.5 Control Charts

The graphic display of control results oncontrol charts is often helpful in interpretingthe control data. The Levey-Jennings type ofchart is most commonly used. Charts that22

make use of cumulative summation techniquesor trend analysis techniques may providebetter displays of systematic shifts and

6

differing patterns of analytical errors. Graphic

6,19

February 1999 NCCLS

8

drifts. When a high number of control established estimate of the standard deviationmeasurements is needed (6 or greater) to should be used with the new lot. The estimateprovide the necessary control for a process, of standard deviation should be reevaluatedmean and range charts may be more periodically.appropriate 25

8.6 Setting Control Limits

Control limits should be calculated from themean and standard deviation that describe thevariation expected when a control material isanalyzed by the methods in use in alaboratory. For example, a 1 control rule3s

would have control limits calculated as themean plus and minus 3 standard deviations.

8.6.1 Values for the Mean and StandardDeviation

The mean and standard deviation of a controlmaterial should be established on the basis ofrepeated measurements on those materials bythe methods in use in the laboratory. Controllimits can then be calculated from the meansand standard deviations observed in thelaboratory.

8.6.2 Assayed Control Materials

If assayed materials are used, the valuesstated on the assay sheets should be usedonly as guides. Actual values for the meanand standard deviation must be established byreplicate testing in the laboratory.

8.6.3 Establishing the Value of the Mean ona New Lot

New lots of control material should beanalyzed for each analyte in parallel with thecontrol material in current use. Ideally, aminimum of at least 20 bottles should beassayed on separate days. If the desired 20data points from 20 days are not available,provisional values may have to be set fromfewer than 20 days. Possible approachesinclude making no more than four controlmeasurements per day for at least fivedifferent days.

8.6.4 Establishing the Value of the StandardDeviation on a New Lot

If there is a history of quality control data froman extended period of stable operation, the

If there is no history of quality control data,the standard deviation should be estimated,preferably with a minimum of 20 data pointsfrom 20 separate days. This value should bereplaced with a better estimate when datafrom a longer period of stable operationbecomes available.

8.6.5 Cumulative Values

Estimates of the standard deviation (and to alesser extent the mean) from monthly controldata are often subject to considerable variationfrom month to month due to inherent difficultyof estimating a standard deviation from theavailable number of measurements (e.g., with20 measurements, the estimate of thestandard deviation might vary up to 30% fromthe true value; even with 100 measurements,the estimate may vary by as much as 10%).26

More representative estimates can be obtainedby cumulating the control data from shorterperiods of time, e.g., combining control datafrom six consecutive one-month periods toprovide six month cumulatives. Care should betaken to ensure that the mean is not changingconsistently lower or consistently higher forthe monthly periods being combined.

8.7 Out-of-Control Situations

Laboratories need to establish guidelines forresponding to out-of-control situations.Responses such as repeating control meas-urements or reanalyzing new control materialsare not productive when QC strategies havebeen carefully planned and control rulesselected to minimize the false rejection ofanalytical runs, as described in guidelines forstatistical quality control proposed by aEuropean working group.27

8.7.1 Eliminate Causes of Problems

When QC has been carefully planned andproperly implemented (which requires reliableestimates of the mean and standard deviationbe used in calculating control limits), falserejections are minimized from the outset. Thebest response to an out-of-control signal is to

23,24

Vol. 19 No. 5 C24-A2

9

identify the cause of the problem, find fail-safe ! implementing a proper action for respon-solutions that eliminate that cause, and ding to out-of-control situations.prevent that problem from occurring in thefuture. 27

8.7.2 Clinical Significance of Analytical Errors

It is better to define the clinical quality that isnecessary in the beginning to guide theplanning of QC strategies rather than be facedwith having to make a judgment on the clinicalimportance of errors during the pressure ofdaily service. Guidelines for planning quality-27

changes in test results have been published byLinnet. Guidelines for planning QCprocedures to satisfy biologic goals have beenprovided by the European working group. 27

8.7.3 Verifying Patient Results

The laboratory should establish a policy thatdefines the appropriate action for verifyingpatient results that may have been affected bya QC fault. This is particularly important whenusing long UDRLs and provides a caution toconsider clinical validation needs as well asstability for defining practical run lengths.

8.7.4 Limitations

This recommended practice for dealing without-of-control situations depends on followingthe other recommendations in this documentand shows the interdependence of all theconcepts, approaches, and practices describedin this document. Implementation of thisrecommendation in isolation from the rest ofthe recommendations in this document will notresult in any improvement in laboratory QC. The laboratory must begin by:

! defining the quality required for each test

! establishing a process for planning QCprocedures

! selecting appropriate control rules andnumbers of control measurements

! establishing appropriate UDRLs

! obtaining reliable estimates of the meansand standard deviations to calculate appro-priate control limits

9 Interlaboratory QC Programs

When laboratories share a common pool (lotnumber) of control materials and report theresults to an interlaboratory program, adatabase is created. This data base yieldsstatistical information, which may be used todescribe or define:

(1) Intralaboratory and interlaboratory impre-cision

(2) Laboratory bias relative to a peer group(3) Relationship of analytical and statistical

parameters of imprecision and relative biasto medical requirements.

For laboratory self-evaluation, peer-related biasand relative imprecision are useful parameters.Participation in an interlaboratory program pro-vides an effective mechanism to complementexternal quality control (proficiency survey)programs. Consequently, laboratories areencouraged to actively participate in interlab-oratory QC programs when such programs areavailable.

28

control strategies to detect medically important

February 1999 NCCLS

10

References

1. Westgard JO, Klee GG. Quality 11. Parvin CA, Gronowski AM. Effect ofManagement. In: Burtis CA and Ashwood ER, analytical run length on quality-control (QC)eds. Tietz Textbook of Clinical Chemistry. 2nd performance and the QC planning process. Clined. Philadelphia: W.B. Saunders Company; Chem. 1997;43:2149-2154.1994:548-592.

2. Woo J, Henry JB. Quality management. management: Practical tools for planning andIn: Henry JB, ed. Clinical Diagnosis and assuring the analytical quality of laboratoryManagement by Laboratory Methods. Phila- testing processes. Clin Lab Manag Review.delphia: W.B. Saunders Company; 1996:125- 1996;10:377-403.136.

3. Cembrowski GS, Carey RN. Laboratory DH, Dowd DE, Barry PL, Westgard JO.Quality Management. Chicago: ASCP Press; Selection of medically useful QC procedures1989:264. for individual tests on a multi-test analytical

4. Westgard JO, Barry PL. Cost-EffectiveQuality Control: Managing the Quality and 14. Mugan K, Carlson IH, Westgard JO.Productivity of Analytical Processes. Washing- Planning QC procedures for immunoassays. Jton, DC: AACC Press;1986:230. Clin Immunoassay. 1994;17:216-22.

5. Haven GT, Lawson NS, and Ross JW. 15. Neubauer A, Wolter C, Falkner C,Quality control outline. Pathologist. 1980;34: Neumeier D. Optimizing the frequency and619-621. number of controls for automatic multichannel

6. Westgard JO, Barry PL, Hunt MR. Amultirule Shewart chart for quality control in 16. Taylor TK. Quality Assurance of Chemicalclinical chemistry. Clin. Chem. 1981;27:493- Measurements. Chelsea, MI: Lewis Publishers501. Inc.;1987.

7. Burnett RW, Westgard JO. Selection of 17. Lawson NS, Haven GT, Williams GW.measurement and control procedures to satisfy Analyte stability in clinical chemistry qualityHCFA requirements and provide cost-effective control materials. CRC Crit. Rev. Clin. Lab.operation. Arch Pathol Lab Med. Sci. 1982;17:1-50. 1992;116:777-782.

8. Skendzel LP, Barnett RN, Platt R. Laboratory Tests. 2nd ed. Oradell, NJ: MedicalMedically useful criteria for analytical Economics Books;1987.performance of laboratory tests. Am J ClinPathol. 1985;83:200-205. 19. Westgard JO, Groth T, Aronsson T, et al.

9. Fraser CG, Hyltoft Petersen P, Ricos C, quality control: Probabilities for false rejectionHaeckel R. Proposed quality specifications for and error detection. Clin. Chem. 1977;23:the imprecision and inaccuracy of analytical 1857-1867.systems for clinical chemistry. Eur J Clin ChemClin Biochem. 1992;30:311-317. 20. Westgard JO, Groth T. A predictive value

10. Hyltoft Petersen P, deVerdier C-H, Groth prevalence of errors on the performance ofT, Aronsson T. Clinically based quality goals; control procedures. Am. J. Clin. Pathol;a NORDKEM project. Eur J Haematol. 1983;80:49-56. 1990;45(Suppl 53):6-8.

12. Westgard JO. Error budgets for quality

13. Koch DD, Oryall JJ, Quam EF, Felbruegge

system. Clin Chem. 1990;36:230-3.

analyzers. Clin Chem. 1998;44:1014-1023.

18. Statland BE. Clinical Decision Levels for

Performance characteristics of rules for internal

model for quality control: Effects of the

Vol. 19 No. 5 C24-A2

11

References (Continued)

21. Westgard JO, Groth T. Power functionsfor statistical control rules. Clin. Chem.1979;25:863-869.

22. Levey S, Jennings ER. The use of controlcharts in the clinical laboratory. AM J ClinPathol. 1950;20:1059-66.

23. Westgard JO, Groth T, Aronsson T,deVerdier C-H. Combined Shewhart-Cusumcontrol chart for improved quality control inclinical chemistry. Clin. Chem. 1977;23:1881-1887.

24. Neuhauer AS. The EWMA control chart:properties and comparison with other quality-control procedures by computer simulation.Clin. Chem. 1997;43:594-601.

25. Hainline A. Quality assurance: Theoreticaland practical aspects. In: Faulkner WR, andMeites S, eds. Selected Methods of ClinicalChemistry, Vol. 9. Selected Methods for theSmall Clinical Chemistry Laboratory. Washing-ton, D.C.: AACC; 1982:17-31.

26. Westgard JO, Carey RN, Wold S. Criteriafor judging precision and accuracy in methoddevelopment and evaluation. Clin Chem.1974;20:825-833.

27. Hyltoft Petersen P, Ricos C, Stockl D,Libeer JC, Baadenhuijsen H, Fraser C, Thien-pont L. Proposed guidelines for the internalquality control of analytical results in themedical laboratory. Eur J Clin Chem ClinBiochem. 1996;34:983-999.

28. Linnet K. Choosing quality-control sys-tems to detect maximum clinically allowableanalytical errors. Clin Chem. 1989;35:284-288.

February 1999 NCCLS

12

Summary of Comments and Committee Responses

C24-A: Internal Quality Control Testing: Principles and Definitions; Approved Guideline

General Comments

1. Most hematologists (and heme lab personnel) have no comprehension of what the chemicaljargon, “matrix” means. You want a reasonably wide audience to benefit from this document.

! The term “matrix” has been defined in Section 3.

Key Words

2. You should distinguish clearly between “calibration interval” and “calibration.”

! The working group and area committee decided to omit the brief section on calibration from C24-A2 rather than expanding this discussion to adequately treat this topic.

Section 2.0

3. The last paragraph in this section points out, in its second sentence, the key considerationrelative to a methods analytical performance. In this day of governmental involvement intesting, the statement is most pertinent and I believe applies very specifically to the congressionaland regulatory discussions relative to the “complexity model of testing.” I realize that currentlythis is a draft document, but it would be nice if we were able to quote this—if not as NCCLSmembers, at least individually or representatives of our organizations.

! Addition of Section 5, on “Planning a Quality Control Procedure,” should help laboratoriesconsider their particular requirements and apply QC to meet the needs of their patients andphysicians.

Section 4.0 (New Section 6.0)

4. The definition of the word “matrix” should be added, in very simple language.

!! Matrix is now defined in Section 3.

5. Definitions for instruments performance, stray light, bandwidth, and filter types should beincluded.

! Definitions have been added in Section 3 for performance terms such as bias, imprecision, randomerror, and systematic error, which are the terms most relevant for this document.

6. Definitions for random error and system error should be added.

! These definitions have been added in Section 3.

7. The discussion of user defined run length (UDRL) suggesting a 24-hour time limit is currentlyinconsistent with the capabilities of modern clinical laboratory testing systems and should becorrected.

! The committee recognizes that instrument systems may be stable for longer than 24 hours.However, 24 hours still seems to be a reasonable maximum based on an instrument’s potentialsusceptibility to problems, which is the other side of the coin and an equally importantconsideration. There often are changes being made in a 24-hour period, such as new reagents,

Vol. 19 No. 5 C24-A2

13

new bottles of calibrators, system maintenance, and different operators. There may be newapproaches developed that will allow run lengths to be established in a more scientific manner,which is now recognized in the new paragraph in Section 6.6. This paragraph provides theflexibility for manufacturers and users to establish run lengths by other approaches that areproperly investigated, peer-reviewed, and documented.

Section 5.2 (New Section 7.2)

8. The “type of QC” should be changed to “the unmodified QC.”

! The committee prefers the original wording. "Type of QC" here refers to statistical control usingstable materials. Modified or "unmodified QC " has a different implication in today's regulatoryenvironment. Both modified and unmodified QC procedures could be statistical control usingstable materials.

Section 5.3 (New Section 7.3)

9. “A different batch (lot number) than the” should be changed to “a different supply of.” Also,“the same lot of material” should be changed to “the same material.”

This section has been modified to emphasize the need for independent calibrators and controls.The present language considers that the "pool used to manufacture the control materials shouldbe a different lot from the pool used to manufacture calibration materials."

Section 5.4.1 (New Section 7.4.1)

10. I would suggest that the following parenthetical statement be added at the end of the secondsentence: (patient specimens assayed in prior runs or on different analyzers may serve asmatrix control specimens as appropriate).

! This document doesn’t cover the details of patient data QC that would be necessary if thisstatement were included. For example, the planning of patient data QC procedures is morecomplicated and the manner in which control limits and control rules are established is different.

Section 5.4.2 (New Section 7.4.2)

11. When selecting a general purpose multiconstituent control, all analytes may not be availablewithin instrument range. Using material that requires dilution diminishes the utility of qualitycontrol testing.

! The intention is to recommend that analytes be selected to reflect important clinical levels "wherepossible."The "where possible" recognizes that there will be some difficulties in doing this for allanalytes when using multiconstituent controls.

Section 5.6 (New Section 8.2)

12. I believe that the information included in the discussion of User Defined Run Length (UDRL) isinconsistent with many comments on this subject expressed at the NCCLS National Congress onCLIA ‘88. The statement in this guideline suggesting a 24-hour time limit has been converted byHCFA into a mandatory time limit in CLIA “88. I suggest the committee quickly review thisconcern as the guideline is currently inconsistent with the capabilities of modern clinical laboratorytesting systems.

! Section 6.4 has been changed in C24-A2. A footnote has been included to indicate that the 24-hour maximum is the current CLIA mandate for U.S. laboratories, rather than a mandate in this

February 1999 NCCLS

14

document. The new Section 6.6 allows some flexibility for manufacturers and users, but requiresthat alternate approaches for establishing run lengths be well documented.

Sections 5.7.1 and 5.7.2 (New Section 8.3)

13. There are statements relative to random and fixed placement of quality control samples withinthe run. However, there is no recommendation as to which is appropriate and when. Thedocument would be improved if some sort of a statement were made in this regard rather thanthe somewhat less than adequate implications.

! These original sections on random and fixed placement have been changed to provide a moregeneric description that identifies some of the factors that will influence the placement of controls.

Section 5.8 (New Section 8.1)

14. In the third sentence of the paragraph, the term “data” is used in a way that could be verybroadly interpreted. My concern is that some readers of the document might assume that rawdata developed by manufacturers might be something to which they are entitled. I believe thatan appropriate qualifying statement relative to the use of the word data would be appropriate.

! That statement has been changed to indicate that data should be available to validate qualitycontrol recommendations.

Section 6.2

15. In sentence number 5, I would strongly suggest the phrase “though not necessarily” be addedafter the word customarily. By doing this, one maintains the intent of the document withoutmaking statistical limits mandatory. I believe that, again, in the light of regulatory involvement,reasonable flexibility must be promoted. Even though the NCCLS documents are not intended forregulatory use, they are frequently quoted by the regulators, and I believe we must be sensitiveto these issues.

! The implication of this comment is that nonstatistical limits may be used. With the addition ofthe new Section 5 on planning a quality control procedure, the medical and analyticalrequirements for quality should be considered up-front and the statistical limits or rules be selectedto assure the defined quality is achieved.

Section 6.3 (New Section 8.4.1)

16. I would suggest strongly that a statement similar to the following be added at the end of thelast paragraph of this section: “Specific rules chosen should be based on the analytical and clinicalgoals of the particular assay and this clearly may be different for different analytes and clinicalneeds.”

! The new Section 5 on planning a quality control procedure recognizes the importance of definingthe analytical or clinical quality needed in the first step of the planning process.

Section 8.2

17. The cost savings and convenience of multiconstituent controls can be offset by problems withcross-reactivity.

! This statement and the separate recommendations for the "low test volume environment" havebeen eliminated in C24-A2.

Vol. 19 No. 5 C24-A2

15

Section 8.3.2

18. I would suggest that the second sentence be modified to read “The process of establishingactual target values must include repeating of analytical testing in the laboratory. Incorporationof manufacturers’ and other information including general laboratory experience may be part ofthe final mechanism for determining the laboratory values.”

! The word "target" has been eliminated in the new document to avoid confusion with the use of"target values" in the CLIA regulations. A strong emphasis is still maintained on the laboratory'sneed to establish its own mean and standard deviation that will reflect the performance beingachieved in the individual laboratory.

February 1999 NCCLS

16

Summary of Delegate Comments and Responses

C24-A2: Statistical Quality Control for Quantitative Measurements: Principles and Definitions; ApprovedGuideline—Second Edition

General

1. Change name “Internal Quality Control” to “In-House” or “User Established” to eliminate theconfusion with “Internal Quality Control” used by manufacturers and FDA, CLIA to mean QCinternal to the device.

! The title of the document has been changed to "Statistical Quality Control for QuantitativeMeasurements: Principles and Definitions." The word "internal" has been replaced with"statistical" throughout the guideline.

2. I would suggest that the relationship between accuracy and systematic error and precision andrandom error be discussed in further detail.

!! Other NCCLS documents, particularly the evaluation protocols series, provide in depth discussionsof accuracy and precision. In this guideline, only standard definitions are included.

3. A section with examples and problem solving would be useful.

! Examples and detailed directions for the implementation of statistical QC and construction ofcontrol charts can be found in References 1-6. Examples of the planning of statistical QCprocedures are provided in References 12-14, 27 and 28.

4. The document should more strongly state the acceptability of alternate QC practices (electronicQC, procedural controls, etc.)

! This document deals only with statistical QC. The third paragraph of the Foreword states thatthis document does not attempt to describe alternatives to statistical process control. Becauseof the focus on statistical QC, there are no recommendations about the acceptability or non-acceptability of alternative QC procedures. Another NCCLS subcommittee is consideringalternative QC practices.

Introduction

5. My concern is that the scope of this guideline is much too broad and does not address the needsof small laboratories (contrary to the first sentence in Scope, 2) that might be using instrumentsand systems that self monitor in part or in the whole. When the laboratory elects to use statisticalmonitoring, this document provides a very useful guide to important considerations and concernsthat need to be controlled and/or monitored.

! The title of the document has been changed to, "Statistical Quality Control for QuantitativeMeasurements: Principles and Definitions; Approved Guideline—Second Edition," to clarify thatthese recommendations and guidelines apply when a laboratory elects to use statisticalmonitoring. Large and small laboratories alike may use this document for guidance inimplementing appropriate statistical QC procedures.

6. The last sentence of the introduction should be changed: "…it is still useful to monitor theoutcome of the whole analytical process by traditional statistical quality control." This might notbe true for several reasons, including the monitoring capability of the system, the economics ofthe testing environment, the specificity and compatibility of the control material. An acceptablechange is "…it is often useful to monitor the outcome of the whole analytical process by

Vol. 19 No. 5 C24-A2

17

traditional statistical quality control. However, alternative acceptable quality practices can beestablished to accommodate such factors as internal or self-monitoring capability, availability ofsuitable control materials, the expectation of when a significant change in performance isanticipated to occur, and economics to optimize testing frequency.

! Statistical QC can be used in most situations, even if "alternative" QC procedures are employed.Statistical QC provides an assessment of performance that is independent of an instrument's ownchecks and internal or self-monitoring capability. It should also be noted that statistical QCprovides a way to monitor operator proficiency, which needs to be documented to satisfyregulatory requirements in some countries.

Section 3

7. Add definition of “internal quality control” to the list of definitions. It wasn’t until I got toSection 4 “Purpose of Internal Quality Control” that I truly understood.

! The title of the document has been changed to eliminate "internal" and add "statistical" tomore clearly focus on statistical QC and provide a term that is commonly understood. Thedocument defines “statistical quality control” in Section 3 and "statistical quality controlstrategy" in Section 5.3.

Section 5.3

8. Specific sentence and section that needs changing. 5.3 Identify Candidate QC Strategies. "Aquality control strategy is defined by what control materials are used,… are evaluated." Thisstatement describes the decisions that should be made once the QC strategy is defined, if thestrategy includes statistical QC using simulated (or real) patient control materials. The introductorystatement should be: "A quality control strategy is defined by the needs of the user of the resultsthat will be reported from the clinical laboratory. The tools might include innovative practices thattake advantage of the technological capability of the system or any unique attributes of the testingprocess. If statistical QC practices are employed, the guidance in this document should beconsulted, and consideration given to what control materials are used,… are evaluated."

! The heading of Section 5.3 was changed to "Identify Candidate Statistical QC Strategies." Thisconfines the recommendation to statistical QC, which is the focus of this document. AlternativeQC practices are currently being considered by the NCCLS Subcommittee on Unit Use Testing.

February 1999 NCCLS

Proposed- and tentative-level documents are being advanced through the NCCLS consensus process; therefore, readers should*

refer to the most recent editions.

18

Related NCCLS Publications*

C27-A Blood Gas Preanalytical Considerations: Specimen Collection, Calibration, and Controls;Approved Guideline (1993). Guidelines for collecting and handling an arterial bloodspecimen for pH and blood gas analysis.

EP5-A Evaluation of Precision Performance of Clinical Chemistry Devices; Approved Guideline(1999). Guidelines for designing an experiment to evaluate the precision performanceof clinical chemistry devices; recommendations on comparing the resulting precisionestimates with manufacturer's precision performance claims, and determining whensuch comparisons are valid; and manufacturer's guidelines for establishing claims.

EP9-A Method Comparison and Bias Estimation Using Patient Samples; Approved Guideline(1995). This document addresses procedures for determining the bias between twoclinical methods, and the design of a method-comparison experiment using split patientsamples and data analysis.

EP10-A Preliminary Evaluation of Quantitative Clinical Laboratory Methods; Approved Guideline(1998). This guideline provides experimental design and data analysis for preliminaryevaluation of the performance of an analytical method or device.

H26-A Performance Goals for the Internal Quality Control of Multichannel HematologyAnalyzers; Approved Standard (1996). Recommended performance goals for analyticalaccuracy and precision based on mathematical models for the following measurements:hemoglobin concentration, erythrocyte count, leukocyte count, platelet count, and meancorpuscular volume.

H42-A Clinical Applications of Flow Cytometry: Quality Assurance and Immunophenotypingof Lymphocytes; Approved Guideline (1998). This document provides guidance for theimmunophenotypic analysis of non-neoplastic lymphocytes by immunofluorescence-based flow cytometry; sample and instrument quality control; and precautions foracquisition of data from lymphocytes

M22-A2 Quality Assurance for Commercially Prepared Microbiological Culture Media—SecondEdition; Approved Standard (1996). Quality assurance procedures for manufacturersand users of ready-to-use microbiological culture media.

M29-A Protection of Laboratory Workers from Instrument Biohazards and Infectious DiseaseTransmitted by Blood, Body Fluids, and Tissue; Approved Guideline (1997). Thisdocument provides guidance on the risk of transmission of hepatitis viruses and humanimmunodeficiency viruses in any laboratory setting; specific precautions for preventingthe laboratory transmission of blood-borne infection from laboratory instruments andmaterials; and recommendations for the management of blood-borne exposure.

NRSCL8-A Terminology and Definitions For Use in NCCLS Documents; Approved Standard (1998).Standard definitions for use in NCCLS standards and guidelines, and for submittingcandidate reference methods and materials to the National Reference System for theClinical Laboratory (NRSCL).

Vol. 19 No. 5 C24-A2

19

NOTES

February 1999 NCCLS

20

NOTES

Vol. 19 No. 5 C24-A2

21

NOTES

February 1999 NCCLS

22

NOTES

NCCLS – 940 West Valley Road – Suite 1400 – Wayne, PA 19087 – USA – PHONE 610.688.0100

FAX 610.688.0700 – E-Mail:[email protected]. ISBN 1-56238-371-X

ABC