This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Accreditation Requirements inMolecular Pathology
May 20, 2009
Helen Fernandes, PhDAssociate Professor
Director, Molecular DiagnosticsUMDNJ / New Jersey Medical School
Phenotypic-Clinical Dimension of PathologyPhenotypicPhenotypic--Clinical Dimension of PathologyClinical Dimension of Pathology
Adapted from Clinical Chemistry 2007: 1118 . Manuel Salto-Tellez
7
Prognosis and TreatmentPrognosis and Treatment
Detection of HPV Infection
Diagnosis of Cervical Cancer
Phenotypic-Molecular-Clinical Dimension of PathologyPhenotypicPhenotypic--MolecularMolecular--ClinicalClinical Dimension of PathologyDimension of Pathology
EGFR mutations
KRAS mutations
Diagnosis of Lung Cancer
KRASmutations
Microsatellite Instability Analysis
Diagnosis of Colorectal
Cancer
Specific Translocations
B & T cell rearrange-
ments
Diagnosis of Lympho-
proliferativeDisorders
20210 A
Factor V-Leiden
Diagnosis of Hypercoaguable
State
ERPR Status
HER2-neuStats
Diagnosis of Breast Cancer
Adapted from Clinical Chemistry 2007: 1118 . Manuel Salto-Tellez
8
Prognosis and TreatmentPrognosis and Treatment
Detection of HPV Infection
Diagnosis of Cervical Cancer
Phenotypic-Molecular-Clinical Dimension of PathologyPhenotypicPhenotypic--MolecularMolecular--Clinical Dimension of PathologyClinical Dimension of Pathology
• Rationale: Establish that the analyte(s) fulfills its intended use in its intended environment– An extensive process– Validate the performance characteristics of
laboratory-developed tests – Examine the components of a test that are
assembled for the intended use without regulatory approval. – ASR
– Establish the analytical performance and clinical (diagnostic) performance of a test as it applies to its intended use.
13
Pre-Validation Considerations for the Design of Laboratory Developed Tests• Stringent design/analysis of primers and
probes• Quality and quantity of extracted nucleic
acid• Appropriate platform for the test• Commutable calibrators and controls• Optimization of amplification and
detection
14
Validating Performance Specifications
• Applies to– Laboratory-developed tests– Approved tests that have been modified by the laboratory
• Before reporting patient results, the laboratory must establish performance characteristics for– Accuracy– Precision– Reference and Reportable ranges– Analytical sensitivity (LOD) (LOQ)– Analytic specificity– Interfering substances– Diagnostic (Clinical) Validity
15
Analytical Sensitivity Can Be Done By…
• Control material of known concentration or copy number (calibrators / standards)
• Dilutions of analyte (microorganism, gene) of known quantity
controls in triplicateHigh Positive and Low positive controls in triplicateInter and Intra run
AnalyticalSensitivity
5-10 specimens at LOQ and LOD
Establish LOQ with 3-5 calibrators run in triplicate and LOD with 30 specimens.
Analytical Specificity
Test related analytes
Evaluate interfering substances
Reportable Range
3-5 specimens at different concentrations
Establish measuring range and linear range with 3-5 calibrators run in triplicate
Reference Range (Normal)
10 -20 specimens 50 – 100 specimens depending on analyte
Modified from Clinical Microbiology Newsletter 29:12 2007
18
Clinical (Diagnostic) ValidationAddress the clinical significance and utility of the test.• Diagnose a disease or disease state • Confirm the results of another laboratory test, or clinical
diagnosis• Monitor and assess disease progression, prognosis or
resolution
• Comparing the test to a gold standard– Another test (culture, EIA, chemistry test, FISH)– Clinical outcome / endpoint (response to therapy)
• Use Positive and Negative Predictive values• Use ethnic variation and geographic distribution• Cite references to clinical studies in the literature
19
Validation of a Modified FDA Test
• Alteration of any process or analyte in an approved test that may affect performance
• Changes in:– type or concentration of specimen used– storage specifications– extraction method– amplification parameters or platforms– manufacturer’s cut off– using the test for another purpose
(quantitative vs. qualitative)
20
Verifying or Validating Performance Specifications
• When multiple instruments are used to perform the same test, the laboratory must compare the performance on both instruments
• If using more than one platform to test the same analyte, the lab must verify or establish performance specifications including agreement.
21
Correlation Coefficient and Agreement
‐1.30
0.00
1.30
2.00 3.00 4.00 5.00 6.00
L og10 average
Log 10 difference
R2 = 0.794
2.50
3.50
4.50
5.50
2.50 3.00 3.50 4.00 4.50 5.00 5.50
Assay A Log10 copies / mL
Ass
ay B
Log
10 c
opie
s / m
L
• Correlation coefficient represents the strength of a linear relationship between two variables
• Agreement as measured by the Bland-Altman model is the Trueness or closeness of agreement between the average and the true value of the analyte
Correlation0.79
Agreement95.6%
2 SD
2 SD
22
MOL.31015 Phase II
• Were validation studies with an adequate number and representative (reasonable) distribution of samples performed for each type of specimen expected for the assay (e.g., blood, fresh/frozen tissue, paraffin-embedded tissue, prenatal specimens?
Verification and Validation studies should be performed on all types of specimens that the test is used for.
23
Outline for Assay ValidationExample - CMV Viral Load
1. Determine the number and type of specimens to be used– A minimum of 30-50 specimens are required for assessment of statistical
significance.– Blood, Body fluids, White Blood Cells
2. Select the test to be used for comparison– CMV antigenemia / culture– Another established quantitative assay
3. Document– Accuracy, Precision, Interfering substances– Use appropriate statistical methods of evaluation (r2, regression, agreement)
4. Establish acceptance criteria for analytical sensitivity– For LOD (qualitative acceptance) - > 95% detectable specimens– For LOQ (lower quantification limit) 0.15 Log10
5. Evaluate analytical specificity– Other members of the herpesvirus family – HSV, EBV
24
LogLog1010 copies/ml in current test (copies/ml in current test (LightcyclerLightcycler))
Log
Log 1
010co
pies
/ml i
n va
lidat
ed te
st (A
BI75
00)
copi
es/m
l in
valid
ated
test
(ABI
7500
)
Direct correlation - PCR
Realtime vs. Realtime
Indirect correlation
PCR vs. Antigenemia
Accuracy - CMV Viral Load
25
Outline for Assay ValidationExample - CMV Viral Load
1. Determine the number and type of specimens to be used– A minimum of 30 specimens are required for assessment of statistical
significance. – Blood, Body fluids, White Blood Cells
2. Select the test to be used for comparison– CMV antigenemia / culture– Another established quantitative assay
3. Document– Accuracy, Precision, Interfering substances– Use appropriate statistical methods of evaluation (r2, regression, agreement)
4. Establish acceptance criteria for analytical sensitivity– For LOD (qualitative acceptance) - > 95% detectable specimens– For LOQ (lower quantification limit) 0.15 Log10
5. Evaluate analytical specificity– Other members of the herpesvirus family – HSV, EBV
26
Design for Assay ValidationExample - CMV Viral Load…
6. Assess (Diagnostic) Clinical performance– Correlate with clinical presentation– Establish baseline / reference range (important for CMV)
7. Resolve discrepancies– Sequencing– Send-out to another facility or reference lab
8. Set the parameters for interpretation of results– Use baseline to guide clinical disease– Define what is a clinically significant change when used for monitoring
disease. Tolerance limit of the assay = Assay variability + biological variation
9. Summarize all of the above in a ”Validation document”– Expand on each section– Statement on acceptance of validation for clinical use – Review & signature by Director
27
The test is accepted for clinical useThe test is accepted for clinical use
28
TOTAL QUALITY MANAGEMENT (QM) TOTAL QUALITY MANAGEMENT (QM)
Pre AnalyticPre Analytic AnalyticAnalytic Post AnalyticPost Analytic
• Clinical validation can be challenging• Limited diagnostic testing guidelines• Results for the same test use different
methodologies that may not be in agreement• Lack of concordance on units of reporting• Interpretation can sometimes be difficult• Some esoteric tests performed in a single lab
31
Challenges in Quality Assurance in Molecular Diagnostics
• Since most technologies involve amplification technical precision is mandatory
• Contamination of specimens, equipment reagents and aerosol contamination are legitimate problems
• Samples can be inhibitory to amplification• Mutations in the nucleic acid can interfere
with testing • Discrepancies could be an issue
32
You need to be in touchYou need to be in touch
QA / QC is an ongoing process that affirms verification & validation.
It needs to be monitored on a regular basis for trends.
Other Analytic QC Monitors• Melting curve temperature- Tm range (+ 2oC) • Blank OD values• For realtime PCR monitor Ct values of controls (qual
or quant)• Lot – Lot check• Failed runs /reactions
41
QC at the POST-ANALYTICAL LEVEL
• Interpretation of Results: – Molecular tests can be subjective – Have more than one person review results
• Difficult problems: – Discrepant analysis– Blips at the end of an amplification curve
• Reporting in different units of measure Copies/mL; IU/mL; genome eq/mL
42
QC Monitors Post Analytical• Turn around time• Notification of rapid tests• Records of quantitative tests when used for
monitoring e.g. Viral Load• Record all new / rare mutations or variations
in a notebook• Record critical results – for rapid tests and
CSF• Correlation between tests• Errors in reporting
43
QC SUMMARY
• Continual Quality Control required• Documentation of all QC monitors on a
regular basis – monthly – Graphing of control values– In-house control in quantitative tests for reagent
and tech variation– Melting temperatures for variants– Ct value for probe integrity– Background OD for contamination
44
Does the test perform the way it was intended to?
Would you use your test to help the clinician make a decision for you or a
family member ??
Overall…..??
45
Most Common Deficiencies in 2008
MOL 31820 For FDA approved assays, are performance criteria verified by the laboratory ? (33.3%)
MOL 32050 Is there a summary statement signed by the lab director documenting the review of validation studies and approval of thetest for clinical use? (4.4%)
46
MOL 10160 For tests which CAP does not provide Proficiency testing does the lab participate in external PT of exercise Alternate PT?
MOL 31475 Do validation studies document analytical sensitivity & precision?
47
Regulatory/accrediting agencies and payers• CLIA• CAP• JCAHO
MM01MM01--A2 A2 Molecular Diagnostic Methods for Genetic DiseasesMolecular Diagnostic Methods for Genetic DiseasesMM02MM02--A2 A2 Immunoglobin and TImmunoglobin and T--Cell Receptor Gene Cell Receptor Gene
Rearrangement AssaysRearrangement AssaysMM03MM03--A2 A2 Molecular Diagnostic Methods for Infectious DiseasesMolecular Diagnostic Methods for Infectious DiseasesMM05MM05--A A Nucleic Acid Amplification Assays for Nucleic Acid Amplification Assays for
HematopathologyHematopathologyMM06MM06--A A Quantitative Molecular Methods for Infectious DiseasesQuantitative Molecular Methods for Infectious DiseasesMM07MM07--A A Fluorescence In Situ Hybridization (FISH) MethodsFluorescence In Situ Hybridization (FISH) MethodsMM09MM09--A A Nucleic Acid SequencingNucleic Acid SequencingMM10MM10--A A Genotyping for Infectious DiseasesGenotyping for Infectious DiseasesMM11MM11--P P Molecular Methods for Bacterial Strain TypingMolecular Methods for Bacterial Strain TypingMM12MM12--A A Diagnostic Nucleic Acid MicroarraysDiagnostic Nucleic Acid MicroarraysMM13MM13--A A Collection, Transport, Preparation, and Storage of Collection, Transport, Preparation, and Storage of
Specimens for Molecular MethodsSpecimens for Molecular MethodsMM14MM14--A A Proficiency Testing (External Quality Assessment) for Proficiency Testing (External Quality Assessment) for
Molecular MethodsMolecular MethodsMM16MM16--P P Use of External RNA Controls in Gene Expression Use of External RNA Controls in Gene Expression
Documents for Reference• Laboratory Developed Tests in Molecular
Diagnostics. Quality Control and Assay ValidationIn WB Coleman and GJ Tsongalis, ed. Molecular Diagnostics: For the Clinical Laboratorian, 2nd edition (2006). Totowa, NJ: Humana Press, Inc.