Applying Medical Literature I: Topics in Diagnostic Testing Mary Ottolini MD, MPH Vice Chair Medicl Education
Applying Medical Literature I: Topics in Diagnostic Testing
Mary Ottolini MD, MPH
Vice Chair Medicl Education
Case
• Darius Smith is an 11 week old male infant with a 2 day history fever to 39.4. He has not been nursing as well and vomited a couple of times, but has kept down 3 oz of pediolyte in the ED. He is uncircumcised.
• He had a CBC and U/A obtained. The CBC at a WBC of 13,000 with an automated diff of 73.5% neutrophils, 3% bands, 22.5% lymphs and 1% monos.
• His U/A had pH: 7; SG: 1.025;(+)Nitrites and LE; 2+ blood; 2+protien; 10-15 RBC and 350 WBCs/HPF.
Case Cont:
• The ED attending wants you to admit him, because he is under 3 months of age at she thinks he has pyelonephritis because his CRP is elevated (44).
• You want to treat him and discharge him home,but you don’t know how good the CRP test is, or if there is another test you can do besides a DMSA scan to accurately predict whether he has pyelonephritis.
Objectives
• Calculate sensitivity, specificity, and likelihood ratios
• Explain how disease prevalence affects positive and negative predictive values
• Calculate and use likelihood ratios to choose among diagnostic tests
• Interpret a receiver operating characteristic curve • Interpret and apply a clinical prediction rule
When Do you do a Diagnostic Test?
CRP
Procalcitonin
Threshold Model of Diagnostic Test Ordering
Test
A B
No TestNo Treatment
No TestTreat
Thresholds are Determined by:
How good is the test? (Sensitivity and Specificity)
Risk of inappropriate treatment
Risk of the diagnostic test
Benefit of the appropriate therapy
0 1Pretest Probability
Questions
• What tests do you use to determine if the patient has a pyelonephritis? How good are these tests?
• Is this baby at increased risk for pyelonephritis? – How do you determine if a young infant likely has
pyelonephritis?– So What?- Do you manage young infants with
pyelonephritis differently than older infants?• What are the risks of inappropriate treatment?• What is the benefit of appropriate treatment?
Prevalence of UTI: Pretest Probabilities:
• Overall prevalence of UTI in age 2mos-2 yrs in fever without source= 5%
• Girls < 1yr= 6.5%; Boys=3.3%
• Girls 1-2 yrs= 8.1%; Boys=1.9%
• Circumcised boys= 0.2%-0.4% (5-20X greater in uncircumcised boys)
Why diagnose and treat?
• Prevent urosepsis
• Prevent renal scarring
• Eliminate acute infection
PICOPatient
Intervention
Control
Outcome
Infant
CRP
Pyelonephritis
How to Use the Article to Manage Your Patient
• Are the results of the study Valid?
• Are the results of the study Important?
• Can you Apply this valid, important evidence in caring for your patient?
Are the Results Valid?
• Was there an independent, blind comparison with a reference standard?
• Did the patient sample include an appropriate spectrum of patients to whom the diagnostic test will be applied in clinical practice?
• Did the results of the test being evaluated influence the decision to perform the reference standard?
• Were the methods for performing the test described in sufficient detail to permit replication?
Did the patient sample include an appropriate spectrum of patients to whom the diagnostic test will be applied in clinical practice?
METHODS:• Prospective study of 100 consecutive children (69 F/31M), 1 mo-13 yo
• 1st febrile UTI: admitted to the Peds Dept of U of Udine, 1/2000 –1/2002,
• Inclusion Criteria: fever (temperature of 38°C) and/or symptoms suggesting UTI, abdominal/flank pain, and for infants < 5 mos nonspecific signs, such as irritability, vomiting, difficulties with feeding, or failure to thrive.
• Definition of UTI: Catheterized or midstream, clean-void UC = single microorganism at 105 CFU/mL.
• Exclusion: Receipt of antibiotics in prior week or documented or suspected previous febrile UTI and/or known abnormalities of the urinary tract.
• All were treated with IV ceftriaxone, (75 mg/kg per day) for 1 to 5 days, followed by oral antibiotics per susceptibility, for a total of 10 days.
What is the Gold/Reference Standard?
How was pyelonephritis defined?
Was there an independent, blind comparison with a reference
standard?
Did the performance of the test being studies influence the interpretation of whether the “Gold
Standard” for disease was met
Did the results of the test being evaluated influence the decision
to perform the reference standard?
Did the CRP or PCT result influence whether a DMSA was performed?
Methods cont:
• All pts had a CBC, ESR, CRP and Procalcitonin
Imaging Studies• Renal ultrasound within 3 d after
admission, and VCUG 1 mo later • DMSA scintigraphy within 5 days
after admission for all children.• The presence/extent of renal lesions
were evaluated independently by 2 investigators, blinded with respect to CRP, PCT levels and the clinical status of the patient.
• Divided into 2 groups based on no to min renal involvment vs renal involvment
Were the methods for performing the test described in sufficient detail to permit replication?
What are the results?
• Are likelihood rations for the test results presented or data necessary for their calculation provided?
What are the results?
• Are likelihood rations for the test results presented or data necessary for their calculation provided?– If not provided: draw a 2x2 table to show how:
• Sensitivity and specificity were derived
• Positive and negative predictive value were derived
• Discuss the likelihood ratios and how you would interpret them
Results:TABLE 2. Sensitivity, Specificity, PPV, and NPV of PCT and CRP Measurements for Prediction of Acute Pyelonephritis
(Cutoff Values of 0.8, 0.5, or 1 ng/mL for PCT and 20 or 50 mg/L for CRP)
Sens Spec PPV NPV
PCT, 0.8 ng/mL 83.3 93.6 93.7 83.0
PCT, 0.5 ng/mL 90.7 70.2 77.7 86.8
PCT, 1 ng/mL 81.4 93.6 93.6 81.4
CRP, 20 mg/L 94.4 31.9 61.4 83.3
CRP, 50 mg/mL 74.0 76.6 78.4 72.0
PPV, positive predictive value; NPV, negative predictive value.
How do you tell how good the test is?
• Sensitivity and specificity
• Positive and negative predictive values
• Likelihood ratios
Sensitivity and Specificity
DiseaseNo
Disease
Test+
TestNeg
True (+)False (+)
False (-) True (-)
a b
c d
a/a+c d/b+d
Sensitivity Specificity
If the patient has disease what is the probability the test will be positive?If the patient does not have disease what is the probability the test will be negative?
Gold Standard
Sensitivity and Specificity
SnNOUTA test with ahigh sensitivitythat is negative rules things out
Specificity: NIH Negative in HealthSensitivity: PID Positive in Disease
SpPINA test with a high specificity that is positive rules things in
Positive and Negative Predictive Value
Disease No Disease
Test (+)
Test (-)
True (+) False (+)
False (-) True (-)
a b
c d
a/a+b
d/c+d
Positive Predictive Value
Negative PredictiveValue
If the test is positive what is the probability of disease?
If the test is negative what is the probability of no disease?
Likelihood Ratios
Disease No Disease
Test (+)
Test (-)
Sensitivity
1-Sens Specificity
1- Spec
1 1Positive LR: Sensitivity/1-Specificity: Prob of true(+)/prob false(+)
Negative LR: 1-Sensitivity/Specificity: Prob of false(-)/prob true (-)
1
1
Likelihood Ratios:
POSITIVE LR: Probability of a positive test in diseaseProbability of a positive test in health
NEGATIVE LR: Probability of a negative test in disease
Probability of a negative test in health
Positive LR Not usefulIf between 1-2More usefulIf >2<10Most useful if>10
Negative LR
Not useful
If between .5-1
More useful
If <.5>.1
Most useful if
<.1
Likelihood ratios+ Likelihood ratio= Probability abnl CRP if disease present
Probability abnl CRP if healthy
=sensitivity/1-specificity=1.38
- Likelihood ratio = Probability nl CRP - if disease presentProbability nl CRP - if healthy
= 1-sensitivity/specificity=0.19
Pretest ProbabilityOf pyelonephritis=50/100
If CRP is positivePosttest probabilityOf pyelo = 60/100
If CRP is negativePosttest probabilityOf pyelo = 15/100
Will the Results Help in Caring for my Patients?
• Will the reproducibility of the test result and its interpretation be satisfactory in my setting?
• Are the results applicable to my patient?
• Will the results change my management?
• Will patient be better off as a result of the test?
Question about treatment
• You are the hospitalist representative to your P&T committee. Your ED group has been lobbying strongly to have only Xopenex available on the hospital formulary because they say based upon their experience with a lot of asthma patients “it works better and has fewer side effects”. You respectfully ask for their evidence.
PICOPatient
Intervention
Control
Outcome
Child
Levalbuterol
Asthma
Racemic Albuterol
Admission Rates
How can the results help my patients?
• Were the study patients similar to my patients?
• Were all clinically important outcomes considered?
• Are the likely benefits of treatment worth the potential harms and costs?
EBM Resources
• Cochrane collaboration (via OVID)
• JAMA EBM reference guides on-line
• www.intensivecare.com
• http://cebm.jr2.ox.ac.uk
• http://depts.washington.edu/pedebm
• www.ped.med.umich.edu/ebm
Other on-line resources• Children’s Hospital Library Site
– Electronic Textbooks– Up to Date
• Countway Digital Library (www.countway.harvard.edu)– MD Consult
• Electronic Textbooks• Medline searching (crude, but quick)
– Pub Med– OVID Full Text
Statistical Significance vs. Clinical Impact
• Are the results of the study Valid?
• Are the results of the study Important? – Measures of Clinical Impact (effect size)
• Point estimates with confidence intervals
• Relative risks, odds ratios
• Absolute risk reduction
• Number needed to treat
• Will results help you to care better for your patients?