Applying Medical Literature I: Topics in Diagnostic Testing Mary Ottolini MD, MPH Vice Chair Medicl Education.

Applying Medical Literature I: Topics in Diagnostic Testing

Mary Ottolini MD, MPH

Vice Chair Medicl Education

Case

• Darius Smith is an 11 week old male infant with a 2 day history fever to 39.4. He has not been nursing as well and vomited a couple of times, but has kept down 3 oz of pediolyte in the ED. He is uncircumcised.

• He had a CBC and U/A obtained. The CBC at a WBC of 13,000 with an automated diff of 73.5% neutrophils, 3% bands, 22.5% lymphs and 1% monos.

• His U/A had pH: 7; SG: 1.025;(+)Nitrites and LE; 2+ blood; 2+protien; 10-15 RBC and 350 WBCs/HPF.

Case Cont:

• The ED attending wants you to admit him, because he is under 3 months of age at she thinks he has pyelonephritis because his CRP is elevated (44).

• You want to treat him and discharge him home,but you don’t know how good the CRP test is, or if there is another test you can do besides a DMSA scan to accurately predict whether he has pyelonephritis.

Objectives

• Calculate sensitivity, specificity, and likelihood ratios

• Explain how disease prevalence affects positive and negative predictive values

• Calculate and use likelihood ratios to choose among diagnostic tests

• Interpret a receiver operating characteristic curve • Interpret and apply a clinical prediction rule

When Do you do a Diagnostic Test?

CRP

Procalcitonin

Threshold Model of Diagnostic Test Ordering

Test

A B

No TestNo Treatment

No TestTreat

Thresholds are Determined by:

How good is the test? (Sensitivity and Specificity)

Risk of inappropriate treatment

Risk of the diagnostic test

Benefit of the appropriate therapy

0 1Pretest Probability

Questions

• What tests do you use to determine if the patient has a pyelonephritis? How good are these tests?

• Is this baby at increased risk for pyelonephritis? – How do you determine if a young infant likely has

pyelonephritis?– So What?- Do you manage young infants with

pyelonephritis differently than older infants?• What are the risks of inappropriate treatment?• What is the benefit of appropriate treatment?

Prevalence of UTI: Pretest Probabilities:

• Overall prevalence of UTI in age 2mos-2 yrs in fever without source= 5%

• Girls < 1yr= 6.5%; Boys=3.3%

• Girls 1-2 yrs= 8.1%; Boys=1.9%

• Circumcised boys= 0.2%-0.4% (5-20X greater in uncircumcised boys)

Why diagnose and treat?

• Prevent urosepsis

• Prevent renal scarring

• Eliminate acute infection

PICOPatient

Intervention

Control

Outcome

Infant

CRP

Pyelonephritis

How to Use the Article to Manage Your Patient

• Are the results of the study Valid?

• Are the results of the study Important?

• Can you Apply this valid, important evidence in caring for your patient?

Are the Results Valid?

• Was there an independent, blind comparison with a reference standard?

• Did the patient sample include an appropriate spectrum of patients to whom the diagnostic test will be applied in clinical practice?

• Did the results of the test being evaluated influence the decision to perform the reference standard?

• Were the methods for performing the test described in sufficient detail to permit replication?

Did the patient sample include an appropriate spectrum of patients to whom the diagnostic test will be applied in clinical practice?

METHODS:• Prospective study of 100 consecutive children (69 F/31M), 1 mo-13 yo

• 1st febrile UTI: admitted to the Peds Dept of U of Udine, 1/2000 –1/2002,

• Inclusion Criteria: fever (temperature of 38°C) and/or symptoms suggesting UTI, abdominal/flank pain, and for infants < 5 mos nonspecific signs, such as irritability, vomiting, difficulties with feeding, or failure to thrive.

• Definition of UTI: Catheterized or midstream, clean-void UC = single microorganism at 105 CFU/mL.

• Exclusion: Receipt of antibiotics in prior week or documented or suspected previous febrile UTI and/or known abnormalities of the urinary tract.

• All were treated with IV ceftriaxone, (75 mg/kg per day) for 1 to 5 days, followed by oral antibiotics per susceptibility, for a total of 10 days.

What is the Gold/Reference Standard?

How was pyelonephritis defined?

Was there an independent, blind comparison with a reference

standard?

Did the performance of the test being studies influence the interpretation of whether the “Gold

Standard” for disease was met

Did the results of the test being evaluated influence the decision

to perform the reference standard?

Did the CRP or PCT result influence whether a DMSA was performed?

Methods cont:

• All pts had a CBC, ESR, CRP and Procalcitonin

Imaging Studies• Renal ultrasound within 3 d after

admission, and VCUG 1 mo later • DMSA scintigraphy within 5 days

after admission for all children.• The presence/extent of renal lesions

were evaluated independently by 2 investigators, blinded with respect to CRP, PCT levels and the clinical status of the patient.

• Divided into 2 groups based on no to min renal involvment vs renal involvment

Were the methods for performing the test described in sufficient detail to permit replication?

What are the results?

• Are likelihood rations for the test results presented or data necessary for their calculation provided?

What are the results?

• Are likelihood rations for the test results presented or data necessary for their calculation provided?– If not provided: draw a 2x2 table to show how:

• Sensitivity and specificity were derived

• Positive and negative predictive value were derived

• Discuss the likelihood ratios and how you would interpret them

Results:TABLE 2. Sensitivity, Specificity, PPV, and NPV of PCT and CRP Measurements for Prediction of Acute Pyelonephritis

(Cutoff Values of 0.8, 0.5, or 1 ng/mL for PCT and 20 or 50 mg/L for CRP)

Sens Spec PPV NPV

PCT, 0.8 ng/mL 83.3 93.6 93.7 83.0

PCT, 0.5 ng/mL 90.7 70.2 77.7 86.8

PCT, 1 ng/mL 81.4 93.6 93.6 81.4

CRP, 20 mg/L 94.4 31.9 61.4 83.3

CRP, 50 mg/mL 74.0 76.6 78.4 72.0

PPV, positive predictive value; NPV, negative predictive value.

How do you tell how good the test is?

• Sensitivity and specificity

• Positive and negative predictive values

• Likelihood ratios

Sensitivity and Specificity

DiseaseNo

Disease

Test+

TestNeg

True (+)False (+)

False (-) True (-)

a b

c d

a/a+c d/b+d

Sensitivity Specificity

If the patient has disease what is the probability the test will be positive?If the patient does not have disease what is the probability the test will be negative?

Gold Standard

Sensitivity and Specificity

SnNOUTA test with ahigh sensitivitythat is negative rules things out

Specificity: NIH Negative in HealthSensitivity: PID Positive in Disease

SpPINA test with a high specificity that is positive rules things in

Positive and Negative Predictive Value

Disease No Disease

Test (+)

Test (-)

True (+) False (+)

False (-) True (-)

a b

c d

a/a+b

d/c+d

Positive Predictive Value

Negative PredictiveValue

If the test is positive what is the probability of disease?

If the test is negative what is the probability of no disease?

Likelihood Ratios

Disease No Disease

Test (+)

Test (-)

Sensitivity

1-Sens Specificity

1- Spec

1 1Positive LR: Sensitivity/1-Specificity: Prob of true(+)/prob false(+)

Negative LR: 1-Sensitivity/Specificity: Prob of false(-)/prob true (-)

1

1

Likelihood Ratios:

POSITIVE LR: Probability of a positive test in diseaseProbability of a positive test in health

NEGATIVE LR: Probability of a negative test in disease

Probability of a negative test in health

Positive LR Not usefulIf between 1-2More usefulIf >2<10Most useful if>10

Negative LR

Not useful

If between .5-1

More useful

If <.5>.1

Most useful if

<.1

Likelihood ratios+ Likelihood ratio= Probability abnl CRP if disease present

Probability abnl CRP if healthy

=sensitivity/1-specificity=1.38

- Likelihood ratio = Probability nl CRP - if disease presentProbability nl CRP - if healthy

= 1-sensitivity/specificity=0.19

Pretest ProbabilityOf pyelonephritis=50/100

If CRP is positivePosttest probabilityOf pyelo = 60/100

If CRP is negativePosttest probabilityOf pyelo = 15/100

Will the Results Help in Caring for my Patients?

• Will the reproducibility of the test result and its interpretation be satisfactory in my setting?

• Are the results applicable to my patient?

• Will the results change my management?

• Will patient be better off as a result of the test?

Question about treatment

• You are the hospitalist representative to your P&T committee. Your ED group has been lobbying strongly to have only Xopenex available on the hospital formulary because they say based upon their experience with a lot of asthma patients “it works better and has fewer side effects”. You respectfully ask for their evidence.

PICOPatient

Intervention

Control

Outcome

Child

Levalbuterol

Asthma

Racemic Albuterol

Admission Rates

How can the results help my patients?

• Were the study patients similar to my patients?

• Were all clinically important outcomes considered?

• Are the likely benefits of treatment worth the potential harms and costs?

EBM Resources

• Cochrane collaboration (via OVID)

• JAMA EBM reference guides on-line

• www.intensivecare.com

• http://cebm.jr2.ox.ac.uk

• http://depts.washington.edu/pedebm

• www.ped.med.umich.edu/ebm

Other on-line resources• Children’s Hospital Library Site

– Electronic Textbooks– Up to Date

• Countway Digital Library (www.countway.harvard.edu)– MD Consult

• Electronic Textbooks• Medline searching (crude, but quick)

– Pub Med– OVID Full Text

Statistical Significance vs. Clinical Impact

• Are the results of the study Valid?

• Are the results of the study Important? – Measures of Clinical Impact (effect size)

• Point estimates with confidence intervals

• Relative risks, odds ratios

• Absolute risk reduction

• Number needed to treat

• Will results help you to care better for your patients?