@2003 Martin L Lesser, PhD

SCREENING AND

DIAGNOSTIC TESTING

Martin L Lesser, PhDBiostatistics Unit

Feinstein Institute for Medical Research

North Shore – LIJ Health System

@2003 Martin L Lesser, PhD

OUTLINE• What is a Screening test? • Objectives of Screening• Features of a Good Screening test?• Diagnostic Testing• Calculations: Sensitivity, Specificity,

PPV, NPV, Accuracy, Prevalence, Bayes’ Theorem, ROC Curves

What is a Screening test?

• A test administered to a group of asymptomatic people to detect the signs of a disease (does not diagnose… if positive, need further evaluation)

• Usually a secondary prevention technique-improve outcome of illness in ‘affecteds’-reduce severity of disease-reduce mortality

SCREENING vs DIAGNOSIS-asymptomatic -possibly symptomatic -usually in a high risk -not necessarily in high

group (f-hx, lifestyle) risk group-community setting -clinical setting-inexpensive -can be expensive-easy to administer -may be complex-less invasive -may be invasive-relatively safe -may be risky-does not diagnose per se -goal is definitive

diagnosis

Features of a Good Screening Test

• Detects disease prior to clinical symptoms• Effective therapy or treatment must exist for

the disease detected (accessible and acceptable to ‘screenee’)

• Early detection would likely lead to a cure or effective treatment

• Safe to administer (and quick)• Not very costly• Must not cause undue anxiety• Preferably, follow-up diagnostic test must

not be harmful, cumbersome or expensive• Results must be valid, reliable and

reproducible

• Serious (important) disease

Screening Test Examples• Sphygmomanometer: Hypertension, CAD,

CVA• Pap Smear: Cervical Cancer• PPD Test: Tuberculosis• Cholesterol test: Hypercholesterolemia,

CAD• Mammogram: Breast Cancer• Chest X-ray: Lung Cancer• Fecal Occult Blood Test: Colon Cancer• PSA: Prostate Cancer

Who should be screened?

-NOBODY -EVERYBODY -SOME

-no benefit -wasteful (low risk)

-harmful (no cure)

-costly

-who might benefit?

-high risk grp! -family hx!

-cheap!

Diseases Appropriate for Screening

• Must be serious

• Beneficial pre-symptomatic treatment

• High prevalence of preclinical disease

Standard 2x2 Table

Disease

Test

Resu

lt

Disease +

Disease -

Test +

Test -

b (FP)

c (FN)

a (TP)

d (TN)

Sensitivity

=a/(a+c)

Specificity

=d/(b+d)

PPV***=a/(a+b)

NPV***=d/(c+d)

***formula applicable only when sampling is cross-sectional -may have to use Bayes’ Rule!!!

PPV***=a/(a+b)

SENSITIVITY

=Persons with the disease who test positive x 100%_______________________________________________

Total number of persons with the disease

=a x 100%

_______________

(a + c )

Standard 2x2 Table

Disease + Disease -

Test + a (TP) b (FP)

Test - c (FN) d (TN)

SPECIFICITY

=Persons without the disease who test negative x 100%_______________________________________________Total number of persons without the disease

=d x 100%

_______________ (b + d)

Standard 2x2 Table

Disease + Disease -

Test + a (TP) b (FP)

Test - c (FN) d (TN)

POSITIVE PREDICTIVE VALUE (PPV)

=Persons with a positive test who have the disease x 100%_______________________________________________

Total number of persons who test positive

=a x 100%

_______________

(a + b)

Standard 2x2 Table

Disease + Disease -

Test + a (TP) b (FP)

Test - c (FN) d (TN)

NEGATIVE PREDICTIVE VALUE (NPV)

=Persons with a negative test who don’t have disease x 100%_______________________________________________

Total number of persons who test negative

=d x 100%

_______________

(c + d)

Standard 2x2 Table

Disease + Disease -

Test + a (TP) b (FP)

Test - c (FN) d (TN)

ACCURACY

=Persons with a correct diagnosis x 100%_______________________________________________

Total number of persons tested

=(a+d) x

100%_______________(a + b + c +

d)

=(TP + TN) x

100%____________________

(a + b + c + d)

Standard 2x2 Table

Disease + Disease -

Test + a (TP) b (FP)

Test - c (FN) d (TN)

Test Characteristics• Fixe

d

• Relative

(Population Independent)

(Population Dependent)

-Sensitivity

-Likelihood that someone with disease has a positive test

-Specificity

-Likelihood that someone without disease has a negative test

-Positive Predictive Value

-Likelihood that someone with a positive test has the disease

-Negative Predictive Value

-Likelihood that someone with a negative test does not have the disease

Fixed Characteristics

• Screening Rule Out High Sensitivity• Diagnosis Rule In High Specificity

Highly Sensitive Test

-picks up most people with disease who truly have disease

Highly Specific Test

-unlikely to mislabel people as having disease when in fact they do not have the disease

-good for screening!!!

-avoid unnecessary treatment!!!

ExamplesExample 1 : PPD Test

for TB Diameter > 1 mm

=> TB+-results in too many

TB+’s (high FP)

Example 2 : CA125 in Ovarian Ca

-elevated CA125 even in non-Ovarian Cancer cases!!!

-unlikely to miss a true case of Ovarian Ca (high sensitivity)

-many people who don’t have the disease will test positive

(low specificity)

Relative Characteristics

PPV and NPV are related to the overall prevalence of the disease in the population you are testing!

NOTE! We normally assume that Sensitivity and Specificity remain constant regardless of the prevalence of the disease in the population you are testing .

Prevalence and PPV

Example: HIV Testing

1. Drug Rehab Center 2. Monastery of St. Claire

Bayes’ TheoremPPV/NPV: influenced by 3

quantities:

•Sensitivity

•Specificity

•Prevalence (prior odds)

**As Prevalence increases->

PPV increases!

**As Prevalence increases->

NPV decreases!

Bayes’ Formula for PPVPPV= Pr (D+| T+) x 100%

=Pr (T+|D+) x Pr (D+) x 100%_______________________________________________{Pr (T+|D+) x Pr (D+)} + {Pr(T+|D-) x

Pr(D-)}

=sens x prev x 100%

_____________________________________{sens x prev} + {(100-spec) x

(100-prev)}

Bayes’ Formula for NPVNPV= Pr (D-| T-) x 100%

=Pr (T-|D-) x Pr (D-) x 100%_______________________________________________{Pr (T-|D-) x Pr (D-)} + {Pr(T-|D+) x

Pr(D+)}

=spec x (100-prev) x 100%

_________________________________________{spec x (100-prev)} + {(100-

sens) x prev}

How to Set Cut Points

-It’s like tuning your radio!

-Want to pick up certain frequencies (disease)Want to catch disease, i.e. Minimize missing disease

Avoid too many false positives

Attain a balance of Sensitivity and Specificity!!!

Receiver Operating Characteristic Curves (ROC Curves)

Legend: England--Battle of Britain -performance of radar receiver

operators

TP: Correct early warning of German planes coming over the English Channel

FP: Receiver operator sent out alarm but no enemy planes appeared

FN: German planes appeared without previous warning from the radar operators.

Constructing an ROC Curve

Simply:

Plot Sensitivity on the Y-axisagainst FP (or 100-Specificity) on the X-axis

Hypothetical Example:Blood Pressure Screening to Predict 10-Year Stroke Risk in Subjects 50

Years and Older-Take single blood pressure measurement in a large number of subjects

-Follow subjects for 10 years to determine stroke status

Cutoff for + Test (SBP)

> 0 mm Hg

Sensitivity

100%

FP

100%

> 500 mm Hg

0% 0%

> 120 mm Hg

?? ??

> 130 mm Hg

?? ?? ?? > 140 mm

Hg

Specificity

0%

100%

??

?? ?? ??

CUTOFF SENSI TI VI TY SPECI FI CI TY FP 0 100 0 100

110 98 12 88 120 95 40 60 130 90 55 45 140 75 80 20 150 58 90 10 160 40 94 6 180 20 97 3 200 10 98 2 500 0 100 0

Competing Screening Tests

-Plot the ROC curves on the same graphExample: SBP vs. Cholesterol vs.

HgbA1c

-Area Under the ROC curve:

is the probability that a randomly selected pair of normal and abnormal subjects can be correctly classified

Do Screening Tests Work?

•Analysis of Outcomes

Survival of those diagnosed by screening prior to symptoms

versus

Survival of those diagnosed at the time of symptomatic presentation

•Other ways: (Randomized trial, Population based study)

Lead Time Bias

Precancerous

Cells

Small Nodule

Advanced Disease

Death

Time from Dx at Screening to Death

Time from Dx at Clinical Presentation to Death

Lead Time

Length Bias-------------------------------------------------------

-----------------------------------------------------------

--------

---------

----------------------------------------------

---------------

---------------

----- --------------

----

------------------------------------------------

-------

---------------------

---------------------------------

Other Sources of Bias (Source: Begg CB, Statistics in Medicine 1987)

Subject Selection•Case Mix•Verification bias•Uninterpretable test results•Inter-observer variation•Temporal changes

Methodology•Influence of clinical factors on interpretation•Variation in positivity criterion•Absence of a definitive reference test•Cutoff point validation bias

REFERENCES• Jekel, JK, Katz, DL, Elmore JG. Epidemiology,

Biostatistics, and Preventive Medicine. 2nd Ed. 2001. WB Saunders Company-Harcourt Health Sciences.

• Hennekens CH MD DrPH, Buring JE, ScD. Edited by Mayrent SL, PhD. Epidemiology in Medicine. 1st Ed. 1987. Little Brown and Company, Boston/Toronto.

• Dawson B, Trapp RG. Basic & Clinical Biostatistics.

3nd Ed. 2000. McGraw-Hill Medical Publishing Division.

• Lesser, ML in Fishman-Javitt MC, MD Stein HL, MD Lovecchio JL, MD (eds). 1990. Imaging of the Pelvis-MRI and Correlations to CT and Ultrasound.

@2002 Cristina P. Sison, PhD

Thanks!

For Statistical consulting, call:

NORTH SHORE-LIJ HEALTH SYSTEM: BIOSTATISTICS UNIT(516) 240-8300CORNELL PEOPLE: (212) 746-8544CORNELL GCRC: (212) 746-6291

@2002 Cristina P. Sison, PhD