EPI-820 Evidence-Based Medicine LECTURE 3: DIAGNOSIS I Mat Reeves BVSc, PhD.

EPI-820 Evidence-Based Medicine

LECTURE 3: DIAGNOSIS I

Mat Reeves BVSc, PhD

Objectives:

• 1. Understand the four main types of clinical diagnostic strategies.

• 2. Define and understand test characteristics (Se & Sp).

• 3. Understand how test characteristics influence ability to rule-in and rule-out disease.

• 4. Understand the construction and application of ROC curves.

• 5. Define and understand PV+ and PV- and the influence of prevalence.

I. Clinical Diagnostic Strategies

• Diagnosis = the process of discovering a patient’s underlying disease by:• ascertaining the patient’s history, signs and

symptoms, • choosing appropriate tests, • interpreting the results and, • making correct conclusions.

• Highly complicated, not well understood process.

Four Basic Methods:

• A. Gestalt or Pattern Recognition• recognition of a pattern of signs and/or symptoms

that leads to an immediate diagnosis e.g., Parkinson’s disease, catatonia, myasthenia gravis

• signs or symptoms immediately rule-in diagnosis = pathognomonic

• very effective but limited to a few diseases. • Equivalent to Specificity = 100% (i.e., no FP’s)

B. Exhaustion

• collect very complete patient history and physical examination.

• then hope diagnosis will become apparent. • routinely taught, but rarely used (inefficient!!).

C. Branching algorithms

• clinical work-up presented as a series of logical steps

• results in a diagnosis, or a management plan.• represents close to the idealized diagnostic

process. • useful esp. if don’t have extensive experience

or rare condition.

D. Hypothetico-deductive reasoning

• strategy used by nearly all clinicians most all the time.

• formulate differential diagnoses list (hypotheses), then use tests to identify final diagnosis. • 1. Formulate explanations (hypotheses) for the patient’s

primary problem.• 2. Consider those that are most likely and/or that are

particularly harmful to miss (e.g., cerebral aneurysm for headache).

• 3. Simultaneously rule-out harmful/catastrophic and rule-in most likely.

• 4. Continue until one candidate has a very high likelihood (i.e., > 90%).

II. Clinical Test Characteristics• A. Sensitivity and Specificity

• "Diagnostic test" can be applied to any piece of clinical information - patient’s history, physical examination or diagnostic procedures.

• Assume, dichotomous disease and diagnostic test results – hence, 4 possible interpretations:

• Two are correct• true positive (TP) and true negative (TN)

• Two are incorrect • false positive (FP) and false negative (FN)

Figure 1. Relationship between Diagnostic Test Result and

Disease Status

FP

DISEASE

PRESENT (D+) ABSENT (D-)

TEST

POSITIVE (T+)

NEGATIVE (T-)

TP

FN TN

a bc d

FP

Sensitivity & Specificity

• Interpretation of diagnostic tests is concerned with comparing the relative frequencies and “costs” of the incorrect results (FNs and FPs) versus the correct results (TPs and TNs).

• Degree of overlap is a measure of the test effectiveness which is quantified by Se and Sp

D - D +

Diagnostic test result (continuous)

T - T +

Cut -point

TN TP

= FN

= FP

Figure 2. Results for a Typical Diagnostic Test Illustrating Overlap Between Disease (D+) and Non-disease (D-) Populations

Sensitivity (Se):

• defined as the proportion of individuals with disease that have a positive test result, or

• Se = TP = a . TP + FN a + c

• conditional probability of being test positive given that disease is present

• Se = P(T+ | D+). • referred to as the true-positive rate. • calculated solely from diseased individuals (LH column).

Specificity (Sp)

• defined as the proportion of individuals without disease that have a negative test result, or

• Sp = TN = d . TN + FP d + b

• conditional probability of being test negative given that disease is absent

• Sp = P(T-|D-)• referred to as the true-negative rate. • calculated solely from non-diseased individuals (RH

Column).

Figure 3. Example of the Sensitivity and Specificity of Impendence Plethysmography and I-125 Fibrinogen Leg Scanning for DVT (Ref: Hull et al, Ann Int Med, 1981)

FP

DVT

PRESENT (D+) ABSENT (D-)

TEST

Either or both POS (T+)

Both NEG (T-)

103

11 152

a bc d

8

Se = 103/114 = 90%

Sp = 152/160 = 95%

Figure 4. Example of a Perfectly Sensitive Test

D - D +

Diagnostic test resultT - T +

Cut -point

TN TP

= FP

Tests with High Sensitivity

• perfectly sensitive test (Se = 100%), all diseased patients are test positive (no FN’s)

• all test negative patients are disease free (TNs) but usually many FPs exist.

• highly sensitive tests are used to rule-out disease - if the test is negative you can be confident that disease is absent (FN results are rare!).

• highly sensitive tests do not tell you if disease is present, because they provides no information regarding FP’s (see Sp).

• SnNout = if a sign, symptom or other diagnostic tests has a sufficiently high Sensitivity, a Negative result rules out disease.

Tests with High Sensitivity

• Three clinical scenarios where high sensitivity test should be used:• 1) Early stages of a diagnostic work-up.

– large number of potential diseases are being considered. – a negative result indicates a particular disease can be dropped

(i.e., ruled out).

• 2) Important penalty for missing a disease. – Examples - TB, syphilis - dangerous but treatable conditions. – don’t want to miss cases, hence avoid false negative results

• 3) Screening tests.– the probability of disease is relatively low (i.e., low prevalence) – want to find as many asymptomatic cases as possible (incr.

yield)

Table 1. Examples of Tests with High Sensitivities

Disease/Condition Test Sensitivity

Duodenal ulcer History of ulcer, 50+ yrs, pain relieved by eating or pain after eating

95%

Favourable prognosis following non-traumatic coma

Corneal reflex 92%

High intracranial pressure Absence of spont. pulsation of retinal veins 100%

Deep vein thrombosis Positive to impedance plethysmography and/or fibrinogen leg scanning

92%

Pancreatic cancer Endoscopic retrograde cholangio- pancreatography (ERCP)

95%

Tests with High Specificity

• perfectly specific test (Sp = 100%), all non-diseased patients test negative (no FP’s)

• all test positive patients have disease (TPs) but usually sizeable number of FNs

• highly specific tests are used to rule-in disease - if the test is positive you can be confident that disease is present (FPs are rare).

• highly specific tests do not tell you if disease is absent, because Sp provides no information regarding FNs (see Se).

• SpPin = if a sign, symptom or other diagnostic tests has a sufficiently high Specificity, a Positive result rules in disease.

D - D +

Diagnostic test result

T - T +

Cut -point

TN TP

= FN

Figure 5. Example of a Perfectly Specific Test

Tests with High Specificity

• Clinical scenarios when high specificity tests should be used:• 1) To rule-in a diagnosis suggested by other tests

– specific tests are therefore used at the end of a work-up to rule-in a final diagnosis e.g., biopsy, culture, CT scan.

• 2) False positive tests results can harm patient– want to be absolutely sure that disease is present.– example, the confirmation of HIV positive status or the

confirmation of cancer prior to chemotherapy.

Table 2. Examples of Tests with High Specificities

Disease/Condition Test Specificity

Alcohol dependency Yes to 3 or more of the 4 CAGE questions 99.7%

Iron-deficiency anemia Serum ferritin 90%

Deep vein thrombosis Negative to impedance plethysmography and/or fibrinogen leg scanning

92%

Pancreatic cancer Endoscopic retrograde cholangio- pancreatography (ERCP)

97%

Breast cancer Fine needle aspirate 98%

Strep throat Pharyngeal gram stain 96%

B. Trade-off Between Sensitivity and Specificity

• inherent trade-off between sensitivity and specificity. • the location of the cut-point is arbitrary, and should

be modified according to the purposes of the test. • trade-off between Sp and Se cannot be avoided • ideal cut-point depends on what the purpose of the

test is - do you want high Se or high Sp or a balance between the two?

D - D +

Diagnostic test result

T - T +

Cut -point

TN TP

= FN

= FP

Figure 6. Trade-off Between Se and Sp: Lowering the Test Cut-point Increases Se but Decreases Sp

III. Receiver Operator Characteristic (ROC) Curves

• an alternative way to demonstrate the relationship between Se and Sp

• constructed by plotting sensitivity (or true positive rate) against the false positive rate (1 - Specificity) using various cut points.

Figure 7. A Receiver Operator Character-istic Curve (The Accuracy of the CK Test in the Diagnosis of Myocardial Infarction)

Uses for the ROC Curve

• 1. Comparing the usefulness of different tests• increase Se and Sp: the further curve is pushed

into the top left hand corner - best tests lie "to the north-west"

• worthless test has equal TP and FP rates - indicated the 45 degree diagonal (LR= 1.0).

• compare discriminating ability of tests by calculating the Area Under the ROC Curve

– AUROCC varies from 0.5 (no ability) to 1.0 (perfect accuracy)

2. Deciding on the Best Cut-point

• best cut-point is influenced by:• the likelihood of disease (i.e., its prevalence), and• the relative costs (or risk-benefit ratio) associated

with FPs and FNs.

• optimal operating position is where the slope of the ROC curve equals:

ACfp P(D-)ACfn P(D+)

IV. Prevalence and Predictive Values

• In terms of conditional probabilities Se and Sp can be defined as:• Se = P(T+|D+)• Sp = P(T-|D-)

• Problem: can only be calculated if the true disease status is known!

• But: the clinician is using a test precisely because the disease status is unknown!

• Clinician actually wants the conditional probability of disease given the test result, OR• P(D+|T+) and P(D-|T-)

Predictive Value Positive (PVP)

• the probability of disease in a patient with a positive (abnormal) test.

• PVP = TP = a . TP + FP a + b

• conditional probability of being diseased given the test was positive, or• PVP = P(D+|T+) • note link between Sp and PVP via FP rate. A

highly specific test rules-in disease because PVP is maximized.

Predictive Value Negative (PVN)

• the probability of not having disease when the test result is negative (normal).

• PVN = TN = d . TN + FN d + c

• conditional probability of not being diseased given the test was negative• PVN = P(D-|T-). • note the link between Se and PVN via the FN rate.

A highly sensitive test rule-out disease because PVN is maximized.

Prevalence

• the proportion of the total population tested that have disease, or

• P = Total Number of Diseased Total Population (N) = TP + FN = a + c .

TP+FN+FP+TN a + b + c + d

• very important

Prevalence

• can have a dramatic influence on predictive values

• Prevalence is the "third force" - often goes unnoticed only to reveal its influence in dramatic fashion!

• Equivalent names:• the likelihood of disease, prior probability, prior

belief, prior odds, pre-test probability and pre-test odds.

Figure 8. The PVP and PVN of the Combined IP and I-125 FS Tests in DVT When Prevalence of DVT is 42%

FP

DVT

PRESENT (D+) ABSENT (D-)

Either or both POS (T+)

Both NEG (T-)

103

11 152

a bc d

8

Se = 90% Sp = 95%

PVP = 103/111 = 92.8%

N = 114 N = 160 N = 274

PVN = 152/163 = 93.3%

Figure 9. The PVP and PVN of the Combined IP and I-125 FS Tests in DVT When Prevalence of DVT is 15%

FP

DVT

PRESENT (D+) ABSENT (D-)

Either or both POS (T+)

Both NEG (T-)

27

3 161

a bc d

9

Se = 90% Sp = 95%

PVP = 27/36 = 75.0%

N = 30 N = 170 N = 200

PVN = 161/164 = 98.2%

.20

.20

0.40

.40

.60

.60

.80

.80

Pre

dict

ive

Val

ue

Prevalence or Prior probability

PV-

1.0

1.0

PV+

Figure 10. The PVP and PVN as a Function of Prevalence for a Typical Diagnostic Test

• As prevalence falls, positive predictive value must

fall along with it, and negative predictive value must rise. Conversely, as prevalence increases, positive predictive value will increase and negative predictive value will fall.

SummaryI. Test Operating Characteristics

  Also Called

Derived From

Useful Result

Most Affects

Sensitivity true positive rate

patients with disease

negative Negative predictive value

Specificity true negative rate

patients without disease

positive Positive predictive value

II. Testing Situations

  Likely disease prevalence

Need good.... Use a test which is...

Rule Out low Negative predictive value

Sensitive

Rule In high Positive predictive value

Specific