SCREENING FOR DISEASE Nigel Paneth. THREE KEY MEASURES OF VALIDITY 1.SENSITIVITY 2.SPECIFICITY 3.PREDICTIVE VALUE.

SCREENING FOR DISEASE

Nigel Paneth

THREE KEY MEASURES OF VALIDITY

1. SENSITIVITY

2. SPECIFICITY

3. PREDICTIVE VALUE

SENSITIVITYSensitivity tells us how well a positive test detects disease.

It is defined as the fraction of the diseased who test positive.

Its complement is the false negative rate, defined as the fraction of the diseased who test negative.

Sensitivity and false negative rate add up to one.

SENSITIVITY AND THE FALSE NEGATIVE RATE ARE

COMPLEMENTARY

N who test positive + N who test negative = 1

All with disease All with disease

SENSITIVITY + FALSE NEGATIVE RATE = 1

SPECIFICITYSpecificity tells us how well a negative test detects non-disease.

It is defined as the fraction of the non-diseased who test negative.Its complement is the false positive rate, defined as the fraction of the non-diseased who test positive.Specificity and the false positive rate add up to one.

SPECIFICITY AND THE FALSE POSITIVE RATE ARE COMPLEMENTARY

N who test negative + N who test positive = 1

All without disease All without disease

SPECIFICITY + FALSE POSITIVE RATE = 1

DENOMINATORS OF THESE RATES

• Note that all the denominators of the four rates so far defined (sensitivity, specificity and the false + and false – rates) are DISEASE STATES

• The denominators of sensitivity and the false negative rate is PEOPLE WITH DISEASE

• The denominators of specificity and the false positive rate is PEOPLE WITHOUT DISEASE

PREDICTIVE VALUE

Positive predictive value is the proportion of all people with positive tests who have the disease.

Negative predictive value is the proportion of all people with negative tests who do not have the disease.

PREDICTIVE VALUES DEFINED

• POSITIVE PREDICTIVE VALUE =

All people with disease

All people with a positive test

• NEGATIVE PREDICTIVE VALUE =

All people without disease

All people with a negative test

POINTS TO NOTE

• Note that the numerators and denominators are reversed compared to sensitivity and specificity. In predictive values, the denominator is the test result, and the numerator is disease or non-disease

• In general, the positive predictive value is the one most used. Positive predictive value and sensitivity are perhaps the two most important parameters in understanding the usefulness of a test under field conditions.

CRITICAL DIFFERENCE BETWEEN DISEASE-DENOMINATORED AND

TEST-DENOMINATORED MEASURES

• Sensitivity and specificity do not vary according to the prevalence of the disease in the population.

• Predictive value of a test, however is HIGHLY DEPENDENT on the prevalence of the disease in the population

CALCULATING THE RATESA test is used in 50 people with disease and 50 people without. These are the results:

Disease

+ -

Test+ 48 3 51

- 2 47 49

50 50 100

Sensitivity = 48/50 = 96%

Specificity = 47/50 = 94%

Positive predictive value = 48/51 = 94%

Negative predictive value = 47/49 = 96%

Disease

+ -

Test+ 48 3 51

- 2 47 49

50 50 100

Now lets take this test out into a population where 2% of people have the disease, not 50% as in the previous example. Assume there are 10,000 people, and the same sensitivity and specificity as before, namely 96% and 94%, respectively

Disease

+ -

Test+ 192 588 780

- 8 9,212 9,220

200 9,800 10,000

• What is the positive predictive value now? 192/780 = 24.6%• When the prevalence of disease is 50%, 94% of positive tests indicate disease. But when prevalence is only 2%, less than one in four test results indicate a person with disease, and 2% actually would represents a quite common disease. • False positives tend to swamp true positives in populations, because most diseases we test for are rare.

CHANGING THE THRESHOLD FOR A TEST When disease is defined by a threshold on a continuous test, the test characteristics can be altered by changing the threshold or cut-off point.

Lowering the threshold improves sensitivity, but often at the price of lowered specificity (i.e. more false-positives).

Raising the threshold improves specificity, but often at the price of lowered sensitivity (i.e. more false negatives).

This can be especially important when the distribution of a characteristic is unimodal, such as blood pressure, cholesterol, weight, etc. (Because the gray area is so large).

PROBLEMS WITH SCREENING1. Do we have the right threshold?

2. Is there a truly effective treatment available for the discovered disease?

3. Is that treatment more effective in screened than non-screened cases?

4. What are the side effects of the screening process?

5. How efficient is screening? i.e. how many people must be screened to obtain a case?

A randomized trial to assess a screening program for colon cancer is instituted. The intervention group gets regular screening, the control group is left to its own devices.

EXAMPLE OF SCREENING ASSESSMENT

After five years it is found that:

1. More cases are discovered in the screened group than in the controls.

2. The cases are discovered at an earlier stage of the cancer in the screened group.

3. Five year survival is higher for the people with cancer in the screened group.

Can we conclude that this screening program is necessarily effective?

NO, THE PROGRAM IS NOT NECESSARILY EFFECTIVE 

The apparent benefits may only demonstrate the effects of LEAD-TIME BIAS. If it is possible to diagnose a condition earlier, but not to improve survival after diagnosis, the screening program will have an over-representation of earlier diagnosed cases, whose survival will be increased by exactly the amount of time their diagnosis was advanced by the screening program. Thus they have not benefited, but the amount of time they know they have cancer has been increased.

Consider how time of diagnosis changes with screening in the scenario below:

unscreened group: Dx Death

Age 50 51 52 53 54 55

screened group: Dx Death

Age 50 51 52 53 54 55

In the previous scenario, incidence of disease is initially higher, diagnosis is made earlier, stage of diagnosis is earlier, and duration of survival from diagnosis is longer. All of these give the impression of benefit from screening.However, the patient does not benefit, as death is not postponed.

The only proper evidence of effectiveness of a screening program is a reduction of total age-specific mortality or morbidity, ideally demonstrated by randomized trial.

MAMMOGRAPHY EXERCISE

The next two slides are answers to

questions in the following website

http://mammography.ucsf.edu/inform/index.cfm

QUESTION 12

Part 1. Under age 50, sensitivity is 75%, over 50, sensitivity is 90%.

Part 2. Under age 50, specificity is about 97%, over 50, about 98.5%.

Part 3. Under age 50, PP+ is about 3%; over 50, about 6-7%. At all ages, about 5%

QUESTIONS 13 AND 14

• These questions raise the concept of -

Number needed to screen

How many women in each age group must be screened to save one life from breast cancer?