Diagnostic Test Studies

Diagnostic Test Studies

Tran The TrungNguyen Quang Vinh

Why we need a diagnostic test?

We need “information” to make a decision “Information” is usually a result from a test Medical tests:

To screen for a risk factor (screen test) To diagnosse a disease (diagnostic test) To estimate a patient’s prognosis (pronostic test)

When and in whom, a test should be done? When “information” from test result have a value.

Value of a diagnostic test

The ideal diagnostic test: Always give the right answer:

Positive result in everyone with the diseaseNegative result in everyone else

Be quick, safe, simple, painless, reliable & inexpensive

But few, if any, tests are ideal. Thus there is a need for clinically useful

substitutes

Is the test useful ?

Reproducibility (Precision) Accuracy (compare to “gold standard”) Feasibility Effects on clinical decisions Effects on Outcomes

Determining Usefulnessof a Medical Test

Question Possible Designs Statistics for Results

1. How reproducible is the test?

Studies of:

- intra- and inter observer &- intra- and inter laboratory

variability

Proportion agreement, kappa, coefficient of variance, mean & distribution of differences (avoid correlation coefficient)



2. How accurate is the test?

Cross-sectional, case-control, cohort-type designs in which test result is compared with a “gold standard”

Sensitivity, specificity, PV+, PV-, ROC curves, LRs


Question Possible Designs

Statistics for Results

3. How often do test results affect clinical decisions?

Diagnostic yield studies, studies of pre-& post test clinical decision making

Proportion abnormal, proportion with discordant results, proportion of tests leading to changes in clinical decisions; cost per abnormal result or per decision change


Question Possible Designs

Statistics for Results

4. What are the costs, risks, & acceptability of the test?

Prospective or retrospective studies

Mean cost, proportions experiencing adverse effects, proportions willing to undergo the test



5. Does doing the test improve clinical outcome, or having adverse effects?

Randomized trials, cohort or case-control studies in which the predictor variable is receiving the test & the outcome includes morbidity, mortality, or costs related either to the disease or to its treatment

Risk ratios, odd ratios, hazard ratios, number needed to treat, rates and ratios of desirable and undesirable outcomes

Common Issues for Studies of Medical Tests

Spectrum of Disease Severity and Test Results: Difference between Sample and Population? Almost tests do well on very sick and very well

people. The most difficulty is distinguishing Healthy &

early, presymtomatic disease. Subjects should have a spectrum of

disease that reflects the clinical use of the test.


Sources of Variation: Between patients Observers’ skill Equipments

=> Should sample several different institutions to obtain a generalizable result.


Importance of Blinding: (if possible) Minimize observer bias Ex. Ultrasound to diagnose appendicitis(It is different to clinical practice)

Studies of Diagnostic tests

Studies of Test Reproducibility Studies of The Accuracy of Tests Studies of The Effect of Test Results on

Clinical Decisions Studies of Feasibility, Costs, and Risks of

Tests Studies of The Effect of Testing on

Outcomes

Studies of Test Reproducibility

The test is to test the precision Intra-observer variability Inter-observer variability

Design: Cross-sectional design Categorical variables: Kappa Continuous variables: coefficient of variance

Compare to it-self (“gold standard” is not required)

Studies of the Accuracy of Tests

Does the test give the right answer? “Tests” in clinical practice:

Symptoms Signs Laboratory tests Imagine testsTo find the right answer.“Gold standard” is required

How accurate is the test? Validating tests against a gold

standard: New tests should be validated by

comparison against an established gold standard in an appropriate subjects

Diagnostic tests are seldom 100% accurate (false positives and false negatives will occur)

Validating tests against a gold standard

A test is valid if: It detects most people with disorder (high Sen) It excludes most people without disorder (high

Sp) a positive test usually indicates that the

disorder is present (high PV+) The best measure of the usefulness of a

test is the LR: how much more likely a positive test is to be found in someone with, as opposed to without, the disorder

A Pitfall of Diagnostic test

A test can separate the very sick from the very healthy does not mean that it will be useful in distinguish patients with mild cases of the disease from others with similar symptoms

Sampling

The spectrum of patients should be representative of patients in real practice.

Example: Which is better? What is the limits? Chest X-ray to diagnose aortic aneurism (AA).

Sample are 100 patients with and 100 without AA that ascertained by CT scan or MRI.

FNA to diagnose thyroid cancer. 100 patients with nodule > 3cm and had indication to thyroidectomy (biopsy was the gold standard).

“Gold standard”

“Gold standard” test: often confirm the presence or absence of the disease : D(+) or D(-).

Properties of “Gold standard”: Ruling in the disease (often doing well) Ruling out the disease (maybe not doing well) Feasible & ethical ? (ex. Biopsy of breast mass) Widely acceptable.

The test result

Categorical variable: Result: Positive or Negative Ex. FNA cytology

Continuous variable: Next step is: find out “cut-off point” by ROC

curve Ex. almost biochemical test: pro-BNP, TR-Ab,..

Analysis of Diagnostic Tests

Sensitivity & Specificity Likelihood ratio: LR (+), LR (-) Posterior probability (Post-test

probability) / Positive, Negative Predictive value (PPV, NPV); given Prior probability (Pre-test probability)

How accurate is the test?

Sensitivity and Specificity

Test Result

Disease D“Gold standard”

++ --

++ a b-- c d

aSensa c

dSpecb d

Positive & Negative Predictive Value

PV (+): positive predictive value

PV (-): negative predictive value

Test Result

Disease D+ -

+ a b- c d

( ) aPVa b

( ) dPVc d

/( )( )/( )a a cLikelihoodRatio LRb b d

Posterior odds

When combined with information on the prior probability of a disease*, LRs can be used to determine the predictive value of a particular test result:

Posterior odds = Prior odds x Likelihood ratio

*expressing the prior probability [p] of a disease as the prior odds [p/(1‑p)] of that disease. Conversely, if the odds of a disease are x/y, the probability of the disease is x / (x + y)

Choice of a cut-off pointfor continuous results

Consider the implications of the two possible errors:

If false‑positive results must be avoided (such as the test result being used to determine whether a patient undergoes dangerous surgery), then the cutoff point might be set to maximize the test's specificity

If false‑negative results must be avoided (as with screening for neonatal phenylketonuria), then the cutoff should be set to ensure a high test sensitivity

Choice of a cut-off pointfor continuous results

Using receiver operator characteristic (ROC) curves: Selects several cut-off points, and determines

the sensitivity and specificity at each point Then, graphs sensitivity (true‑positive rate) as

a function of 1‑specificity (false‑positive rate) Usually, the best cut-off point is where the

ROC curve "turns the corner”

RECEIVER OPERATING CHARACTERISTIC (ROC) curve

ROC curves (Receiver Operator Characteristic)

Ex. SGPT and Hepatitis

1-Specificity

Sensitivity1

1

SGPTSGPT D +D + D -D - SumSum< 50< 50 1010 190190 20020050-9950-99 1515 135135 150150100-149100-149 2525 6565 9090150-199150-199 3030 3030 6060200-249200-249 3535 1515 5050250-299250-299 120120 1010 130130>>300300 6565 55 7070SumSum 300300 450450 750750

Diagnostic Test Studies

Documents