EBM Prof Darwin 4. Prognosis EBM_dr Kuntjoro.ppt

AppraisingPrognostic Study

Introduction - Prognosis• Important phase of a disease progression of

a disease. • Patient’s, doctor’s, insurance’s concern • Prognosis: the prediction of the future course

of events following the onset of disease. – can include death, complications,

remission/recurrence, morbidity, disability and social or occupational function.

Introduction - Prognosis

• Possible outcomes of a disease and the frequency with which they can be expected to occur.– Natural history: the evolution of disease

without medical intervention. – Clinical course: the evolution of disease in

response to medical intervention.

Natural History Studies• Degree to which natural history can be studied

depends on the medical system (Scandinavia) and the type of disease (rare, high risk).

• The natural history of some diseases can be studied because:

• remain unrecognized (i.e., asymptomatic) e.g., anemia, hypertension.

• considered “normal” discomforts e.g., arthritis, mild depression.

Natural History Studies

• Natural history studies permit the development of rational strategies for:– early detection of disease

• e.g., Invasive Cervical CA.

– treatment of disease • e.g. Diabetes

Prognosis

Patients at risk of target eventPatients at risk of target event

Prognosticfactor

Prognosticfactor TimeTime

Suffer targetoutcome

Suffer targetoutcome

Do not suffertarget outcome

?

?

1. Was a defined, representative sample of patients assembled at a common (usually early) point in the course of their disease?

2. Was the follow-up of the study patients sufficiently long and complete?

3. Were objective outcome criteria applied in a blind fashion?

4. If subgroups with different prognoses are identified, was there adjustment for important prognostic factors and validation in an independent “test set” patients?

A. ARE THE RESULTS OF THIS A. ARE THE RESULTS OF THIS PROGNOSIS STUDY VALID? PROGNOSIS STUDY VALID? A. ARE THE RESULTS OF THIS A. ARE THE RESULTS OF THIS PROGNOSIS STUDY VALID? PROGNOSIS STUDY VALID?

• How well defined the individuals in the study – criteria - representative of the underlying population. – inclusion, exclusion – sampling method

• similar, well-defined point in the course of their disease cohort

A.1. Was a defined, representative sample of patients assembled at a common (usually early) point in the course of their disease?

A.1. Was a defined, representative sample of patients assembled at a common (usually early) point in the course of their disease?

A.2. Was follow-up sufficiently long and complete? A.2. Was follow-up sufficiently long and complete?

• Ideal follow-up period – Until EVERY patient recovers or has one of the other

outcomes of interest, – Until the elapsed time of observation is of clinical interest to

clinicians or patients. • Short follow up time too few study patients with outcome of

interest little information of use to patient • Loss to follow up influence the estimate of the risk of the

outcome validity?. – Patients are too ill (or too well); Die; Move, etc

• Most journals require at least 80% follow-up for a prognosis study to be considered valid.

• Best and worst case scenario (sensitivity analysis)

A.3. Were objective outcome criteria applied in a blind fashion?A.3. Were objective outcome criteria applied in a blind fashion?

• Investigators making judgments about clinical outcomes are kept “blind” to subjects’ clinical characteristics and prognostic factors.

• Minimize measurement bias!

Measurement bias• Measurement bias can be minimized by:

– ensuring observers are blinded to the exposure status of the patients.

– using careful criteria (definitions) for all outcome events.

– apply equally rigorous efforts to ascertain all events in both exposure groups.

• Prognostic factors: factors associated with a particular outcome among disease subjects. Can predict good or bad outcome

• Need not necessarily cause the outcome, just be associated with them strongly enough to predict their development – examples includes age, co-morbidities, tumor size, severity

of disease etc. – often different from disease risk factors e.g., BMI and pre-

menopausal breast CA.

A.4. If subgroups with different prognoses are identified, was there adjustment for important prognostic factors and validation in an independent “test set” patients?

A.4. If subgroups with different prognoses are identified, was there adjustment for important prognostic factors and validation in an independent “test set” patients?

• Risk factors – distinct from prognostic factors, – include lifestyle behaviors and environmental exposures

that are associated with the development of a target disorder.

– Ex: smoking = important risk factor for developing lung cancer, but tumor stage is the most important prognostic factor in individuals who have lung cancer.

A.4. If subgroups with different prognoses are identified, was there adjustment for important prognostic factors and validation in an independent “test set” patients?

A.4. If subgroups with different prognoses are identified, was there adjustment for important prognostic factors and validation in an independent “test set” patients?

Bias in Follow-up Studies A. Selection or Confounding Bias

1. Assembly or susceptibility bias: when exposed and non-exposed groups differ other than by the prognostic factors under study, and the extraneous factor affects the outcome of the study.

• Examples: – differences in starting point of disease (survival cohort)– differences in stage or extent of disease, co-morbidities, prior

treatment, age, gender, or race.

Bias in Follow-up StudiesA. Selection or Confounding Bias

2. Migration bias: • patients drop out of the study (lost-to-follow-up).

usually subjects drop out because of a valid reason e.g., died, recovery, side effects or disinterest.

• these factors are often related to prognosis. • asses extent of bias by using a best/worst case

analysis. • patients can also cross-over from one exposure group

to another • if cross-over occurs at random = non-differential

misclassification of exposure

Bias in Follow-up StudiesA. Selection or Confounding Bias

3. Generalizability bias• related to the selective referral of

patients to tertiary (academic) medical centers.

• highly selected patient pool have different clinical spectrum of disease.

• influences generalizability

Survival Cohorts

• Survival cohort (or available patient cohort) studies can be very biased because:– convenience sample of current patients are likely to be at

various stages in the course of their disease.– individuals not accounted for have different experiences

from those included e.g., died soon after trt.

• Not a true inception cohort e.g., retrospective case series.

Surv

ival

Coh

orts

B

ias

True CohortTrue Cohort

Survival CohortSurvival Cohort

ObservedImprovement

TrueImprovement

AssembleCohortN=150

Measure OutcomesImproved = 75Not improved = 75

50% 50%

80% 50%Measure OutcomesImproved = 40Not improved = 10

Assemble patients

BeginFollow-upN = 50

Not ObservedN = 100

Dropouts:Improved = 35Not improved = 65

II. Bias in Follow-Up Studies

B. Measurement bias– Measurement (or assessment) bias occurs when

one group has a higher (or lower) probability of having their outcome measured or detected.

• likely for softer outcomes – side effects, mild disabilities, subclinical disease or – the specific cause of death.

B. Are the results of this study important?B. Are the results of this study important?

1. How likely are the outcomes over time?

2. How precise is this prognostic estimate?

B.1. How likely are the outcomes over time? B.1. How likely are the outcomes over time?

• % of outcome of interest at a particular point in time (1 or 5 year survival rates)

• Median time to the outcome (e.g. the length of follow-up by which 50% of patients have died)

• Event curves (e.g. survival curves) that illustrate, at each point in time, the proportion of the original study sample who have not yet had a specified outcome.

Survival Rate 1 year survivalA. Good

B. 20%

C. 20%

D. 20%

Median survivalA. ?

B. 3 months

C. 9 months

D. 7.5 months

B.2 How precise is this prognostic estimate? B.2 How precise is this prognostic estimate?

• Precision 95% confidence interval– The narrower the confidence interval, the more

precise is the estimate.

• If survival over time is the outcome of interest shorter follow-up periods results in more precision follow up period important to be clinically important

C. Can we apply this valid, important evidence about prognosis to our patients?

1. Is our patient so different from those in the study that its results cannot apply?

2. Will this evidence make a clinically important impact on our conclusions about what to offer or tell our patient?

Is our patient so different from those in the study that its results cannot apply?Is our patient so different from those in the study that its results cannot apply?

• How well do the study results generalize to the patients in your practice? – Compare patients' important clinical characteristics, – Read the definitions thoroughly – The closer the match between the patient before

you and those in the study, the more confident you can be in applying the study results to that patient.

• For most differences, the answer to this question is “no”, we can use the study results to inform our prognostic conclusions.

C.2 Will this evidence make a clinically important impact on our conclusions about what to offer or tell our patient?

C.2 Will this evidence make a clinically important impact on our conclusions about what to offer or tell our patient?

• Useful for – Initiating or not therapy, – monitoring therapy that has been initiated, – deciding which diagnostic tests to order. – providing patients and families with the

information they want about what the future is likely to hold for them and their illness.

C.2 Will this evidence make a clinically important impact on our conclusions about what to offer or tell our patient?

C.2 Will this evidence make a clinically important impact on our conclusions about what to offer or tell our patient?

• Communicating to patients their likely fate

• Guiding treatment decisions• Comparing outcomes to make

inferences about quality of care

Conclusion• Prognosis study beneficial

– Communicating to patients their likely fate– Guiding treatment decisions– Comparing outcomes to make inferences

about quality of care