OBSERVATIONAL STUDIES Instructor: Fabrizio D’Ascenzo [email protected] Role MD.

OBSERVATIONAL STUDIES

Instructor: Fabrizio D’[email protected]

www.emounito.orgwww.metcardio.org

Role MD

CONFLICT OF INTEREST

None

AIM OF THE COURSE

A critical appraisal

- Theorical- Practical

of observational studies

TODAY’S PROGRAM: FIRST PART

1) Literature: clinical general concepts

2) Literature: clinical methodological concepts

3) Quick assessment of an observational study

4) Complete assessment of on observational study

HOW TO READ and WRITE A STUDY

Two points of view:

- Clinical

- Methodological

CLINICAL

- Strenght of association- Temporality- Consistency

- Theorical Plausibility- Coherence

- Specificity in the cause- Dose-response

- Experimental evidence- Analogy

STRENGHT OF ASSOCIATION

Size of the association as measured by appropriate statistical tests

Example Odds Ratio, Relative Risk

But

strength of association depends on the prevalence of other potential confounding

factors

TEMPORALITY

Exposure should always precede the outcome

CONSISTENCY

The association is consistent when results are replicated

in

studies in different settings using different methods. 

If a relationship is causal, we would expect to find it

consistently in different studies and among different

populations. 

THEORICAL PLAUSIBILITY andCOHERENCE

The association agrees with currently accepted

understanding of pathological processes. 

A causal association is increased if a biological gradient or

dose-response curve can be demonstrated.

The association should be compatible with existing theory

and knowledge. 

IS THIS ENOUGH?

RELIABLE EVIDENCE?

METHODOLOGICAL

GRADING THE EVIDENCE

WHY TO PERFORM AND READ NOT RANDOMIZED EVIDENCE?

• to save economical resources

• to create hypothesis, especially for non

randomizable patients

• to shed light on the generalizability of results

from existing randomized experiments

HOW TO EVALAUTE NON RANDOMIZED EVIDENCE?

QUICK ASSESSMENT OF AN OBSERVATIONAL STUDY

3 CRUCIAL CONCEPTS

- DESIGN OF THE STUDY

- BIAS

- MULTIVARIATE ANALYSIS

THREE DIFFERENT DESIGNS

COHORT

Advantages: chances to appraise different

outcomes

Disvantages: if events/outcomes are unfrequent,

large number of patient is needed

CASE-CONTROL

Advantages: studies for infrequent outcomes

Disvantages: controls patients need to be

selected from the whole population

CROSS SECTIONAL

Advantages: easy to perform

Disvantages: limited function

OR EASIER

• Retrospective>means testing an hypothesis on datasets

- already present- built for that hypothesis but not at the time of

patients’assessment

• Prospective>means testing an hypothesis on datasets built for it, to evaluate, study and insert data of the patients at the moment of their hospitalization/drug assumption/intervention

REASON FOR ASSOCIATIONS

REASON FOR ASSOCIATIONS

• Bias

• Confounding

• Chance

• Cause

BIAS

Measure of association between exposure and

outcome is systematically wrong

Two directions:

- bias away from the null

- bias towards the null

SELECTION BIAS

Unintended systematic difference between

the two or more groups, which is associated

with the exposure.

FOR EXAMPLE

Inclusion of too selected patients:

> patients with more severe disease presentation are

often excluded

TO

obtain larger benefits

If reported:

How many patients attain a complete follow up>

if a patient is lost at follow up, he/her may have dead (more probably) or alive

ATTRITION BIAS

1192 consecutive patients undergoing PCI in our center

between January 2009 and January 2011

1116 patients with follow up data derived from Piedmont Region

dedicated registry (AURA)

37 not detectable

(30 not European….)

1155 at follow up of 787 days (median;474-1027)

Medical folders of each patient, and for re-hospitalizations were re-analyzed by a

physician

76 patients not recorded in Piedmont Region dedicated registry:

39 recovered through phone call

Figure 1.

If reported:

who adjudicate the events:

- A blinded central committee

- Non blinded researchers

ADJUDICATION BIAS

an error in measuring exposure or

outcome may cause information bias>lower

risk if the study is multicenter

ANALITICAL/INFORMATION BIAS

IF REPORTED….

CHANCE

The precision of an estimate of the association between

exposure and outcome is usually expressed as a confidence

interval

(usually a 95% confidence interval)

The width of the confidence

interval is determined by the number of subjects with the outcome of interest,

which in turn is determined by the sample size.

With 200 ptsVariables in the Equation

.069 .582 .014 1 .906 1.071 .342 3.351

.488 .567 .739 1 .390 1.629 .536 4.950

.769 .565 1.855 1 .173 2.158 .713 6.527

.010 .747 .000 1 .990 1.010 .233 4.368

2.111 .547 14.886 1 .000 8.256 2.825 24.126

DIABETE

PREGRESS

RICOVERO

V21

GSP_POSI

B SE Wald df Sig. Exp(B) Lower Upper

95.0% CI for Exp(B)

Variables in the Equation

.069 .238 .084 1 .773 1.071 .672 1.706

.488 .232 4.436 1 .035 1.629 1.034 2.564

.010 .305 .001 1 .975 1.010 .555 1.836

.769 .231 11.131 1 .001 2.158 1.373 3.390

2.111 .223 89.317 1 .000 8.256 5.329 12.791

DIABETE

PREGRESS

V21

RICOVERO

GSP_POSI

B SE Wald df Sig. Exp(B) Lower Upper

95.0% CI for Exp(B)

With 1000 pts

CONFOUNDINGThe aim of an observational study is to examine

the effect of the exposure,

but

sometimes the apparent effect of the exposure

is

actually the effect of another characteristic

which is associated with the exposure and

with the outcome.

MULTIVARIATE ANALYSIS

Multivariable analysis aims to explore the

relationship

between a dependent variable

and

two or more independent variables appraised

simultaneously.

ARE ALL MULTIVARIATE ANALYSIS THE SAME?

• Logistic regression

• Cox Multivariate adjustement

• Propensity score

HOW TO CHOOSE VARIABLESTo avoid:

- automatic algorithms with stepwise selection

To choose established association from:

- prior well conducted experimental or clinical studies

- strong associations (e.g.p<0.10 or p<0.05 at

univariate analysis)

LOGISTIC REGRESSION: THE SIMPLEST ONE

The logit function transforms a dependent

variable ranging between 0 and 1 such as a

probability of an event

into a variable stemming from −∞ to +∞.

Thus, event probabilities can be appraised as a

linear regression function

to

appraise the logit of the probability of an event

(dependent variable) given one or more

dependent variables

LOGISTIC REGRESSION: THE SIMPLEST ONE

LOGISTIC REGRESSION: THE SIMPLEST ONE: LIMITS

Overfit model can be highly predictive in the

dataset in which the model was developed, but

not in one in which it is validated or tested.

Multicollinearity, whereby covariate present in

the model are unduly associated

Does not correct for time

COX PROPORTIONAL HAZARD ANALYSIS: THE MOST USED ONE

• It addresses differences in follow-up duration and

censored data

• It is based on The hazard function, which forms

the basis of Cox analysis: the event rate at time t

conditional on survival until time t or late

CENSORED DATA

Censored patients are exploited to compute

hazards and are assumed in the Cox model

to fail at the same rate as the non censored,

but are not supposed to survive to the next

time point.

The term right censored implies that the event of

interest (i.e., the time-to-failure) is to the right of

our data point. In other words, if the units were to

keep on operating, the failure would occur at

some time after our data point (or to the right on

the time scale)

RIGHT CENSORED DATA

INTERVAL CENSORED DATA

If we inspect a certain unit at 100 hours and find

it operating

and perform another inspection at 200 hours to

find that the unit is no longer operating,

then the only information we have is that the unit

failed at some point in the interval between

100 and 200 hours.

A failure time is only known to be before a certain time.

LEFT CENSORED DATA

PROPENSITY SCORES: THE NEW ONE

conditional probability of receiving an

exposure or treatment given a vector of

measured covariates

Courtesy of American Heart Association

Propensity scores act as a proxy between

cases and covariates influencing exposure,

and thus can be used instead of such

covariates to simplify the analysis plan and

increase robustness

PROPENSITY SCORES: THE NEW ONE

How to do it:

a logistic regression in a non-parsimonious fashion

results of this non-parsimonious logistic regression are

then exploited to build the propensity score

THEN

insert in multivariate adjustment to increase accuracy

matching

PROPENSITY SCORES: THE NEW ONE

Different methods:

- calipers of width of 0.2 of the standard deviation of

the logit of the propensity score

- Mahalanobis metric

Matching

-greedy matching

MATCHING

MATCHING

calipers of width of 0.2 of the standard deviation

of the logit

of the propensity score and the use of calipers of

width 0.02 and 0.03 tended to have superior

performance for estimating treatment effects

Calibration

Whether the distances between the observed (treatment—yes or

no) and the predicted outcome from the model (propensity

score) are small and unsystematic. This is usually formally

appraised with the Hosmer–Lemeshow goodness of fit test.

PROPENSITY SCORES: THE NEW ONE

Discrimination

How well the predicted probabilities derived from the

model classify patients into their actual treatment group.

This is usually quantified with c-statistic, receiver

operator characteristic, and area under the curve.

PROPENSITY SCORES: THE NEW ONE

IS THIS THE SAME?

It is important to keep in mind that even propensity

score methods can only adjust for observed

confounding covariates and not for unobserved

ones.

IS EVERYTHING SO PERFECT?

ACCURATE ASSESSMENT OF AN OBSERVATIONAL STUDY

VARIABLES

Clearly define all outcomes, exposures, predictors,

potential confounders, and effect modifiers.

Give diagnostic criteria, if applicable

DATA SOURCES/ MEASUREMENT

For each variable of interest, give sources of data and

details of

methods of assessment (measurement).

Describe comparability of

assessment methods if there is more than one group.

STUDY SIZE

Explain how the study size was arrived at

HOW TO DO IT?

RESULTS

• Report numbers of individuals at each stage of

study—eg numbers potentially eligible, examined

for eligibility, confirmed eligible, included in the

study, completing follow-up, and analysed

• Give reasons for non-participation at each stage

• Consider use of a flow diagram

DISCUSSION• Summarise key results with reference to study objectives

• Discuss limitations of the study, taking into account sources of

potential bias or imprecision. Discuss both direction and

magnitude of any potential bias

• Give a cautious overall interpretation of results considering

objectives, limitations, multiplicity of analyses, results from

similar studies, and other relevant evidence

• Discuss the generalisability (external validity) of the study

results

FUNDING

Give the source of funding and the role of the

funders for the present study and, if

applicable, for the original study on which

the present article is based

TAKE HOME MESSAGES

- Check for biological and methodological

Pitfalls

- Remember that multivariate analysis is multivariate analysis

- Remember that multivariate analysis is “only” multivariate analysis

THANKS A LOT!!!!