Evaluation of evidence [compatibility mode]

EVALUATION OF EVIDENCE

Shikur MohammedShikur Mohammed

(BSc, MPH)

2/2/2015 1By Shikur Mohammed (BSc, MPH)

Evaluation of Evidence

� Evaluation is a process of determining the

usefulness, reliability and validity of something

against explicit predetermined standards


Common Problems in observed findings

� Inadequacy of the observed sample (The role ofchance)

� Inappropriate selection of study subjects & unfair

data collection methods (Bias)data collection methods (Bias)

� Comparing unequal (Confounding)


1. The role of chance

� The larger the sample on which the estimate is

based, the less variability and the more reliable the

inference

� This is done by performing an appropriate test of

statistical significancestatistical significance

� A measure that is often reported from all tests of

statistical significance is the P-value


The role of chance cont…

� P < 0.05 - statistically significant

� P > 0.05 – no statistically significant association (

chance can not be excluded as a likely explanation)chance can not be excluded as a likely explanation)


The role of chance cont…

� It is always advisable to report the actual P-value

rather than merely that the results did or did not

achieve statistical significanceachieve statistical significance

� confidence interval (CI) is far more informative

measure than P-value to evaluate the role of chance


Bias

� It is any systematic error in the design, conduct or

analysis of a study

� Types of biasTypes of bias

1. Selection bias,

2. Information (Observation) bias


1. Selection bias

� It is a bias introduced while selecting the study

participants

� It is a particular problem in case control and� It is a particular problem in case control and

retrospective cohort studies. E.g. If the way in

which participants are selected into the study is

different for cases and controls


Examples of selection bias

a) Diagnostic bias: Diagnostic approach related to

knowing exposure status

For example

� women who take oral contraceptives (OCs) may be

screened more often for breast cancer than women who

do not take OCs because of the suspected link between

oral contraceptive and breast cancer


selection bias…

b). Volunteer bias/ Compliance bias: People who

accept to participate in a study or people who refuse

to participate are often quite different from the

general population

c). Non-response bias: This is due to differences in thec). Non-response bias: This is due to differences in the

characteristics between the responders and non-

responders to the study


selection bias cont…

d). Loss to follow up: Difference in completeness of

follow-up between comparison groups

e). Berkson’s bias: Studies carried out exclusively in

hospital settings are subject to selection biashospital settings are subject to selection bias

attributable to the fact that risks of hospitalization

can combine in patients who have more than one

condition


Ways of minimizing selection bias

• Population-based studies are preferable

• Avoid the inclusions as study subjects of people

who have volunteered on their own

In case-control study, it is useful to select several• In case-control study, it is useful to select several

different control groups

• keep losses to follow-up to minimum


2. Information/Observation bias

� Refers to bias which arises during the data collection

process

� It occur because of mistakes in categorizing study� It occur because of mistakes in categorizing study

subjects with respect to their exposure or disease

status


Examples of information bias

a) Investigator bias/ Interviewer bias/ Observer bias:

Occurs when investigators collect information

differently in different comparison groups

b) Response bias/Recall bias: Occurs as a result of

difficulty to recall prior exposuresdifficulty to recall prior exposures

c) Social desirability bias: Occurs because subjects are

systematically more likely to provide a socially

acceptable response


Examples information bias….

d). Placebo effect: tendency for individuals to report favorable

response to any therapy regardless of the physiologic efficacy

of what they received

e). Hawthorn effect: Refers to the change in the dependent

variable which may be due to the process of measurement or

observation itself


Ways of minimizing information bias

� Blinding: the study subjects doesn't know to which

group they are assigned

� using placebo: Use of placebo minimizes the bias in the

ascertainment of both subjective disease outcomes and

side effects. It facilitates that both groups in the study

gain equal attention

� Using standard procedures, instruments, questionnaires,

interviewing techniques


Confounding

� Confounding variable is a variable that can cause or

prevent the outcome of interest, is not an

intermediate variable, and is associated with the

factor under investigation


Confounding …

� Confounding variable must fulfill each of the following criteria:1. the variable must be associated with the exposure and,

independent of that exposure, be a risk factor for thediseasedisease

2. The distribution (frequency) of the confounding variableshould vary between the groups that are compared

3. Confounder must not be an intermediate link in a causalpathway between exposure and outcome


Confounding …….Example

� An observed association between drinking alcohol

and increased risk of lung cancer (LC) could be due

to the effect of cigarette smoking, since alcohol

drinking is associated with smoking and,drinking is associated with smoking and,

independent of alcohol consumption, smoking is a

risk factor for LC


Control for Confounding VariablesControl for Confounding VariablesControl for Confounding VariablesControl for Confounding Variables

� In the design:• Randomization• Restriction• Matching

� During analysis:� During analysis:• Standardization• Stratification• Multivariate analysis


Establishing causal relationships

� The following formal criteria are widely used to evaluate the

likelihood that an association is causal

1. Strength of the association

2. Dose-response relationship

3. Consistency of the relationship3. Consistency of the relationship

4. Temporal relationship

5. Specificity of the association

6. Biological plausibility (coherence with existing

information)

7. Prevention


Conclusion about causation

� The above criteria are the ones most frequently employed in

trying to establish causation

� None provides in itself a perfect means of providing

causation, and each has its limitationscausation, and each has its limitations

� However, when they are considered together, the weight of

the evidence may allow a tentative conclusion to be reached


Evaluation of evidence [compatibility mode]

Health & Medicine

shikur mohammed bsc

volunteer bias compliance

berksons bias

nonresponse bias

selection biasattributable

mphthe role of chance

significant p

role of chance222015