Thomas Songer, PhD Introduction to Research Methods In the Internet Era Bias Assessing Validity of Association.

Thomas Songer, PhD

Introduction to Research MethodsIn the Internet Era

Bias

Assessing Validity of Association

Learning Objectives:

1. Identify the possible alternative explanations for statistical associations:

--- Chance--- Bias--- Confounding

2. Distinguish between the major types of bias in epidemiologic studies.

2

Research Process

Research question

Hypothesis

Identify research design

Data collection

Presentation of data

Data analysis

Interpretation of data

Polgar, Thomas 3

Epidemiologic Reasoning

Assess validity of association• true relationship between the exposure and disease

- Does the observed association really exist?- Is the association valid?

- Are there alternative explanations for the association?- Chance (Random Error)- Bias (Systematic Error)- Confounding

4

A “valid” statistical association implies “Internal Validity” in the study

Internal Validity: The results of an observationare correct for the particular group being studied

What about “external validity”?

Do the results of the study apply (“generalize”) toto people who were not in the study (e.g. the target population)?

5

Evaluating Associations

• Internal Validity – Strength of the measurement tools,

assessment methods of exposure and outcome variables in the study, and control for study effects

• External Validity -- strength of the study sample with regards to generalizability

6

Evaluating Associations

• Threats to validity in research studies• Random error

• Sample size

• Systematic error– Selection bias– Measurement bias– Loss to follow-up

• Hawthorne Effect

• Confounding

• Regression to the mean

7

Note: DO NOT compromise internal validity in the goal of generalization

* An invalid result cannot be generalized

* Thus, internal validity should never becompromised in an attempt to achievegeneralizability

8

Evaluating Associations

Note: Keep in mind that even if chance, bias, and confounding have been sufficiently ruled out (or taken into account), it does not necessarily mean that the valid association observed is causal.

The observed association may simply be a coincidence.coincidence.

(i.e. In the last 10, years, incidence rates for prostate cancer have increased, as have sales of plasma TV screens).

9

Evaluating Associations

How do we know that the associations observed in

epidemiologic studies are real?

10

Evaluating Associations

Evaluating the validity of an association:

In any epidemiologic study, there are at least 3alternative explanations for the observed results:

1. CHANCE (random error)2. BIAS (systematic error)3. CONFOUNDING

These explanations are not mutually exclusive --more than one can be present in the same study

11

Bias or Systematic Error• Systematic, non-random, deviation of

results from the truth

high systematic error low systematic error12

Bias

• Potential biases must be considered and addressed in all epidemiologic studies

• We often assume that exposed and unexposed groups are comparable

• This is not necessarily true

13

Systematic Error (Bias)

BIAS: Systematic error in the design, conduct, or analysis of a study that results in a mistaken estimate of an exposure/disease relationship

1. SELECTION BIAS

2. INFORMATION BIAS

* Recall Bias* Interviewer Bias* Reporting Bias* Surveillance Bias

14

Selection Bias

• A distortion in a measure of disease frequency or association resulting from the manner in which subjects are selected for the study

• Result of deficiencies in study design

• E. g. - Case-control study - exposure status may influence selection of subjects to a different extent in cases and controls

- self-selection bias

15

Bias

SELECTION BIAS: Any systematic error that arises in the process of identifying the two 2 study groups to be compared)

• Results in the study groups being non-comparable, unless some type of statistical adjustment can be made

16

Selection Bias

EXAMPLE: Case Control Study

Outcome: Hemorrhagic strokeExposure: Appetite suppressant products that

contain Phenylpropanolamine (PPA)

Cases: Persons who experienced a strokeControls: Persons in the community without stroke

Bias: Control subjects were recruited by random-digit dialing from 9:00

AM to 5:00 PM. This resulted in over- representation of unemployed persons who may not represent the study base in terms of use of appetite suppressant products.17

EXAMPLE: Non-Response

• If refusal or non-response is related to exposure, the estimate of effect may be biased. For example, if controls are selected by use of a household survey, non-response may be related to demographic and lifestyle factors associated with employment.

• Responders often differ systematically from persons who do not respond.

18

Selection Bias

Berkson’s Bias

• A form of selection bias that affects hospital-based epidemiology studies.

• People in hospital are likely to suffer from multiple diseases and engage in unhealthy behaviours (e.g. smoking)

• As a result, they are atypical of the population in the community

19

Healthy Worker Effect

• A form of selection bias that affects epidemiology studies of workers.

• Ill and disabled people are likely to be unemployed. The employed (workers) are healthier than other segments of the population.

• As a result, they are atypical of the population in the community

20

Information Bias

Definition: Systematic differences in the way in which data on exposure and outcome are obtained from the various study groups.

Some Types/Sources of Information Bias:

• Bias in abstracting records• Bias in interviewing• Bias from surrogate interviews• Surveillance bias• Reporting and recall bias

21

Information Bias

• Results from systematic differences in the way data on exposure or outcome are obtained

• May result from measurement defects or questionnaires or interviews that do not measure what they claim to

• Examples of information bias– Recall bias : self-reported information may be

inaccurate due to low levels of recall

22

Recall Bias

DEFINITION: Study group participants systematically differ in the way data on exposure or outcome are recalled

• Particularly problematic in case-control studies

• Individuals who have experienced a disease or adverse health outcome may tend to think about possible “causes” of the outcome. This can lead to differential recall

23

Recall Bias - Example

Outcome: Cleft palateExposure: Systemic infection during

pregnancyCases: Mothers giving birth to children with

cleft palateControls: Mothers giving birth to children

free of cleft palate

Bias: Mothers who have given birth to a child with cleft palate may recall more thoroughly colds and other infections experienced during pregnancy

24

Interviewer Bias

DEFINITION: Systematic difference in the soliciting, recording, or interpretation of information from study participants

• Can affect every type of epidemiologic study

• May occur when interviewers are not “blinded” to exposure or outcome status of participants.

25

• Interviewer’s knowledge of subjects’ disease status may result in differential probing of exposure history

• Similarly, interviewer’s knowledge of subjects’ exposure history may result in differential probing and recording of the outcome under examination

• Placebo control is one method used to maintain observer blindness in randomized trials. 26

Interviewer Bias

Reporting Bias

DEFINITION: Selective suppression or revealing of information such as past history of sexually transmitted disease.

• Often occurs because subject reluctance to report an exposure due to attitudes, beliefs, and perceptions

• “Wish bias” may occur among subjects who have developed a disease and seek to show that the disease “is not their fault.”

27

Surveillance Bias

• If a population is monitored over a period of time, disease ascertainment may be better in the monitored population than in the general population (“surveillance bias”).

• May lead to biased estimate of exposure/disease relationship.

28

Misclassification BiasDEFINITION: Erroneous classification of the

exposure or disease status of an individual into a category to which it should not be assigned

Misclassification of the exposure or outcomeExample:--- Cases incorrectly classified as controls--- Controls incorrectly classified as cases--- Exposed incorrectly classified as non-

exposed--- Non-exposed incorrectly classified as

exposed29

Control of Bias

Can only be prevented and controlled

during the design and conduct of a study

• Choice of a study population

• Methods of data collection

• Sources of case ascertainment and risk factor information

Sever 30

Good Study Design Protects Against All Forms of Error

31