Economics 105: Statistics • Any questions? • No GH due Monday.
Feb 25, 2016
Economics 105: Statistics• Any questions?• No GH due Monday.
Multiple-Group Threats to Internal Validity
• When you move from single to multiple group research the big concern is whether the groups are comparable.
• Usually this has to do with how you assign units (e.g., persons) to the groups (or select them into groups).
• We call this issue selection or selection bias.
The Central Issue
Administerprogram
Measureoutcomes
Measurebaseline
Alternativeexplanations
Alternativeexplanations
X OOOO
Do not administerprogram
Measureoutcomes
Measurebaseline
The Multiple Group Case
• Diabetes education for adolescents
• Pre-post comparison group design
• Measures (O) are standardized tests of diabetes knowledge
Example
• Any other event that occurs between pretest and posttest that the groups experience differently.
• For example, kids in one group pick up more diabetes concepts because they watch a special show on Oprah related to diabetes.
X OOOO
Selection-History Threat
• Differential rates of normal growth between pretest and posttest for the groups.
• They are learning at different rates, even without program.
X OOOO
Selection-Maturation Threat
• Differential effect on the posttest of taking the pretest.
• The test may have “primed” the kids differently in each group or they may have learned differentially from the test, not the program.
X OOOO
Selection-Testing Threat
• Any differential change in the test used for each group from pretest and posttest
• For example, change due to different forms of test being given differentially to each group, not due to program
X OOOO
Selection-Instrumentation Threat
• Differential nonrandom dropout between pretest and posttest.
• For example, kids drop out of the study at different rates for each group.
• Differential attrition
X OOOO
Selection-Mortality Threat
• Different rates of regression to the mean because groups differ in extremity.
• For example, program kids are disproportionately lower scorers and consequently have greater regression to the mean.
X OOOO
Selection-Regression Threat
“Social Interaction” Threats to Internal Validity
• All are related to social pressures in the research context, which can lead to posttest differences that are not directly caused by the treatment itself.
• Most of these can be minimized by isolating the two groups from each other, but this leads to other problems (for example, hard to randomly assign and then isolate, or may reduce generalizability).
What Are “Social” Threats?
• Controls might learn about the treatment from treated people (for example, kids in the diabetes educational group and control group share the same hospital cafeteria and talk with one another).
Diffusion or Imitation of Treatment
• Administrators give a compensating treatment to controls.
• Researchers feel badly and give control group kids a video to watch pertaining to diabetes. Contaminates the study!
=
Compensatory Equalization of Treatment
• Controls compete to keep up with treatment group.
Compensatory Rivalry
• Controls "give up" or get discouraged
• Likely to exaggerate the posttest differences, making your program look more effective than it really is
Resentful Demoralization
What is a Clinical Trial?• “A prospective study comparing the effect and
value of intervention(s) against a control in human beings.”
• Prospective means “over time”; vs. retrospective• It is attempting to change the natural course of a
disease• It is NOT a study of people who are on drug X
versus people who are not
• http://www.clinicaltrials.gov/info/resources
Model of Two-Group Randomized Clinical Trial
What are the characteristics of a Clinical Trial?• Begins with a primary research question, and the trial
design flows from this question (constrained by practicalities)
• Everything must be exhaustively defined in advance (to prevent accusations of fishing for a positive finding)
• The hypothesis (“-es”)• Population to be studied• inclusion criteria• exclusion criteria• contraindications to therapy• indications to therapy• Treatment strategy (treatment, exact dosage, dosage
schedule, etc)• The outcome(s)
Beta-Blocker Heart Attack Trial (BHAT)• Published in Journal of the American Medical AssociationJAMA 1982; 247: 1701 - 1714JAMA 1983; 250: 2814 – 2819• Up until about 25 years ago, the treatment of myocardial
infarction consisted of bed rest, alleviation of symptomatic pain, possible administration of early antiarrhythmics
• But a third of people who have a heart attack die from it ‘suddenly’
• In 1976, NIH sponsored a conference to discuss potential agents to be used in either a primary or secondary prevention setting to reduce sudden death, for which there was no treatment.
• The conference made an official recommendation to do a clinical trial.
Example: Job Corps• What is Job Corps? http://jobcorps.doleta.gov/
• January 5, 2006 Thursday Late Edition – Final
SECTION: Section C; Column 1; Business/Financial Desk; ECONOMIC SCENE; Pg. 3
HEADLINE: New (and Sometimes Conflicting) Data on the Value to Society of the Job Corps
BYLINE: By Alan B. Krueger.
Alan B. Krueger is the Bendheim professor of economics and public affairs at Princeton University. His Web site is www.krueger.princeton.edu.
He delivered the 2005 Cornelson Lecture in the Department of Economics here at Davidson (that’s the big econ lecture each year).
Example: Job Corps• Quotations from “New (and Sometimes Conflicting) Data on the Value
to Society of the Job Corps” by Alan B. Krueger.
• Since 1993, Mathematica Policy Research Inc. has evaluated the performance of the Job Corps for the Department of Labor.
• Its evaluation is based on one of the most rigorous research designs ever used for a government program. From late 1994 to December 1995, some 9,409 applicants to the Job Corps were randomly selected to be admitted to the program and another 6,000 were randomly selected for a control group that was excluded from the Job Corps.
• Those admitted to the program had a lower crime rate, higher literacy scores and higher earnings than the control group.
RCT for Credit Card Offers
Source: Agarwal, et al. (2010), Journal of Money, Credit & Banking, 42 (4)
RCT for Education in India
Source: Banerjee, et al. (2007), Quarterly Journal of Economics
RCT for Education in India
RCT for the Effect of High Rewards on Performance
Source: Ariely, Gneezy, Loewenstein, and Mazar (2009), Review of Economic Studies
RCT for the Effect of High Rewards on Performance
Recommended Reading
Amazon link Amazon link
Amazon link
Correlation vs. Regression• A scatter plot can be used to show the
relationship between two variables• Correlation analysis is used to measure
strength of the association (linear relationship) between two variables– Correlation is only concerned with strength of the
relationship – No causal effect is implied with correlation
Introduction to Regression Analysis• Regression analysis is used to:
– Predict the value of a dependent variable based on the value of at least one independent variable
– Explain the impact of changes in an independent variable on the dependent variable
• Dependent variable: the variable we wish to predict or explain ... outcome variable, Y.
• Independent variables: the variables used to explain variation in Y ... covariates, explanatory variables, r.h.s. vars, X-variables
Simple Linear Regression Model
• Only one independent variable, X• Relationship between X and Y is
described by a linear function• Changes in Y are assumed to be
caused by changes in X
Types of Relationships
Y
X
Y
X
Y
Y
X
X
Linear relationships Curvilinear relationships
Types of Relationships
Y
X
Y
X
Y
Y
X
X
Strong relationships Weak relationships
(continued)
Types of Relationships
Y
X
Y
X
No relationship
(continued)
Theoretical Linear Models• The basis of “causality” in models
– Time ordering– Co-variation– Non-spuriousness
• Examples– Fire Deaths = f (# of fire trucks at the scene)– Job Retention = f (current job satisfaction)– Income = f (education)
Deterministic Linear Models•Theoretical Model:– b0 and b1 are constant terms
• b0 is the intercept
• b1 is the slope
– Xi is a predictor of Yia
bb0
Xi
Yi
Linear component
Stochastic Simple Linear Population Regression Model
Population Y intercept
Population SlopeCoefficient
Population Random Error term
Outcome Variable
Explanatory Variable
Random Error component
(continued)
Pop Random Error for this Xi value
Y
X
Observed Value of Y for Xi
Xi
Pop Slope = β1
Pop Intercept = β0
εi
Stochastic Simple Linear Population Regression Model
The Multiple Regression Model
Idea: Examine the linear relationship between 1 dependent (Y) & 2 or more independent variables (Xi)
Multiple Regression Model with k Independent Variables:
Y-intercept Population slopes Random Error
• Endogenous explanatory variables
Modeling Exercise examples• What is the effect of your roommate’s SAT
scores on your grades? The effect of studying?
• Do police reduce crime?
• Does more education increase wages?
• What is the effect of school start time on academic achievement?
• Does movie violence increase violent crime?
Endogenous Explanatory Variable• Causes of endogenous explanatory variables
include …• Wrong functional form• Omitted variable bias … occurs if both the
1. Omitted variable theoretically determines Y2. Omitted variable is correlated with an included X
• Errors-in-variables (aka, measurement error)• Sample selection bias• Simultaneity bias (Y also determines X)