I231B QUANTITATIVE METHODS
ANOVA continued and Intro to Regression
Agenda2
Exploration and Inference revisited
More ANOVA (anova_2factor.do)
Basics of Regression (regress.do)
3
It is "well known" to be "logically unsound and practically misleading" to make inference as if a model is known to be true when it has, in fact, been selected from the same data to be used for estimation purposes.
- Chris Chatfield in "Model Uncertainty, Data Mining and Statistical Inference", Journal of the Royal Statistical Society, Series A, 158 (1995), 419-486 (p 421)
Never mix exploratory analysis with inferential modeling of the same variables
in the same dataset.4
Exploratory model building is when you hand-pick some variables of interest and keep adding/removing them until you find something that ‘works’.
Inferential models are specified in advance: there is an assumed model and you are testing whether it actually works with the current data.
(ONE IV AND ONE DV)
5
Basic Linear Regression
Regression versus Correlation6
Correlation makes no assumption about one whether one variable is dependent on the other– only a measure of general association
Regression attempts to describe a dependent nature of one or more explanatory variables on a single dependent variable. Assumes one-way causal link between X and Y.
Thus, correlation is a measure of the strength of a relationship -1 to 1, while regression measures the exact nature of that relationship (e.g., the specific slope which is the change in Y given a change in X)
Basic Linear Model7
Yi = b0 + b1xi + ei.
X (and X-axis) is our independent variable(s)
Y (and Y-axis) is our dependent variable
b0 is a constant (y-intercept)
b1 is the slope (change in Y given a one-unit change in X)
e is the error term (residuals)
Basic Linear Function8
Slope9
But...what happens if B is negative?
Statistical Inference Using Least Squares
10
We obtain a sample statistic, b, which estimates the population parameter.
We also have the standard error for b
Uses standard t-distribution with n-2 degrees of freedom for hypothesis testing.
YYii = b = b0 0 + b+ b11xxii + e + eii..
Why Least Squares?11
For any Y and X, there is one and only one line of best fit. The least squares regression equation minimizes the possible error between our observed values of Y and our predicted values of Y (often called y-hat).
Data points and Regression12
http://www.math.csusb.edu/faculty/stanton/m262/regress/regress.html