CS 559: Machine Learning Fundamentals and Applications 8 th Set of Notes Instructor: Philippos Mordohai Webpage: www.cs.stevens.edu/~mordohai E-mail: [email protected] Office: Lieb 215 1
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
CS 559: Machine Learning Fundamentals and Applications
8th Set of Notes
Instructor: Philippos MordohaiWebpage: www.cs.stevens.edu/~mordohaiE-mail: [email protected]
Office: Lieb 215
1
Project Proposal• Dataset
– How many instances• Classes (or what is being predicted)• Inputs
– Include feature extraction, if needed– If your inputs are images or financial data, this must
be addressed • Methods
– At least one simple classifier (MLE with Gaussian model, Naïve Bayes, kNN)
– At least one advanced classifier (SVM, Boosting, Random Forest, CNN)
2
Project Proposal
• Typical experiments– Measure benefits due to advanced classifier
compared to simple classifier– Compare different classifier settings
• k in kNN• Different SVM kernels• AdaBoost vs. cascade• Different CNN architectures
– Measure effects of amount of training data available
– Evaluate accuracy as a function of the degree of dimensionality reduction using PCA
3
Project Proposal
• Email me a pdf with all these• I must say “approved” in my response,
otherwise address my comments and resubmit
4
Overview
• Linear Regression– Barber Ch. 17– HTF Ch. 3
5
Simple Linear Regression
• How does a single variable of interest relate to another (single) variable?– Y = outcome variable (response,
dependent...)– X = explanatory variable (predictor, feature,
independent...)
• Data: n pairs of continuous observations (X1,Y1) … (Xn,Yn)
6
Example• How does systolic blood pressure (SBP) relate to age?
• Graph suggests that Y relates to X in an approximately linear way
7
Regression: Step by Step
1. Assume a linear model: Y = β0 + β1 X2. Find the line which “best” fits the data, i.e.
estimate parameters β0 and β1
3. Does variation in X help describe variation in Y ?
4. Check assumptions of model5. Draw inferences and make predictions
8
Straight-line Plots
9
Assumptions of Linear Regression
• Five basic assumptions1. Existence: for each fixed value of X, Y is
a random variable with finite mean and variance
2. Independence: the set of Yi are independent random variables given Xi
10
Assumptions of Linear Regression
3. Linearity: the mean value of Y is a linear function of X
|
11
Assumptions of Linear Regression
4. Homoscedasticity: the variance of Y is the same for any X
5. Normality: For each fixed value of X, Y has a normal distribution (by assumption 4, σ2 does not depend on X)
12
Formulation
• Yi are linear function of Xi plus some random error
13
Linear Regression
14
Estimating β0 and β1
• Find “best” line• Criterion for “best”: estimate β0 and β1 to
minimize:
• This is the residual sum of squares, sum of squares due to error, or sum of squares about regression line
• Least Squares estimator15
Rationale for LS Estimates
• 2 measures the “deviance” of Yi from the estimated model
• The “best” model is the one from which the data deviate the least
16
Least Squares Estimators• Taking derivatives with respect to β, we obtain
• The residual variance is
17
Example: SBP/age data
18
Using the Model
• Using the parameter estimates, our best guess for any Y given X is
• Hence at
• Every regression line goes through ( , )• Also
19
Correlation and Regression Coefficient
20
Example
21
Example
22
Example
23
Example
24
Related Documents
