Page 1
Introduction Objective / Motivation Schedule / Structure Homework / Exercises Material ... Background examples Questions
CS7616 - Pattern Recognition - Introduction
Henrik I Christensen
Robotics & Intelligent Machines @ GTGeorgia Institute of Technology,
Atlanta, GA [email protected]
Henrik I Christensen (RIM@GT) CS7616 - PR 1 / 35
Page 2
Introduction Objective / Motivation Schedule / Structure Homework / Exercises Material ... Background examples Questions
Outline
1 Introduction
2 Objective / Motivation
3 Schedule / Structure
4 Homework / Exercises
5 Material ...
6 Background examples
7 Questions
Henrik I Christensen (RIM@GT) CS7616 - PR 2 / 35
Page 3
Introduction Objective / Motivation Schedule / Structure Homework / Exercises Material ... Background examples Questions
Introduction
Welcome to CS7616
Pattern Recognition
Today:
Outline of the course - Objective / MotivationSchedule of lecturesStyle of the courseExercises / ProjectsMaterial to be used in the course
Henrik I Christensen (RIM@GT) CS7616 - PR 3 / 35
Page 4
Introduction Objective / Motivation Schedule / Structure Homework / Exercises Material ... Background examples Questions
Information
Class website:http://www.cc.gatech.edu/∼hic/CS7616Schedule, Material, Slide copies, General Information
T-Square - Usual stuff, announcements, ...
Slides - PDF copy will be posted after class with summary
Piazza - You will receive an invitation for the class forumUse it for general questions / discussions
Henrik I Christensen (RIM@GT) CS7616 - PR 4 / 35
Page 5
Introduction Objective / Motivation Schedule / Structure Homework / Exercises Material ... Background examples Questions
Staffing
Henrik I ChristensenLecturer
Sidd Choudhary, TA Steven Hickson, TA Ruffin White, TA
Henrik I Christensen (RIM@GT) CS7616 - PR 5 / 35
Page 6
Introduction Objective / Motivation Schedule / Structure Homework / Exercises Material ... Background examples Questions
Outline
1 Introduction
2 Objective / Motivation
3 Schedule / Structure
4 Homework / Exercises
5 Material ...
6 Background examples
7 Questions
Henrik I Christensen (RIM@GT) CS7616 - PR 6 / 35
Page 7
Introduction Objective / Motivation Schedule / Structure Homework / Exercises Material ... Background examples Questions
Objective
Get a solid knowledge of key methods in pattern recognition
Discuss state of the art methods / techniques in pattern recognition
Explore a few representative data sets that illustrate use of patternrecognition
Explore increasingly complex methods over the semester
This is not a general machine learning course
Henrik I Christensen (RIM@GT) CS7616 - PR 7 / 35
Page 8
Introduction Objective / Motivation Schedule / Structure Homework / Exercises Material ... Background examples Questions
Motivation
PR is used everywhere in daily lives
Henrik I Christensen (RIM@GT) CS7616 - PR 8 / 35
Page 9
Introduction Objective / Motivation Schedule / Structure Homework / Exercises Material ... Background examples Questions
Speech Recognition
Henrik I Christensen (RIM@GT) CS7616 - PR 9 / 35
Page 10
Introduction Objective / Motivation Schedule / Structure Homework / Exercises Material ... Background examples Questions
Financial trading
Henrik I Christensen (RIM@GT) CS7616 - PR 10 / 35
Page 11
Introduction Objective / Motivation Schedule / Structure Homework / Exercises Material ... Background examples Questions
Medical Diagnostics
Henrik I Christensen (RIM@GT) CS7616 - PR 11 / 35
Page 12
Introduction Objective / Motivation Schedule / Structure Homework / Exercises Material ... Background examples Questions
Face Recognition
Henrik I Christensen (RIM@GT) CS7616 - PR 12 / 35
Page 13
Introduction Objective / Motivation Schedule / Structure Homework / Exercises Material ... Background examples Questions
Scene Labeling
Henrik I Christensen (RIM@GT) CS7616 - PR 13 / 35
Page 14
Introduction Objective / Motivation Schedule / Structure Homework / Exercises Material ... Background examples Questions
Netflix Movie Recommendation
Henrik I Christensen (RIM@GT) CS7616 - PR 14 / 35
Page 15
Introduction Objective / Motivation Schedule / Structure Homework / Exercises Material ... Background examples Questions
Generic Problem Structure
Feature Extraction Classification Decision
Henrik I Christensen (RIM@GT) CS7616 - PR 15 / 35
Page 16
Introduction Objective / Motivation Schedule / Structure Homework / Exercises Material ... Background examples Questions
Outline
1 Introduction
2 Objective / Motivation
3 Schedule / Structure
4 Homework / Exercises
5 Material ...
6 Background examples
7 Questions
Henrik I Christensen (RIM@GT) CS7616 - PR 16 / 35
Page 17
Introduction Objective / Motivation Schedule / Structure Homework / Exercises Material ... Background examples Questions
Structure
A mixture of foundational lectures and
Group discussions of influential/current papers
We will divide class into 4 groups for smaller discussionsEvery student is expected to present 1 paper during term as part of thegroup discussions
Large group lectures are a challenge for in-depth discussions
Henrik I Christensen (RIM@GT) CS7616 - PR 17 / 35
Page 18
Introduction Objective / Motivation Schedule / Structure Homework / Exercises Material ... Background examples Questions
Lecture topics
1 Bayes Decision Theory
2 Linear Methods for Classification
3 Sub-space Methods
4 Ensemble Methods
5 Hidden Markov Models
6 Prototype/memory based methods
7 Kernels and other tricks
8 Tree based techniques
9 Large Margin Classifiers
10 Deep Learning
Henrik I Christensen (RIM@GT) CS7616 - PR 18 / 35
Page 19
Introduction Objective / Motivation Schedule / Structure Homework / Exercises Material ... Background examples Questions
Discussion Sessions
Discuss two papers per class:
Each paper:
Student presentation of paper ≈ 15 minutes introGroup: What are the main lessons/key insight from the paperGroup: How could it be improved / what would you do differently?TA/Lecturer: guide discussion / presentation
Henrik I Christensen (RIM@GT) CS7616 - PR 19 / 35
Page 20
Introduction Objective / Motivation Schedule / Structure Homework / Exercises Material ... Background examples Questions
Outline
1 Introduction
2 Objective / Motivation
3 Schedule / Structure
4 Homework / Exercises
5 Material ...
6 Background examples
7 Questions
Henrik I Christensen (RIM@GT) CS7616 - PR 20 / 35
Page 21
Introduction Objective / Motivation Schedule / Structure Homework / Exercises Material ... Background examples Questions
Homework
Leverage of a set of datasets - varying in complexity, ...
A homework assignment roughly every month
4 assignments in total
First three will use the common datasets (Gaussian / Ensemble /Temporal)
Final homework - option to use your own dataset - large margin /deep learning
Henrik I Christensen (RIM@GT) CS7616 - PR 21 / 35
Page 22
Introduction Objective / Motivation Schedule / Structure Homework / Exercises Material ... Background examples Questions
Credit / Grading
45% Homeworks 1-3
25% Homework 4
25% Class Presentation / Discussions
5% Class participation
There will no final exam!
Henrik I Christensen (RIM@GT) CS7616 - PR 22 / 35
Page 23
Introduction Objective / Motivation Schedule / Structure Homework / Exercises Material ... Background examples Questions
Submission of material
Please submit home work on time.
Late submissions will be 75% for 1 day late, 50% for 2 days late andthen 25% after that
You can ask for permission with a good motivation, but have to do itwell ahead of time (not an hour before!)
Do not expect that we are online the last hour before a deadline.Unfair to the TAs and others.
Henrik I Christensen (RIM@GT) CS7616 - PR 23 / 35
Page 24
Introduction Objective / Motivation Schedule / Structure Homework / Exercises Material ... Background examples Questions
Outline
1 Introduction
2 Objective / Motivation
3 Schedule / Structure
4 Homework / Exercises
5 Material ...
6 Background examples
7 Questions
Henrik I Christensen (RIM@GT) CS7616 - PR 24 / 35
Page 25
Introduction Objective / Motivation Schedule / Structure Homework / Exercises Material ... Background examples Questions
Book 1: Elements of Statistical Learning
Main textbookElements of Statistical LearningT. Hastie, R. Tibshirani & J.FreiedmanSpringer Verlag, 2nd Edition, 2009http://www-stat.stanford.edu/
~tibs/ElemStatLearn
Henrik I Christensen (RIM@GT) CS7616 - PR 25 / 35
Page 26
Introduction Objective / Motivation Schedule / Structure Homework / Exercises Material ... Background examples Questions
Book 2: Machine Learning - Kevin Murphy
Machine LearningK. MurphyMIT-Press, 2013
Henrik I Christensen (RIM@GT) CS7616 - PR 26 / 35
Page 27
Introduction Objective / Motivation Schedule / Structure Homework / Exercises Material ... Background examples Questions
Book 3: Duda, Hart and Stork
Pattern ClassificationR. O. Duda, P. E. Hart and D. G.StorkWiley Interscience, 2nd, 2001, ISBN0-471-05669-3
Henrik I Christensen (RIM@GT) CS7616 - PR 27 / 35
Page 28
Introduction Objective / Motivation Schedule / Structure Homework / Exercises Material ... Background examples Questions
Software
You can use Matlab or Python - we will try to support both
Some demonstrations using Matlab / K. Murphy Toolkit
https://github.com/probml/pmtk3
Some examples using Scikit-Learn Toolkit
http://scikit-learn.org
Still try to finalize 2-3 datasets for homework
Henrik I Christensen (RIM@GT) CS7616 - PR 28 / 35
Page 29
Introduction Objective / Motivation Schedule / Structure Homework / Exercises Material ... Background examples Questions
Outline
1 Introduction
2 Objective / Motivation
3 Schedule / Structure
4 Homework / Exercises
5 Material ...
6 Background examples
7 Questions
Henrik I Christensen (RIM@GT) CS7616 - PR 29 / 35
Page 30
Introduction Objective / Motivation Schedule / Structure Homework / Exercises Material ... Background examples Questions
Model based recognition
Google has 2.5 million objects inthe 3D object warehouse
Can we use these for recognitionof objects?
Can we provide context forobject recognition?
Henrik I Christensen (RIM@GT) CS7616 - PR 30 / 35
Page 31
Introduction Objective / Motivation Schedule / Structure Homework / Exercises Material ... Background examples Questions
Gesture based recognition
Tracking of hands for person forrobot interaction
Color classification of hands andhead of user
Tracking of objects usingKalman filter
HMM based recognition ofgestures
Henrik I Christensen (RIM@GT) CS7616 - PR 31 / 35
Page 32
Introduction Objective / Motivation Schedule / Structure Homework / Exercises Material ... Background examples Questions
Recognition of daily activities
Images of standard objects torecognize daily activities
Example application forassistance to people withmemory challenges
Using Deep Learning forRecognition of situations
Henrik I Christensen (RIM@GT) CS7616 - PR 32 / 35
Page 33
Introduction Objective / Motivation Schedule / Structure Homework / Exercises Material ... Background examples Questions
Outline
1 Introduction
2 Objective / Motivation
3 Schedule / Structure
4 Homework / Exercises
5 Material ...
6 Background examples
7 Questions
Henrik I Christensen (RIM@GT) CS7616 - PR 33 / 35
Page 34
Introduction Objective / Motivation Schedule / Structure Homework / Exercises Material ... Background examples Questions
Next Lecture
Thursday - Bayes Decision Theory
DHS: Chapter 2 (2.1-2.6)
We will provide the initial list of papers for class discussions
Discuss the datasets for home work
Henrik I Christensen (RIM@GT) CS7616 - PR 34 / 35
Page 35
Introduction Objective / Motivation Schedule / Structure Homework / Exercises Material ... Background examples Questions
Questions?
Questions?
Henrik I Christensen (RIM@GT) CS7616 - PR 35 / 35