Dec 28, 2015
The Initials
Time: Wednesday: 16 hrs to 18 hrs Friday : 16 hrs to 18 hrs
Course website:
http://delta.cs.cinvestav.mx/~debrup/Machine_Learning.html
Books:
1) Pattern Classification: Duda, Hart and Stork
2) Machine Learning: Mitchell
3) Neural Networks : Haykin
The Initials (contd.)
Grading policies:
4 homeworks (20%)
2 exams (30%)
1 project/term paper ( 50%)
The lectures would be in English, I am really sorry about it.
We are good at recognizing patterns
We can recognize faces with ease
We can understand spoken words
We can read handwritings
……… many more
It would be nice to make machines perform these tasks.
Informally pattern recognition deals with techniques and methods to make machines recognize patterns.
Satellite image of Kolkata, in 4 channels: Red, Green, Blue and Infrared
Which parts are land and which are water?
Where is the airport?
Protein Fold Prediction1) Proteins are sequences of amino acid molecules.
There are 20 distinct type of amino acids and their sequences can form many-many protein molecules.
2) A amino acid chain must fold in a certain manner which helps it in its specified activity.
3) The activity and function of a protein depends on the way it folds. Thus the fold information is necessary for determining the function of a protein.
Problem: Given an amino acid sequence, tell me the fold it will undergo.
Text Categorization
Given a large corpus of text documents (Say news items).
Tell me the categories of the documents.
Similar problems:
Classify music in a music corpus
Classify images in a image corpus
Retrive documents/music/images from a database which are similar to x.
Other Problems
•Predict whether a patient hospitalized due to heart attack will have a second heart attack. The prediction is to be based on demographic, diet and clinical measurements of the patient.
•Predict the price of a stock in 6 months from now on the basis of company performance measures and econimic data.
•Identify the numbers in a handwritten zip code (CP) from digitized images.
•Estimate the amount of glucose in the blood of a diabetic patient from the infra red absorption spectrum of the blood.
•Estimate the risk factors for prostrate cancer based on clinical and demographic variables.
Pattern Recognition
(What the experts say?)
Duda Hart, 1973 : “A field concerned with machine recognition of meaningful regularities”.
Bezdek, 1981: “Pattern recognition is the search for structure in data”
Object Data – Numeric Features
The characteristics of an object is encoded in a vector called the Feature Vector
Each component of the vector represents some attribute of the object. These components are called features.
Example:
Iris data : A data in R4. Representing iris flowers. The features are the sepal length, sepal width, petal length and petal width of 150 iris flowers of 3 different types.
Multichannel satellite Image: Images captured by different sensors which captures the frequency information of the electromagnetic radiation from earths surface.
Relational Data
A relational matrix whose values are assigned by humans or computed from features:
R r rij n n ij
Russian German Chinese Japanese
1.0 0.6 0.2 0.25
0.6 1.0 0.1 0.2
0.2 0.1 1.0 0.8
0.25 0.2 0.8 1.0
Russian
German
Chinese
Japanese
Pattern Recognition Systems
Preprocessing
Feature extraction
Feature analysis
The main recognition task
Recognition Involves Learning
Learning in this context means:
1) Extracting knowledge from past experience
2) Representing the knowledge efficiently
3) Using the knowledge for future predictions/recognitions
Types of Learning
Supervised learning
Learning with a teacher
Unsupervised learning
Learning without a teacher
Reinforcement learning
Learning with a critic
Supervised Learning (Contd.)
Supervised learning systems varies according to the type of function learned.
Basic types:
Function approximation systems (Numeric outputs)
qp RRh :
Classifier systems (outputs are classes)
qp RRh :
Supervised Learning (Contd.)
)(xy S
It is assumed that x and y bear a (unknown) functional relationship say
It is assumed that (xi, yi) is generated from a fixed (but unknown) time invariant probability distribution.
L i Ni i x y, , , , ...,1 2The Training set
Goal: To find h which closely resembles S given L.
Supervised Learning (Contd.)
How to measure whether h resembles S.
Training error: The error on training data points
Test error (Generalization Error): The error on points not in the training set (difficult to measure)
))((1
ii
N
ii yxhJ