Dr. Ghassabi [email protected] Tehran Markaz University Spring 2015 [email protected] Statistical Pattern Recognition Session 1 (Introduction)

Dr. Ghassabi [email protected] Tehran Markaz University Spring 2015 [email protected] Statistical Pattern Recognition Session 1 (Introduction) 1 Course Link: cipcv.ir/lessons

Grades Home work 20% Exam- 40% Presentation and Participation- 40% If you write a good paper, your grade will become 20 2

Homework TA: Ali Rahmani, Email: [email protected]@gmail.com Format (email(subject)& exercise(name)): SPR-NAME-Family-Exe1 Deadline: 1 week Group works: More than 1 week 3

Paper Presenting a paper (Survey or Research paper) An overview of Statistical pattern recognition techniques for speaker verification An overview of Statistical pattern recognition techniques for speaker verification Research paper (2010,,2014) Comparing results of several papers (an evaluation study) Writing a research paper 4 http://www.matlabsite.com/1127/mvpacw9210-choice-of-topic-and-thesis-writing.html http://www.matlabsite.com/1748/fvacw9304-practical-thesis-preparation-and-academic- paper-publication.html

Presentation Example: http://research.cs.queensu.ca/home/xiao/pr.html http://research.cs.queensu.ca/home/xiao/pr.html Scope of subjects: Image Processing and Computer Vision: Face Recognition, Object Recognition, Document analysis, Biometric, remote sensing Speech Recognition Datamining and machine learning 5

Computer Vision Robotics Neuroscience Graphics Computational Photography Machine Learning Medical Imaging Human Computer Interaction Optics Image Processing Feature Matching Recognition

How To find a paper? IEEE, Elsevier, springer http://cipcv.ir/related-magazine 7

How to find a Project Subject? ICPR (International Conference on Pattern Recognition) ICPR 2013, 2012, 2010 http://www.icpr2012.org/overview.html 8

How to find a Project Subject? See Scope In Call Papers http://www.icpr2012.org/cfp.html 9

How to find a Project Subject? http://cipcv.ir/related-magazine Journals (IEEE, Elsevier, Springer) IEEE Transactions on pattern analysis and machine intelligence Pattern Recognition Pattern Recognition Letter 10

Goals Course Website: http://cipcv.ir/lessonshttp://cipcv.ir/lessons To make the graduate students acquainted with the fundamental concepts of statistical pattern recognition and its applications in computational intelligence Image Processing Computer Vision 11

Links Courses on Statistical Pattern Recognition SPR-Sharif University A Statistical Learning/Pattern Recognition Glossary A Statistical Learning/Pattern Recognition Glossary Tutorials on Topics in Statistical Pattern Recognition Tutorials on Topics in Statistical Pattern Recognition 12

Books Main books: Pattern Recognition books website (By Theodoridis and Koutroumbas) Pattern Recognition books website (By Theodoridis and Koutroumbas) Pattern classification (By duda) An introduction to statistical pattern recognition(By fukunaga) Other books: Fuzzy Models and Algorithms for Pattern Recognition and Image Processing( by James C. Bezdek, James Keller, Raghu Krisnapuram,Nikhil R. Pal) Fuzzy Models and Algorithms for Pattern Recognition and Image Processing( by James C. BezdekJames KellerRaghu KrisnapuramNikhil R. Pal Pattern Recognition and Machine Learning books website (By Bishop) Statistical Data Mining Tutorials (By Andrew Moore) Probabilistic Graphical Methods books website (By Koller and Friedman) The Elements of Statistical Learning books website (By Hastie, Tibshirani and Friedman) The Elements of Statistical Learning books website (By Hastie, Tibshirani and Friedman) 13

Papers A. K. Jain, R. P. W. Duin, J. Mao, "Statistical Pattern Recognition: A Review" (local copy), IEEE Trans. on Pattern Analysis and Machine Intelligence, 22(1):4-37, January 2000."Statistical Pattern Recognition: A Review"local copy Chi Hau Chen, Pei-Gee Peter Ho, Statistical pattern recognition in remote sensing, Pattern Recognition 41 (2008) 2731 2741 14

Matlab Toolboxes Statistical Pattern Recognition Toolbox Statistical Pattern Recognition Toolbox PRTools: The Matlab Toolbox for Pattern Recognition PRTools: The Matlab Toolbox for Pattern Recognition Bayes Net Toolbox Bayes Net Toolbox HMM Toolbox HMM Toolbox LIBSVM A Library for SVM LIBSVM A Library for SVM 15

Outline of PR course Introduction to PR and its applications Bayes Theory Neural Networks (Perceptron, Multi layer perceptron, RBF) Non-parametric methods (Decision Tree) SVM Clustering methods Feature Extraction and Selection 16

Outline of PR course Introduction to PR and its applications Bayes Theory Neural Networks (Perceptron, Multi layer perceptron, RBF) Non-parametric methods (Decision Tree) SVM Clustering methods Feature Extraction and Selection 17

What is Pattern? Opposite to chaos; it is an entity, object, process or event, vaguely defined, that can be given a name or label. Examples of Patterns: A fingerprint image A handwritten cursive word A human face A speech signal Texture Etc. 18

What is PR? Systems to put patterns into pre-specified categories Relate Perceived Patterns to previously Perceived Patterns Classification What is a pattern? A fingerprint images, A handwritten cursive word, A human face, A speech signal What kinds of category we have? 19

An Example N pattern categories (classes) Sky, buildings, mountain, trees, Common attributes (features) Color Contrast Texture Goal Observing some labeled pixels, We wish to assign a label to each new (unlabeled) pixel. 20

Application of PR Machine Vision Character Recognition Computer aided diagnosis Speech Recognition Datamining and knowledge Discovery Others: fingerprint identification, signature authentication, text retrieval, and face and gesture recognition. 21

Application of PR 22

Exercise 1 What is the difference between PR and Machine Learning (ML)? http://www.icpr2014.org/tutorialpages/philosophicalaspects http://www.icpr2014.org/tutorialpages/philosophicalaspects http://stats.stackexchange.com/questions/5026/what-is-the-difference-between-datamining-statistics-machine-learning-and-ai http://stats.stackexchange.com/questions/5026/what-is-the-difference-between-datamining-statistics-machine-learning-and-ai http://www.37steps.com/638/machine-learning-and-pattern-recognition/ http://www.37steps.com/638/machine-learning-and-pattern-recognition/ http://www.quora.com/What-is-the-difference-between-pattern-recognition-and- machine-learning http://www.quora.com/What-is-the-difference-between-pattern-recognition-and- machine-learning http://www.inf.ed.ac.uk/teaching/courses/mlpr/lectures/mlpr-prelim.pdf http://www.inf.ed.ac.uk/teaching/courses/mlpr/lectures/mlpr-prelim.pdf 23

Components of a PR system Patterns: Images Classes: (A, B) 24 Examples of image regions corresponding to (a) class A and (b) class B. a benign lesionmalignant (cancer),

Components of a PR system What measurable quantities make these two regions distinct from each other? 25 Plot of the mean value versus the standard deviation for a number of different images originating from class A (o) and class B (+). a straight line separates the two classes. the unknown pattern is more likely to belong to classA than class B

Components of a PR system Which features are discriminative? 26 Defective Cookies Non-Defective Cookies Are the mean value versus the standard deviation suitable features to separate defective and non-defective cookies?

Components of a PR system Which features are suitable to recognize vessel and non-vessel features? 27 http://cvip.computing.dundee.ac.uk/papers/Lupascu2009.pdf

improfile: : : imshow(retina1.tif);improfile; 28

Components of a PR system the mean value and the standard deviation are known as features l features x i, i =1, 2,..., l are used, and they form the feature vector x [x1, x2,..., xl ] T Each of the feature vectors identifies uniquely a single pattern (object). The straight line is known as the decision line, and it constitutes the classifier whose role is to divide the feature space into regions that correspond to either class A or class B. 29

Components of a PR system How are the features generated? Feature Extraction What is the best number l of features to use? Feature Selection Having adopted the appropriate, for the specific task, features, how does one design the classifier? Classifier stage design Once the classifier has been designed, how can one assess the performance of the designed classifier? That is, what is the classification error rate? system evaluation stage. 30

Components of a PR system The design of a pattern recognition system essentially involves the following three aspects: data acquisition data representation decision making 31 The basic stages involved in the design of a classification system.

Components of a PR system Categories (Classes): Supervised Classification Discriminant Analysis Unsupervised Classification Clustering 32

Design cycle Data collection Feature Choice Model Choice Training Evaluation Computational Complexity 33

Image Categorization Training Labels Training Images Classifier Training Training Image Features Testing Test Image Trained Classifier Outdoor Prediction

Learning a classifier Given some set of features with corresponding labels, learn a function to predict the labels from the features xx x x x x x x o o o o o x2 x1

Formulation: binary classification Formulation +1 x1x1 x2x2 x3x3 xNxN x N+1 x N+2 x N+M ??? Training data: each image patch is labeled as containing the object or background Test data Features x = Labels y = Where belongs to some family of functions Classification function Minimize misclassification error (Not that simple: we need some guarantees that there will be generalization)

Many classifiers to choose from SVM Neural networks Nave Bayes Bayesian network Decision Trees K-nearest neighbor Etc. Which is the best one?

PR approaches Statistical PR Based on underlying statistical properties/model of patterns and pattern classes. Use numerical features for distinguishing between classes. Bayesian Methods Neural Networks Decision Trees Support Vector Machines Etc. Structural (or syntactic) PR Based on explicit or implicit representation of a classs structure Pattern classes represented by means of formal structures as grammers, automata, strings, graphs, trees, etc. Reference: Syntactic and structural pattern recognition: theory and applications, By Horst Bunke, Alberto Sanfeliu 38

PR approaches 39

Next Session Classifiers Based on Bayes Decision Theory 40

Dr. Ghassabi [email protected] Tehran Markaz University Spring 2015 [email protected] Statistical Pattern Recognition Session 1 (Introduction)

Documents

pal pattern recognition

pattern recognition

object recognition

face recognition

pattern analysis

irlessons slide

springer http

papers http