Csc446: Pattren Recognition (LN1)

Slide - 1

CSC446 : Pattern Recognition

Prof. Dr. Mostafa G. M. Mostafa Faculty of Computer & Information Sciences

Computer Science Department

AIN SHAMS UNIVERSITY

Lecture Note 1:

Course Organization &

Chapter 1: Introduction to PRS

ASU-CSC446 : Pattern Recognition. Prof. Dr. Mostafa Gadal-Haqq

Slide - 2

CSC446: Patt Recog - Course Outline

• Introduction to PRS

• Mathematical Foundations

• Supervised Learning

• Bayesian Decision Theory

• Maximum Likelihood Estimation

• Non-Parametric Methods

• Linear Discriminant Functions & NN

• Unsupervised Learning

• K-mean Clustering


Slide - 3

• Text Book:

– Duda, Hart, and Stork. “Pattern

Classification”, 2nd ed. Wiley, 2001.

• Reference Book:

– C. M. Bishop. “Pattern Recognition & Machine

Learning”. Springer, 2007.

– A. Webb. “Statistical Pattern Recognition”. Arnold,

1999.

• Lab book: Handout materials + “Matlab

Getting Started” and “Building GUI” tutorials.

CSC446 : Course Organization & Guidelines


http://books.google.com.sa/books?id=YCMicH3-JGQC&printsec=frontcover&source=gbs_summary_r&cad=0

Slide - 4

Prerequisites:

– CSW150: Structured Programming.

– SCC223 : Probability & Statistics

– SCC332 : Numerical Methods

– CSC343 : Artificial Intelligence


Refresh your Information


Slide - 5

Grading:

• Midterm Exam (10 points)

• Assignments, Quizzes (10 points)

• Final Project, Lab test (15 points)

• Final Exam (65 points)



Slide - 6


Lecture Protocol:

Feel free to interrupt and ask ME.

DON’T ask/talk to your colleagues.

Programming and homework assignments

•Late answers are given 50% of the mark.

Slides are available in pdf format.


Slide - 7


How to pass this course?

– You will learn a lot during this course, but you

will have to work hard to pass it!

– Don’t accumulate …

– Do it yourself …

– Ask for help …


Slide - 8


Warning:

– Working policy: You are encouraged to collaborate

in study groups. But submitting a copy or slightly

changed others’ solutions or codes is Cheating.

– Cheating will be punished severely

• Assignments: All get 0

• Midterm or Final: you will get Fail


Slide - 9


Resources (Most important):

Societies:

IAPR: http://www.iapr.org/

Journals:

PAMI: http://www.computer.org/tpami/

PR: http://www.elsevier.com/locate/pr

PRL: http://www.elsevier.com/locate/prl

Web Sites:

PRInfo: http://www.ph.tn.tudelft.nl/PRInfo/


Slide - 10

Chapter 1:

Introduction to Pattern

Recognition

CSC446 : Pattern Recognition

(Read all sections in Chapter 1)


Slide - 11

• Objectives of Pattern Recognition Systems

• Applications of Pattern Recognition Systems

• What is a Pattern Recognition System?

• An intuitive Example

• Components of Pattern Recognition Systems

• The Design Cycle

• Learning and Adaptation

– Supervised, Unsupervised, and Reinforcement Learning.

Intro Pattern Recognition - Outline


Slide - 12

PRS Objective:

• Building a machine that can learn and

recognize patterns as human,

• Having such a machine is immensely useful

to mankind.

Pattern Recognition System


Slide - 13

Pattern Recognition Systems

What is a Pattern?

What is a Pattern Recognition

System?


Slide - 14

What is a Pattern?

• A pattern is a set of instances which share

some regularities, and are similar to each

other in the set.

• A pattern should occur repeatedly.

• A pattern is observable, sometimes partially,

by some sensors with noise and distortions.


Slide - 15

Examples of Patterns


Slide - 16

Examples of Patterns

Speech Signal


Slide - 17

What is Pattern Recognition?

• Definition: “The act of taking in raw data

and taking an action based on the category

of the pattern found in the data.”

an object

Decision raw data

Pattern

Recognition

System (Cylinder)


Slide - 18

Applications of PRS

Applications: Robotics

Photo by courtesy of US Department of Energy .

Robot Manny is developed

at Battelle's Pacific

Northwest Laboratories in

Richand, Washington. It

took 12 researchers 3

years (1986-1989) and $2

million to develop this

robot. Manny was built

for the U.S. Army in the

late 1980s to test protective

clothing.


http://www.doedigitalarchive.doe.gov/

Slide - 19

Applications of PRS

Applications: In Military

A remote-controlled

bomb disposal robot

in action.

Photo by courtesy of US Airforce .


http://www.af.mil/photos/

Slide - 20

Applications of PRS

• OCR: Handwritten/printed optical characters recognition.


Slide - 21

Applications of PRS –Dictation machines, Voice Command, HCI

HCI,

Archiving Dictation

Voice

Command


Slide - 22

Applications of PRS •Investigation: Lie detector,


Slide - 23

Applications of PRS

•Other Applications:

– Manufacturing:

• Defect detection in chip manufacturing

• Fruit/vegetable recognition

– Biometrics: voice, iris, fingerprint, face, gait

recognition

– Medical diagnosis

–Smell recognition (e-nose, sensor networks)

–Bioinformatics: classification of DNA sequences.

–Security: intrusion detection


Slide - 24

Machine Learning

Concepts

Readings: Chapter 1 in Bishop’s PRML


Slide - 25

Machine Learning

• Learning Process:

–Learner (a computer program; an agent) processes data D

representing past experiences and tries to either develop an

appropriate response to future data, or describe the seen

data in some meaningful way.

• Example:

– Learner sees a set of patient records with corresponding

diagnoses. It can either try to :

– predict the presence of a disease for future patients.

– describe the dependencies between diseases, symptoms.


Slide - 26

Types of Machine Learning • Supervised learning

– Learning mapping between input x and desired output y

– Teacher presents some samples of pairs (x, y)

• Unsupervised learning

– Learning relations between data components

– No teacher signal.

• Reinforcement learning

– Learning mapping between input x and desired output y

– Teacher gives a Critic signal (reinforcement) of

how good the response was.


Slide - 27

Supervised Learning

• Data: A set of n examples

𝑿 = {𝒙𝟏, 𝒙𝟐, …, 𝒙𝒏} & 𝑻 = {𝒕𝟏, 𝒕𝟐, …, 𝒕𝒏}

x is input vector, and t is desired output (given by a teacher).

• Objective: Learn the mapping 𝑭: 𝑿 → 𝒀

That is to find: 𝒚𝒊 ≈ 𝒇(𝒙𝒊) for all 𝒊 = 𝟏, … , 𝒏

• Two types of problems:

Regression: X discrete or continuous Y is continuous

Classification: X discrete or continuous Y is discrete


Slide - 28

Supervised Learning Examples

• Regression: Y continuous

Debt/equity

Earnings company stock price

Future product orders


Slide - 29

Supervised Learning Examples

• Classification: Y discrete

{ a, b, c, …, x, y, z}

X is a vector/sequence of values

{ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9 }


Slide - 30

Unsupervised Learning

• Data: A set of n examples

𝑿 = {𝒙𝟏, 𝒙𝟐, …, 𝒙𝒏}

desired output NOT GIVEN (no teacher).

• Objective:

– Learn the relation between data components

• Two types of problems:

Clustering: Group “similar” examples together

Density Estimation : Model probabilistically the samples


Slide - 31

Unsupervised Learning Examples

• Clustering: Group “similar” examples together


Slide - 32

Unsupervised Learning Examples

• Density Estimation: Find probability density p(x)

Model used : Mixture of Gaussians


Slide - 33

Reinforcement Learning

• We want to learn: 𝑭: 𝑿 → 𝒀

• We only see samples of x but not y

• Instead of getting y we get a feedback

(reinforcement) from a critic about how good our

output was.

• Example:

– Real time strategic (RTS) games.


Slide - 34

Next Time

Introduction to Pattern

Recognition


Csc446: Pattren Recognition (LN1)

Data & Analytics