Pattern Recognition 1: Introduction€¦ · Arne Leijon Pattern Recognition 1: Introduction. Classification Summary Intro Example Probability Density Decision Boundary and Decision

ClassificationSummary

Pattern Recognition 1: Introduction

Arne Leijon

KTH Sound and Image Processing

Aug 27, 2012

Arne Leijon Pattern Recognition 1: Introduction

ClassificationSummary

IntroExampleProbability Density

A Pattern-Classification System

Feature

ExtractorClassifier

Transducer

Input

signal

Output

decision

Arne Leijon Pattern Recognition 1: Introduction

ClassificationSummary

IntroExampleProbability Density

Source Categories

Sub-source #1

Sub-source #2

Sub-source #

€

Ns

Noise

Noise

Noise

Noise

Signal Source

Noise

S

Random State

Switch

Transducer

Arne Leijon Pattern Recognition 1: Introduction

ClassificationSummary

IntroExampleProbability Density

Optimal Classification Mechanism

Select index

of largest

Discriminant Function

Discriminant Function

#1

#K

x

Feature Extraction

Classifier

†

g1 x( )

†

gNdx( )

Arne Leijon Pattern Recognition 1: Introduction

ClassificationSummary

IntroExampleProbability Density

First Classifier Example: Man or Woman?

150 160 170 180 190 20050

60

70

80

90

100

x1= Height / cm

x 2=

Weig

ht / kg

Arne Leijon Pattern Recognition 1: Introduction

ClassificationSummary

IntroExampleProbability Density

Two Gaussian Feature Probability Density Functions

Women

150160

170180

190200

60

80

1000

1

2

3

4

5

x 10!3

x1= Height / cmx

2= Weight / kg

Men

150160

170180

190200

60

80

1000

1

2

3

4

5

x 10!3

x1= Height / cmx

2= Weight / kg

Arne Leijon Pattern Recognition 1: Introduction

ClassificationSummary

IntroExampleProbability Density

Contours of Equal Probability Density

Women

x1= Height / cm

x 2=

Weig

ht / kg

150 160 170 180 190 20050

60

70

80

90

100

Men

x1= Height / cm

x 2=

Weig

ht / kg

150 160 170 180 190 20050

60

70

80

90

100

Arne Leijon Pattern Recognition 1: Introduction

ClassificationSummary

IntroExampleProbability Density

Decision Boundary and Decision Regions

x1= Height / cm

x 2=

Weig

ht / kg

150 160 170 180 190 20050

60

70

80

90

100

Arne Leijon Pattern Recognition 1: Introduction

ClassificationSummary

IntroExampleProbability Density

Gaussian Probability Density (2-dim)

x1

x2

!5 !4 !3 !2 !1 0 1 2 3 4 5!5

!4

!3

!2

!1

0

1

2

3

4

5

cov [X ] =

✓32 00 12

◆=

=

✓9 00 1

◆

Arne Leijon Pattern Recognition 1: Introduction

ClassificationSummary

IntroExampleProbability Density

Gaussian Probability Density (2-dim)

x1

x2

!5 !4 !3 !2 !1 0 1 2 3 4 5!5

!4

!3

!2

!1

0

1

2

3

4

5

cov [X ] =

✓12 00 32

◆=

=

✓1 00 9

◆

Arne Leijon Pattern Recognition 1: Introduction

ClassificationSummary

IntroExampleProbability Density

Gaussian Probability Density (2-dim)

x1

x2

!5 !4 !3 !2 !1 0 1 2 3 4 5!5

!4

!3

!2

!1

0

1

2

3

4

5

cov [X ] =P

✓32 00 12

◆P

T =

=

✓5 44 5

◆

where

P =1p2

✓1 11 �1

◆

Arne Leijon Pattern Recognition 1: Introduction

ClassificationSummary

IntroExampleProbability Density

Gaussian Probability Density (K -dim)

f

X

(x) =1

(2⇡)K/2pdetC

e

� 12(x�µ)TC�1(x�µ)

Arne Leijon Pattern Recognition 1: Introduction

ClassificationSummary

IntroExampleProbability Density

General Probability Density: Gaussian Mixture (1-dim)

!2 0 2 4 60

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

x

Pro

b. D

ensi

ty

Arne Leijon Pattern Recognition 1: Introduction

ClassificationSummary

IntroExampleProbability Density

General Probability Density: Gaussian Mixture (2-dim)

Arne Leijon Pattern Recognition 1: Introduction

ClassificationSummary

IntroExampleProbability Density

General Probability Density: Gaussian Mixture (K -dim)

f

X

(x) =MX

m=1

w

m

1

(2⇡)K/2pdetC

m

e

� 12(x�µ

m

)TC�1m

(x�µm

)

Arne Leijon Pattern Recognition 1: Introduction

ClassificationSummary

Summary: Important Concepts

Feature Vector: vector x containing all input data to the classifier

Observation Space: set of all possible feature-vector values

Decision Function: d(x) maps feature vector x into discretedecision output, coded as integer

Decision Region: set of feature-vector values giving same decision:⌦i

= {x : d(x) = i}Discriminant Function: g

i

(x) maps any feature vector x into realnumber, used for classification

General Optimal Classifier: d(x) = argmaxk

g

k

(x)

Two-category Classifier: can be implemented using a singleDiscriminant Function: d(x) = sgn g(x)

Gaussian Mixture Model: can approximate any distribution

Arne Leijon Pattern Recognition 1: Introduction