Top Banner
발표자 : 백승호
73

1. Introductiondmqm.korea.ac.kr/uploads/seminar/Open Set Recognition... · 2020-01-03 · - 20 / 72 - Data 無 Label有 Knowns Unknowns s s Knowns Knowns (KKC) Knowns Unknowns (KUC)

Aug 01, 2020

Download

Documents

dariahiddleston
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: 1. Introductiondmqm.korea.ac.kr/uploads/seminar/Open Set Recognition... · 2020-01-03 · - 20 / 72 - Data 無 Label有 Knowns Unknowns s s Knowns Knowns (KKC) Knowns Unknowns (KUC)

발표자 : 백승호

Page 2: 1. Introductiondmqm.korea.ac.kr/uploads/seminar/Open Set Recognition... · 2020-01-03 · - 20 / 72 - Data 無 Label有 Knowns Unknowns s s Knowns Knowns (KKC) Knowns Unknowns (KUC)

- 2 / 72 -

Page 3: 1. Introductiondmqm.korea.ac.kr/uploads/seminar/Open Set Recognition... · 2020-01-03 · - 20 / 72 - Data 無 Label有 Knowns Unknowns s s Knowns Knowns (KKC) Knowns Unknowns (KUC)

1. Introduction

Page 4: 1. Introductiondmqm.korea.ac.kr/uploads/seminar/Open Set Recognition... · 2020-01-03 · - 20 / 72 - Data 無 Label有 Knowns Unknowns s s Knowns Knowns (KKC) Knowns Unknowns (KUC)

- 4 / 72 -

Page 5: 1. Introductiondmqm.korea.ac.kr/uploads/seminar/Open Set Recognition... · 2020-01-03 · - 20 / 72 - Data 無 Label有 Knowns Unknowns s s Knowns Knowns (KKC) Knowns Unknowns (KUC)

- 5 / 72 -

Page 6: 1. Introductiondmqm.korea.ac.kr/uploads/seminar/Open Set Recognition... · 2020-01-03 · - 20 / 72 - Data 無 Label有 Knowns Unknowns s s Knowns Knowns (KKC) Knowns Unknowns (KUC)

- 6 / 72 -

Page 7: 1. Introductiondmqm.korea.ac.kr/uploads/seminar/Open Set Recognition... · 2020-01-03 · - 20 / 72 - Data 無 Label有 Knowns Unknowns s s Knowns Knowns (KKC) Knowns Unknowns (KUC)

- 7 / 72 -

Page 8: 1. Introductiondmqm.korea.ac.kr/uploads/seminar/Open Set Recognition... · 2020-01-03 · - 20 / 72 - Data 無 Label有 Knowns Unknowns s s Knowns Knowns (KKC) Knowns Unknowns (KUC)

- 8 / 72 -

English Math

ScienceArt

Page 9: 1. Introductiondmqm.korea.ac.kr/uploads/seminar/Open Set Recognition... · 2020-01-03 · - 20 / 72 - Data 無 Label有 Knowns Unknowns s s Knowns Knowns (KKC) Knowns Unknowns (KUC)

- 9 / 72 -

English Math

ScienceArt

Page 10: 1. Introductiondmqm.korea.ac.kr/uploads/seminar/Open Set Recognition... · 2020-01-03 · - 20 / 72 - Data 無 Label有 Knowns Unknowns s s Knowns Knowns (KKC) Knowns Unknowns (KUC)

- 10 / 72 -

Training

Page 11: 1. Introductiondmqm.korea.ac.kr/uploads/seminar/Open Set Recognition... · 2020-01-03 · - 20 / 72 - Data 無 Label有 Knowns Unknowns s s Knowns Knowns (KKC) Knowns Unknowns (KUC)

- 11 / 72 -

Test

Page 12: 1. Introductiondmqm.korea.ac.kr/uploads/seminar/Open Set Recognition... · 2020-01-03 · - 20 / 72 - Data 無 Label有 Knowns Unknowns s s Knowns Knowns (KKC) Knowns Unknowns (KUC)

- 12 / 72 -

Test

Page 13: 1. Introductiondmqm.korea.ac.kr/uploads/seminar/Open Set Recognition... · 2020-01-03 · - 20 / 72 - Data 無 Label有 Knowns Unknowns s s Knowns Knowns (KKC) Knowns Unknowns (KUC)

- 13 / 72 -

Test

Page 14: 1. Introductiondmqm.korea.ac.kr/uploads/seminar/Open Set Recognition... · 2020-01-03 · - 20 / 72 - Data 無 Label有 Knowns Unknowns s s Knowns Knowns (KKC) Knowns Unknowns (KUC)

- 14 / 72 -

Test

Page 15: 1. Introductiondmqm.korea.ac.kr/uploads/seminar/Open Set Recognition... · 2020-01-03 · - 20 / 72 - Data 無 Label有 Knowns Unknowns s s Knowns Knowns (KKC) Knowns Unknowns (KUC)

- 15 / 72 -

Page 16: 1. Introductiondmqm.korea.ac.kr/uploads/seminar/Open Set Recognition... · 2020-01-03 · - 20 / 72 - Data 無 Label有 Knowns Unknowns s s Knowns Knowns (KKC) Knowns Unknowns (KUC)

2. Recent Advances in Open Set Recognition

Page 17: 1. Introductiondmqm.korea.ac.kr/uploads/seminar/Open Set Recognition... · 2020-01-03 · - 20 / 72 - Data 無 Label有 Knowns Unknowns s s Knowns Knowns (KKC) Knowns Unknowns (KUC)

- 17 / 72 -

Knowns Unknowns

Kn

ow

ns

Un

kn

ow

ns

Knowns Knowns

(KKC)

Knowns Unknowns

(KUC)

Unknowns Knowns

(UKC)

Unknowns Unknowns

(UUC)

Knowledge of State(Data available)

Kn

ow

led

ge o

f D

om

ain

(Un

der

stan

din

g)

High Low

Low

Hig

h

Data 有

Label 無

Data 有

Label 有

Data 無

Label 有

Data 無

Label 有

Page 18: 1. Introductiondmqm.korea.ac.kr/uploads/seminar/Open Set Recognition... · 2020-01-03 · - 20 / 72 - Data 無 Label有 Knowns Unknowns s s Knowns Knowns (KKC) Knowns Unknowns (KUC)

- 18 / 72 -

Training

Page 19: 1. Introductiondmqm.korea.ac.kr/uploads/seminar/Open Set Recognition... · 2020-01-03 · - 20 / 72 - Data 無 Label有 Knowns Unknowns s s Knowns Knowns (KKC) Knowns Unknowns (KUC)

- 19 / 72 -

Data 無

Label 有

Knowns Unknowns

Kn

ow

ns

Un

kn

ow

ns

Knowns Knowns

(KKC)

Knowns Unknowns

(KUC)

Unknowns Knowns

(UKC)

Unknowns Unknowns

(UUC)

Knowledge of Domain(Understanding)

Kn

ow

led

ge o

f St

ate

(Dat

a av

aila

ble

)

High Low

Low

Hig

h

Data 有

Label 無

Data 有

Label 有

Data 無

Label 有

Page 20: 1. Introductiondmqm.korea.ac.kr/uploads/seminar/Open Set Recognition... · 2020-01-03 · - 20 / 72 - Data 無 Label有 Knowns Unknowns s s Knowns Knowns (KKC) Knowns Unknowns (KUC)

- 20 / 72 -

Data 無

Label 有

Knowns Unknowns

Kn

ow

ns

Un

kn

ow

ns

Knowns Knowns

(KKC)

Knowns Unknowns

(KUC)

Unknowns Knowns

(UKC)

Unknowns Unknowns

(UUC)

Knowledge of Domain(Understanding)

Kn

ow

led

ge o

f St

ate

(Dat

a av

aila

ble

)

High Low

Low

Hig

h

Data 有

Label 無

Data 有

Label 有

Data 無

Label 有

Page 21: 1. Introductiondmqm.korea.ac.kr/uploads/seminar/Open Set Recognition... · 2020-01-03 · - 20 / 72 - Data 無 Label有 Knowns Unknowns s s Knowns Knowns (KKC) Knowns Unknowns (KUC)

- 21 / 72 -

Data 無

Label 有

Knowns Unknowns

Kn

ow

ns

Un

kn

ow

ns

Knowns Knowns

(KKC)

Knowns Unknowns

(KUC)

Unknowns Knowns

(UKC)

Unknowns Unknowns

(UUC)

Knowledge of Domain(Understanding)

Kn

ow

led

ge o

f St

ate

(Dat

a av

aila

ble

)

High Low

Low

Hig

h

Data 有

Label 無

Data 有

Label 有

Data 無

Label 有

Page 22: 1. Introductiondmqm.korea.ac.kr/uploads/seminar/Open Set Recognition... · 2020-01-03 · - 20 / 72 - Data 無 Label有 Knowns Unknowns s s Knowns Knowns (KKC) Knowns Unknowns (KUC)

- 22 / 72 -

Data 無

Label 有

Knowns Unknowns

Kn

ow

ns

Un

kn

ow

ns

Knowns Knowns

(KKC)

Knowns Unknowns

(KUC)

Unknowns Knowns

(UKC)

Unknowns Unknowns

(UUC)

Knowledge of Domain(Understanding)

Kn

ow

led

ge o

f St

ate

(Dat

a av

aila

ble

)

High Low

Low

Hig

h

Data 有

Label 無

Data 有

Label 有

Data 無

Label 有

심해물고기 ??

Page 23: 1. Introductiondmqm.korea.ac.kr/uploads/seminar/Open Set Recognition... · 2020-01-03 · - 20 / 72 - Data 無 Label有 Knowns Unknowns s s Knowns Knowns (KKC) Knowns Unknowns (KUC)

- 23 / 72 -

Data 無

Label 有

Knowns Unknowns

Kn

ow

ns

Un

kn

ow

ns

Knowns Knowns

(KKC)

Knowns Unknowns

(KUC)

Unknowns Knowns

(UKC)

Unknowns Unknowns

(UUC)

Knowledge of Domain(Understanding)

Kn

ow

led

ge o

f St

ate

(Dat

a av

aila

ble

)

High Low

Low

Hig

h

Data 有

Label 無

Data 有

Label 有

Data 無

Label 有

심해물고기 ??아몰랑~

Page 24: 1. Introductiondmqm.korea.ac.kr/uploads/seminar/Open Set Recognition... · 2020-01-03 · - 20 / 72 - Data 無 Label有 Knowns Unknowns s s Knowns Knowns (KKC) Knowns Unknowns (KUC)

- 24 / 72 -

Training Testing Goal

Traditional classification KKC KKC Classifying KKC

Classification

with Reject OptionKKC KKC

Classifying KKC &

rejecting samples of

low confidence

One-class Classification

(Anomaly Detection)

KKC & few or none outliers

from KUCs

KKC &

few or none outliersDetecting outliers

One-shot Learning

(Few-shot Learning)

KKC & a limited number of

UKCs’ samplesUKC Identifying UKC

Zero-shot LearningKKC &

semantic-informationKKC & UKC Identifying KKC & UKC

Open Set Recognition KKC KKC & UKCIdentifying KKC &

rejecting UKC

SettingTask

Page 25: 1. Introductiondmqm.korea.ac.kr/uploads/seminar/Open Set Recognition... · 2020-01-03 · - 20 / 72 - Data 無 Label有 Knowns Unknowns s s Knowns Knowns (KKC) Knowns Unknowns (KUC)

- 25 / 72 -

Train Test [ Goal ]

Dog

Cat

Traditional classification

Classification

with Reject Option

One-class Classification

(Anomaly Detection)

One-shot Learning

(Few-shot Learning)

Zero-shot Learning

Open Set Recognition

SettingTask

Page 26: 1. Introductiondmqm.korea.ac.kr/uploads/seminar/Open Set Recognition... · 2020-01-03 · - 20 / 72 - Data 無 Label有 Knowns Unknowns s s Knowns Knowns (KKC) Knowns Unknowns (KUC)

- 26 / 72 -

Train Test [ Goal ]

Dog

Cat

Traditional classification

Classification

with Reject Option

One-class Classification

(Anomaly Detection)

One-shot Learning

(Few-shot Learning)

Zero-shot Learning

Open Set Recognition

SettingTask

SampleProbability

ClassDog Cat

1 0.97 0.03 Dog

2 0.01 0.99 Cat

3 0.23 0.77 Cat

4 0.37 0.63 Cat

𝑝𝑖 =𝑒𝑍𝑖

σ𝑗=1𝑘 𝑒𝑍𝑗

ሿL𝑎𝑏𝑒𝑙 𝑖𝑛𝑑𝑒𝑥 = 𝑎𝑟𝑔𝑚𝑎𝑥[𝑝1, 𝑝2, … , 𝑝𝑘

Page 27: 1. Introductiondmqm.korea.ac.kr/uploads/seminar/Open Set Recognition... · 2020-01-03 · - 20 / 72 - Data 無 Label有 Knowns Unknowns s s Knowns Knowns (KKC) Knowns Unknowns (KUC)

- 27 / 72 -

Train Test [ Goal ]

Dog

Cat

Unknown

Traditional classification

Classification

with Reject Option

One-class Classification

(Anomaly Detection)

One-shot Learning

(Few-shot Learning)

Zero-shot Learning

Open Set Recognition

SettingTask

Page 28: 1. Introductiondmqm.korea.ac.kr/uploads/seminar/Open Set Recognition... · 2020-01-03 · - 20 / 72 - Data 無 Label有 Knowns Unknowns s s Knowns Knowns (KKC) Knowns Unknowns (KUC)

- 28 / 72 -

Train Test [ Goal ]

Dog

Cat

Unknown

SampleProbability

ClassDog Cat

1 0.97 0.03 Dog

2 0.01 0.99 Cat

3 0.23 0.77 Cat

4 0.37 0.63 Unknown

Class = { reject, if Softmax(xi) < 𝜃

argmaxli , if Softmax(xi) ≥ 𝜃

𝜃 = ThresholdTraditional classification

Classification

with Reject Option

One-class Classification

(Anomaly Detection)

One-shot Learning

(Few-shot Learning)

Zero-shot Learning

Open Set Recognition

SettingTask

Page 29: 1. Introductiondmqm.korea.ac.kr/uploads/seminar/Open Set Recognition... · 2020-01-03 · - 20 / 72 - Data 無 Label有 Knowns Unknowns s s Knowns Knowns (KKC) Knowns Unknowns (KUC)

- 29 / 72 -

Train Test [ Goal ]

Dog

Unknown

Outlier

Traditional classification

Classification

with Reject Option

One-class Classification

(Anomaly Detection)

One-shot Learning

(Few-shot Learning)

Zero-shot Learning

Open Set Recognition

SettingTask

Page 30: 1. Introductiondmqm.korea.ac.kr/uploads/seminar/Open Set Recognition... · 2020-01-03 · - 20 / 72 - Data 無 Label有 Knowns Unknowns s s Knowns Knowns (KKC) Knowns Unknowns (KUC)

- 30 / 72 -

Train Test [ Goal ]

Dog

Unknown

Outlier

Traditional classification

Classification

with Reject Option

One-class Classification

(Anomaly Detection)

One-shot Learning

(Few-shot Learning)

Zero-shot Learning

Open Set Recognition

SettingTask

Page 31: 1. Introductiondmqm.korea.ac.kr/uploads/seminar/Open Set Recognition... · 2020-01-03 · - 20 / 72 - Data 無 Label有 Knowns Unknowns s s Knowns Knowns (KKC) Knowns Unknowns (KUC)

- 31 / 72 -

Train Test [ Goal ]

Dog

Unknown

Outlier

Traditional classification

Classification

with Reject Option

One-class Classification

(Anomaly Detection)

One-shot Learning

(Few-shot Learning)

Zero-shot Learning

Open Set Recognition

SettingTask

Page 32: 1. Introductiondmqm.korea.ac.kr/uploads/seminar/Open Set Recognition... · 2020-01-03 · - 20 / 72 - Data 無 Label有 Knowns Unknowns s s Knowns Knowns (KKC) Knowns Unknowns (KUC)

- 32 / 72 -

Train Test [ Goal ]

Dog

Unknown

Outlier

Traditional classification

Classification

with Reject Option

One-class Classification

(Anomaly Detection)

One-shot Learning

(Few-shot Learning)

Zero-shot Learning

Open Set Recognition

SettingTask

Page 33: 1. Introductiondmqm.korea.ac.kr/uploads/seminar/Open Set Recognition... · 2020-01-03 · - 20 / 72 - Data 無 Label有 Knowns Unknowns s s Knowns Knowns (KKC) Knowns Unknowns (KUC)

- 33 / 72 -

Train Test [ Goal ]

Dog

Unknown

Outlier

Traditional classification

Classification

with Reject Option

One-class Classification

(Anomaly Detection)

One-shot Learning

(Few-shot Learning)

Zero-shot Learning

Open Set Recognition

SettingTask

Page 34: 1. Introductiondmqm.korea.ac.kr/uploads/seminar/Open Set Recognition... · 2020-01-03 · - 20 / 72 - Data 無 Label有 Knowns Unknowns s s Knowns Knowns (KKC) Knowns Unknowns (KUC)

- 34 / 72 -

Train Test [ Goal ]

Dog

Unknown

Outlier

Traditional classification

Classification

with Reject Option

One-class Classification

(Anomaly Detection)

One-shot Learning

(Few-shot Learning)

Zero-shot Learning

Open Set Recognition

SettingTask

Page 35: 1. Introductiondmqm.korea.ac.kr/uploads/seminar/Open Set Recognition... · 2020-01-03 · - 20 / 72 - Data 無 Label有 Knowns Unknowns s s Knowns Knowns (KKC) Knowns Unknowns (KUC)

- 35 / 72 -

Train Test [ Goal ]

Knife

Traditional classification

Classification

with Reject Option

One-class Classification

(Anomaly Detection)

One-shot Learning

(Few-shot Learning)

Zero-shot Learning

Open Set Recognition

SettingTask

Ship

Air craft

Page 36: 1. Introductiondmqm.korea.ac.kr/uploads/seminar/Open Set Recognition... · 2020-01-03 · - 20 / 72 - Data 無 Label有 Knowns Unknowns s s Knowns Knowns (KKC) Knowns Unknowns (KUC)

- 36 / 72 -

Train Test [ Goal ]

Knife

Traditional classification

Classification

with Reject Option

One-class Classification

(Anomaly Detection)

One-shot Learning

(Few-shot Learning)

Zero-shot Learning

Open Set Recognition

SettingTask

Ship

Air craft

Page 37: 1. Introductiondmqm.korea.ac.kr/uploads/seminar/Open Set Recognition... · 2020-01-03 · - 20 / 72 - Data 無 Label有 Knowns Unknowns s s Knowns Knowns (KKC) Knowns Unknowns (KUC)

- 37 / 72 -

Traditional classification

Classification

with Reject Option

One-class Classification

(Anomaly Detection)

One-shot Learning

(Few-shot Learning)

Zero-shot Learning

Open Set Recognition

SettingTask

[ Why Zero Shot ? ]

1) AI 전성시대 & Data 홍수의시대(너무많은 Class가 필요)

2) 일부 Class에 대한 Label은 전문가만이가능

3) 결국기존 Classification의 성능강화

송이

영지

표고

흰주름

Page 38: 1. Introductiondmqm.korea.ac.kr/uploads/seminar/Open Set Recognition... · 2020-01-03 · - 20 / 72 - Data 無 Label有 Knowns Unknowns s s Knowns Knowns (KKC) Knowns Unknowns (KUC)

- 38 / 72 -

Traditional classification

Classification

with Reject Option

One-class Classification

(Anomaly Detection)

One-shot Learning

(Few-shot Learning)

Zero-shot Learning

Open Set Recognition

SettingTask

[ Why Zero Shot ? ]

1) AI 전성시대 & Data 홍수의시대(너무많은 Class가 필요)

2) 일부 Class에 대한 Label은 전문가만이가능

3) 결국기존 Classification의 성능강화

송이

영지

표고

흰주름

Page 39: 1. Introductiondmqm.korea.ac.kr/uploads/seminar/Open Set Recognition... · 2020-01-03 · - 20 / 72 - Data 無 Label有 Knowns Unknowns s s Knowns Knowns (KKC) Knowns Unknowns (KUC)

- 39 / 72 -

Traditional classification

Classification

with Reject Option

One-class Classification

(Anomaly Detection)

One-shot Learning

(Few-shot Learning)

Zero-shot Learning

Open Set Recognition

SettingTask

[ Why Zero Shot ? ]

1) AI 전성시대 & Data 홍수의시대(너무많은 Class가 필요)

2) 일부 Class에 대한 Label은 전문가만이가능

3) 결국기존 Classification의 성능강화

송이

영지

표고

흰주름

Page 40: 1. Introductiondmqm.korea.ac.kr/uploads/seminar/Open Set Recognition... · 2020-01-03 · - 20 / 72 - Data 無 Label有 Knowns Unknowns s s Knowns Knowns (KKC) Knowns Unknowns (KUC)

- 40 / 72 -

Train Test

Traditional classification

Classification

with Reject Option

One-class Classification

(Anomaly Detection)

One-shot Learning

(Few-shot Learning)

Zero-shot Learning

Open Set Recognition

SettingTask

Husky

Bull

Goal

Husky

Bull

Unknown

Page 41: 1. Introductiondmqm.korea.ac.kr/uploads/seminar/Open Set Recognition... · 2020-01-03 · - 20 / 72 - Data 無 Label有 Knowns Unknowns s s Knowns Knowns (KKC) Knowns Unknowns (KUC)

- 41 / 72 -

Train Test

Traditional classification

Classification

with Reject Option

One-class Classification

(Anomaly Detection)

One-shot Learning

(Few-shot Learning)

Zero-shot Learning

Open Set Recognition

SettingTask

Husky

Bull

Goal

Husky

Bull

Unknown

Page 42: 1. Introductiondmqm.korea.ac.kr/uploads/seminar/Open Set Recognition... · 2020-01-03 · - 20 / 72 - Data 無 Label有 Knowns Unknowns s s Knowns Knowns (KKC) Knowns Unknowns (KUC)

- 42 / 72 -

Train Test

Traditional classification

Classification

with Reject Option

One-class Classification

(Anomaly Detection)

One-shot Learning

(Few-shot Learning)

Zero-shot Learning

Open Set Recognition

SettingTask

Husky

Bull

Goal

Husky

Bull

Unknown

Page 43: 1. Introductiondmqm.korea.ac.kr/uploads/seminar/Open Set Recognition... · 2020-01-03 · - 20 / 72 - Data 無 Label有 Knowns Unknowns s s Knowns Knowns (KKC) Knowns Unknowns (KUC)

- 43 / 72 -

Testing

Page 44: 1. Introductiondmqm.korea.ac.kr/uploads/seminar/Open Set Recognition... · 2020-01-03 · - 20 / 72 - Data 無 Label有 Knowns Unknowns s s Knowns Knowns (KKC) Knowns Unknowns (KUC)

- 44 / 72 -

Training

Page 45: 1. Introductiondmqm.korea.ac.kr/uploads/seminar/Open Set Recognition... · 2020-01-03 · - 20 / 72 - Data 無 Label有 Knowns Unknowns s s Knowns Knowns (KKC) Knowns Unknowns (KUC)

3. What I do in DMQA ?

Page 46: 1. Introductiondmqm.korea.ac.kr/uploads/seminar/Open Set Recognition... · 2020-01-03 · - 20 / 72 - Data 無 Label有 Knowns Unknowns s s Knowns Knowns (KKC) Knowns Unknowns (KUC)

- 46 / 72 -

[ Structure ] [ How to find 𝜃 ]

Page 47: 1. Introductiondmqm.korea.ac.kr/uploads/seminar/Open Set Recognition... · 2020-01-03 · - 20 / 72 - Data 無 Label有 Knowns Unknowns s s Knowns Knowns (KKC) Knowns Unknowns (KUC)

- 47 / 72 -

Convolution + ReLU Max pooling 2x2

Input WBM

… …

40x40x1

20x20x64

10x10x128

5x5x256

Normal

Center

Edge

Full fail

Probability> Threshold

Probability > Threshold Probability < Threshold

Fully Connected

Scratch

Ring

225 0 9 0 0 0

5 78 5 1 0 0

16 4 272 4 0 0

1 1 3 20 1 0

0 3 2 1 5 0

0 2 0 15 68 0

Tru

e la

bel

Rin

g(N

ew)

Full

fail

Edge

Cen

ter

No

rmal

Scra

tch

Predict label

Original Classification Proposed Classification (by max) Proposed Classification (by stdev)

Number of samples found a new pattern The number of samples that require reclassification

Normal Center Edge Full fail Ring(New)

Scratch

199 0 4 0 0 31

3 69 1 0 0 16

4 2 256 1 0 33

0 1 1 16 0 8

0 1 0 1 3 6

0 0 0 2 36 47

Tru

e la

bel

Rin

g(N

ew)

Full

fail

Edge

Cen

ter

No

rmal

Scra

tch

Predict label

Normal Center Edge Full fail Ring(New)

Scratch

205 0 7 0 0 22

3 73 2 0 0 11

5 2 260 1 0 28

0 1 1 17 0 7

0 2 0 1 4 4

0 0 0 3 43 39

Tru

e la

bel

Rin

g(N

ew)

Full

fail

Edge

Cen

ter

No

rmal

Scra

tch

Predict label

Normal Center Edge Full fail Ring(New)

Scratch

Page 48: 1. Introductiondmqm.korea.ac.kr/uploads/seminar/Open Set Recognition... · 2020-01-03 · - 20 / 72 - Data 無 Label有 Knowns Unknowns s s Knowns Knowns (KKC) Knowns Unknowns (KUC)

- 48 / 72 -

성과 한계점

1) 신규 불량 패턴 등장 인식 가능

2) 잘못된 분류의 감소

(후속 처리 품질 위험 감소)

1) 지정된 범주의 이중 불량 발생

탐지 불가

2) 신규 불량 패턴의 종류를 고려하지

않고 모두 신규 패턴(1 Class)으로 분류

(2개 이상의 신규 패턴 등장 ??)

Page 49: 1. Introductiondmqm.korea.ac.kr/uploads/seminar/Open Set Recognition... · 2020-01-03 · - 20 / 72 - Data 無 Label有 Knowns Unknowns s s Knowns Knowns (KKC) Knowns Unknowns (KUC)

4. DOC : Deep Open Classificationof Text Documents

Page 50: 1. Introductiondmqm.korea.ac.kr/uploads/seminar/Open Set Recognition... · 2020-01-03 · - 20 / 72 - Data 無 Label有 Knowns Unknowns s s Knowns Knowns (KKC) Knowns Unknowns (KUC)

- 50 / 72 -

Page 51: 1. Introductiondmqm.korea.ac.kr/uploads/seminar/Open Set Recognition... · 2020-01-03 · - 20 / 72 - Data 無 Label有 Knowns Unknowns s s Knowns Knowns (KKC) Knowns Unknowns (KUC)

- 51 / 72 -

Normal

Donut

Edge

Location

Center

각 Class 별로 Sigmoid를 최종활성화함수로갖는다.

(고전적인 Multi-task 분류기는 Softmax 사용)

각범주별확률을따로계산

Page 52: 1. Introductiondmqm.korea.ac.kr/uploads/seminar/Open Set Recognition... · 2020-01-03 · - 20 / 72 - Data 無 Label有 Knowns Unknowns s s Knowns Knowns (KKC) Knowns Unknowns (KUC)

- 52 / 72 -

Normal

Donut

Edge

Location

Center

각 Class 별로 Sigmoid를 최종활성화함수로갖는다.

(고전적인 Multi-task 분류기는 Softmax 사용)

각범주별확률을따로계산

Page 53: 1. Introductiondmqm.korea.ac.kr/uploads/seminar/Open Set Recognition... · 2020-01-03 · - 20 / 72 - Data 無 Label有 Knowns Unknowns s s Knowns Knowns (KKC) Knowns Unknowns (KUC)

- 53 / 72 -

Normal

Donut

Edge

Location

Center

각 Class 별로 Sigmoid를 최종활성화함수로갖는다.

(고전적인 Multi-task 분류기는 Softmax 사용)

각범주별확률을따로계산

Page 54: 1. Introductiondmqm.korea.ac.kr/uploads/seminar/Open Set Recognition... · 2020-01-03 · - 20 / 72 - Data 無 Label有 Knowns Unknowns s s Knowns Knowns (KKC) Knowns Unknowns (KUC)

- 54 / 72 -

Normal

Donut

Edge

Location

Center

Y

0

0

1

0

0

Loss 1

Loss 3

Loss 5

Loss 2

Loss 4

Total loss = σ𝑖=15 𝐿𝑜𝑠𝑠 𝑖

Page 55: 1. Introductiondmqm.korea.ac.kr/uploads/seminar/Open Set Recognition... · 2020-01-03 · - 20 / 72 - Data 無 Label有 Knowns Unknowns s s Knowns Knowns (KKC) Knowns Unknowns (KUC)

- 55 / 72 -

Normal

Donut

Edge

Location

Center

Y

0

0

1

0

0

Loss 1

Loss 3

Loss 5

Loss 2

Loss 4

Total loss = σ𝑖=15 𝐿𝑜𝑠𝑠 𝑖

Page 56: 1. Introductiondmqm.korea.ac.kr/uploads/seminar/Open Set Recognition... · 2020-01-03 · - 20 / 72 - Data 無 Label有 Knowns Unknowns s s Knowns Knowns (KKC) Knowns Unknowns (KUC)

- 56 / 72 -

Normal

Donut

Edge

Location

Center

Y

0

0

1

0

0

Loss 1

Loss 3

Loss 5

Loss 2

Loss 4

Total loss = σ𝑖=15 𝐿𝑜𝑠𝑠 𝑖

𝜃𝑜𝑟𝑖𝑔𝑖𝑛𝑎𝑙

𝜽DOC

Page 57: 1. Introductiondmqm.korea.ac.kr/uploads/seminar/Open Set Recognition... · 2020-01-03 · - 20 / 72 - Data 無 Label有 Knowns Unknowns s s Knowns Knowns (KKC) Knowns Unknowns (KUC)

- 57 / 72 -

Normal

Donut

Edge

Location

Center

Y

0

0

1

0

0

Loss 1

Loss 3

Loss 5

Loss 2

Loss 4

Total loss = σ𝑖=15 𝐿𝑜𝑠𝑠 𝑖

𝜃𝑜𝑟𝑖𝑔𝑖𝑛𝑎𝑙

Page 58: 1. Introductiondmqm.korea.ac.kr/uploads/seminar/Open Set Recognition... · 2020-01-03 · - 20 / 72 - Data 無 Label有 Knowns Unknowns s s Knowns Knowns (KKC) Knowns Unknowns (KUC)

- 58 / 72 -

Page 59: 1. Introductiondmqm.korea.ac.kr/uploads/seminar/Open Set Recognition... · 2020-01-03 · - 20 / 72 - Data 無 Label有 Knowns Unknowns s s Knowns Knowns (KKC) Knowns Unknowns (KUC)

- 59 / 72 -

1) Class i의예측확률 𝒑(𝒚 = 𝒍𝒊|𝒙𝒋, 𝒚𝒋=𝒍𝒊)이

가우시안분포의절반을따른다고가정

1

Page 60: 1. Introductiondmqm.korea.ac.kr/uploads/seminar/Open Set Recognition... · 2020-01-03 · - 20 / 72 - Data 無 Label有 Knowns Unknowns s s Knowns Knowns (KKC) Knowns Unknowns (KUC)

- 60 / 72 -

1) Class i의예측확률 𝒑(𝒚 = 𝒍𝒊|𝒙𝒋, 𝒚𝒋=𝒍𝒊)이

가우시안분포의절반을따른다고가정

1

2) 기존의점 𝒑(𝒚 = 𝒍𝒊|𝒙𝒋, 𝒚𝒋=𝒍𝒊)에대해

평균 “1”에미러링된 점을만든다

𝒎𝒊𝒓𝒓𝒐𝒓𝒆𝒅 𝒑𝒐𝒊𝒏𝒕 = 𝟏 + (𝟏 − 𝒑(𝒚 = 𝒍𝒊|𝒙𝒋, 𝒚𝒋=𝒍𝒊))

ex) 𝒑(𝒚 = 𝒍𝒊|𝒙𝒋, 𝒚𝒋=𝒍𝒊)→ 𝒎𝒊𝒓𝒓𝒐𝒓𝒆𝒅 𝒑𝒐𝒊𝒏𝒕 = 𝟏. 𝟐

Page 61: 1. Introductiondmqm.korea.ac.kr/uploads/seminar/Open Set Recognition... · 2020-01-03 · - 20 / 72 - Data 無 Label有 Knowns Unknowns s s Knowns Knowns (KKC) Knowns Unknowns (KUC)

- 61 / 72 -

1) Class i의예측확률 𝒑(𝒚 = 𝒍𝒊|𝒙𝒋, 𝒚𝒋=𝒍𝒊)이

가우시안분포의절반을따른다고가정

1

2) 기존의점 𝒑(𝒚 = 𝒍𝒊|𝒙𝒋, 𝒚𝒋=𝒍𝒊)에대해

평균 “1”에미러링된 점을만든다

𝒎𝒊𝒓𝒓𝒐𝒓𝒆𝒅 𝒑𝒐𝒊𝒏𝒕 = 𝟏 + (𝟏 − 𝒑(𝒚 = 𝒍𝒊|𝒙𝒋, 𝒚𝒋=𝒍𝒊))

ex) 𝒑(𝒚 = 𝒍𝒊|𝒙𝒋, 𝒚𝒋=𝒍𝒊)→ 𝒎𝒊𝒓𝒓𝒐𝒓𝒆𝒅 𝒑𝒐𝒊𝒏𝒕 = 𝟏. 𝟐

Page 62: 1. Introductiondmqm.korea.ac.kr/uploads/seminar/Open Set Recognition... · 2020-01-03 · - 20 / 72 - Data 無 Label有 Knowns Unknowns s s Knowns Knowns (KKC) Knowns Unknowns (KUC)

- 62 / 72 -

1) Class i의예측확률 𝒑(𝒚 = 𝒍𝒊|𝒙𝒋, 𝒚𝒋=𝒍𝒊)이

가우시안분포의절반을따른다고가정

1

2) 기존의점 𝒑(𝒚 = 𝒍𝒊|𝒙𝒋, 𝒚𝒋=𝒍𝒊)에대해

평균 “1”에미러링된 점을만든다

𝒎𝒊𝒓𝒓𝒐𝒓𝒆𝒅 𝒑𝒐𝒊𝒏𝒕 = 𝟏 + (𝟏 − 𝒑(𝒚 = 𝒍𝒊|𝒙𝒋, 𝒚𝒋=𝒍𝒊))

ex) 𝒑(𝒚 = 𝒍𝒊|𝒙𝒋, 𝒚𝒋=𝒍𝒊)→ 𝒎𝒊𝒓𝒓𝒐𝒓𝒆𝒅 𝒑𝒐𝒊𝒏𝒕 = 𝟏. 𝟐

Page 63: 1. Introductiondmqm.korea.ac.kr/uploads/seminar/Open Set Recognition... · 2020-01-03 · - 20 / 72 - Data 無 Label有 Knowns Unknowns s s Knowns Knowns (KKC) Knowns Unknowns (KUC)

- 63 / 72 -

1) Class i의예측확률 𝒑(𝒚 = 𝒍𝒊|𝒙𝒋, 𝒚𝒋=𝒍𝒊)이

가우시안분포의절반을따른다고가정

1

2) 기존의점 𝒑(𝒚 = 𝒍𝒊|𝒙𝒋, 𝒚𝒋=𝒍𝒊)에대해

평균 “1”에미러링된 점을만든다

𝒎𝒊𝒓𝒓𝒐𝒓𝒆𝒅 𝒑𝒐𝒊𝒏𝒕 = 𝟏 + (𝟏 − 𝒑(𝒚 = 𝒍𝒊|𝒙𝒋, 𝒚𝒋=𝒍𝒊))

ex) 𝒑(𝒚 = 𝒍𝒊|𝒙𝒋, 𝒚𝒋=𝒍𝒊)→ 𝒎𝒊𝒓𝒓𝒐𝒓𝒆𝒅 𝒑𝒐𝒊𝒏𝒕 = 𝟏. 𝟐

Page 64: 1. Introductiondmqm.korea.ac.kr/uploads/seminar/Open Set Recognition... · 2020-01-03 · - 20 / 72 - Data 無 Label有 Knowns Unknowns s s Knowns Knowns (KKC) Knowns Unknowns (KUC)

- 64 / 72 -

1) Class i의예측확률 𝒑(𝒚 = 𝒍𝒊|𝒙𝒋, 𝒚𝒋=𝒍𝒊)이

가우시안분포의절반을따른다고가정

1

2) 기존의점 𝒑(𝒚 = 𝒍𝒊|𝒙𝒋, 𝒚𝒋=𝒍𝒊)에대해

평균 “1”에미러링된 점을만든다

𝒎𝒊𝒓𝒓𝒐𝒓𝒆𝒅 𝒑𝒐𝒊𝒏𝒕 = 𝟏 + (𝟏 − 𝒑(𝒚 = 𝒍𝒊|𝒙𝒋, 𝒚𝒋=𝒍𝒊))

ex) 𝒑(𝒚 = 𝒍𝒊|𝒙𝒋, 𝒚𝒋=𝒍𝒊)→ 𝒎𝒊𝒓𝒓𝒐𝒓𝒆𝒅 𝒑𝒐𝒊𝒏𝒕 = 𝟏. 𝟐

Page 65: 1. Introductiondmqm.korea.ac.kr/uploads/seminar/Open Set Recognition... · 2020-01-03 · - 20 / 72 - Data 無 Label有 Knowns Unknowns s s Knowns Knowns (KKC) Knowns Unknowns (KUC)

- 65 / 72 -

1) Class i의예측확률 𝒑(𝒚 = 𝒍𝒊|𝒙𝒋, 𝒚𝒋=𝒍𝒊)이

가우시안분포의절반을따른다고가정

1

2) 기존의점 𝒑(𝒚 = 𝒍𝒊|𝒙𝒋, 𝒚𝒋=𝒍𝒊)에대해

평균 “1”에미러링된 점을만든다

𝒎𝒊𝒓𝒓𝒐𝒓𝒆𝒅 𝒑𝒐𝒊𝒏𝒕 = 𝟏 + (𝟏 − 𝒑(𝒚 = 𝒍𝒊|𝒙𝒋, 𝒚𝒋=𝒍𝒊))

ex) 𝒑(𝒚 = 𝒍𝒊|𝒙𝒋, 𝒚𝒋=𝒍𝒊)→ 𝒎𝒊𝒓𝒓𝒐𝒓𝒆𝒅 𝒑𝒐𝒊𝒏𝒕 = 𝟏. 𝟐

3) 기존점과생성된점을사용하여표준편차 (𝝈)를추정

𝝈

Page 66: 1. Introductiondmqm.korea.ac.kr/uploads/seminar/Open Set Recognition... · 2020-01-03 · - 20 / 72 - Data 無 Label有 Knowns Unknowns s s Knowns Knowns (KKC) Knowns Unknowns (KUC)

- 66 / 72 -

1) Class i의예측확률 𝒑(𝒚 = 𝒍𝒊|𝒙𝒋, 𝒚𝒋=𝒍𝒊)이

가우시안분포의절반을따른다고가정

1

2) 기존의점 𝒑(𝒚 = 𝒍𝒊|𝒙𝒋, 𝒚𝒋=𝒍𝒊)에대해

평균 “1”에미러링된 점을만든다

𝒎𝒊𝒓𝒓𝒐𝒓𝒆𝒅 𝒑𝒐𝒊𝒏𝒕 = 𝟏 + (𝟏 − 𝒑(𝒚 = 𝒍𝒊|𝒙𝒋, 𝒚𝒋=𝒍𝒊))

ex) 𝒑(𝒚 = 𝒍𝒊|𝒙𝒋, 𝒚𝒋=𝒍𝒊)→ 𝒎𝒊𝒓𝒓𝒐𝒓𝒆𝒅 𝒑𝒐𝒊𝒏𝒕 = 𝟏. 𝟐

3) 기존점과생성된점을사용하여표준편차 (𝝈)를추정

𝝈

4) 𝜽𝑫𝑶𝑪 𝒕𝒊 = 𝒎𝒂𝒙 𝟎. 𝟓 , 𝟏 − 𝜶 ∙ 𝝈𝒊

𝜶는일반적으로 3을사용

𝜽𝑫𝑶𝑪(𝒕𝒊) = 𝒎𝒂𝒙(𝟎. 𝟓 , 𝟏 − 𝜶 ∙ 𝝈𝒊)

Page 67: 1. Introductiondmqm.korea.ac.kr/uploads/seminar/Open Set Recognition... · 2020-01-03 · - 20 / 72 - Data 無 Label有 Knowns Unknowns s s Knowns Knowns (KKC) Knowns Unknowns (KUC)

- 67 / 72 -

일반적인 Sigmoid func.의임계값은 0.5이다

Train에 참가하지않은 Unseen 표본에대한확률을 고려必

DOC는잠재적인 Unseen 표본에대한확률을고려 하여임계값을재설정

각임계값은 Class 별로차등 적용

Page 68: 1. Introductiondmqm.korea.ac.kr/uploads/seminar/Open Set Recognition... · 2020-01-03 · - 20 / 72 - Data 無 Label有 Knowns Unknowns s s Knowns Knowns (KKC) Knowns Unknowns (KUC)

- 68 / 72 -

Normal

Donut

Edge

Location

Center

Page 69: 1. Introductiondmqm.korea.ac.kr/uploads/seminar/Open Set Recognition... · 2020-01-03 · - 20 / 72 - Data 無 Label有 Knowns Unknowns s s Knowns Knowns (KKC) Knowns Unknowns (KUC)

- 69 / 72 -

Data-set 1 Data-set 2

DOCU. News Review

Class 20 50

# of Samples(in 1 class)

1000 1000

Train60%

Test30%

Valid10%

Page 70: 1. Introductiondmqm.korea.ac.kr/uploads/seminar/Open Set Recognition... · 2020-01-03 · - 20 / 72 - Data 無 Label有 Knowns Unknowns s s Knowns Knowns (KKC) Knowns Unknowns (KUC)

- 70 / 72 -

Unknown을많이가지고있더라도성능저하가적다

DOC가 DOC(𝒕 = 𝟎. 𝟓)보다성능이높다

Page 71: 1. Introductiondmqm.korea.ac.kr/uploads/seminar/Open Set Recognition... · 2020-01-03 · - 20 / 72 - Data 無 Label有 Knowns Unknowns s s Knowns Knowns (KKC) Knowns Unknowns (KUC)

5. Conclusion

Page 72: 1. Introductiondmqm.korea.ac.kr/uploads/seminar/Open Set Recognition... · 2020-01-03 · - 20 / 72 - Data 無 Label有 Knowns Unknowns s s Knowns Knowns (KKC) Knowns Unknowns (KUC)

- 72 / 72 -

Page 73: 1. Introductiondmqm.korea.ac.kr/uploads/seminar/Open Set Recognition... · 2020-01-03 · - 20 / 72 - Data 無 Label有 Knowns Unknowns s s Knowns Knowns (KKC) Knowns Unknowns (KUC)

감사합니다.