Towards Open World Recognition Abhijit Bendale, Terrance Boult University of Colorado of Colorado Springs Poster no 85.

Towards Open World Recognition

Abhijit Bendale, Terrance BoultUniversity of Colorado of Colorado Springs

Poster no 85

2

Multi-Class Classification System

W Scheirer, A Rocha, A Sapkota, T Boult “Towards Open Set Recognition” IEEE TPAMI 2013

3

Out in the Real-WorldDetect New Category

Pool Table

Bowling Pin

Boxing glove

Calculator

Chess board

?

?

?

?

4

Open World Recognition

• World with Knowns (K) &Unknowns Unknowns (UU)

Detect as Unknown

• NU: Novel Unknowns

Label Data• LU: Labeled

Unknowns

Incremental Class Learning

• K: Known

Scale

5

Open Set Learning

Incremental Learning

Scalable Learning

Ristin+ (CVPR’14)

Yeh+ (CVPR’08)

Li+ CVPR’07)

Mensink+(PAMI’13)

Related Work

Jain+ (ECCV’14)

Scheirer+ PAMI’13)

Scheirer+(PAMI’14)

Deng+ (NIPS’11)

Marszalek+(ECCV’08)

Liu+ (CVPR’13)

Open World Recognition

6

Open Space in Classification

??

?

?

?

?

? ? ??

??

??

Open Set Recognition Empirical risk

functionRegularization constant

Open space risk

W Scheirer, A Rocha, A Sapkota, T Boult “Towards Open Set Recognition” IEEE TPAMI 2013

Closed SpaceOpen Space

7

NCM – Metric Learning

NCM Classifier with Metric Learning

T Mensink, J Verbeek, F Perronin, G Csurka “Distance based Image Classification: Generalizing to New Classes at Near Zero Cost” IEEE TPAMI 2013M Ristin, M Guillaumin, J Gall, L Van Gool “Incremental Learning of NCM Forests for Large-Scale Image Classification” CVPR 2014

8

Compact Abating Probability (CAP) Models

W Scheirer, L Jain, T Boult “Probability Models for Open Set Recognition” IEEE TPAMI 2014

Class Mean

9

Theorem 1: Open Space Risk for Model Combination

10

Theorem 2: Open Space Risk for Transformed Spaces

11

Opening an Existing Algorithm:Nearest Non-Outlier (NNO) Algorithm

W = Linear Transformation (weight matrix from metric learning)

Standard gamma functionIn volume of m-D ball Class mean for class i

τ is threshold for open world

Probability

12

Open World Evaluation

Training phase

Testing phase

Parameter Learning Phase Incremental Learning Phase

Closed Set TestingUnknown Categories

Known Categories

Open Set Testing

? ?

13

Training for Open World

NCM - ML NNO

• Parameter Learning with initial set of categories

• Estimation of τ for open set learning to balance open space risk• Optimize for Known vs Unknown Errors• Incrementally add new categories

14

ExperimentsDatasets

• ILSVRC’10: 1.2M training images, 1000 classes• ILSVRC’12: 1.2M training images, 1000 classes

Features• Dense SIFT features, Quantized into 1000 Bag of Visual Words• Publically available features• LBP, HOG, Dense SIFT (for ILSVRC’12)

Algorithms• Nearest Class Mean - ML Classifier (NCM) [Mensink etal PAMI 2013]• Nearest Non-Outlier Algorithm (NNO) [This Paper]• 1vSet [Scheirer etal PAMI 2013]• Linear SVM [Liblinear, Fan etal JMLR 2008]

50 Initial Categories

Increasing # of unknown categories during testing i.e. increasing openness of problem

Incrementally addingcategories during training

Closed Settesting

16

200 Initial Categories

Increasing # of unknown categories during testing i.e. increasing openness of problem

Incrementally addingcategories during training

Closed Settesting

500 known + 500 unknown categories

17

• Formalized Open World Recognition and showed how to “Open” an existing algorithm.

• NNO allows construction of scalable systems that can be updated incrementally

Conclusion & Future Work

See us at Poster no 85 …!!!!

• Exploring sophisticated novelty detectors, open world detection, “opening” other baseline algorithms etc.

• Open World Deep Learning methods• Happy to Collaborate…!!!