Top Banner
Roles of local and global information in image classification Devi Parikh
12

Roles of local and global information in image classification Devi Parikh.

Dec 16, 2015

Download

Documents

Ada Anderson
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: Roles of local and global information in image classification Devi Parikh.

Roles of local and global information in image classification

Devi Parikh

Page 2: Roles of local and global information in image classification Devi Parikh.

Coast Highway Mountain Street

Forest Inside city Open country Tall buildings

Bathroom Bedroom Dining room Gym

Kitchen Living room Movie theater Stair case

Aeroplane Car-rear Face

Ketch Motorbike Watch

OSR

ISR

CAL

2

Page 3: Roles of local and global information in image classification Devi Parikh.

Jumbled Images

3

Page 4: Roles of local and global information in image classification Devi Parikh.

Human Study: Intact Images

Page 5: Roles of local and global information in image classification Devi Parikh.

Human Study: Jumbled Images

Page 6: Roles of local and global information in image classification Devi Parikh.

Human Recognition of Jumbled Images

OSR ISR CAL0

10

20

30

40

50

60

70

80

90

100

Intact

JumbledAccu

racy

6

Page 7: Roles of local and global information in image classification Devi Parikh.

How do we recognize them?

Majority Vote of Local Independent

Decisions?

MRF Inference?

Are existing machine implementations of this model good enough?

7

Page 8: Roles of local and global information in image classification Devi Parikh.

Human Studies: Individual Blocks

8

Page 9: Roles of local and global information in image classification Devi Parikh.

Human Recognition of Individual Blocks

OSR ISR CAL25

30

35

40

45

50

Accu

racy

9

Page 10: Roles of local and global information in image classification Devi Parikh.

Machine Experiments

• Assign blocks to codewords

• Accumulate p(scene|codewords)

• Pick class with highest votes

Page 11: Roles of local and global information in image classification Devi Parikh.

Majority Vote ResultsPr

opor

tion

of m

atch

ed re

spon

ses

OSR ISR CAL0

10

20

30

40

50

60

70

80

90

100

Random

Human Most Confident Block

Human Majority-Vote

Machine Majority-Vote

Inter-Human Agreement

11

Page 12: Roles of local and global information in image classification Devi Parikh.

Classification of Jumbled Images

OSR ISR CAL0

10

20

30

40

50

60

70

80

90

100

Human

Machine

Accu

racy

Need advanced modeling of global information12