Top Banner
Grammar of Image Zhaoyin Jia, 03-30-2009
62

Grammar of Image Zhaoyin Jia, 03-30-2009. Problems Enormous amount of vision knowledge: Computational complexity Semantic gap …… Classification,

Dec 22, 2015

Download

Documents

Rosalyn Webster
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: Grammar of Image Zhaoyin Jia, 03-30-2009. Problems  Enormous amount of vision knowledge:  Computational complexity  Semantic gap …… Classification,

Grammar of Image

Zhaoyin Jia, 03-30-2009

Page 2: Grammar of Image Zhaoyin Jia, 03-30-2009. Problems  Enormous amount of vision knowledge:  Computational complexity  Semantic gap …… Classification,

Problems Enormous amount of vision knowledge:

Computational complexity

Semantic gap

……

20 40 60 80 100 120 140 160

20

40

60

80

100

120

Classification,Recognition

Page 3: Grammar of Image Zhaoyin Jia, 03-30-2009. Problems  Enormous amount of vision knowledge:  Computational complexity  Semantic gap …… Classification,

Task of image parsing

Page 4: Grammar of Image Zhaoyin Jia, 03-30-2009. Problems  Enormous amount of vision knowledge:  Computational complexity  Semantic gap …… Classification,

Objectives in this paper Framework for vision

And-Or Graph

Algorithm for this framework Top-down/bottom-up computation

Generalization of small sample Use Monte Carlos simulation to

synthesis more configurations

Fill the semantic gap

Page 5: Grammar of Image Zhaoyin Jia, 03-30-2009. Problems  Enormous amount of vision knowledge:  Computational complexity  Semantic gap …… Classification,

Grammar Language: co-occurance of s is more than

chance

Image: Parallel; T-junction

( | )1

( | ) ( | )

p s A B

p s A p s B

CONSTANTINOPLE

Page 6: Grammar of Image Zhaoyin Jia, 03-30-2009. Problems  Enormous amount of vision knowledge:  Computational complexity  Semantic gap …… Classification,

Formulation of grammar Start symbol: S Non-terminal nodes: VN

Reproduction Rule: R

Terminal nodes: VT

Page 7: Grammar of Image Zhaoyin Jia, 03-30-2009. Problems  Enormous amount of vision knowledge:  Computational complexity  Semantic gap …… Classification,

Formulation of grammar Start symbol: S Non-terminal nodes: VN

Reproduction Rule: R

Terminal nodes: VT

Page 8: Grammar of Image Zhaoyin Jia, 03-30-2009. Problems  Enormous amount of vision knowledge:  Computational complexity  Semantic gap …… Classification,

Formulation of grammar Start symbol: S Non-terminal nodes: VN

Reproduction Rule: R

Terminal nodes: VTS NP VP

VP VP PPVP V NP

……

Page 9: Grammar of Image Zhaoyin Jia, 03-30-2009. Problems  Enormous amount of vision knowledge:  Computational complexity  Semantic gap …… Classification,

Formulation of grammar Start symbol: S Non-terminal nodes: VN

Reproduction Rule: R

Terminal nodes: VT

Page 10: Grammar of Image Zhaoyin Jia, 03-30-2009. Problems  Enormous amount of vision knowledge:  Computational complexity  Semantic gap …… Classification,

Formulation of grammar Start symbol: S Non-terminal nodes: VN

Reproduction Rule: R

Terminal nodes: VT

Page 11: Grammar of Image Zhaoyin Jia, 03-30-2009. Problems  Enormous amount of vision knowledge:  Computational complexity  Semantic gap …… Classification,

Image grammar Start symbol: S

Reproduction Rules

Non-terminal nodes: VN

Terminal nodes: VT

Page 12: Grammar of Image Zhaoyin Jia, 03-30-2009. Problems  Enormous amount of vision knowledge:  Computational complexity  Semantic gap …… Classification,

Overlapping parts/Ambiguity

Page 13: Grammar of Image Zhaoyin Jia, 03-30-2009. Problems  Enormous amount of vision knowledge:  Computational complexity  Semantic gap …… Classification,

Similar color, occlusion, etc.

Overlapping parts/Ambiguity

Page 14: Grammar of Image Zhaoyin Jia, 03-30-2009. Problems  Enormous amount of vision knowledge:  Computational complexity  Semantic gap …… Classification,

For each VN , we have reproduction rules:

with a probability associated with each one:

Probability of parsing tree:

Probability of sentence:

Stochastic Context Free Grammar

Page 15: Grammar of Image Zhaoyin Jia, 03-30-2009. Problems  Enormous amount of vision knowledge:  Computational complexity  Semantic gap …… Classification,

Stochastic Grammar with Context From left to right: bi-gram model (Markov

chain)a sentence with n words:

Non-local relations: tree model

Page 16: Grammar of Image Zhaoyin Jia, 03-30-2009. Problems  Enormous amount of vision knowledge:  Computational complexity  Semantic gap …… Classification,

New issues in Image Grammar Loss of “left to right” order: region adjacency

graph

Page 17: Grammar of Image Zhaoyin Jia, 03-30-2009. Problems  Enormous amount of vision knowledge:  Computational complexity  Semantic gap …… Classification,

New issues in Image Grammar Scaling makes different terminal in parsing

tree

Page 18: Grammar of Image Zhaoyin Jia, 03-30-2009. Problems  Enormous amount of vision knowledge:  Computational complexity  Semantic gap …… Classification,

New issues in Image Grammar Switch between texture and structure

Page 19: Grammar of Image Zhaoyin Jia, 03-30-2009. Problems  Enormous amount of vision knowledge:  Computational complexity  Semantic gap …… Classification,

Building the image grammar Visual Vocabulary:

primitives, sketch graph, textons… Relations and configurations:

co-occurance, attached, hinged, supported, occluded…

And-or Graph representationembedding image grammar

Learning /testing the parse graphfind the possible inference

Page 20: Grammar of Image Zhaoyin Jia, 03-30-2009. Problems  Enormous amount of vision knowledge:  Computational complexity  Semantic gap …… Classification,

Database Lotus Hill Institute Dataset

636,748 images, 3,927,130 Physical Objects

A few hundred are free

Benjamin Yao, Xiong Yang, and Song-Chun Zhu, “Introduction to a large scale general purpose ground truth dataset: methodology, annotation tool, and benchmarks.” EMMCVPR, 2007http://www.imageparsing.com/

Page 21: Grammar of Image Zhaoyin Jia, 03-30-2009. Problems  Enormous amount of vision knowledge:  Computational complexity  Semantic gap …… Classification,

Free Data

6 categories, 145 subsetsManmade Object 75 Nature Object 40 Objects in

Scene 6 Transportation 9 UCLA Aerial Image 5 UIUC Sport

Activity 10

Outline & segmentation of the object

http://yoshi.cs.ucla.edu/yao/data/

Page 22: Grammar of Image Zhaoyin Jia, 03-30-2009. Problems  Enormous amount of vision knowledge:  Computational complexity  Semantic gap …… Classification,

Free Data

6 categories, 145 subsetsManmade Object 75 Nature Object 40 Objects in

Scene 6 Transportation 9 UCLA Aerial Image 5 UIUC Sport

Activity 10

Segmentation of a scene (street)

http://yoshi.cs.ucla.edu/yao/data/

Page 23: Grammar of Image Zhaoyin Jia, 03-30-2009. Problems  Enormous amount of vision knowledge:  Computational complexity  Semantic gap …… Classification,

Free Data

6 categories, 145 subsetsManmade Object 75 Nature Object 40 Objects in

Scene 6 Transportation 9 UCLA Aerial Image 5 UIUC Sport

Activity 10

Physical parts of the object

http://yoshi.cs.ucla.edu/yao/data/

OBJECT1:truck

OBJECT1:truck PART1:truck:body PART2:truck:windshield PART3:truck:headlight PART4:truck:headlight PART5:truck:headlight PART6:truck:headlight

PART7:truck:rearview mirror PART8:truck:rearview mirror PART9:truck:rear light PART10:truck:window PART11:truck:frontal left wheel PART12:truck:frontal right wheel

PART13:truck:back wheel PART14:truck:back wheel PART15:truck:carriage

Page 24: Grammar of Image Zhaoyin Jia, 03-30-2009. Problems  Enormous amount of vision knowledge:  Computational complexity  Semantic gap …… Classification,

Visual Vocabulary The “Lego Land”

Language

Page 25: Grammar of Image Zhaoyin Jia, 03-30-2009. Problems  Enormous amount of vision knowledge:  Computational complexity  Semantic gap …… Classification,

Visual Vocabulary

: function of image primitives

: a) geometry transformation

b) appearance

: bond between each primitives

Page 26: Grammar of Image Zhaoyin Jia, 03-30-2009. Problems  Enormous amount of vision knowledge:  Computational complexity  Semantic gap …… Classification,

Visual Vocabulary Sketch and Texture

SK NSK SK NSKI I I

S. C. Zhu, Y. N. Wu, and D. B. Mumford, “Minimax entropy principle and its applications to texture modeling,” Neural Computation, vol. 9, no. 8, pp. 1627–1660, November 1997

Page 27: Grammar of Image Zhaoyin Jia, 03-30-2009. Problems  Enormous amount of vision knowledge:  Computational complexity  Semantic gap …… Classification,

Primal sketch model

Input image

Sketch graph

Texture pixelsC. E. Guo, S. C. Zhu, and Y. N. Wu, “Primal sketch: Integrating texture and structure,” in

Proceedings of International Conference on Computer Vision,2003.

Page 28: Grammar of Image Zhaoyin Jia, 03-30-2009. Problems  Enormous amount of vision knowledge:  Computational complexity  Semantic gap …… Classification,

Primal sketch model

C. E. Guo, S. C. Zhu, and Y. N. Wu, “Primal sketch: Integrating texture and structure,” in Proceedings of International Conference on Computer Vision,2003.

Page 29: Grammar of Image Zhaoyin Jia, 03-30-2009. Problems  Enormous amount of vision knowledge:  Computational complexity  Semantic gap …… Classification,

High level visual vocabulary Cloth: collar, left/right sleeves, hands

H. Chen, Z. J. Xu, Z. Q. Liu, and S. C. Zhu, “Composite templates for cloth modeling and sketching,” in Proceedings of IEEE Conference on Pattern Recognition and Computer Vision, New York, June 2006

Page 30: Grammar of Image Zhaoyin Jia, 03-30-2009. Problems  Enormous amount of vision knowledge:  Computational complexity  Semantic gap …… Classification,

Relations and configurations Definition of relation:

bonds:relations: , : structure, : compatibility

Three types of relations Bonds and connections Joints and junctions Object interactions/semantics

Definition of configurations:

{( , )}s t S S

{( , ; , ) : , }E s t s t S

,C V E { : ( ( , ; ), ) };i i i iV A A x y

Page 31: Grammar of Image Zhaoyin Jia, 03-30-2009. Problems  Enormous amount of vision knowledge:  Computational complexity  Semantic gap …… Classification,

Relations Bonds and connections

connects primitives into bigger graphs

intensity/color compatibility

{ , 1,2,..., , 1, 2,..., ( )}ijS i n j n i

( ) {( , ; , )}bond ij ijE S

( , , )x y

Page 32: Grammar of Image Zhaoyin Jia, 03-30-2009. Problems  Enormous amount of vision knowledge:  Computational complexity  Semantic gap …… Classification,

Relations Joint and junctions

Page 33: Grammar of Image Zhaoyin Jia, 03-30-2009. Problems  Enormous amount of vision knowledge:  Computational complexity  Semantic gap …… Classification,

Relations Object interactions

Page 34: Grammar of Image Zhaoyin Jia, 03-30-2009. Problems  Enormous amount of vision knowledge:  Computational complexity  Semantic gap …… Classification,

Configuration Spatial layout of entities at a certain level

Primal sketch – parts – object – scene

,C V E { : ( ( , ; ), ) };i i i iV A A x y

Page 35: Grammar of Image Zhaoyin Jia, 03-30-2009. Problems  Enormous amount of vision knowledge:  Computational complexity  Semantic gap …… Classification,

Reconfigurable graphs Treat bonds as random variables: address

nodes

Page 36: Grammar of Image Zhaoyin Jia, 03-30-2009. Problems  Enormous amount of vision knowledge:  Computational complexity  Semantic gap …… Classification,

Inference of the configuration Have the primal sketch of the image Detect the ‘T-junction’ Simulated annealing to infer the Gestalt Law

R. X. Gao and S. C. Zhu, “From primal sketch to 2.1D sketch,” Technical Report, Lotus Hill Institute, 2006

Red dot: connect region

Black line: known edge

Green line: inferred connection

Page 37: Grammar of Image Zhaoyin Jia, 03-30-2009. Problems  Enormous amount of vision knowledge:  Computational complexity  Semantic gap …… Classification,

Reconfigurable graphs

Ru-Xin Gao1, Tian-Fu Wu, Song-Chun Zhu, and Nong Sang, “Bayesian Inference for Layer Representation with Mixed Markov Random Field ”

Source image

T-junction

Inferred connection

Layer extractio

n

Page 38: Grammar of Image Zhaoyin Jia, 03-30-2009. Problems  Enormous amount of vision knowledge:  Computational complexity  Semantic gap …… Classification,

Reconfigurable graphs

R. X. Gao and S. C. Zhu, “From primal sketch to 2.1D sketch,” Technical Report, Lotus Hill Institute, 2006

Page 39: Grammar of Image Zhaoyin Jia, 03-30-2009. Problems  Enormous amount of vision knowledge:  Computational complexity  Semantic gap …… Classification,

And-Or Graph Parse graph of the image

pt: parse tree of vocabulary E: relations Inference the parse graph:

( , )pg pt E

* argmax ( | )pg p pg I

Z. J. Xu, L. Lin, T. F. Wu, and S. C. Zhu, “Recursive top-down/bottom up algorithm for object recognition,” Technical Report, Lotus Hill Research Institute, 2007.

Page 40: Grammar of Image Zhaoyin Jia, 03-30-2009. Problems  Enormous amount of vision knowledge:  Computational complexity  Semantic gap …… Classification,

Contain all the valid parse graphs

And node, Or node, leaf-node

Relation between children of And node

Parse tree: assigning label on Or node

And-Or Graph

Z. J. Xu, L. Lin, T. F. Wu, and S. C. Zhu, “Recursive top-down/bottom up algorithm for object recognition,” Technical Report, Lotus Hill Research Institute, 2007.

Page 41: Grammar of Image Zhaoyin Jia, 03-30-2009. Problems  Enormous amount of vision knowledge:  Computational complexity  Semantic gap …… Classification,

Definition: image primitives relations at all level : probability model defined on the And-Or

graph : valid configuration of terminal nodes

And-Or Graph, , , , ,and or N TG S V V R P

and orNV V V 1 2, ,...or And AndV V V 1 2| , ,...And Or Or

TV V V V

{ ( , ; ), )TV x y

{( , ; , )}m s t st stR E v v

P

Page 42: Grammar of Image Zhaoyin Jia, 03-30-2009. Problems  Enormous amount of vision knowledge:  Computational complexity  Semantic gap …… Classification,

Stochastic Model on And-Or graph Terminal (leaf) node: And-Or node: Set of links: Switch variable at Or-node: Attributes of primitives:

( )T pg

( ), ( )or andV pg V pg

( )E pg

1( ; , , ) exp( ( ))

( )p pg R pg

Z

( ) ( ) ( )

( , ) ( )

( ) ( ( )) ( ( ))

( , , , )

Or andv t

v V pg v V pg T pg

ij i j ij iji j E pg

pg w v t

v v

( )w t

( )t

Page 43: Grammar of Image Zhaoyin Jia, 03-30-2009. Problems  Enormous amount of vision knowledge:  Computational complexity  Semantic gap …… Classification,

Stochastic Model on And-Or graph Terminal (leaf) node: And-Or node: Set of links: Switch variable at Or-node: Attributes of primitives:

( )T pg

( ), ( )or andV pg V pg

( )E pg

1( ; , , ) exp( ( ))

( )p pg R pg

Z

( ) ( ) ( )

( , ) ( )

( ) ( ( )) ( ( ))

( , , , )

Or andv t

v V pg v V pg T pg

ij i j ij iji j E pg

pg w v t

v v

( )w t

( )t

SCFG: weigh the frequency at the children of or-nodes

Page 44: Grammar of Image Zhaoyin Jia, 03-30-2009. Problems  Enormous amount of vision knowledge:  Computational complexity  Semantic gap …… Classification,

Stochastic Model on And-Or graph Terminal (leaf) node: And-Or node: Set of links: Switch variable at Or-node: Attributes of primitives:

( )T pg

( ), ( )or andV pg V pg

( )E pg

1( ; , , ) exp( ( ))

( )p pg R pg

Z

( ) ( ) ( )

( , ) ( )

( ) ( ( )) ( ( ))

( , , , )

Or andv t

v V pg v V pg T pg

ij i j ij iji j E pg

pg w v t

v v

( )w t

( )t

Weigh the local compatibility of primitives (geometric and appearance)

Page 45: Grammar of Image Zhaoyin Jia, 03-30-2009. Problems  Enormous amount of vision knowledge:  Computational complexity  Semantic gap …… Classification,

Stochastic Model on And-Or graph Terminal (leaf) node: And-Or node: Set of links: Switch variable at Or-node: Attributes of primitives:

( )T pg

( ), ( )or andV pg V pg

( )E pg

1( ; , , ) exp( ( ))

( )p pg R pg

Z

( ) ( ) ( )

( , ) ( )

( ) ( ( )) ( ( ))

( , , , )

Or andv t

v V pg v V pg T pg

ij i j ij iji j E pg

pg w v t

v v

( )w t

( )t

Spatial and appearance between primitives (parts or objects)

Page 46: Grammar of Image Zhaoyin Jia, 03-30-2009. Problems  Enormous amount of vision knowledge:  Computational complexity  Semantic gap …… Classification,

Learning And-Or Graph

Learning the vocabulary Learning the relation set R, given Learning the parameters , given R and

1( ; , , ) exp( ( ))

( )p pg R pg

Z

( , ) ( )( ) ( ) ( )

( ) ( ( )) ( ( )) ( , , , )Or and

v t ij i j ij iji j E pgv V pg v V pg T pg

pg w v t v v

Page 47: Grammar of Image Zhaoyin Jia, 03-30-2009. Problems  Enormous amount of vision knowledge:  Computational complexity  Semantic gap …… Classification,

Learning And-Or Graph

Learning the vocabulary , and hierarchic And-Or Graph

Learning the relation set R, given Learning the parameters , given R and

1( ; , , ) exp( ( ))

( )p pg R pg

Z

( , ) ( )( ) ( ) ( )

( ) ( ( )) ( ( )) ( , , , )Or and

v t ij i j ij iji j E pgv V pg v V pg T pg

pg w v t v v

Discussed in the paper

Page 48: Grammar of Image Zhaoyin Jia, 03-30-2009. Problems  Enormous amount of vision knowledge:  Computational complexity  Semantic gap …… Classification,

Learning And-Or Graph

Learning and Pursuing Relation Set R: Start from Stochastic

Context Free Graph (a)

Learn the relations that maximally reduce the KL divergence to the observation (b-e)

( , )f I pgObservation:

Learning model:

( ; , , )p pg R

J. Porway, Z. Y. Yao, and S. C. Zhu, “Learning an And–Or graph for modeling and recognizing object categories,” Technical Report, Department of Statistics,2007

Page 49: Grammar of Image Zhaoyin Jia, 03-30-2009. Problems  Enormous amount of vision knowledge:  Computational complexity  Semantic gap …… Classification,

Learning graph parameter Approximating to Similar to texture synthesis

( ; , , )p pg R ( , )f I pg

S. C. Zhu, Y. N. Wu, and D. B. Mumford, “Minimax entropy principle and its applications to texture modeling,” Neural Computation, vol. 9, no. 8, pp. 1627–1660, November 1997

Learning And-Or Graph

Page 50: Grammar of Image Zhaoyin Jia, 03-30-2009. Problems  Enormous amount of vision knowledge:  Computational complexity  Semantic gap …… Classification,

Case I: Rectangle Nodes: Rectangle

Two vanishing points, four edge direction

Rules:

F. Han and S. C. Zhu, “Bottom-up/top-down image parsing by attribute graph grammar”. Proceedings of International Conference on Computer Vision, Beijing,China, 2005.

Page 51: Grammar of Image Zhaoyin Jia, 03-30-2009. Problems  Enormous amount of vision knowledge:  Computational complexity  Semantic gap …… Classification,

Case I: Rectangle Get the primal sketch of the

scene

Find the ‘strong’ rectangular (bottom-up, red)

Weigh (score) different hypothesis (top-down, blue) Weight is the compatibility of the

image with the proposed rectangular (primal-sketch)

Accept the best one

Do the previous 3 steps until all the weigh is small. (negative)F. Han and S. C. Zhu, “Bottom-up/top-down image parsing by attribute graph grammar”. Proceedings of International Conference on Computer Vision, Beijing,China, 2005.

2

( , )

2

(( ( , ) ( , ))

~ exp( )2

kx y

I x y B x y

Page 52: Grammar of Image Zhaoyin Jia, 03-30-2009. Problems  Enormous amount of vision knowledge:  Computational complexity  Semantic gap …… Classification,

Case I: Rectangle Inference process

Page 53: Grammar of Image Zhaoyin Jia, 03-30-2009. Problems  Enormous amount of vision knowledge:  Computational complexity  Semantic gap …… Classification,

Case I: Rectangle

F. Han and S. C. Zhu, “Bottom-up/top-down image parsing by attribute graph grammar”. Proceedings of International Conference on Computer Vision, Beijing,China, 2005.

Page 54: Grammar of Image Zhaoyin Jia, 03-30-2009. Problems  Enormous amount of vision knowledge:  Computational complexity  Semantic gap …… Classification,

Case II: Human Cloth Use And-Or graph to generate a matching

model

Vocabulary (training dataset)

Matching using the And-

or Graph

Matching using the And-

or Graph

Page 55: Grammar of Image Zhaoyin Jia, 03-30-2009. Problems  Enormous amount of vision knowledge:  Computational complexity  Semantic gap …… Classification,

Case II: Human Cloth The And-Or

Graph

H. Chen, Z. J. Xu, Z. Q. Liu, and S. C. Zhu, “Composite templates for cloth modeling and sketching,” in Proceedings of IEEE Conference on Pattern Recognition and Computer Vision, New York, June 2006.

Novel Configuration

Page 56: Grammar of Image Zhaoyin Jia, 03-30-2009. Problems  Enormous amount of vision knowledge:  Computational complexity  Semantic gap …… Classification,

Inference process

Case II: Human Cloth

H. Chen, Z. J. Xu, Z. Q. Liu, and S. C. Zhu, “Composite templates for cloth modeling and sketching,” in Proceedings of IEEE Conference on Pattern Recognition and Computer Vision, New York, June 2006.

Localize face, then estimate the parts of the body

Bottom-up: a coarse matching of the parts

Top-down: refine the matching using the relation

Page 57: Grammar of Image Zhaoyin Jia, 03-30-2009. Problems  Enormous amount of vision knowledge:  Computational complexity  Semantic gap …… Classification,

Case II: Human Cloth Inference result

H. Chen, Z. J. Xu, Z. Q. Liu, and S. C. Zhu, “Composite templates for cloth modeling and sketching,” in Proceedings of IEEE Conference on Pattern Recognition and Computer Vision, New York, June 2006.

Page 58: Grammar of Image Zhaoyin Jia, 03-30-2009. Problems  Enormous amount of vision knowledge:  Computational complexity  Semantic gap …… Classification,

Case II: Human Cloth Inference result

H. Chen, Z. J. Xu, Z. Q. Liu, and S. C. Zhu, “Composite templates for cloth modeling and sketching,” in Proceedings of IEEE Conference on Pattern Recognition and Computer Vision, New York, June 2006.

Hands are not exactly the same: find the best matching in the dataset

Page 59: Grammar of Image Zhaoyin Jia, 03-30-2009. Problems  Enormous amount of vision knowledge:  Computational complexity  Semantic gap …… Classification,

Case III: RecognitionZ. J. Xu, L. Lin, T. F. Wu, and S. C. Zhu, “Recursive top-down/bottomup algorithm for object recognition,” Technical Report, Lotus Hill Research Institute, 2007.

Page 60: Grammar of Image Zhaoyin Jia, 03-30-2009. Problems  Enormous amount of vision knowledge:  Computational complexity  Semantic gap …… Classification,

Conclusion Enormous amount of vision knowledge: (Add-Or graph)

……

Page 61: Grammar of Image Zhaoyin Jia, 03-30-2009. Problems  Enormous amount of vision knowledge:  Computational complexity  Semantic gap …… Classification,

Conclusion Computational complexity :

Remain open for scheduling bottom-up/top-down procedure

Semantic Gap Learning the And-Or Graph Learning the vocabulary , and its attributes

After all, we are not supposed to define so many things:

ideal vision words:

what we have now:

20 40 60 80 100 120 140 160

20

40

60

80

100

120

Page 62: Grammar of Image Zhaoyin Jia, 03-30-2009. Problems  Enormous amount of vision knowledge:  Computational complexity  Semantic gap …… Classification,

Thank you

Zhaoyin Jia