Complex Networks for Representation and Characterization of Images For CS790g Project Bingdong Li 9/23/2009.

Complex Networks for Representation and

Characterization of Images

For CS790g ProjectBingdong Li9/23/2009

Outline• Background• Motivation• Current States (CS):

– Representation– Characterization

Using examples from – Backes, Casanova, and Bruno’s Approach using local information – Kim, Faloutsos and Hebert’s Approach using global information

• Comparison of Two Approaches• Summary• Questions and Comments

Background: Complex Network

Source: cs790: complex network lecture

Background: Image

Source: CS674 Image Processing Lecture

Background: Image Processing

Source: CS674 Image Processing Lecture

Background: Image Representation

Source: CS674 Image Processing Lecture

Outline

• Background • Motivation

Motivation

• Belief:

• Computer vision is one of the most difficult problem remains, how can we represent and characterize image in the way of complex network so that we analysis it?

For a given problem, if it can be described in the

way of mathematics, it is half way to solve

the problem.

Outline

• Background • Motivation• Current States (CS):

– Representation– Characterization

Using examples from – Backes, Casanova, and Bruno’s Approach using

local information

CS: Backes’ Approach

• Construction of graph, – Vertices: points of shape boundary are modeled as

fully connected network,– Weight: the Euclidean distance d– through a sequential thresholds Tl (d< Tl), the fully

connected network becomes a dynamic complex network, the topological features of the growth of the dynamic network are used as a shape descriptor (or signature)

CS: Backes’ Approach

CS: Backes’ Approach

• Properties of the complex network– High clustering coefficient– The small world property

CS: Backes’ Approach

CS: Backes’ Approach

• Dynamic evolution signature• F: T T where

Tini and TQ, respectively, the initial and final threshold

CS: Backes’ Approach

• Characterization– Degree descriptor

kμ average degree, Kk max degree

CS: Backes’ Approach

• Evolution by a threshold T=0.1, .15, .20

CS: Backes’ Approach

Process of extraction of degree descriptor from an Image

CS: Backes’ Approach

• Advantage of Degree Descriptors– Rotation and scale inveriance– Noise tolerance– Robustness

CS: Backes’ Approach

Representation of rotate invariance

CS: Backes’ Approach

Representation of scale invariance

CS: Backes’ Approach

• Characterization– Joint Degree descriptor

Is the concatenation of the entropy(H), energy(E), and average joint degree(P) at each instant threshold T

CS: Backes’ Approach

• Advantage of Joint Degree Descriptors– Rotation and scale inveriance– Noise tolerance– Robustness

– Normalization of vertex is irrelevant because the joint degree concerns the probability distribution P(ki,k’)i

CS: Backes’ Approach

CS: Backes’ Approach

CS: Backes’ Approach

CS: Backes’ Approach

CS: Backes’ Approach

CS: Backes’ Approach

• Weakness of Backe’s Approach:– Initial and final threshold

Outline

• Background• Motivation• Current States (CS):

– Representation– Characterization

Using examples from – Backes, Casanova, and Bruno’s Approach using local

information – Kim, Faloutsos and Hebert’s Approach using global

information

CS: Kim’s Approach

• Construct Visual Similarity Network (VSN)– Vertices (V): features of from training images– Edges (E): link features that matched across

images– Weights (W): consistence of correspondence with

all other correspondences in matching image Ia and Ib

VSN = (V, E, W)

CS: Kim’s Approach

• Construction of VSN– Vertices: can be any unit of local visual

information. In this approach, features detected using Harris-Affine point detector and the SIFT descriptor

CS: Kim’s Approach

• Construction of VSN– Edges: established between features in different

images. • Spectral matching algorithm is used to each pair of

image (Ia, Ib)

• A new edge is established between feature ai and bj

CS: Kim’s Approach

CS: Kim’s Approach

CS: Kim’s Approach

• Construction of VSN– Edge weights

– M n*n is a spare weight matrix, M(ai , bj) is the weight value

A small part of VSN

CS: Kim’s Approach

• Characterization– Ranking of information

• Remove noisy• Measure the importance

P is the PageRank vector

CS: Kim’s Approach

• Characterization– Structural similarity“similar nodes are highly likely to exhibit similar link

structures in the graph” p.4The similarity is computed by using link analysis

algorithm

CS: Kim’s Approach

• CharacterizationLink analysis algorithmGiven a VSN G, a node ai , the neighborhood

subgraph Gai either pointed to ai or point to by ai M, the adjacency matrix of G ai.

CS: Kim’s Approach

The left image is extracted features, the right image shows top20% high-ranked features

CS: Kim’s Approach

• Weakness of Kim’s Approach– Using threshold in computing edge weights– Mystery constant α =0.1– Category partition to pre-determined K groups– The difference of objects appearance in the

training data set is too big, make the conclusion weak

Outline

• Background• Motivation• Current States (CS):• Comparison of Two Approaches

Comparison• Backes’s Approach

– Unsupervised approach– using local information – Dynamic complex network– More task on complex network, less work on image

processing• Kim’s Approach

– Supervised approach– using global information– Static complex network– More work on image processing, less work on complex

network• Both using threshold, but Backe’s approach based on

initial and final value,

Outline

• Background• Motivation• Current States (CS):• Comparison of Two Approaches• Summary

Summary

• In both approaches using complex network for representation and characterization of image,– provide a unique way for object classification and

analysis, – present better results than traditional and state-

of-art methods, – demonstrate the potential of complex network

analysis to computer vision.

Questions and Comments

Thanks