Top Banner
Content Based Image Organization CS491 Spring 2006 Prof. Chengyu Sun Kelly Breed
17

Content Based Image Organization CS491 Spring 2006 Prof. Chengyu Sun Kelly Breed.

Dec 22, 2015

Download

Documents

Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: Content Based Image Organization CS491 Spring 2006 Prof. Chengyu Sun Kelly Breed.

Content Based Image Organization

CS491 Spring 2006Prof. Chengyu Sun

Kelly Breed

Page 2: Content Based Image Organization CS491 Spring 2006 Prof. Chengyu Sun Kelly Breed.

Overview of FindPic

What it does

How it works (from a user standpoint)

Modes

– grayscale– quantized color– texture for both

Page 3: Content Based Image Organization CS491 Spring 2006 Prof. Chengyu Sun Kelly Breed.

Technology

Objective-C

Cocoa framework

Local application

Self Organizing Maps

– T. Kohonen– 1980s

Page 4: Content Based Image Organization CS491 Spring 2006 Prof. Chengyu Sun Kelly Breed.

Technique: SOM Overview

Gather image data (feature vectors)

Create matrix (with same size vectors)

For each image find best match in matrix

Change matrix node and surrounding nodes

Reduce neighborhood size and change factor

Re-run for a number of epochs

Page 5: Content Based Image Organization CS491 Spring 2006 Prof. Chengyu Sun Kelly Breed.

Technique: Image Data

Reduction

Features

– e.g. grayscale histogram

Storage

Page 6: Content Based Image Organization CS491 Spring 2006 Prof. Chengyu Sun Kelly Breed.

Setting up the Map

2D Array of matrix nodes

Same feature vectors

Initialized to random values

Mapping begins

Page 7: Content Based Image Organization CS491 Spring 2006 Prof. Chengyu Sun Kelly Breed.

Finding the Best Match

Euclidean distance between the image (input) vectors and the matrix vectors

D mV 0 iV 0 2 mV 1 iV 1 2 ... mV n 1 iV n 1 2

Page 8: Content Based Image Organization CS491 Spring 2006 Prof. Chengyu Sun Kelly Breed.

Technique: Mapping

Find best match and change that node

Page 9: Content Based Image Organization CS491 Spring 2006 Prof. Chengyu Sun Kelly Breed.

Technique: Mapping

Change the surrounding nodes

Page 10: Content Based Image Organization CS491 Spring 2006 Prof. Chengyu Sun Kelly Breed.

Technique: Mapping

Change the surrounding nodes

Page 11: Content Based Image Organization CS491 Spring 2006 Prof. Chengyu Sun Kelly Breed.

Making Changes

After best match is found change the matrix node is made to be more like the input node

e.g. matrixVector[0] = 50, imageVector[0] = 30, cf = 0.9

matrixVector[0] = 50 – 0.9(50 – 30) = 32

e.g. matrixVector[0] = 30, imageVector[0] = 50, cf = 0.9

matrixVector[0] = 30 – 0.9(30 – 50) = 48

matrixVector i matrixVector i cf matrixVector i imageVector i

Page 12: Content Based Image Organization CS491 Spring 2006 Prof. Chengyu Sun Kelly Breed.

Finding the Neighbors

Inverse parabola

Filters for neighbors

Also determines change factor

Page 13: Content Based Image Organization CS491 Spring 2006 Prof. Chengyu Sun Kelly Breed.

Change Over Time

Inverse parabola determines neighborhood and change factor

yx2

Mt Where M=matrix dimension and t = time

Page 14: Content Based Image Organization CS491 Spring 2006 Prof. Chengyu Sun Kelly Breed.

Sample Neighborhood

Best Matching unit is set to position (1,1) with t = 1

0.8 0.9 0.8 0.5 0.0 -0.7 0.9 1.0 0.9 0.6 0.1 -0.6 0.8 0.9 0.8 0.5 0.0 -0.7 0.5 0.6 0.5 0.2 -0.3 -1.0 0.0 0.1 0.0 -0.3 -0.8 -1.5-0.7 -0.6 -0.7 -1.0 -1.5 -2.2

Page 15: Content Based Image Organization CS491 Spring 2006 Prof. Chengyu Sun Kelly Breed.

Sample Neighborhood

Best Matching unit is set to position (1,1) with t = 0.3

0.1 0.2 0.1 -0.2 -0.7 -1.4 0.2 0.3 0.2 -0.1 -0.6 -1.3 0.1 0.2 0.1 -0.2 -0.7 -1.4-0.2 -0.1 -0.2 -0.5 -1.0 -1.7-0.7 -0.6 -0.7 -1.0 -1.5 -2.2-1.4 -1.3 -1.4 -1.7 -2.2 -2.9

Page 16: Content Based Image Organization CS491 Spring 2006 Prof. Chengyu Sun Kelly Breed.

End Results

Images with similar feature vectors should be grouped into the same matrix cells.

Surrounding cells should also contain images that are similar.

Page 17: Content Based Image Organization CS491 Spring 2006 Prof. Chengyu Sun Kelly Breed.

Conclusions

SOM results compared to human categorized results

Determining closeness of match

Average percentages

– Grayscale: 41.46% – Quantized color: 55.14%– Grayscale with texture: 50.01%– Quantized with texture: 56.03%