Content Based Image Retrieval

CONTENT BASED IMAGE RETRIEVAL

SUYOG DUTT JAIN

Dept. of CSE

Manipal Institute of Technology

Presentation Outline

Objective of the seminar

Basics of Content Based Image Retrieval

Existing Systems

Algorithms based on the feature – color

Applications

Conclusion

References

Objective of the seminar

To throw light upon research in Image Processing

Discussion of Content Based Image Retrieval [CBIR]

Detailed discussion of some basic but very efficient algorithms for Image Retrieval

Basics of Content Based Image Retrieval

Definition

Retrieval of images based on visual features such as color, texture and shape.

Steps Involved:

(i) Feature Extraction (ii) Feature Comparison

Challenging Tasks:

(i) Maintaining and Searching through database

(ii) Formulation of exact query

(iii) Evaluation of close results

Basics of Content Based Image Retrieval : Retrieval Methods Color

Feature Extraction : Color Intensities

Feature Comparison: Color Histograms

Texture

Feature Extraction : Relative Brightness etc.

Feature Comparison: Degree of Contrast etc.

Shape

Feature Extraction : Aspect Ratio, Local Features

Feature Comparison: Directional Histograms

Basics of Content Based Image Retrieval: Architecture

Basics of Content Based Image Retrieval : Concepts

Digital Image

Pixel

Image Quantization

Color Histogram

Grayscale Images

Conversion of Color Image to Grayscale Image

Basics of Content Based Image Retrieval : Query Format

Query by example Using a test image

Query by low level features Using a sketch drawn by user

Existing Systems

QBIC

VIRAGE

EXCALIBUR

Algorithm Color Histogram Matching : Basics

The histogram of a digital image with gray levels in the range 0…L-1 is given by a discrete function

Hist (rk) = nk

Where,

rk is the kth gray level

k=0 …L-1,

L is the number of intensity levels.

nk = number of pixels at gray level rk.

Algorithm Color Histogram Matching : Feature ExtractionFor height of bitmap to 0

For 0 to width of bitmap

Read pixel

if pixel value=ith gray level

increment (histogram at ith gray level)

else

continue

Algorithm Color Histogram Matching : Feature Matching

255

histdist [dataset] = Σ |hist_database[j]-hist_query[j]|

j=0

where, 

j denotes the various gray levels

hist_query is the histogram of query image,

hist_database is histogram of the database image.

histdist is error difference or distance metric.

The nearest matching database images with the query image has the least distance metric. The exact match is the one with the zero distance metric.

Algorithm Bit Plane Histogram Matching : Basics Image is composed of eight one bit planes,

bit plane 0 for the Least Significant Bit ..

bit plane 7 for Most Significant Bit.

Visually significant information is contributed by higher order bits and least significant bits contribute less.

When total number of pixels over the entire image is counted in which a particular ith bit is set to 1, it gives the ith bit plane Histogram

Algorithm Bit Plane Histogram Matching : Feature Extraction

Pixel

0th bit hist

7th bit set

6th bit set

0th bit set

7th bit hist

6th bit hist

yes

yes

yes

no

no

Algorithm Bit Plane Histogram Matching : Feature ExtractionFor height of bitmap to 0

For 0 to width of bitmap

Read pixel

if pixel value at ith bit plane is 1

increment (histogram [ith bit plane])

else

continue

Algorithm Bit Plane Histogram Matching : Feature Matching Bit plane histogram is computed for the query image

Distance between histograms of database images and query image is as shown:

hist (|i|, database) =|hist_database[i]-hist_query[i] | i=7, 6...0

hist_database[i] and hist_query[i] are the bit plane histograms i.e. number of pixels in the image having ith bit as 1.

Algorithm Hierarchical Bit Plane Histogram Matching

Step1: Compute bit plane histogram error difference at 7th bit plane between query feature and n number of database features. Apply threshold and reduce.

Step2: Computing 6th bit plane histogram error difference on images obtained in step 1.

…………..

Continue till 0th bit plane.

One arithmetic operation per bit plane

Needs less computational time and power

Algorithm Spatial Histogram Matching : Basics It is common that the major object in an image is

located in the central position

Image is divided in to sub-regions that are of equal-size in terms of percentile area

The color histograms for each sub-region are computed for all the database images and query image

Sub Region 1

Sub Region 2Sub Region 3Sub Region 4

Algorithm Spatial Histogram Matching : Defining Regionsclass Point

{ int x,y;

Point()

{x=0;y=0;}}

class Region

{ Point tl=new Point();

Point tr=new Point();

Point bl=new Point();

Point br=new Point();

Region()

{

tl.x=0;tl.y=0;

tr.x=0;tr.y=0;

bl.x=0;bl.y=0;

br.x=0;br.y=0;

}

};

Algorithm Spatial Histogram Matching : Defining Regionspublic void defineRegion()

{ int w1=iw/8;

int h1=ih/8;

//Region 1

r1.tl.x=w1*3;r1.tl.y=h1*3;

r1.tr.x=w1*5;r1.tr.y=h1*3;

r1.bl.x=w1*3;r1.bl.y=h1*5;

r1.br.x=w1*5;r1.br.y=h1*5;

//Region 2

r2.tl.x=w1*2;r2.tl.y=h1*2;

r2.tr.x=w1*6;r2.tr.y=h1*2;

r2.bl.x=w1*2;r2.bl.y=h1*6;

r2.br.x=w1*6;r2.br.y=h1*6;

//Region 3

r3.tl.x=w1*1;r3.tl.y=h1*1;

r3.tr.x=w1*7;r3.tr.y=h1*1;

r3.bl.x=w1*1;r3.bl.y=h1*7;

r3.br.x=w1*7;r3.br.y=h1*7;

//Region 4

r4.tl.x=0;r4.tl.y=0;

r4.tr.x=iw;r4.tr.y=0;

r4.bl.x=0;r4.bl.y=ih;

r4.br.x=iw;r4.br.y=ih; }

Algorithm Spatial Histogram Matching : Finding Region of Pixel(x,y)public String findRegion(int x,int y) { String region=""; if((x>=r1.tl.x&&x<=r1.tr.x)&&(y>=r1.tl.y&&y<=r1.bl.y)) { region="Region1"; }

else if((x>=r2.tl.x&&x<=r2.tr.x)&&(y>=r2.tl.y&&y<=r2.bl.y)) { region="Region2"; }

Algorithm Spatial Histogram Matching : Finding Region of Pixel(x,y)

else if((x>=r3.tl.x&&x<=r3.tr.x)&&(y>=r3.tl.y&&y<=r3.bl.y))

{

region="Region3";

}

else

{

region="Region4";

}

return region;

}

Algorithm Spatial Histogram Matching : How it Works

Calculate the grayscale histogram features for all the four sub regions. Then compare the Histogram features starting from innermost sub region to outermost sub region, hierarchically.

Step1: Compute gray scale histogram error difference at sub-region1 between query feature and n number of databases features. Apply threshold and reduce.

Step2: Compute gray scale histogram error difference at sub-region2 between query feature and features of reduced set of images obtained in step1. Apply threshold and reduce.

Algorithm Spatial Histogram Matching : How it Works

Step3: Compute gray scale histogram error difference at sub-region3 between query feature and features of reduced set of images obtained in step2. Apply threshold and reduce.

Step4: Compute gray scale histogram error difference at sub-region4 between query feature and features of reduced set of images obtained in step3. Apply threshold and reduce.

Finally, least distance metrics will represent the similar Image to the query Image

Applications of Content Based Image Retrieval

Search Engines

Object Recognition and tracking

Crime Investigation

Art Collections

Medical Records

Conclusion

This research area is growing very rapidly

Current systems are still in prototype stage and lack reliability

Current techniques are based on low level features and there is a huge semantic gap existing

Much more research work is needed for coming out with a reliable and semantically competent system

References[1]. Manjunath KN, Renuka A, “Bit plane histogram matching for CBIR”, National Level Technical

paper presentation, Kadi, Gujarat.

[2]. Manjunath KN, Renuka A, Harischandra Hebbar N, “Hierarchical Bit plane histogram matching for CBIR”, IEEE’s Signal Processing Society, EMBS, TIFAC-CORE sponsored National Conference on Image Processing, MSRSAS, Bangalore.

[3]. Manjunath KN, Renuka A, Harischandra Hebbar N, “Spatial Bit plane histogram matching for CBIR”, AICTE, ISTE New Delhi sponsored National Conference on Graphics, Vision and Image Processing, J.N.N College of Engineering, Shimoga.

[4]. Kato, T., Database architecture for content-based image retrieval in: Jambardino, A. A., and Niblack, W. R., (Eds.), Image Storage 439 and Retrieval Systems. Proc SPIE 1662, 112–123, 1992.  

[5]. Swain, M. J., and Ballard, D. H., Color indexing. Int. J. Comput. Vis. 7(1):11–32, 1991.  

[6]. Stricker, M., and Orengo, M., Similarity of color images. In: Niblack, W. R., and Jain, R. C., (Eds.), Storage and Retrieval for Image and Video Databases III. Proc SPIE 2420, pp 381–392, 1995.  

[7]. Stricker, M., and Dimai, A., Color indexing with weak spatial constraints. In: Storage and Retrieval for Image and Video Databases IV. Proc SPIE 2670, 29–40, 1996.

[8]. Gonzalez, R. C., and Woods, R. E., Digital image processing, 2004 2nd Edition, pp 94–103.  

[9]. Flickner, M et al “Query by image and video content: the QBIC system” IEEE Computer 28(9), 23-92