Top Banner
Embedded lattices tree: An efficient indexing scheme for content based retrieval on image databases Mahmoud Mejdoub * , Leonardo Fonteles, Chokri BenAmar, Marc Antonini I3S Laboratory, UMR 6070 CNRS and University of Nice, Sophia Antipolis, Nice, France REGIM: Research Group on Intelligent Machines, Engineering National School of Sfax (ENIS), BP W, 3038 Sfax, Tunisia article info Article history: Received 28 May 2008 Accepted 17 December 2008 Available online 25 December 2008 Keywords: Multimedia databases Content based image retrieval Indexing structure Wavelets Lattice Vector quantization Salient points Feature extraction abstract One of the challenges in the development of a content-based multimedia indexing and retrieval applica- tion is to achieve an efficient indexing scheme. To retrieve a particular image from a large scale image database, users can be frustrated by the long query times. Conventional indexing structures cannot usu- ally cope with the presence of a large amount of feature vectors in high-dimensional space. This paper addresses such problems and presents a novel indexing technique, the embedded lattices tree, which is designed to bring an effective solution especially for realizing the trade off between the retrieval speed up and precision. The embedded lattices tree is based on a lattice vector quantization algorithm that divides the feature vectors progressively into smaller partitions using a finer scaling factor. The efficiency of the similarity queries is significantly improved by using the hierarchy and the good algebraic and geometric properties of the lattice. Furthermore, the dimensionality reduction that we perform on the feature vectors, trans- lating from an upper level to a lower one of the embedded tree, reduces the complexity of measuring sim- ilarity between feature vectors. In addition, it enhances the performance on nearest neighbor queries especially for high dimensions. Our experimental results show that the retrieval speed is significantly improved and the indexing structure shows no sign of degradations when the database size is increased. Ó 2008 Elsevier Inc. All rights reserved. 1. Introduction The development of internet and multimedia devices have caused a rapid increase in the size of digital information that is used, and stored via several applications. In order to overcome such problems, efficient image retrieval tools are required in vari- ous domain including medicine, publishing, architecture, etc. For this purpose, many content based image retrieval systems (CBIR) [1,2,4,3] have been developed. These systems have a common characteristics: the images are represented as vectors of d numeric features and similarity retrieval are performed by conducting near- est neighbor queries in the feature space. To apply the content based image retrieval to large size image databases, it is needed to develop multidimensional indexing structures efficiently sup- porting nearest neighbor retrieval. A straightforward way to per- form similarity matching is the sequential search algorithm (SSA) where every feature vector in the database is scanned to find if it satisfies the query requirement or not. However, SSA can be very costly since the running time of SSA is proportional to the feature space dimension and the database size. There are many methods to organize the feature vectors of images in the database such that a ranked list of nearest neighbors can be retrieved without perform- ing an exhaustive comparison with all the database image feature vectors. Actually, many researches are done on indexing high- dimensional vectors focused mainly on the vector quantization approach and the multidimensional indexing approach. We can divide the multidimensional indexing techniques into two categories, space-partitioning methods and data partitioning methods. These techniques are formed mostly in a hierarchical tree structure used to divide the data space progressively into smaller partition. The main difference between the two categories is in the way the partitioning is performed. In the first category, the kd-tree [5] divides the feature space into predefined hyper-planes regardless of the feature vectors dis- tribution. Such regions are disjoint and their union covers the en- tire space. The major inconvenient of the kd-tree is that it identifies the position of the feature vector in the data space using, in each level of the tree, one coordinate at one time. Besides, the rigid partitioning of the space, can lead to the consultation of few populated or empty clusters especially in high dimensions. In the second category, the most popular data partitioning tech- niques include the R-tree [7], SS-tree [8] and SR-tree [9]. Unlike the kd-tree structure, the feature space is divided according to the database items distribution. The R-tree applies hyper-rectangles, 1047-3203/$ - see front matter Ó 2008 Elsevier Inc. All rights reserved. doi:10.1016/j.jvcir.2008.12.003 * Corresponding author. E-mail addresses: [email protected] (M. Mejdoub), [email protected] (L. Fonteles), [email protected] (C. BenAmar), [email protected] (M. Antonini). J. Vis. Commun. Image R. 20 (2009) 145–156 Contents lists available at ScienceDirect J. Vis. Commun. Image R. journal homepage: www.elsevier.com/locate/jvci
12

Embedded lattices tree: An efficient indexing scheme for content based retrieval on image databases

May 02, 2023

Download

Documents

Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: Embedded lattices tree: An efficient indexing scheme for content based retrieval on image databases

J. Vis. Commun. Image R. 20 (2009) 145–156

Contents lists available at ScienceDirect

J. Vis. Commun. Image R.

journal homepage: www.elsevier .com/locate / jvc i

Embedded lattices tree: An efficient indexing scheme for content based retrievalon image databases

Mahmoud Mejdoub *, Leonardo Fonteles, Chokri BenAmar, Marc AntoniniI3S Laboratory, UMR 6070 CNRS and University of Nice, Sophia Antipolis, Nice, FranceREGIM: Research Group on Intelligent Machines, Engineering National School of Sfax (ENIS), BP W, 3038 Sfax, Tunisia

a r t i c l e i n f o

Article history:Received 28 May 2008Accepted 17 December 2008Available online 25 December 2008

Keywords:Multimedia databasesContent based image retrievalIndexing structureWaveletsLatticeVector quantizationSalient pointsFeature extraction

1047-3203/$ - see front matter � 2008 Elsevier Inc. Adoi:10.1016/j.jvcir.2008.12.003

* Corresponding author.E-mail addresses: [email protected] (M. Me

(L. Fonteles), [email protected] (C. BenAmar), Chokri.Ben

a b s t r a c t

One of the challenges in the development of a content-based multimedia indexing and retrieval applica-tion is to achieve an efficient indexing scheme. To retrieve a particular image from a large scale imagedatabase, users can be frustrated by the long query times. Conventional indexing structures cannot usu-ally cope with the presence of a large amount of feature vectors in high-dimensional space. This paperaddresses such problems and presents a novel indexing technique, the embedded lattices tree, whichis designed to bring an effective solution especially for realizing the trade off between the retrieval speedup and precision.

The embedded lattices tree is based on a lattice vector quantization algorithm that divides the featurevectors progressively into smaller partitions using a finer scaling factor. The efficiency of the similarityqueries is significantly improved by using the hierarchy and the good algebraic and geometric propertiesof the lattice. Furthermore, the dimensionality reduction that we perform on the feature vectors, trans-lating from an upper level to a lower one of the embedded tree, reduces the complexity of measuring sim-ilarity between feature vectors. In addition, it enhances the performance on nearest neighbor queriesespecially for high dimensions. Our experimental results show that the retrieval speed is significantlyimproved and the indexing structure shows no sign of degradations when the database size is increased.

� 2008 Elsevier Inc. All rights reserved.

1. Introduction

The development of internet and multimedia devices havecaused a rapid increase in the size of digital information that isused, and stored via several applications. In order to overcomesuch problems, efficient image retrieval tools are required in vari-ous domain including medicine, publishing, architecture, etc. Forthis purpose, many content based image retrieval systems (CBIR)[1,2,4,3] have been developed. These systems have a commoncharacteristics: the images are represented as vectors of d numericfeatures and similarity retrieval are performed by conducting near-est neighbor queries in the feature space. To apply the contentbased image retrieval to large size image databases, it is neededto develop multidimensional indexing structures efficiently sup-porting nearest neighbor retrieval. A straightforward way to per-form similarity matching is the sequential search algorithm (SSA)where every feature vector in the database is scanned to find if itsatisfies the query requirement or not. However, SSA can be verycostly since the running time of SSA is proportional to the featurespace dimension and the database size. There are many methods to

ll rights reserved.

jdoub), [email protected]@enis.rnu.tn (M. Antonini).

organize the feature vectors of images in the database such that aranked list of nearest neighbors can be retrieved without perform-ing an exhaustive comparison with all the database image featurevectors. Actually, many researches are done on indexing high-dimensional vectors focused mainly on the vector quantizationapproach and the multidimensional indexing approach.

We can divide the multidimensional indexing techniques intotwo categories, space-partitioning methods and data partitioningmethods. These techniques are formed mostly in a hierarchical treestructure used to divide the data space progressively into smallerpartition. The main difference between the two categories is inthe way the partitioning is performed.

In the first category, the kd-tree [5] divides the feature spaceinto predefined hyper-planes regardless of the feature vectors dis-tribution. Such regions are disjoint and their union covers the en-tire space. The major inconvenient of the kd-tree is that itidentifies the position of the feature vector in the data space using,in each level of the tree, one coordinate at one time. Besides, therigid partitioning of the space, can lead to the consultation of fewpopulated or empty clusters especially in high dimensions.

In the second category, the most popular data partitioning tech-niques include the R-tree [7], SS-tree [8] and SR-tree [9]. Unlike thekd-tree structure, the feature space is divided according to thedatabase items distribution. The R-tree applies hyper-rectangles,

Page 2: Embedded lattices tree: An efficient indexing scheme for content based retrieval on image databases

146 M. Mejdoub et al. / J. Vis. Commun. Image R. 20 (2009) 145–156

represented as nodes in the tree, to divide the space. Children of anode then divide the space inside the hyper-rectangle with smallerhyper-rectangles. It is known in previous works [5] that the R-treesoutperform the kd-trees in high dimensional nearest neighbor que-ries. However, the most serious problem of the R-tree is thatbounding rectangles can overlap in higher dimensions. In orderto prevent this, White and Jain proposed the SS-tree [6], an alterna-tive to R-tree structure, which uses minimum bounding spheres in-stead of rectangles. Even though SS-tree outperforms R-tree, theoverlapping in the high dimensions still occurs. The SR-tree is animprovement of the R-tree and SS-tree. One of its most importantfeatures is that it employs the intersection of hyper-spheres andhyper-rectangles to determine the shape of a partition.

Generally, the main drawback of the multidimensional indexingtechniques is that they do not scale up well to high-dimensionalspaces due to the phenomenon called ‘‘the curse of dimensional-ity”. Furthermore, the number of partitions increases exponentiallywith the dimensionality. As it is reported in [10], these multidi-mensional indexing structures are mostly useful for mediumdimensional feature spaces. To avoid the curse dimensionalityproblem, the principal component analysis (PCA) [11] and the la-tent semantic indexing (LSI) method [12], inspired from the textretrieval field and adapted to the image retrieval, are used to re-duce the dimensionality of feature vectors. Lower dimensionaltransformed feature vectors are used to approximate the originalfeature vectors. However, the major drawback of these methodsis the risk of the pertinent information loss in the feature spaceresulting from the reduction of the dimensionality. An other mul-tidimensional indexing structures shortcoming is that the dissimi-larity distance between two points has to be based on a distancecomputation which is costly and CPU intensive especially for highdimensional data spaces. Moreover, the performance of a nearestneighbor query algorithm degrades if a query point is located neara partition border because there is two decisions to take. The first isto ignore the neighboring partitions decreasing the retrieval preci-sion. The second is to take into account the neighboring partitionsresulting in increasing computational requirements. In addition tothe multidimensional indexing technique inconveniences statedabove, one of their major limit is the incremental construction ofthe indexing tree that could lead, depending on the order of the ob-jects insertion, to significantly varying performances during theindexing phase.

As an alternative to the multidimensional techniques, manyvector quantization schemes have been proposed in order to pro-duce an image index. Existing solution which improve over stan-dard k-means include the use of mean shift based clustering [13],hierarchical k-means [14], agglomerative clustering [15], random-ized trees [16] and self-organizing maps (SOM) [17]. Indeed, thesevector quantization algorithm provide a partitioning of the datainto groups or clusters so that data items into a certain group aremore similar to each other than to data items in other groups. Eachgroup is then represented by its centroid or sometimes a singlerepresentative data item. Thus, instead of the original data items,the query point is compared to the centroids or the group repre-sentatives. The best group or groups, according to the used similar-ity measure, are then selected and the data items belonging tothose groups are evaluated to return the k nearest neighbors. Thesebased clustering algorithms suffer from the expensive distancecomputation especially if the dimensionality of the input vectorsis high. Moreover, many of these clustering algorithms assumedthat the number of clusters, the initial partitions and the learningweights were known prior to clustering, but this is rarely the case,especially in the indexing framework. Also, except the SOM algo-rithm, the majority of the data clustering algorithms does not pre-serve the topological ordering of the data space.

Taking into account the drawbacks cited above, and with an aimto accelerate nearest neighbors search, we propose an accurate andfast solution based on the lattice vector quantization and whichdiffers significantly from existing solutions, as it does not rely onclustering techniques. Due to the good geometric and algebraicproperties of the lattice Zn [18,19], the proposed indexing structuredoes not need learning parameters initialization and expensive on-line distance computation. Besides, it allows us to explore effi-ciently feature space and neighborhood relations between featurevectors preserving the topological ordering. By using the proposedindexing structure, the overlapping is avoided and the quality ofclusters is preserved. Moreover, the hierarchy structure of the pro-posed indexing tree and the dimensionality reduction proposedmethod allow us to obtain an efficient partitioning of the featurespace.

The paper is organized as follows. In Section 2, we present theadapted method to extract a fuzzy feature vector that describesthe visual content of the database images. The extracted fuzzy fea-ture vector is based on local descriptors of the prominent objects.Besides, the competitive agglomeration algorithm (CA) [20] is ap-plied in order to group the obtained local descriptors into catego-ries of visually similar regions. In Section 3, we outline theproposed algorithm used to index the obtained fuzzy feature vec-tor. We present our proposed hierarchical indexing method basedon the regular lattice in Section 4. In Section 5, we describe the pro-posed method to build the proposed hierarchical embedded lat-tices tree. Some experimental results are presented in Section 6with the aim to illustrate the effectiveness of the proposed index-ing method comparing it to SR-tree [9] and SOM [17].

2. Feature extraction

In this paper, we focus on local feature vectors extraction in or-der to describe the image prominent objects. First, a robust salientfeature detector based is designed. The resulting points are notconfined to corners, but indicate where ‘‘something” happens atdifferent the image. Second, a compact descriptor is computedfor each salient point, by analyzing the signal within the supportregion located in the neighborhood of the salient points. Third,all extracted salient descriptors are grouped into categories of visu-ally similar regions using the competitive agglomeration algorithmCA. In what follows, we enumerate in details the different stepsused to extract a fuzzy feature vector associated to each databaseimages:

2.1. First step: image conversion in the CIE-Lab

We convert the database images in the CIE-Lab color space.

2.2. Second step: extraction of the low level feature vectors

We use two methods to derive the low level feature vectorscomputed on a small neighborhood around each salient point:

2.2.1. Wavelet based feature extraction methodThe salient points extraction algorithm is based on the wavelet

representation of the image at different scales introduced by Lou-pias and Sebe in [23]. In the proposed approach, the salient pointsextraction is established by applying, on the CIE-Lab convertedimages, a new proposed wavelet: the biorthogonal Beta wavelet[21,22,26]. An example showing a comparison between the perfor-mances of three wavelets basis (Daubechies, Filtre9-7 [24] andbiorthogonal Beta) in extracting the salient points is given inFig. 1. We observe that the salient points extracted by the biorthog-onal Beta wavelet are spreader and better localized around the im-

Page 3: Embedded lattices tree: An efficient indexing scheme for content based retrieval on image databases

M. Mejdoub et al. / J. Vis. Commun. Image R. 20 (2009) 145–156 147

age objects than those extracted by Daubechies and Filtre9-7wavelets.

After the salient point extraction step, first we consider the pix-els in a small neighborhood of 25 � 25 around each salient point toform a salient object. Second, we transform each color componentof the salient object in the spatio-frequency domain which is morerepresentative than the spatial domain. To do this, we use the bior-thogonal Beta wavelet, which outperforms the Daubechies and theFiltre9-7 in the feature vector extraction scheme [25,26]. Accord-ingly, the biorthogonal Beta wavelet is applied with one decompo-sition level on each color component of the salient object. Third, wecompute the standard deviation of the coefficients of the four ob-tained bands (one approximation band and three details bands).Since we have four standard deviations for each color component,we obtain a low level feature vector xj with 12 components, foreach extracted salient object.

2.2.2. SIFT [27] based feature extraction methodThe SIFT descriptor is derived, computing distributions over

gradient orientations for different subpatches (Fig. 2). We experi-mented with 4� 4 subpatches, with eight different discretizationlevels for the orientations, resulting in a low level feature vectorxj with 128 components, associated to each salient point.

2.3. Third step: salient objects categorization

To achieve salient objects categorization, an efficient clusteringscheme is required. The competitive agglomeration algorithm (CA),presented in [20] was chosen. In fact, compared to FCM [28] algo-rithm, the main CA advantage resides on the automatic determina-tion of the cluster numbers. We group the obtained low levelfeature vectors xj extracted for each salient object from all databaseimages in a matrix X ¼ fxj; j ¼ 1; . . . ;Ngwith N denotes the numberof feature vectors to be classified. Then, we apply the CA algorithmto partition the matrix X into C clusters. The obtained clusters de-fine the salient objects categories.

Fig. 1. Results of the salient points

The CA is performed by minimizing the following objectivefunction j based on entropy measures:

j ¼ j1 þ aj2 ¼XC

i

XN

j

l2ijd

2ij � a �

XC

i

pi log pi ð1Þ

witha: a parameter used to equilibrate the contributions of j1 and j2.lij represents the membership degree of feature xj to cluster i.di;j is the Mahalanobis distance between xj and bi is the center of

the cluster i. It is given by:

di;j ¼ dðxj;biÞ with bi ¼PN

j l2ijxjPN

j l2ij

pi ¼ NiN denotes the probability of the cluster i and Ni ¼

PNj li;j de-

notes the fuzzy cardinality of the cluster i.The objective function j is subject to the membership

constraint:

XC

i¼1

li;j ¼ 1; 81 6 j 6 N ð2Þ

The initial CA partition has an over-specified number of clus-ters, which is dynamically reduced as the algorithm progresses.At the convergence, the final partition has the optimal number ofclusters. The objective function combines two components j1 andj2. The first one is similar to the FCM objective function and hasa global minimum when each data point is in a separate cluster.The global minimum of the entropy measures given by j2 isachieved when all points are in the same cluster such that it con-trols the number of clusters. Optimal cluster number is determinedby a balance between the opposite effects terms j1 and j2. The twocomponents are combined by the parameter a which is chosen toequilibrate their contributions. The value of a decreases slowlysuch that it favors agglomeration in the first iterations while

extraction based on wavelets.

Page 4: Embedded lattices tree: An efficient indexing scheme for content based retrieval on image databases

Fig. 2. Results of the salient points extraction based on SIFT.

148 M. Mejdoub et al. / J. Vis. Commun. Image R. 20 (2009) 145–156

emphasizing objective function in the latest ones. So minimizing jwith an over-specified number of initial clusters optimizes simul-taneously the data partition and the number of classes.

2.4. Fourth step: extraction of the fuzzy feature vector associated to thesalient object

In order to translate each low level feature vector in X to a fuzzyfeature vector, we use the membership matrix U given by:

U ¼ fli;j;1 6 i 6 C;1 6 j 6 Ng ð3Þ

The coefficient li;j assigned to the row i and the column j of U indi-cates the degree of membership of xj in the cluster i. Each column ofthe matrix U represents then the fuzzy feature vector associated tothe low level feature vector xj. The parameter C determines thedimensionality of the fuzzy feature space U.

2.5. Fifth step: image fuzzy feature extraction

To extract the fuzzy feature vector of an image I, we computethe mean of the fuzzy feature vectors associated to the salient ob-jects belonging to I. Thus, we obtain for each image a fuzzy featurevector.

Fig. 3 illustrates an example of fuzzy feature extraction for 3clusters (C ¼ 3).

Fig. 3. Extraction of the fuzzy feature vector.

3. Indexing the feature vectors

Once the fuzzy feature vectors are extracted, they must be in-dexed in such a way that a fast and efficient retrieval in the data-base is guaranteed. For that purpose, we quantize each featurevector using a lattice vector quantizer (LVQ) and assign the indexof the lattice vector to the corresponding feature vector. We pro-pose to use the Zn lattice in order to guarantee a fast navigationover the points in the n-dimensional space and allow a fast corre-spondence between a lattice vector and its index. More details aregiven hereinafter.

A lattice K in Rn is composed of all integral combination of a setof linearly independent vectors ai (the basis of the lattice) suchthat:

K ¼ fxjx ¼ u1a1 þ u2a2 þ � � � unang ð4Þ

where the ui are integers. The partition of the space is hence regularand depends only on the chosen basis vectors ai 2 Rm (m P n). Sucha regular structure permits to identify the nearest vectors in thespace using fast algorithms. The LVQ is an efficient algorithm forfinding the closest lattice vector x of a query feature vector v. Inthe case of a Zn lattice, the closest lattice vector with a precisionc is given by:

x ¼ vc

� �ð5Þ

where [�] stands for the ‘round’ operator, and c is a scaling factor.Feature vectors are obtained as described in Section 2. Once the

feature vectors are quantized into lattice vectors x, we may attri-bute a unique and decodable index for each x. The proposed index-ing method computes an index by classifying the lattice vectorsaccording to their norm and the geometrical properties of the lat-tice. Taking into account the properties of the feature vectors, anindex is composed by four indices: an index for the norm (IN), anindex for the leader (IL), an index for the permutation (Ip) and final-ly an index for the sign (IS). Let us detail each of these indices in thefollowing:

� The index for the norm (IN) is given by the l1 normkxk1 ¼

Pni¼1jxij of the lattice vector x. It classifies the different

lattice vectors x in different hyper-pyramids (shells). The l1

norm is chosen because the sum of the used fuzzy feature vec-tors coordinates is a constant and it is equal to 1.

� The lattice vectors lying on the same shell with index IN are sub-divided to a few number of vectors, called leaders. The leadersare vectors from which all the other lattice vectors of the corre-sponding shell can be generated by permutations and signchanges of its coordinates. The Section 3.1 explains in detailshow the leader indices IL are computed.

� The index for the permutation (IP) is computed by the Schalk-wijk algorithm as in [18].

Page 5: Embedded lattices tree: An efficient indexing scheme for content based retrieval on image databases

M. Mejdoub et al. / J. Vis. Commun. Image R. 20 (2009) 145–156 149

� The index for the sign (IS) represents the change of octant of thehyper-space in which the vector lies. For example, the vector(7,�3) of dimension 2 is in the fourth quadrant (octant indimension 2), while the vector (�7,�3) is in the third one. Thesevectors are symmetrical with respect to the y -axis. For moredetails in how to compute the index of sign see [18].

This indexing method creates an hierarchical tree adapted tothe database indexing framework (see Sections 4.1 and 5.1).

3.1. Proposed leader indexing

The proposed algorithm classifies all the leader indices [29,30]in such a way that the indexing is no longer based on a greedysearch algorithm or direct addressing, but on low-cost enumera-tion algorithm which just depends on the quantity of leaders in-stead of on the explicit knowledge of all of them.

A hyper-pyramid of radius r and dimension n is composed byall the vectors v such that jjvjj1 ¼ r. As said before, leaders arethe elementary vectors of a hyper-surface from which operationsof permutations and sign changes lead to all the other vectorslying on this hyper-surface. Indeed, the leaders are vectors withpositive coordinates sorted in increasing (or decreasing) order.Therefore, leaders for l1 norm are vectors which verify the con-ditions below:

(1)Pn

i¼1v i ¼ r;(2) 0 6 v i 6 v j; for alli < j.

In the case of a l1 norm, one can note that those conditionsare linked to the theory of partitions in number theory [31]. In-deed, in number theory a partition of a positive integer r is away of writing r as a sum of d positive integers (also called part).The number of partitions of r is given by the partition functionpðrÞ such that:

X1r¼0

pðrÞyr ¼Y1d¼1

11� yd

� �; ð6Þ

which corresponds to the reciprocal of the Euler’s function [31].Further mathematical development lead to representations of thepðrÞ function that allow faster computation. Interested readersshould refer to [31].

However, we are usually interested in shells of l1 norm equals tor in a d-dimensional lattice with r–d. In this case, one can use thefunction qðr; dÞ [31] which computes the number of partitions of rwith at most d parts (in partition theory it is equivalent to thenumber of partitions of r with no element greater than d withany number of parts). Then, for a hyper-pyramid of norm r ¼ 5and dimension d ¼ 3, we have qð5;3Þ ¼ 5, i.e. five leaders givenby: (0,0,5), (0,1,4), (0,2,3), (1,1,3), and (1,2,2).

The function qðr; dÞ can be computed from the recurrence rela-tion: [31]:

qðr;dÞ ¼ qðr;d� 1Þ þ qðr � d; dÞ; ð7Þ

with qðr;dÞ ¼ pðrÞ for d P r; qð1;dÞ ¼ 1 and qðr; 0Þ ¼ 0.

3.1.1. Using function qðr; dÞ to index the leadersAs we will see in the following, Eq. (7) not only gives the total

number of leaders lying on a given hyper-pyramid but can alsobe used to provide unique indices for these leaders. To illustratethe principle of the proposed algorithm, let us suppose that theleaders of a given hyper-pyramid have been classified in a lexico-graphical order as:

Index value Leader0 ð0; . . . ;0; 0; rnÞ1 ð0; . . . ;0;1; rn � 1Þ2 ð0; . . . ;0;2; rn � 2Þ3 ð0; . . . ;1;1; rn � 2Þ... ..

.

In this way, the index of a leader L corresponds to the number ofleaders that appear before it. For example, the leaderð0; . . . ;1;1; rn � 2Þ should be assigned to index 3.

Consider a leader L ¼ ðx1; x2; . . . ; xn�1; xnÞ of dimension n andnorm rn ¼

Pni¼1xi. Since the leaders are sorted in a lexicographical

order, all the leaders with the largest coordinate gn verifyingxn þ 1 6 gn 6 rn appear before L. The number of leaders with larg-est coordinate equal to xn þ t (t P 1) and norm rn can be easily cal-culated using the function q of Eq. (7) and is given byqðrn � ðxn þ tÞ;n� 1Þ. Clearly, computing the number of leaderswith the largest coordinate equal to xn þ t, with norm r ¼ rn anddimension d ¼ n, is equivalent to calculate the number of leadersof norm rn � ðxn þ tÞ with dimension n� 1.

By introducing the function qðr; d; kÞ, which counts all the parti-tions of a number r with at most d parts not greater than k, we canshow that the index of a leader can be computed using the follow-ing formula:

IL ¼Xn�2

j¼0while xn�ðjþ1Þ–0

Xmin½xn�ðj�1Þ ;rn�j �

i¼xn�jþ1

qðrn�j � i; n� ðjþ 1Þ; iÞ; ð8Þ

with xnþ1 ¼ þ1 and qð0;dÞ ¼ qð0; d; kÞ ¼ 1. Note that, when rn�j � i isless than or equal to i, qðrn�j � i;n� ðjþ 1Þ; iÞ = qðrn�j � i; n� ðjþ 1ÞÞ,because in that case all vectors counted by qðr;nÞ are leaders.

In this section, we have proposed an efficient solution for index-ing lattice vectors. In our scheme, indexing a lattice vector is re-duced to indexing its corresponding leader and somepermutations and sign changes. The number of leaders being smal-ler than the cardinality of a hyper-surface, indexing a leader can bedone even for huge vector dimension which is impossible whenindexing is done directly on all the vectors of a hyper-surface(due to computer precision requirement and memory utilization).

4. The proposed indexing structure based on the regular lattice

Lattice vector quantization based on Zn divides the data spaceinto hypercubes. The centroid of each hypercube is a lattice point.Each feature vector of the feature space U obtained as explained inSection 2 is quantized in a lattice point. The query procedure is gi-ven as follows:

(1) The query feature vector is computed.(2) The query feature vector is quantized in a lattice point.(3) We exploit the good properties of the lattice space to deter-

mine the nearest lattice points of the quantized query fea-ture vector.

(4) We collect the feature vectors of the database quantized bythe nearest lattice points.

(5) We perform SSA on the feature vectors of the collectedimages to get only the k nearest query feature vectors.

4.1. Building the regular lattice tree

Supposing F is a fuzzy feature space, F is firstly quantized, in aregular lattice Zn, using a scaling factor ci:

Page 6: Embedded lattices tree: An efficient indexing scheme for content based retrieval on image databases

150 M. Mejdoub et al. / J. Vis. Commun. Image R. 20 (2009) 145–156

G ¼ Fci

� �ð9Þ

where [�] stands for the ‘round’ operator. Then, each quantized fea-ture vector in a lattice point pj is indexed with the indexðIN; IL; IP ; ISÞ corresponding in the regular lattice tree to the branchBðIN ;IL ;IP ;ISÞ. This branch contains, respectively, the indices of norm,leader, permutation and sign [30,31]. Note that we use the indexof sign only when the feature vectors have negative components.The index of norm is a prefix index that classifies the feature vectorsin different hyper-pyramids of the lattice. Indeed, the feature vec-tors having the same IN are located in the same hyper-pyramid.After that, the indices of leader, permutation and sign are used todetermine the position of the feature vectors in one fixed hyper-pyramid. These indices are obtained as explained in Section 3. Theregular lattice tree has four stages and one leaf node. Each stagecontains a certain number of tables. We have four kinds of tablesi.e. tableIN , tableIL , tableIP and tableIS associated, respectively, to theindices of norm, leader, permutation and sign and located in thefirst, second, third and fourth stages of the tree.ðIN; IL; IP; ISÞ is used to index the feature vectors quantized in a

lattice point pj. That’s why, we denote it by ðIN; IL; IP ; ISÞpjand we

store in the leaf node of the branch BðIN ;IL ;IP ;ISÞpjthe two following

structures:

� The structure CðpjÞ contains the feature vectors of the imagedatabase indexed by the same index ðIN; IL; IP; ISÞpj

.� The structure ImðpjÞ contains the images associated to the fea-

ture vectors belonging to the structure CðpjÞ.

Where 1 6 j 6 nbpoints and nbpoints is the number of full latticepoints in the lattice. We mean by full lattice point, a point in whichare quantized feature vectors of the database images. We give inFig. 4 an example of a regular lattice indexing tree explaining morethe proposed approach. The first, second, third and fourth stages ofthe tree correspond, respectively, to the norm indices IN , the leaderindices IL, the permutation indices IP and the sign indices IS. In theleaf node of the branch BðIN¼9;IL¼5;IP¼6;IS¼1Þp1

are stored the two struc-tures Cðp1Þ and Imðp1Þ:

� Cðp1Þ corresponds to the set of feature vectors quantized in thelattice point p1 having as index ðIN ¼ 9; IL ¼ 5; IP ¼ 6; IS ¼ 1Þ.

� Imðp1Þ corresponds to the set of database images whose featurevectors are quantized in the lattice point p1.

We can also observe in Fig. 4, that as the lattice points p2 and p3

have the same IN ¼ 9, the IL of p2 and p3 are in the same tableIL lo-

Fig. 4. Example of a re

cated in the level 2 and pointed with nodeIN¼9. Similarly, since thelattice points p3 and p4 have the same indices IN ¼ 9 and IL ¼ 1, theIP of p2 and p3 are in the same tableIP located in the level 3 andpointed by the branch BðIN¼9;IL¼1Þ.

4.2. Getting the k nearest query feature vectors

We present in the following the proposed method to browse theneighborhood of the lattice point pj in which is quantized the queryfeature vector:

� We define the nearest lattice vectors x of the lattice origin vector(the null vector 0) as:

maskd ¼ x 2 K;X

i

x2i ¼ d; d 2 P � N

( )ð10Þ

which defines the lattice vectors at the square distance d from theorigin. For example, as showed in Fig. 5 for a lattice Z2, the neigh-bors at a square distance d ¼ 4 from the origin of the lattice are

given by mask4 ¼0�2

� �;�20

� �;

02

� �;

20

� �� �. They correspond to

the third neighborhood of the null vector bf 0.� The set of lattice points located at a neighborhood correspond-

ing to a maximum distance D of the lattice point pj is given by:

VðpjÞ ¼ ½V0;V1;V2; . . . ;Vd; . . . ;VD� for d 2 P ð11Þ

with:

Vd ¼ fv 2 K; v ¼ vpjþ x; x 2 maskdg; ð12Þ

vpjrepresents the lattice vector of the lattice point pj and V0 con-

tains the lattice point pj.

4.3. Advantages of the indexing structure based on the regular lattice

The advantages of the proposed indexing structure based on theregular lattice comparing to multidimensional indexing techniquesand vector quantization techniques based on clustering are de-tailed here:

1. Reduced size of the proposed hierarchical index (we propose anindex of only four components: ðIN; IL; IP; ISÞ).

2. Finding the closest lattice vector of a query vector using a sim-ple analytic vector quantization method given in Eq. 5 withoutneeding a sequential scan of the cluster representative vector.

gular lattice tree.

Page 7: Embedded lattices tree: An efficient indexing scheme for content based retrieval on image databases

Fig. 5. The lattice points located at the neighborhood of the lattice origin in Z2.

M. Mejdoub et al. / J. Vis. Commun. Image R. 20 (2009) 145–156 151

3. Finding the closest lattice vector of a query vector using a sim-ple analytic vector quantization method given in Eq. 5 withoutneeding a distance computation between the query vector andthe clusters representatives.

4. Immediate dictionary construction avoid the problems of parti-tion and learning parameters initialization.

5. Preserving the topological ordering of feature vectors.6. Taking into account the correlation between feature

dimensions.7. Non overlap between partitions.8. Independence of feature vectors insertion order.

We present in Table 1, the advantages of the proposed indexingstructure compared to the data partitioning techniques such as SR-tree, the space partitioning techniques such as kd-tree, the vectorquantization algorithm derived from k-means and SOM.

The essentially drawback of the proposed indexing regular lat-tice tree is the rigid partitioning of the space. This can lead to theconsultation of few populated or empty clusters during the neigh-borhood lattice browsing. Indeed, the performance of the nearestneighbor query based on a regular lattice is dependent on the lat-tice density which is determined according to the scaling factor c.

� If c is great then the source will be contracted and the regular latticewill be very dense. Consequently, the error of quantization will beimportant and the precision of the image retrieval process willdecrease. Besides, the number of the feature vectors quantized ina lattice point will be great so we must perform SSA on a large num-ber of feature vectors and the speed up will be degraded.

� If c is small then the source is expanded. Consequently, the errorof quantization is lower and the feature vectors are partitionedinto lattice points that have a short number of feature vectorsquantized on it. So having a small c increases the performanceof the nearest neighbor search in term of accuracy. However,the regular lattice will be less dense and the number of featurevectors quantized in a lattice point will be very small, so forretrieving the k nearest images the retrieval engine has to parsean important number of nearest lattice points. Thus, the speedup cost will be more important.

In order to overcome such problems and provide efficient solu-tions to the aforementioned shortcomings of the indexing struc-ture based on a regular lattice, we propose an indexing methodbased on the embedded lattices that avoid the risk of getting

Table 1Advantages of the tree regular lattice.

1 2

Data partitioning techniques �Space partitioning techniques �vector quantization algorithm derived from k-means � �SOM � �

empty and few populated clusters during the lattice neighborhoodbrowsing operation.

5. The proposed indexing method based on embedded lattices

The embedded lattices tree has a hierarchic structure, which isformed into one or more levels. In the first level, we use the tree(built as explained in Section 4.1) to index the fuzzy feature vectorsquantized in the first level regular lattice. The other levels are capa-ble of holding one or more regular lattice trees.

The lattices embedding is processed basically with the threesteps presented below:

� First, in the level i of the embedded lattices tree, we quantize thefeature vectors of the database with a scaling factor ci.

� Second, we reduce the feature vectors dimensionality selectingonly the feature dimensions that achieve the maximal similaritybetween the feature vectors which are quantized in the samelattice point.

� Third, in the level iþ 1, we re-quantize the obtained feature vec-tors of reduced the dimensionality in a regular lattice with afiner scaling factor ciþ1 (ciþ1 < ci).

The selection of the most relevant feature dimensions and thefiner scaling factor allow us to obtain a better feature vectors dis-tribution in the lower level lattices.

5.1. Building the embedded lattices tree

We present here in details the different steps used to build theembedded lattices tree:

1. i ¼ 1: with i indicates the level in the embedded latticestree.

2. The feature vectors obtained as explained in Section 2 arequantized with the scaling factor c1 in a lattice Zn locatedin the first level of the embedded lattices tree.

3. Associate to each quantized feature vector pj;1 of the firstlevel lattice Zn, one index ðIN; IL; IPÞpj;1

with IN , IL, IP repre-senting, respectively, the norm, leader and permutationindices of the lattice point pj;1. Note that we do not considerthe index of sign in indexing the lattice vectors of the firstlevel lattice since their coordinates are all positive.

4. Make the two following tests cited in 4.1 and 4.24.1 if (i ¼¼ 1) then: each obtained index ðIN; IL; IPÞpj;i

isassociated to a branch Bpj;i

¼ BðIN ;IL ;IP Þpj;iwith three

nodes. These branches form the regular lattice treelocated in the first level of the embedded tree.

4.2 if (i! ¼ 1) then: each obtained index ðIN ; IL; IP ; ISÞpj;iis

associated to a branch Bpj;i¼ BðIN ;IL ;IP ;ISÞpj;i

with fournodes. These branches form the regular lattice treelocated in the ith level of the embedded tree.

5. For each branch Bpj;iobtained from the step 4, we execute

the steps from 6 to 13:6. In the leaf of the branch Bpj;i

, we store the two followingstructures:

3 4 5 6 7 8

� � �� �

� � �� �

Page 8: Embedded lattices tree: An efficient indexing scheme for content based retrieval on image databases

152 M. Mejdoub et al. / J. Vis. Commun. Image R. 20 (2009) 145–156

� The matrix Cðpj;iÞ containing the feature vectors of theimage database quantized in the lattice point pj;i.

� The vector Imðpj;iÞ containing the images associated tothe feature vectors belonging to the matrix Cðpj;iÞ.

7. If (i is equal to levelmax) we exit, else we execute the stepsfrom 8 to 13 in order to quantize the feature vectors ofCðpj;iÞ with a finer scaling factor ciþ1. Note that levelmax isthe maximal number of levels in the embedded latticestree.

8. Center the feature vectors of Cðpj;iÞ according to pj;i usingthe Eq. (13). We obtain then the matrix Ccentrðpj;iÞ.

Ccentrðpj;iÞ ¼ Cðpj;iÞ=ci � pj;i: ð13Þ

9. Form the vector DIMðpj;iÞ containing the most pertinentdimensions that achieve the maximal similarity betweenthe feature vectors of Cðpj;iÞ. To determine the most perti-nent dimensions, we make for each dimension the meanL1 distance between the feature vectors coordinates associ-ated to this dimension and we keep only the dimensionscorresponding to a distance below a certain threshold ti.Then, include the vector DIMðpj;iÞ in the leaf of Bpj;i

(seeFig. 6).

10. Project the feature vectors of the matrix Ccentrðpj;iÞ on thevector DIMðpj;iÞ to reduce the dimensionality of the featurevectors of Ccentrðpj;iÞ. We obtain then the matrixCreduc;centrðpj;iÞ.

11. Quantize the feature vectors of Creduc;centrðpj;iÞ in a level iþ 1lattice childðpj;iÞ using a scaling factor ciþ1 ¼ ci=a. Note thata is an odd entire in order to get a perfect embeddedlattices.

12. Compute the index ðIN; IL; IP ; ISÞpj;iþ1of each quantized fea-

ture vector in a lattice point pj;iþ1 of the lattice childðpj;iÞ.Each obtained index ðIN; IL; IP; ISÞpj;iþ1

presents a branch

BðIN ;IL ;IP ;ISÞpj;iþ1of four nodes in the regular lattice tree of level

iþ 1. This tree is the child of the branch BðIN ;IL ;IP ;ISÞpj;i.

13. i ¼ iþ 1 and go to the step 4.

Fig. 6 presents an example of an embedded lattices tree com-posed of a parent in level 1 and three children in level 2. The first,second and third stages of the first level regular lattice tree corre-spond, respectively, to the indices of norm IN , indices of leader IL

Fig. 6. Example of the em

and indices of permutation IP . In the second level of the embeddedlattices tree we have 3 regular lattice trees, each one with fourstages. The first, second, third and fourth stages of these regularlattices tree correspond, respectively, to the indices of norm IN ,indices of leader IL, indices of permutation IP and indices of signIS. The regular lattice trees located in the level two of theembedded tree are the children of the branches ðBðIN ;IL ;IP Þpj;1

Þ16j63

which form the first level regular lattice tree. For example, the fea-ture vectors that are quantized with a scaling factor c1 in the firstlevel lattice in a same lattice point p1;1 having as indexðIN ¼ 9; IL ¼ 5; IP ¼ 6Þp1;1

are re-quantized in the lattice childðp1;1Þin two lattice points p1;2 and p2;2 having as index, respectivelyðIN ¼ 3; IL ¼ 6; IP ¼ 1; IS ¼ 3Þp1;2

and ðIN ¼ 10; IL ¼ 3; IP ¼ 6;

IS ¼ 3Þp2;2. Consequently, the branches BðIN¼3;IL¼6;IP¼1;IS¼3Þp1;2

and

BðIN¼10;IL¼3;IP¼6;IS¼3Þp2;2are the children of the branch BðIN¼9;IL¼5;IP¼6Þp1;1

.

The use of a finer scaling factor c2 and the elimination of theirrelevant dimensions allow us to have a more precise distributionof the feature vectors in the level two regular lattices.

Fig. 7 shows an example of transition from the level i to thelevel iþ 1 during the building of the embedded lattices tree illus-trated in Fig. 6. We can observe that the feature vectors of the 3dimensionality matrix Ccentrðp1;1Þ centered around p1;1 are quan-tized in a lattice point p1;1 with a scaling factor c1. Then, these fea-ture vectors are firstly projected on DIMðp1;1Þ ¼ fx1; y1g obtainingCreduc;centrðp1;1Þ and secondly quantized with a scaling factorciþ1 ¼ ci=a in two lattice points of the regular lattice childðp1;1Þ.Consequently, as presented in Fig. 6, the branch Bp1;1

has two chil-dren branches forming the tree associated to the regular latticechildðp1;1Þ.

5.2. Advantages of the proposed method

Due to the problem known by ‘‘curse of dimensionality”, in themost cases, dimensionality reduction is applied as a preprocessingstep and then the multidimensional indexing technique is per-formed on the obtained lower dimensional space. The major incon-venient of a such approach is the loss of local proximityinformation between data items. The advantage of our reductiondimensionality approach is that it takes into account the distribu-tion of the data. Indeed, it is applied separately in each similargroup of vectors quantized in the same lattice point. Furthermore,

bedded lattices tree.

Page 9: Embedded lattices tree: An efficient indexing scheme for content based retrieval on image databases

Fig. 7. Quantization of Cðp1;1Þ in the lattice childðp1;1Þ.

M. Mejdoub et al. / J. Vis. Commun. Image R. 20 (2009) 145–156 153

in comparison with the regular lattice, the proposed embedded lat-tices tree realizes the trade off between the number of images inwhich the SSA is applied and the number of the browsed latticepoints. This trade off is obtained in the lower level regular latticessince they guarantee in the same time the two requirements ofdensity and low error of quantization. The density requirement isachieved thanks to the dimensionality reduction applied whenwe move from the upper levels to the lower ones. While, the lowerror of quantization is obtained thanks to the finer scaling factorused to quantize the feature vectors. Indeed, in the latticechildðpj;iÞ of level iþ 1, we take into account only the dimensionsthat realize the maximum similarity between the feature vectorsquantized in pj;i. Therefore, the feature vectors of reduced dimen-sionality quantized in the lattice childðpj;iÞ are more similar and thisallows not only to increase the density of the lower level latticesbut also to improve the feature vectors distribution in these lat-tices. Besides, The SSA is no more applied on a large number of fea-ture vectors quantized in a lattice point pj;i but it is restricted to aportion of feature vectors quantized in the lattice childðpj;iÞ locatedin the level iþ 1. Consequently, the proposed embedded latticestree enhances the nearest neighbor search performances in termof retrieval precision and speed up.

6. Experimentation and results

In experiments, we use the Corel [2] database, formed by 10 im-age categories (African people and villages, beach, buildings, buses,dinosaurs, elephants, flowers, horses, mountains and glaciers, food)each containing 100 images. We conducted our tests on two fea-ture vectors databases. The first is a database of 1000 fuzzy featurevectors of dimension Ncluster ¼ 60. Each fuzzy feature vector is ob-tained as explained in the fifth step of Section 2. The second data-base contains 1,000,000 vectors of dimension Ncluster ¼ 60. Vectorsare fuzzy local descriptors based on points of interest derived fromthe Nim ¼ 1000 images of the initial Corel database. From each im-age database, we extract Np ¼ 1000 points of interest and conse-quently 1000 fuzzy feature vectors are obtained as explained inthe fourth step of Section 2. Thus, we obtain a database ofNp � Nim ¼ 1;000;000 vectors. Experiments were performed on apersonal computer with configurations: Intel Pentium M 725(1.6 GHZ), 512 MB memory DDR 333 MHz, 80 GB HDD. We testedthe performance of our proposed image retrieval method taking

into account both the retrieval process accuracy and speed up.For the accuracy evaluation, we use the recall–precision graph(recall ¼ d=r with d is the number of correctly detected imagesand r the number of relevant images for a given query;precision ¼ d=k with k is the desired number of retrieved images,in experiments k ¼ 20;40;60;80) and for the speed up evaluation,we define the speed up parameter as:

speed up ¼ tðindexingÞ ð14Þ

with tðindexingÞ is the elapsed time for the indexing methods.For the 1000 feature vectors database, 1000 query vectors are

used to evaluate both retrieval accuracy and speed up on a subsetof 800 feature vectors taken randomly (80 images are taken ran-domly from each category) from the 1000 sized database featurevectors. For the second database, the evaluations are made on asubset of 800,000 feature vectors taken randomly (80 images aretaken randomly from each category) from the 1,000,000 sizeddatabase feature vectors. In order to evaluate the speed up, we con-sider the mean response time for searching k nearest neighbors of1000 query vectors selected randomly from the database. While,the accuracy is evaluated, using 100 images from each databasecategory. For each image, we extract 1000 salient objects. Afterthat, each salient object is used as a query. Then, from the 1000queries made, we determine the potential categories. For that,we compute the number of returned salient objects that belongto each category. Finally the system returns only the nearestimages that belong to the potential categories.

We built an embedded lattices tree with three levels using threedifferent scaling factors (c1 ¼ 1=2, c2 ¼ 1=6 and c3 ¼ 1=18) for eachlevel. Besides, we created three trees associated to three regular lat-tices. For the first one, we use the scaling factor c ¼ 1=2 (method1).For the second one, we use the scaling factor c ¼ 1=6 (method2). Forthe third one, we use the scaling factor c ¼ 1=18 (method3).

In the first experiment that we made, we compared the behav-ior of the proposed embedded lattices tree structure using the pro-posed fuzzy feature extraction method based on wavelet and thefuzzy feature extraction method based on SIFT. Even though, theproposed wavelet based feature extraction method use a low levelfeature vector with only 12 dimensions against 128 dimensions forthe SIFT descriptor, similar performances with the SIFT are ob-tained as well as for the accuracy (Fig. 8) and for the speed up per-formances (Fig. 9) in case of both 1000 and 1,000,000 featurevectors databases size. In the second experiment, we evaluatedthe performances of the proposed embedded lattices tree compar-ing it to SOM and SR-tree using the database of 1000 feature vec-tors. Experimentations (Fig. 10) show that:

(1) SOM outperforms SR-tree in term of retrieval accuracy.(2) SR-tree outperforms SOM in term of speed up.(3) The embedded lattices tree outperforms SOM and SR-tree in

term of retrieval accuracy and speed up.

Consequently, the proposed embedded lattices tree yields thebest trade off between retrieval accuracy and speed up.

In the third experiment, we evaluated the performance behav-ior using the database containing 1,000,000 feature vectors. Let callnbðnearestÞ the number of neighboring lattice points visited whenretrieving the k nearest feature vectors and nbðSSAÞ the number offeature vectors in which we perform the SSA. Note that the featuresvectors that form nbðSSAÞ are quantized in the neighboring latticepoints that are visited when retrieving the k nearest feature vec-tors. Table 2 shows that the embedded lattices tree produces thebest trade off between nbðSSAÞ and nbðnearestÞ.

We can observe in Fig. 11 that among method(1,2,3), method1produces the worst performances. The low performance achievedin term of accuracy can be explained by the great value of the scal-

Page 10: Embedded lattices tree: An efficient indexing scheme for content based retrieval on image databases

0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.50.48

0.5

0.52

0.54

0.56

0.58

0.6

0.62

recall

prec

isio

n

SIFT based methodwavelet based method

0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5 0.55 0.60.54

0.55

0.56

0.57

0.58

0.59

0.6

0.61

0.62

0.63

recall

prec

isio

n

SIFT based methodwavelet based method

Fig. 8. Accuracy comparison between wavelet and SIFT based feature extraction using databases of 1000 (left) and 1,000,000 (right) images.

20 30 40 50 60 70 80 90 1001

1.5

2

2.5

3

3.5

rank

spee

d (in

milli

seco

nds)

SIFT based methodwavelet based method

50 100 150 200 25015

20

25

30

35

40

45

50

rank

spee

d (in

milli

seco

nds)

SIFT based methodwavelet based method

Fig. 9. Speed comparison between wavelet and SIFT based feature extraction using a database of 1000 (left) and 1,000,000 (right) images.

0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5

0.35

0.4

0.45

0.5

0.55

0.6

recall

prec

isio

n

SOMSR−treeEmbedded lattices tree

20 30 40 50 60 70 80 90 1001

2

3

4

5

6

7

8

9

10

11

rank

spee

d (in

milli

seco

nds)

SOMSR−treeEmbedded lattices tree

Fig. 10. Accuracy (left) and speed (right) comparison between the embedded lattices tree, SOM and SR-tree using a database of 1000 images.

154 M. Mejdoub et al. / J. Vis. Commun. Image R. 20 (2009) 145–156

ing factor c1 while the great value of nbðSSAÞ explains the low per-formance in term of speed up. We can observe also that method2yields the best speed up among method(1,2,3) and method3 pro-duces the best accuracy among method(1,2,3). The speed up per-formance of method2 comparing to method1 and method3 can

be explained by the fact that method2 gives the best trade off be-tween nbðSSAÞ and nbðnearestÞ. While, the accuracy performance ofmethod3 is obtained by using the finest scaling factor c3. However,due to the great number of nbðnearestÞ, the search time increases inmethod3. Experimental results in Fig. 11 show that the embedded

Page 11: Embedded lattices tree: An efficient indexing scheme for content based retrieval on image databases

Table 2Comparison of the speed up results obtained for the scaling factors c (1/2, 1/6 and 1/18) corresponding, respectively, to method(1,2,3) and the embedded lattices tree (k isthe number of images required by the user).

k Trees nb(nearest) nb(SSA) t(indexing)in milliseconds

50 Method1 1.03 6014 902.2Method2 3.4 2440 366.34Method3 2001.7 128 219.37Embedded lattices tree 4.7 127 19.52

100 Method1 1.03 6014 902.2Method2 3.4 2440 366.34Method3 2001.7 128 219.37Embedded lattices tree 4.7 127 19.52

200 Method1 1.03 6014 902.2Method2 3.4 2440 366.34Method3 2632.2 198 292.92Embedded lattices tree 8.6 258 39.56

250 Method1 1.03 6014 902.2Method2 3.4 2440 366.34Method3 2812.5 218 313.95Embedded lattices tree 12.3 298 45.39

M. Mejdoub et al. / J. Vis. Commun. Image R. 20 (2009) 145–156 155

lattices structure, comparing to method1, method2 and method3yields the best trade off between retrieval precision and speedup. We can explain the obtained results by the fact that embedded

0 0.1 0.2 0.3 0.4 0.5 0.6 0.70.35

0.4

0.45

0.5

0.55

0.6

0.65

0.7

recall

prec

isio

n

method1method2method3Embedded lattices tree

Fig. 11. Accuracy (left) a and speed up (right) comparison betw

0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5 0.55 0.60.4

0.45

0.5

0.55

0.6

0.65

recall

prec

isio

n

SR−treeSOMEmbedded lattices tree

spee

d (in

milli

seco

nds)

Fig. 12. Accuracy (left) and speed (right) comparison between the embedde

lattices tree verifies the two requirements of density and low errorof quantization. The density is obtained thanks to the reduction ofdimensionality that eliminates the irrelevant dimensions allowingus to reduce nbðnearestÞ in the lower level lattices. While, the lowerror of quantization is obtained thanks to the finer scaling factorused in the lower level lattices allowing us to reduce nbðSSAÞ.

Moreover, the embedded lattices tree outperforms SOM, SR-treein both retrieval speed up and accuracy (Fig. 12). In the case of thedatabase containing 1000 feature vectors, the embedded latticestree performs the queries 1.6 times faster than the SR-tree and3.1 times faster than the SOM. While, in the case of the databasecontaining 1,000,000 feature vectors, the embedded lattices treeperforms the queries 4.9 times faster than the SR-tree and 7.59times faster than the SOM. Therefore, the speed up of the SOMand SR-tree, comparing it to the proposed embedded lattices tree,grows when the database population increases. The performancesimprovement of the embedded lattices tree in large databases,compared to SOM and SR-tree is due to the fact that, unlike theseindexing structures, the index computation in the proposedembedded lattices tree does not depend of the database size. In-deed, the index is derived only from the fuzzy feature vector itselfand independently of the other database feature vectors. On thememory usage side, as each feature vector is indexed with a shortindex of four components (norm, leader, permutation and sign),the proposed indexing structure requires a low memory cost. It

50 100 150 200 2500

100

200

300

400

500

600

700

800

900

1000

rank

spee

d (in

milli

seco

nds)

method1method2method3Embedded lattices tree

een method(1,2,3,4) using a database of 1,000,000 images.

20 30 40 50 60 70 80 90 1000

50

100

150

200

250

300

rank

SOMSR−treeEmbedded lattices tree

d lattices tree, SOM and SR-tree using a database of 1,000,000 images.

Page 12: Embedded lattices tree: An efficient indexing scheme for content based retrieval on image databases

156 M. Mejdoub et al. / J. Vis. Commun. Image R. 20 (2009) 145–156

is equal to 0:041 Mbyte for the 1000 feature vectors database andequal to 24:78 Mbyte for the 1,000,000 feature vectors database.

7. Conclusion

A new hierarchical indexing method called ‘‘The embedded lat-tices tree”, was proposed in this paper. The proposed indexingmethod has a remarkable tolerance for high input dimensionalityand an innate ability to perform feature selection thanks to theelimination of irrelevant feature dimensions. Fuzzy feature vectorsare extracted based on local characterization of the image and thefuzzy partitioning algorithm CA. The extracted fuzzy feature vec-tors are indexed in a simple way based on the method of the lead-ers. The obtained indexes are used to build the hierarchicalindexing structure. We proposed a first indexing structure basedon the regular lattice. The basic advantages of this structure isthe reduced size of the index and the elimination of on line dis-tance computations. But, its major limit is its performances depen-dence of the lattice density. In order to overcome such limit andprovide efficient solutions, we proposed the embedded lattices treethat verifies the two requirements of density and low error ofquantization. Thus, it successes to realize the trade off betweenthe accuracy and the speed up of the retrieval process. Experimen-tal results show that the proposed indexing structure outperformsthe SR-tree and SOM in both accuracy and speed of the retrieval.The perspectives of this work is to incorporate semantic featuresto discriminate between the feature vectors that belongs to differ-ent categories of the image database.

References

[1] Jia Li, James Z. Wang, Automatic linguistic indexing of pictures by a statisticalmodeling approach, IEEE Transactions on Pattern Analysis and MachineIntelligence 25 (2003) 1075–1088.

[2] R. Shi, H. Feng, T.S. Chua, C.H. Lee, An adaptive image content representationand segmentation approach to automatic image annotation, in: InternationalConference on Image and Video Retrieval (CIVR), 2004, pp. 545–554.

[3] S. Kiranyaz, K. Caglar, O. Guldogan, E. Karaoglu, MUVIS: a multimediabrowsing, indexing and retrieval framework, in: Proceedings of ThirdInternational Workshop on Content Based Multimedia Indexing, CBMI 2003,September 2003, pp. 22–24.

[4] M. Bilenko, S. Basu, R.J. Mooney, Integrating constraints and metric learning insemi-supervised clustering, in: Proceedings of the 21st InternationalConference on Machine Learning (ICML), 2004, pp. 81–88.

[5] J. Bentley, Multidimensional binary search trees in database applications, IEEETransactions on Software Engineering (1979) 333–340.

[6] V. Gaede, O. Günther, Multidimensional access methods, ACM ComputingSurveys 30 (2) (1998).

[7] J. Ren, Y. Shen, L. Guo, A novel image retrieval based on representative colors,Proceedings of the Image and Vision Computing (2003) 102–107.

[8] D. White, R. Jain, Similarity indexing with the SS-tree, in: Proceedings of the12th International Conference on Data Engineering, 1996, pp. 516–523.

[9] N. Bouteldja, V. Gouet-Brunet, M. Scholl, Evaluation of strategies for multiplesphere queries with local image descriptors, in: IST/SPIE Conference onMultimedia Content Analysis, Management, and Retrieval, San Jose, CA, USA,2006.

[10] S. Kiranyaz, M. Gabbouj, A dynamic content-based indexing method formultimedia databases hierarchical cellular tree, in: Conference on ImageProcessing (ICIP), 2005

[11] L.V. Tran, R. Lenz, PCA-based representation of color distributions for color-based image retrieval, in: International Conference in Image Processing(ICIP’01), 2001, pp. 697–700.

[12] P. Praks, J. Dvorsky, V. Snasel, Latent semantic indexing for image retrievalsystems, in: SIAM Linear Algebra, Proceedings, Philadelphia: InternationalLinear Algebra Society (ILAS), 2003, pp. 697–700.

[13] F. Jurie, B. Triggs, Creating efficient codebooks for visual recognition, in: ICCV,2005.

[14] D. Nister, H. Stewenius, Scalable recognition with a vocabulary tree, in: IEEEConference on Computer Vision and Pattern Recognition, 2006, pp. 2161–2168.

[15] B. Leibe, K. Mikolajczyk, B. Schiele, Efficient clustering and matching for objectclass recognition, in: British Machine Vision Conference, 2006.

[16] F. Moosmann, B. Triggs, F. Jurie, Randomized clustering forests for building fastand discriminative visual vocabularies, in: NIPS, 2006.

[17] S. Kaski, J. Kangas, T. Kohonen, Bibliography of self-organizing map (som),Neural Computing Surveys (1998) 1981–1997.

[18] J. Moureaux, P. Loyer, M. Antonini, Low complexity indexing method for Zn andDn lattice quantizers, IEEE Transactions on Communication 46 (12) (1998)1602–1609.

[19] T. Tuytelaars, C. Schmid, Vector quantizing feature space with a regular lattice,in: Proceedings International Conference on Computer Vision, 2007.

[20] Nizar Grira, Michel Crucianu, Nozha Boujemaa, Active Semi Supervisedfuzzy clustering for image database categorization, in: 7th ACM SIGMMInternational Workshop on Multimedia Information Retrieval (MIR’05),2005.

[21] W. Bellil, C.B. Amar, M. Alimi, Beta wavelet based image compression, in:International Conference on Signal System and Design SSD03, 2003, pp. 77–82.

[22] C.B. Amar, M. Zaied, M.A. Alimi, Beta wavelets, synthesis and application tolossy image compression, Advances in Engineering Software, Elsevier Edition36 (7) (2005) 459–474.

[23] E. Loupias, N. Sebe, Wavelet-based salient points for image retrieval, TechnicalReport TR. 99.11, Laboratoire Reconnaissance de Formes et Vision, INSA Lyon,1999.

[24] M. Antonini, M. Barlaud, P. Mathieu, I. Daubechies, Image coding using wavelettransform, IEEE Transactions on Pattern Analysis and Machine Intelligence 1(22) (1992) 205–220.

[25] M. Mejdoub, C. BenAmar, M. Antonini, Extraction d’une signature floue sebasant sur la combinaison de différentes bases d’ondelettes, in: Traitement etanalyse d’images méthodes et applications TAIMA, 2007.

[26] M. Mejdoub, C. BenAmar, Image retrieval system based on the beta wavelettransform, in: International Conference on Signal System and Devices SSD,2007.

[27] D. Lowe, Distinctive image features from scale-invariant keypoints,International Journal of Computer Vision 60 (2) (2004) 91–110.

[28] F. Hoppner, F. Klawonn, R. Kruse, T. Runkler, Fuzzy cluster analysis: methodsfor classification, Data Analysis and Image Recognition (1999) 41–53.

[29] M. Mejdoub, L. Fonteles, C. BenAmar, M. Antonini, Fast algorithm for imagedatabase indexing based on lattice, in: 15th European Signal ProcessingConference EUSIPCO, Pologne, 2007, pp. 1799–1803.

[30] F.L. Hidd, M. Antonini, Indexing Zn lattices vectors for generalized Gaussiandistributions, in: IEEE International Symposium on Information Theory (ISIT),2007.

[31] G.E. Andrews, The Theory of Partitions, Cambridge University Press, 1998.Reprint Edition July 28.