Multimedia Indexing and Retrieval Kowshik Shashank Project Advisor: Dr. C.V. Jawahar
Dec 19, 2015
Multimedia Indexing and Retrieval
Kowshik
Shashank
Project Advisor: Dr. C.V. Jawahar
Problem Statement
“Develop efficient algorithms for a real time, private multimedia Develop efficient algorithms for a real time, private multimedia databasedatabase”
Applications
Defense systems
Surveillance systems
Image/Video collections (under copyright notices)
Web 2.0
Web image search
FeatureExtraction Indexing
QueryFeature
Extraction
Database
SimilarityMeasure
Result
Indexing Schemes
Hierarchical Structures
Vocabulary Trees
Hashing
Private Retrieval In Hierarchical Structures
Querying in CBIR
……..
Feature vector
Query Image
Private Content Based Image Retrieval1. The user extracts the feature vector of the query image, say fquery.
2. The user asks for the data stored in the root node of the indexing structure.
3. fquery and the information are used to decide whether to access the left or the right sub-tree.
4. The user frames a Query Qi to access the node at level i.
5. The database replies with Ai for the query Qi .
6. The user performs a function f( Ai ) to obtain the information at the node.
Go to step 3.
Private Content Based Image Retrieval
A2Q2
Q1 A1
Feature vector (fquery)
……..
Root Info
fquery, f(A1)
fquery, f(A2)
Quadratic Residuosity assumption Consider a natural number N = p. q where p, q are large prime
numbers.
Construct a set
`y` is called a Quadratic Residue (QR), if x | y = x2 and x, y else `y` is called a Quadratic Non-Residue (QNR).
Construct a set YN with equal number of QRs and QNRs
.1,gcd,1|* xNNxxzN
*NZ
*NZ
Quadratic Residuosity Assumption:
Given a number `y` YN, it is predictably hard to decide whether `y` is a QR or a QNR.
Basic Rules
QNR * QNR = QRQNR * QR = QNRQR * QR = QR
Viewing the nodes in a level
QR QR QNR QR
User
ith1 0 1 0
Database
QR2 QR QNR2 QR
User
ith1 0 1 0
Database
Now the user decides on the ith element as QR or QNR and decides upon the data at the ith index in the database.
Q1 Q2 Q3 Q4
Q12 Q2 Q32 Q4
Querying on a Linear Database
A[i] = Q[i] if 0
A[i] = Q[i]2 if 1
Q
A
Converting to 2D database
QNR
QR
…..
….
….
QR
QNR
m x n
Frame a query of length ‘m’ with a QNR in the position of the row in which the node occurs
QR
…..
….
…..
….
The database forms a m x n matrix with the first bit of information
QR
00
1
0
1 1
1 1 QR
…..
….
….
QNR
m x n
0
…..
1
0
….
…..
….
QRQR2
QNR
QR2 QR2
QR2 QR2 QR
…..
….
….
QNR
m x n
Put the square of the number if the bit value is 1 else retain the same number
QNR
…..
QNR2
QR
….
…..
Multiply along the columns
QNR QNRQR …..Ai
….
Framing the Query and Reply
If the user is interested in the data at node (x,y)
Frame a query of length m in which the xth value is a QNR and rest are QR.
The database computes the reply Ai of length n and returns to the user.
If the value of Ai[y] is a QR then the value is 1 else 0.
Complexity of the algorithm
The communication complexity is O(m) on the user side and O(n) on the server side. Hence the communication complexity is O(max(m,n))
If m = n = , the communication complexity isi2
Extension to other Hierarchical Structures Hierarchical Structures
Number of nodes at each level. Information at a node.
Any number of nodes can be converted into a ‘m x n’ matrix.
Any information can be represented in binary format.
If the user has the data about the indexing structure and the format of the information stored at a node, the algorithm can be simulated for any hierarchical structure.
Results
KD Tree and Corel Database Corel Database consists of 9907 images. Color feature extracted as color histogram with 768 dimensions. Average Retrieval Time: 0.596 secs Sample Results
Results
Vocabulary Tree and Nister Dataset Nister Dataset consists of 10,200 images. SIFT features used to obtain visual words. Vocabulary size of 10000 visual words. Average Retrieval Time: 0.320 secs Sample Results
Results Vocabulary size was varied to test the scalability of the algorithm.
As the size increases, the size of the tree increases causing more data to be exchanged, thus increasing the average retrieval time.
Results LSH and Corel Dataset
LSH – Locality Sensitive Hashing 90 hash functions each having 450 bins on an average. Two level hierarchy. Average Retrieval Time: 0.221 secs Confusion metric was varied to obtain various levels of privacy. As confusion metric decreases, the data exchanged decreases thus giving faster
retrieval times.
Results
The algorithm was tested for its scalability. Synthetic datasets to the tune of a million images were used to test the practicality of
the algorithm.
Dataset Size Query Time(in secs)
210 0.005832
212 0.008856
214 0.012004
216 0.037602
218 0.129509
220 0.261255
Conclusion
We have addressed the problem of private retrieval in Image databases.
The algorithm is shown to be customizable for all hierarchical structures as well as Hash based Indexing.
Experimental study shows that the algorithm is accurate, efficient and scalable.
Algorithm is fully private and feasible on large image databases using the state of art indexing schemes.
Demonstrated a near linear operating region for image databases, where the trade off between privacy and speed is feasible.
m x n m x n Qi
0 1
1 1
1 …..
….
1
…..
0
0
….
…..
….
0 1
1 1
1 QR
QR
…..
….
….
QNR1 1
…..
0
0…
.
…..
….
1
QR QR2
QR2 QR2
QR2 …..
….
m x n
QNR2 QNR2
…..
QNR
QR
….
…..
….
QR QR2
QR2 QR2
QR2 …..
….
m x n
QNR2 QNR2
…..
QNR
QR
….
…..
….
QR QR…..QNRAi