Transcript
IntentSearch: Capturing User
Intention for One-Click Internet
Image Search
Presented by
DILSHA V V
IT09106008
MESCE
Guided by
NISHA T M
Objective
Search engine which helps to interpret users search
intention by using ONE-CLICK query image
Contents
Introduction
Existing System
Proposed System
4 steps used
Search techniques
Visual feature design
Adaptive Weight Schema
Features for query categorization
Image Clustering
Advantages
Future enhancement
Conclusion
References
Introduction•Novel Internet image search
•Search engine which helps to interpret users‟ search
intention by using ONE-CLICK query image
•Uses 4 steps for image searching
•Text based information of query word and visual content of
query image to expand the image pool
Introduction(cont..)
User search intention only by query keywords is
difficult because text based image search suffers from..
•Ambiguity of query keywords
•User doesn't have enough knowledge
•Hard for users to describe the visual content of target
images
•Easier search by using both textual and visual content
of query
•Web-scale image search engines mostly rely on
surrounding text features.
•Users‟ search intention by only by query keywords
Existing System
Proposed system
•Image search on the basis of both textual and visual
content of images
•Image pool is re-ranked based on textual and visual
features
Fig. 1: Top-ranked images returned from „Bing‟ using
“apple” as query
4 steps used
Key contribution is to capture the users‟ search intention from
this one-click query image in four steps
•Adaptive similarity
•Keyword expansion
•Visual query expansion
•Image pool expansion
Adaptive similarity
User always has specific intention when submitting a
query image
Categorized into one of the predefined adaptive weight
categories, such as “portrait” and “scenery.”
Correspondence between the query image and its
proper similarity measurement reflects the user
intention.
Keyword expansion
Query keywords are expanded to capture users‟ search
intention inferred from the visual content of query
images
A word w is suggested as an expansion of the query
Image pool expansion
Retrieved by text-based search accommodates images
with a large variety of semantic meanings
More accurate query by keywords is needed to narrow
the intention and retrieve more relevant images.
The user to click on one of the suggested keywords
Both visual and textual information captured are
automatically added into the text query and enlarge the
image pool
Visual query expansion
One query image is not diverse enough to capture the
user‟s intention.
To learn visual and textual similarity metrics, which
are more robust and more specific to the query, for
image reranking.
Search techniques
The user first submits query keywords q.
A pool of image is retrieved by text-based search
User is asked to select the query image from image pool
The query image is classified as one of the predefined
adaptive weight categories
Images in the pool are re-ranked based on their visual
similarities to the query image
Similarities are computed using the weight schema
Visual feature design
Existing features : Gist ,
SIFT,
Daubechies Wavelet ,
Histogram of Gradient (HoG)
New features : Attention guided Color Signature,
Color spatialet (CSpa) ,
Multilayer Rotation Invarient ( EOH),
Facial Feauter
Adaptive Weight Schema
•Weight schema is used for similarity calculations
•Lets take image i and j…
Adaptive similarity between i & j
Sq(i , j) = ∑fm=1
αmq sm(i , j)
where sm(i , j) similarity between i and j on feature m
f is the visual feature
αmq is the express the importance of feature m for
measuring similarity
Existence of faces, the number of faces in the image
Percentage of the image frame taken up by the face region
Coordinate of face center relative to the centre of image
Directionality
Color Spatial Homogeneousness (variance of values in
different blocks of Color Spatialet)
Total energy of edge map obtained from Canny Operator
Edge Spatial Distribution
Features for query categorization
•Image is divided into clusters
•Each word wi has ti clusters
C(wi)= { ci,1 ,.............,ci,ti }
•Visual distance between the query image and a cluster c is
calculated as the mean of the distances between the query
image and the images in c.
•The cluster Ci,j with the minimal distance is chosen as
visual query expansion and its corresponding word wi .
q = wi + q‟
Image Clustering
• Duplicate images
• User friendly
• Easy search for a particular image(on the internet)
• Can find the image is real or not
Advantages
Disadvantages
Future enhancement
•Query log data, which provides valuable co-occurrence
information of keywords , for keyword expansion
•Can be improved by including duplicate detection in
the future work
Conclusion
•Internet image search approach which only requires
one-click user feedback
•Intention specific weight schema
•Without additional human feedback
•Possible for industrial scale image search by both text
and visual content
References
[1] Y. Zhang, Z. Jia, and T. Chen, “Image Retrieval with
Geometry-Preserving Visual Phrases,” Proc. IEEE Int‟l
Conf. Computer Vision and Pattern Recognition, 2011.
[2] J. Cui, F. Wen, and X. Tang, “IntentSearch:
Interactive On-Line Image Search Re-Ranking,” Proc.
16th ACM Int‟l Conf. multimedia,2008.
[3] “Bing Image Search,” http://www.bing.com/images,
2012.
[4] J. Deng, A.C. Berg, and L. Fei-Fei, “Hierarchical
Semantic Indexing for Large Scale Image Retrieval,”
Proc. IEEE Int‟l Conf.Computer Vision and Pattern
Recognition, 2011.
References(cont…)
[5] Y. Cao, C. Wang, Z. Li, L. Zhang, and L. Zhang, “Spatial-
Bag-of-Features,” Proc. IEEE Int‟l Conf. Computer Vision
and Pattern Recognition, 2010.
[6] J. Deng, A.C. Berg, and L. Fei-Fei, “Hierarchical
Semantic Indexing for Large Scale Image Retrieval,” Proc.
IEEE Int‟l Conf.Computer Vision and Pattern Recognition,
2011.
[7] Y. Huang, Q. Liu, S. Zhang, and D.N. Metaxas, “Image
Retrieval via Probabilistic Hypergraph Ranking,” Proc. IEEE
Int‟l Conf.Computer Vision and Pattern Recognition, 2011
THANK YOU
top related