Transcript
Large-scale Visual Search大規模画像検索
NGUYEN ANH TUANtuannguyen.research@gmail.com
2016/08/21
About me• 東京大学 情報理工学系研究科修士2年生
• テーマ：Object Retrieval，情報検索等
• 趣味：水泳，囲碁• ブログ：
https://imsmarxen68.tumblr.com/
1
Outline• Visual Search• Introduction of a visual search framework• Large-scale problems
2
A picture is worth a thousand words
Visual Search
Searcher
Image index
Query image
Result image
1st2nd
3rd
4th
RankImage credit: http://ai.stanford.edu/~jkrause/cars/car_dataset.html
4
Large-scale image retrieval
Handwritten document retrievalImage credits: ImageCLEF2016 (http://www.imageclef.org/2016/handwritten)
Medical image retrievalImage credits: [1] J. Wang et al., "Bag-of-Features Based Medical Image Retrieval via Multiple Assignment and Visual Words Weighting," in IEEE Transactions on Medical Imaging, vol. 30, no. 11, pp. 1996-2011, Nov. 2011
MarketingImage credits: http://ai.stanford.edu/~jkrause/cars/car_dataset.html
Feature extractionFeature
extractionFeature
aggregationFeature
matching Re-ranking
Preliminaryresults
Finalresults
Image credits: http://ai.stanford.edu/~jkrause/cars/car_dataset.html
A picture is worth a thousand
words
6
Number plate
TyreTyre
Feature extraction
Featureaggregation
Featurematching Re-ranking
Preliminaryresults
Finalresults
Statistical kernels 7
Number plate
TyreTyre
Bag-of-Features (BoF)
Goal: one image → one vector Clustering
Feature extraction
Featureaggregation
Featurematching Re-ranking
Preliminaryresults
Finalresults
Statistical kernels
• GMM clustering → Fisher Vector[1]
• K-means clustering→BoF, VLAD[2]
Image credits: http://www.mathworks.com/matlabcentral/
[1] F. Perronnin, C. Dance, “Fisher Kernels on Visual Vocabularies for Image Categorization,” in Proc. CVPR, IEEE, 2007[2] H. Jegou, F. Perronnin, M. Douze, J. Sanchez, P. Perez, C. Schmid, “Aggregating Local Image Descriptors into Compact Codes,” IEEE Trans. Pattern Anal. Mach. Intell. 34 (2012) 1704–1716.
8
Goal: one image → one vector
Feature extraction
Featureaggregation
Featurematching Re-ranking
Preliminaryresults
Finalresults
Image matching = Feature matching
• Feature matching→Nearest Neighbor Search– Inverted files for faster search– Compressed data for better memory usage [3]
[3] H. Jégou, M. Douze, C. Schmid, Product quantization for nearest neighbor search., IEEE Trans. Pattern Anal. Mach. Intell. 33 (2011) 117–28. Data Compression
9
Query
Large-scale Visual Search
Large-scale problems• Features have high dimensionality
– 100~100,000• Too many images then, too many features
– Million-scale of images = billion-scale of features
11
A picture is worth a
thousand words
Problems• Memory
– Image as compact features– Vector compression
• Speed– Inverted indexing techniques– Approximate feature search
• Learning time– Clustering time
• Accuracy– Improve image representations– Reduce compression error– Multi-modal search, fine-grained systems.
12
Thank you for listening
HTTP://WWW.SLIDESHARE.NET/ANHTUAN68/20160821
本日のスライドは下記のURLにあります。
top related