合同勉強会20160821

Large-scale Visual Search大規模画像検索

NGUYEN ANH [email protected]

2016/08/21

About me• 東京大学 情報理工学系研究科修士2年生

• テーマ：Object Retrieval，情報検索等

• 趣味：水泳，囲碁• ブログ：

https://imsmarxen68.tumblr.com/

1

Outline• Visual Search• Introduction of a visual search framework• Large-scale problems

2

A picture is worth a thousand words

Visual Search

Searcher

Image index

Query image

Result image

1st2nd

3rd

4th

RankImage credit: http://ai.stanford.edu/~jkrause/cars/car_dataset.html

4

Large-scale image retrieval

Handwritten document retrievalImage credits: ImageCLEF2016 (http://www.imageclef.org/2016/handwritten)

Medical image retrievalImage credits: [1] J. Wang et al., "Bag-of-Features Based Medical Image Retrieval via Multiple Assignment and Visual Words Weighting," in IEEE Transactions on Medical Imaging, vol. 30, no. 11, pp. 1996-2011, Nov. 2011

MarketingImage credits: http://ai.stanford.edu/~jkrause/cars/car_dataset.html

Feature extractionFeature

extractionFeature

aggregationFeature

matching Re-ranking

Preliminaryresults

Finalresults

Image credits: http://ai.stanford.edu/~jkrause/cars/car_dataset.html

A picture is worth a thousand

words

6

Number plate

TyreTyre

Feature extraction

Featureaggregation

Featurematching Re-ranking

Preliminaryresults

Finalresults

Statistical kernels 7

Number plate

TyreTyre

Bag-of-Features (BoF)

Goal: one image → one vector Clustering

Feature extraction

Featureaggregation

Featurematching Re-ranking

Preliminaryresults

Finalresults

Statistical kernels

• GMM clustering → Fisher Vector[1]

• K-means clustering→BoF, VLAD[2]

Image credits: http://www.mathworks.com/matlabcentral/

[1] F. Perronnin, C. Dance, “Fisher Kernels on Visual Vocabularies for Image Categorization,” in Proc. CVPR, IEEE, 2007[2] H. Jegou, F. Perronnin, M. Douze, J. Sanchez, P. Perez, C. Schmid, “Aggregating Local Image Descriptors into Compact Codes,” IEEE Trans. Pattern Anal. Mach. Intell. 34 (2012) 1704–1716.

8

Goal: one image → one vector

Feature extraction

Featureaggregation

Featurematching Re-ranking

Preliminaryresults

Finalresults

Image matching = Feature matching

• Feature matching→Nearest Neighbor Search– Inverted files for faster search– Compressed data for better memory usage [3]

[3] H. Jégou, M. Douze, C. Schmid, Product quantization for nearest neighbor search., IEEE Trans. Pattern Anal. Mach. Intell. 33 (2011) 117–28. Data Compression

9

Query

Large-scale Visual Search

Large-scale problems• Features have high dimensionality

– 100~100,000• Too many images then, too many features

– Million-scale of images = billion-scale of features

11

A picture is worth a

thousand words

Problems• Memory

– Image as compact features– Vector compression

• Speed– Inverted indexing techniques– Approximate feature search

• Learning time– Clustering time

• Accuracy– Improve image representations– Reduce compression error– Multi-modal search, fine-grained systems.

12

Thank you for listening

HTTP://WWW.SLIDESHARE.NET/ANHTUAN68/20160821

本日のスライドは下記のURLにあります。