SheepDog – Group and Tag Recommendation for Flickr Photos by Automatic Search-based Learning Introduction People who use album on sharing websites always like their photos to be: Popular : more people view their photos & good appreciation Easy management : while attach to groups & add tags are big trouble (about 15000 groups related to “dog” on Flickr) Our Goal: recommend photo(s) to suitable and popular groups and attach relevant tags to each photo automatically Prediction result ● Photo-level data collection Group-level data collection (b) Training Data Acquisition Feature extraction (d) Concept Detection SVM training (c) Model Learning Flickr photo Database concept detector Feature extraction top-n concepts c 1 c n … g 1,1 g 1,p … g n,p g n,1 … (e) Group Recommendation (f) Tag Recommendation Input test image … T 1,1 T 1,2 T 1,d … T n,1 T n,2 T n,d … Recommend to user Recommend to user (a) Concept Definition -- “dog”, “tiger”, “flower”, … Concept Detection Compare two source of pseudo-positive training data from Flickr Photo-Level data mechanism Group-Level data mechanism Provide a new idea of “how to acquire reliable search-based data ” The SVM predictor gives each concept a probability value to indicate the degree that the input photo fits this concept. For the top-n concepts, we recommend users the most popular groups and tags related to these concepts using our ranking algorithm. Animal Architect ure Nature Scene Portrai t Plant Color Oriente d Overall Average Photo level 1.10 1.51 1.72 1.07 1.79 1.51 1.55 Group level 1.24 1.68 1.90 1.40 1.92 1.56 1.69 tiger cat animals dog monkey snake rabbit portrait bird horse 0% 4% 8% 12% 16% 20% Photo-level probability distribution average tiger animals dog cat snake monkey portrait rabbit bird wood 0% 4% 8% 12% 16% 20% Group-level probability distribution average Subjective test result The score for the top-3 concepts recommendation results [1]: S = (P*1 + R*0.5 +W*0)/N c (N c = 15) [1] L. S. Kennedy, S.-F. Chang, and I. V. Kozintsev, "To search or to label?: predicting the performance of search-based automatic image classifiers," in Proc. ACM MIR’06, pp.249 – 258, 2006. Hong-Ming Chen, Ming-Hsiu Chang, Ping-Chieh Chang, Ming-Chun Tien, Winston H. Hsu, and Ja-Ling Wu Communications and Multimedia Lab, National Taiwan University {blacksmith,cmhsiu,pingchieh,trimy,winston,wjl}@cmlab.csie.n tu.edu.tw