Context-based Visual Concept Context-based Visual Concept Detection Using Domain Detection Using Domain Adaptive Semantic Diffusion Adaptive Semantic Diffusion Yu-Gang Jiang ‡ , Jun Wang ‡ , Shih-Fu Chang ‡ , Chong-Wah Ngo † † VIREO Research Group (VIREO), City University of Hong Kong ‡ Digital Video and Multimedia Lab (DVMM), Columbia University 1 NIST TRECVID Workshop, Nov. 2009
22
Embed
Context-based Visual Concept Detection Using Domain Adaptive Semantic Diffusion Yu-Gang Jiang, Jun Wang, Shih-Fu Chang, Chong-Wah Ngo VIREO Research Group.
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Context-based Visual Concept Context-based Visual Concept Detection Using Domain Adaptive Detection Using Domain Adaptive
Semantic DiffusionSemantic Diffusion
Yu-Gang Jiang‡, Jun Wang‡, Shih-Fu Chang‡, Chong-Wah Ngo† † VIREO Research Group (VIREO), City University of Hong Kong‡ Digital Video and Multimedia Lab (DVMM), Columbia University
1NIST TRECVID Workshop, Nov. 2009
Overview: framework
Local FeatureLocal Feature Global FeatureGlobal Feature
SVM ClassifiersSVM Classifiers
66
55
1-41-4
VIREO-374:374 LSCOM
concept detectors
VIREO-374:374 LSCOM
concept detectors
Domain Adaptive Semantic DiffusionDomain Adaptive
Semantic Diffusion
3
Overview: performance
DASDLocal + global features
Local feature alone
Local feature is still the most powerful component (MAP=0.150) Global features help a little bit (MAP=0.156) DASD further contributes incrementally to the final detection
Overview: framework
Local FeatureLocal Feature Global FeatureGlobal Feature
SVM ClassifiersSVM Classifiers
66
55
1-41-4
VIREO-374:374 LSCOM
concept detectors
VIREO-374:374 LSCOM
concept detectors
Domain Adaptive Semantic DiffusionDomain Adaptive
Semantic Diffusion
Local feature representation
5
Chang et al TRECVID 2008; Jiang, Yang, Ngo & Hauptmann, IEEE TMM, to appear
Keypoint extraction
Visual word vocabulary 1
SIF
T fe
atur
e sp
ace
......... .........
Visual word vocabulary 2
DoG Hessian Affine
+ + +
+ + ++ + ++ + +
+
- - -
- - -
- - -
- - - - - -
- - - - - - - - -
SVM classifiers
......
+ + +
+ + ++ + ++ + +
+
- - -
- - -
- - -
- - - - - -
- - - - - - - - -
+ + +
+ + ++ + +
- - -
- - -
- - -
- - - - - -
- - - - - - - - -
Vocabulary Generation BoW Representation
+ + +
+ + ++ + +
- - -
- - -
- - -
- - - - - -
- - - - - - - - -
BoW histograms Using Soft-weighting
Context-based concept detection
Local FeatureLocal Feature Global FeatureGlobal Feature
SVM ClassifiersSVM Classifiers
66
55
1-41-4
VIREO-374:374 LSCOM
concept detectors
VIREO-374:374 LSCOM
concept detectors
DASD: Domain Adaptive Semantic
Diffusion
DASD: Domain Adaptive Semantic
Diffusion
DASD - motivation
• Most existing methods aim at the assignment of concept labels individually– but concepts do not occur in isolation!
military personnel
smoke
explosion_fire
road outdoor
vehicle
building
7
8
Documentary Videos
Broadcast News Videos
• Most existing methods aim at the assignment of concept labels individually– but concepts do not occur in isolation!
• Domain change between training and testing data was not considered