Taxonomic classification for web-based videos Author: Yang Song et al. (Google) Presenters: Phuc Bui & Rahul Dhamecha
Feb 23, 2016
Taxonomic classification for web-based videos
Author: Yang Song et al. (Google)Presenters:
Phuc Bui & Rahul Dhamecha
1. Introduction
Taxonomic classification for web-based videos
Web-based Video Classification
• Web-based Video (e.g. Youtube)– Over 800 million unique users visit / month– Over 4 billion hours of video are watched / month– 72 hours of video are uploaded / minute
• Classification– Improve User experience– Increase Website profit
What’s interesting?
• Large-scale classification– Taxonomy of categories– Unlimited domain
• Combined Approach– Text
• Labeled Web documents• Labeled Video
– Video• Content-based features
Overview Approach
• Multi-labels Classification– One classifier for each category
• Classifiers– Text-based Classifier• from Web-based Documents
– Combined Classifier• Text-based Classifier• Video content-based features
2. Algorithms
TAXONOMIC CLASSIFICATION: - THE VARIOUS CATEGORIES.
TRAINING SET OF EACH CATEGORY
Pre-trained text based classifiers of each category used for porting videos Labeled Video data is used for training these classifiersNo. of Classifiers = No. of CategoriesAda-boosting is deployed to aggregate these weak classifiers to a Strong Classifier
MIGRATION FROM TEXT TO VIDEO
Feature Extraction
Text Based Features.
President Obama: the Real Mitt Romney - Denver, Colorado
Title
Description
Keywords
Content Based Features
Moments from multi-scale analysis
Color HistogramMean, variance of each channel.
Difference between mean of center and boundary
Content Based Features contd…
Edge DetectionCanny Edge Detection Algorithm
Content Based Features contd…
Color Motion Features
Cosine Difference of the histograms of subsequent frames.
Content Based Features contd…
Shot Boundary Features
TypesHard CutFadeDissolveWipe
Hard Cut
instantaneous transition from one scene to the next
Fade
A Fade which is a gradual between a scene and a constant image (fade-out) or between a constant image and a scene (fade-in).
Dissolve
A Dissolve is a gradual transition from one scene to another in which the first scene fade-out and the second scene fade-in. so it is a combination of fade-in and fade-out.
Wipe
A Wipe is a gradual transition in which a line move across the screen, with the new scene appearing behind the line.
Integration
Labled Videos
Text Based Feature
Extraction
Apply Pre-trained Text Classifiers
F Score from Classifiers
Labled Videos
Content Based Feature
Extraction
F Score and Content Based Features are
combined
A new Classifier is trained.
3. Experiments
Data
• 5789 videos• 9087 labels• 565 categories
• 80% training• 20% evaluation
Evaluation
• Precision
• Recall
• F-score
Results
• Sample videos
Results
• 80-category classifiers • 1037-category classifiers
Results
Results
• Adaption + Content-based features classifiers
• Content-based features-only classifiers
4. Conclusion & Dicussion
• Video features– Content-based– Associated texts
• Web-documents based text classifier
• Semi-supervised learning
• Image-based classifiers– ImageNet