KEYFRAME-BASED VIDEO SUMMARIZATION DESIGNER Carlos Ramos Caballero Advisors: Horst Eidenberger and Xavier Giró I Nieto
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
KEYFRAME-BASED VIDEO SUMMARIZATION DESIGNER
Carlos Ramos Caballero
Advisors: Horst Eidenberger and Xavier Giró I Nieto
Contents
Introduction
State of the art
Methodology
Results assessment
Conclusions
2
The application: Designer Master
DEMONSTRATION
3
Contents
Introduction State of the art
Methodology
Results assessment
Conclusions
4
Introduction
Motivation
Designer Master: keyframe-based video summarization interface Object Maps: system for automatic video summarization
5
Graphical User Interface (Designer Master)
Computer Vision Engine (Object Maps)
Introduction
Goals of the thesis
6
Introduction
Goals of the thesis Improving the keyframe extraction module
7
Introduction
Goals of the thesis Improving the keyframe extraction module Assessing the improvement
8
Contents
Introduction
State of the art Methodology
Results assessment
Conclusions
9
State of the art
Shot segmentation
10
Hierarchical decomposition and representation of video content [1]
[1] http://www.scholarpedia.org/article/Video_Content_Structuring
State of the art
Shot segmentation example
11
Shot boundary detection example [2].
[2] Martos, M. “Content-based Video Summarization to Object Maps”, Vienna University of Technology, Austria (2013).
State of the art
Shot segmentation techniques
Pixel-to-pixel methods
• Global pixel-to-pixel • Cumulative pixel-to-pixel
Histogram-based methods • Simple histogram • Maximum histogram • Weighted histogram
Hausdorff method
12
Contents
Introduction
State of the art
Methodology Results assessment
Conclusions
13
Methodology: Implemented solution
System architecture overview
14
Methodology : Implemented solution
Uniform sampling
𝑓𝑝𝑠𝑖: frame rate of the input video.
𝐿𝑖: total number of frames of the input video.
𝑁0: total number of frames we want to keep (𝑁0=100).
15
Methodology : Implemented solution
Gray scale domain
16
Color model transformation RGB to YIQ.
Methodology : Implemented solution
Difference computation
Where 𝐼(𝑡,𝑖,𝑗) represents the intensity value at frame t in pixel(𝑖,𝑗).
X and Y are the width and height of the video frames, respectively.
17
Methodology : Implemented solution
Normalization
Where 𝑑 ̂ is the normalized value, 256 is the number of grey levels, X and Y are the width and height of the video frames, respectively.
18
Methodology : Implemented solution
Decision making
The threshold value used in our application is 𝜏 = 0.1 (as defined in [2]).
19
[2] Martos, M. “Content-based Video Summarization to Object Maps”, Vienna University of Technology, Austria (2013).
Methodology: Environment
Environment
20
Contents
Introduction
State of the art
Methodology
Results assessment Conclusions
21
Results assessment
TEST 1: Testing the applications + ‘in situ’ survey 11 participants Test data: The intouchables trailer
22
Results assessment
Example: pair of summaries
23
Designer Master v1 Designer Master v2
Results assessment
TEST 2: web-based survey 43 participants Test data: The Intouchables trailer
24
Results assessment
EVALUATION Quality of the generated summaries Representativeness of the generated summaries Mean Opinion Score
• 1. Unacceptable • 2. Poor • 3. Good • 4. Very good • 5. Excellent
25
Results assessment
Quality generated summaries
“Please, rate summary 1”
26
“Please, rate summary 2”
Results assessment
Quality generated summaries
27
MOS MOS – scores distribution
Results assessment
Representativeness of the summaries
“Which summary let you better recognize the video content?”
28
Results assessment
Representativeness of the summaries
29
Results assessment
Ease-of-use of the application
“Do you think the application is intuitive and easy to use?”
30
Results assessment
Ease-of-use of the application
31
Results assessment
Execution time
32
Contents
Introduction
State of the art
Methodology
Results assessment
Conclusions
33
Conclusions
Accomplishment of the initial goals Improving the keyframe extraction module by integrating both
projects. Assessing the improvement.
34
Conclusions
Accomplishment of the initial goals Improving the keyframe extraction module by integrating both
projects. Assessing the improvement.
Our work has slightly improved Designer Master Users can create better video summaries and easily due the better
quality of the extracted keyframes.
35
Conclusions
Accomplishment of the initial goals Improving the keyframe extraction module by integrating both
projects. Assessing the improvement.
Our work has slightly improved Designer Master Users can create better video summaries and easily due the better
quality of the extracted keyframes.
It is hoped to develop this work into a product for the Austrian Broadcasting station ORF
36
Conclusions
Accomplishment of the initial goals Improving the keyframe extraction module by integrating both
projects. Assessing the improvement.
Our work has slightly improved Designer Master Users can create better video summaries and easily due the better
quality of the extracted keyframes.
It is hoped to develop this work into a product for the Austrian Broadcasting station ORF
37
Thank you very much for your attention!
Danke schön!
Moltes gràcies!
38
Related Documents
