Motivation Moving from lab-controlled saliency detection to real world application requires more than “better AUC numbers” Comparing different saliency detection algorithms requires a mechanism to evaluate uncertainty and statistical significance To be applicable to wide range of videos, estimation needs to be HVS-inspired rather than data-fitted, hence unsupervised. UNSUPERVISED UNCERTAINTY ANALYSIS FOR VIDEO SALIENCY DETECTION Tariq Alshawi, Zhiling Long and Ghassan AlRegib {talshawi, zhiling.long, alregib}@gatech.edu School of Electrical and Computer Engineering, Georgia Institute of Technology, Atlanta, GA 30332, USA 1. Chenlei Guo; Liming Zhang, "A Novel Multiresolution Spatiotemporal Saliency Detection Model and Its Applications in Image and Video Compression," in Image Processing, IEEE Transactions on , vol.19, no.1, pp.185-198, Jan. 2010 2. F. W. M. Stentiford, “Attention based Auto Image Cropping,” Workshop on Computational Attention and Applications, ICVS, Bielefeld, March 21-24, 2007. 3. K. Debattista, L.P. Santos, A. Chalmers, Accelerating the irradiance cache through parallel component-based rendering, in 6th Eurographics Symposium on Parallel Graphics Visualization. Eurographics, May 2006, pp. 27-34. Uncertainty Framework Proposed Method Experiments Data Public CRCNS database 50 video clips (640 x 480), 5-90 seconds each, 30 frames/sec Street scenes, TV programs, video games, etc. Ground truth eye fixation data from human subjects under freeview condition Feature Extraction Feature Extraction Localized 3D FFT Spectral Decomposition Center-Surround Comparison F F t F s Center-Surround Comparison + S t S s S E t E s f t f x f y ᶱ M (a 0 , b 0 , c 0 ) a 0 b 0 c 0 N (a 0 , b 0 , 0) O Auto-Cropping 2 Rendering 3 Video Processing Algorithm Saliency Detection Algorithm Decision Making Module Risk Assessment Module Uncertainty Estimation Input Video Saliency Map Uncertainty Map Decision Map Output Video Cost Compression 1 Evaluation Methodology Input Video Saliency Detection Evaluation Dataset Results Expanded Eye- fixation map Saliency Map True Uncertainty - Uncertainty Estimation Estimated Uncertainty Fixed Threshold Receiver Operation Characteristics (ROC) M N D Saliency Map S Spatial Neighbors Frame# N Frame# N+1 Frame# N–1 Pixel of Interest Temporal Neighbors 0.6 1 0.6 1 -0.75 1 0.6 1 0.6 Estimated Uncertainty U e True Uncertainty U tr for beverly05, frame 5 Saliency value at pixel x Average value around pixel x
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
-
Motivation
Moving from lab-controlled saliency detection to real world application requires more than “better AUC numbers”
Comparing different saliency detection algorithms requires a mechanism to evaluate uncertainty and statistical significance
To be applicable to wide range of videos, estimation needs to be HVS-inspired rather than data-fitted, hence unsupervised.
UNSUPERVISED UNCERTAINTY ANALYSIS FOR VIDEO SALIENCY DETECTIONTariq Alshawi, Zhiling Long and Ghassan AlRegib
{talshawi, zhiling.long, alregib}@gatech.eduSchool of Electrical and Computer Engineering, Georgia Institute of Technology, Atlanta, GA 30332, USA
1. Chenlei Guo; Liming Zhang, "A Novel Multiresolution Spatiotemporal Saliency Detection Model and Its Applications in Image and Video Compression," in Image Processing, IEEE Transactions on , vol.19, no.1, pp.185-198, Jan. 2010
2. F. W. M. Stentiford, “Attention based Auto Image Cropping,” Workshop on Computational Attention and Applications, ICVS, Bielefeld, March 21-24, 2007.3. K. Debattista, L.P. Santos, A. Chalmers, Accelerating the irradiance cache through parallel component-based rendering, in 6th Eurographics Symposium on
Parallel Graphics Visualization. Eurographics, May 2006, pp. 27-34.
Uncertainty Framework
Proposed Method Experiments
Data
Public CRCNS database
50 video clips (640 x 480), 5-90 seconds each, 30 frames/sec
Street scenes, TV programs, video games, etc.
Ground truth eye fixation data from human subjects under freeview condition
Feature
Extraction
Feature
Extraction
Localized
3D FFT
Spectral
Decomposition
Center-Surround
ComparisonF
Ft
Fs
Center-Surround
Comparison
+
St
Ss
S
Et
Es
ft
fx
fy
ᶱ
M (a0, b0, c0)
a0
b0
c0
N (a0, b0, 0)
O
Auto-Cropping2
Rendering3
Video Processing Algorithm
Saliency Detection Algorithm
Decision Making Module
Risk Assessment
Module
Uncertainty Estimation
Input Video
Saliency Map
UncertaintyMap
DecisionMap
Output Video
Cost
Compression1
Evaluation Methodology
Input Video
Sa
liency D
ete
ctio
nE
va
luatio
n D
ata
se
tR
esu
lts
Expanded Eye-fixation map
Saliency Map
True Uncertainty
-
Uncertainty Estimation
Estimated Uncertainty
Fixed Threshold
Receiver Operation Characteristics
(ROC)
M
N D
Saliency Map S
Spatial
Neighbors
Frame# N Frame# N+1Frame# N–1
Pixel of
Interest
Temporal
Neighbors
0.6 1 0.6
1 -0.75 1
0.6 1 0.6
Estimated
U
ncertain
ty Ue
Tru
e U
nce
rtai
nty
Utr
for beverly05, frame 5
Saliency value at pixel x
Average value around pixel x
Related Documents
