Temporal Action Localization in Untrimmed Videos via Multi- Stage CNNs Slides by Alberto Montes Computer Vision Group Reading Group , June 13th, 2016 [arXiv ] [code ] Zheng Shou, Dongang Wang and Shih-Fu Chang
Temporal Action Localization in Untrimmed Videos via Multi Stage CNNs
Jan 12, 2017
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Temporal Action Localization in Untrimmed Videos via Multi-Stage CNNs
Slides by Alberto MontesComputer Vision Group Reading Group,
June 13th, 2016
[arXiv] [code]
Zheng Shou, Dongang Wang and Shih-Fu Chang
Introduction
Previous Work
Improved Dense Trajectory (iDT)
Fisher Vector2D Convolution
Segment-CNN
Segment-CNN
Segment-CNN
Segment-CNN
Problem Definition
Video:
frame # frames
Annotations:
Candidates:
action category
action categorystart and ending frame
Multi-Scale Segment Generation
◉ Each frame resized to 171x128 pixels◉ Temporal sliding windows:
○ 16, 32, 64, 128, 256, 512 frames○ 75% overlap
◉ Construct segment s by uniformly sampling 16 frames
Network Architecture
C3D Network
Training Proposal and Classification Network
◉ lr=0.0001 except fc8 lr=0.01, momentum=0.9, weight decay factor=0.0005
◉ Drop lr by factor of 2 every 10K iterations
Proposal Network:
● fc8: 2 nodes
Classification Network:
● fc8: K+1 nodes
Localization Network
Add Custom Loss function
Localization Network
true class label
overlap sensitivity
Try to boost segments with high overlap
Works best with: λ = 1, α = 0.25
Localization Network
Learning target:
Localization Network
Prediction and Post-processing
◉ Keep segments with Ppro
> 0.7◉ Remove background segments◉ P
loc multiply with class-specific frequency of
occurrence for each window length in the training data to leverage window length distribution patterns
◉ NMS based on Ploc
to remove redundancy.
(θ - 0.1)
Experiments
MEXaction2
“Bull Charge Cape” and
“Horse Riding” videos
77 hours of videos
Training set: 1336 instances
Validation set: 310 instances
Test set: 329 instances
Datasets
THUMOS 2014
Temporal Action Detection Task
20 categories
Training set: 2755 videos
Validation set: 1010 videos and 3007 instances
Test set: 1574 videos and 3358 instances
Results MEXaction2
DFT: Dense Trajectory Features + SVM
Results MEXaction2
Results MEXaction2
Evaluation
Evaluation
Evaluation
Impact of individual networks:
Conclusions
Propose a multi-stage framework Semgent-CNN to address temporal action location
“
Thank you!Questions?
Related Documents
