This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Fine-grained Walking Activity Recognition via Driving Recorder Dataset
National Institute of Advanced Industrial Science and Technology (AIST) † Keio University
‡ National Traffic Safety and Environment Laboratory (NTSEL)
http://www.hirokatsukataoka.net/
Background • ADAS; Advanced Driver Assistance Systems – A large amount of technologies have been proposed – The pedestrian deaths are on the rise – Detection systems, environment, autonomous driving car
@Pedestrian and vehicle detec0on @Lane detec0on (Environment understanding)
@Autonomous driving in Google
ADAS technologies are highly required!
Pedestrian detection • Vision-based detection is one of the important techniques – Pedestrian detection survey [Benenson+, ECCVW2014] • They implemented and compared 40+ detection approaches
– Deep Learning is applied to detect pedestrians [Sermanet+, CVPR2013] • Convolutional neural networks (CNN) • Automatic feature training and classifier
Better
Detection rate has been improving
New step toward “pedestrian analysis” • High-performance pedestrian localization – Task-assistant CNN (TA-CNN) [Tian+, CVPR2015] • The framework is consist of CNN feat. & attribute (e.g. background, location)
• Limitations of pedestrian safety systems – Pedestrian detection at present – Detection range: width of the vehicle
Going to the next “pedestrian analysis” researches!
Motivation • Fine-grained pedestrian activity recognition in addition to pedestrian detection – More detailed activity analysis – Pedestrian activity intention understanding
Probability map of danger
1.0 second is crucial time in ADAS
Why fine-grained?
Walking along a sidewalk
Turning
Crossing a roadway
Process flow • Fine-grained walking activity recognition
1. Pedestrian localization 2. Activity analysis
Improved dense trajectories (iDT)
Pedestrian detection
x x x x x x x x x x x x x x x
x x x
Trajectory (in t + L frames)
Feature extraction (HOG, HOF, MBH, Traj.)
Bag-of-words (BoW)
iDT
Detection system • Per-frame CNN feature and NMS – Region of interesting (ROI) – VGGNet feature in the detection problem – Non-maximum suppression for combining detection windows
・・・~
~・・・
NMS
Activity Recognition • Improved Dense Trajectories (iDT) [Wang+, ICCV2013] – Pyramidal image sequences and flow tracking – Feature descriptors on trajectories – Feature representation with bag-of-words (BoW)
Walking Crossing Turning
Experiments • Fine-grained walking activity recognition – Understanding small changes while people walking • Walking along a side walk & Crossing a road way • Walking straight & turning • Walking & riding a bicycle
(a) crossing (b) walking (c) turning (d) bicycle
Datasets and implementations • NTSEL dataset & Near-miss dataset
Conclusion • Fine-grained walking activity analysis for the new step of pedestrian intention understanding – State-of-the-art motion analysis algorithms are implemented – High-performance localization and recognition on the traffic datasets – Pedestrian analysis are executed in detail
• More flexible models and intention understanding – We need more data in learning step – Transition model or more strong temporal feature should be implemented