1September 8, 2014
Optimization of computer vision algorithms in codesign methodologies
Marcos Nieto, Juan Diego Ortega, Oihana Otaegui, Andoni Cortés
Vicomtech-IK4
1- ITS World Congress 2014,
September 8 -
2September 8, 2014
Overview
• Motivation - ADAS• Computer vision for ADAS• Codesign Methodology• Test case
– Vehicle detection Prototype #1– Vehicle detection Prototype #2– Vehicle detection Prototype #3
• Results• Conclusions
1
3September 8, 2014
↑ Safety ↑ Traffic Mobility ↓ Energy Consumption
ADASAdvanced Driver Assistance Systems
Motivation - ADAS
• Advanced Driver Assistance Systems (ADAS)
2
4September 8, 2014
Computer vision for ADAS
• Camera sensors– Cheap electronics– Greater quantity and quality of information– Perception similar to human– Can be fused with other sensors/systems: GPS, RADAR, V2V, V2I
Traffic sign recognition Safety distance Driver Monitoring Lane Keeping Pedestrian detection
3
5September 8, 2014
Computer vision for ADAS
• Embedding computer vision algorithms is a challenge
– (HW) Specify (camera, illumination, algorithm...)
– (SW) Optimize algorithm
– (HW) Select target processor
– (SW/HW) Fine tune algorithm parameters
– (SW/HW) Optimize code• Fit process flow to HW capabilities• Exploit massive parallelization HW
– (SW/HW) Add installation / calibration / maintenance / data logging procedures
C++ C, Assembly (ARM)C++ VHDL, CUDA (FPGA, GPU)
4
6September 8, 2014
Codesign Methodology
• Coordinating SW and HW developments– Using Re-configurable HW platforms– Exploiting platform-independent SW libraries
User requirements
Design SW Design/Select HW
Integrate SW in HW - Prototype
Profile SW in HW
Protpotype Validation
User requirements
Design SW Design/Select HW
Integrate SW in HW - Prototype
Profile SW in HW
Protpotype Validation
Itera
te
Last iteration Itera
te
Iterate
Co-design development cycle Traditional SW/HW development cycle
5
7September 8, 2014
Codesign Methodology
• SW and HW developments can run in parallel– More efficient joint development– SW design can be revised after profiling reports in target HW– Expensive code migration/optimization is significantly minimized
“It is better to revise the algorithm rather than spend months optimizing the code”
• The development cycle– First SW prototype can be directly tested in HW– Profiling provides bottleneck information– Revision of SW design solves detected bottleneck– Iterate until SW and HW meet requirements– Optimize final SW in target HW
6
8September 8, 2014
Vehicle detection Prototype #1
Multiscale
HOG
Adaboost Clustering SVM 1.0 Tracking
Shadows, edges,
symmetry
Training Training
Images
Brute- force Mu ltiscale Sca nning
• Multiscale scanning and detection-by-classification– Traditional (and successful) approach in computer vision– A car model is trained with a large dataset of example
images– New images are scanned at different scales and each
candidate region is compared with the model
7
9September 8, 2014
Vehicle detection Prototype #2
• Bottleneck #1: – It can propose absurd hypotheses according to size
and position that are anyway compared with the model
• Solution:– Exploit perspective information to create a grid of possible locations of
vehicles
Perspective Adaboost Clustering SVM 2.0 Tracking
TrainingTraining
HOG
Shadows, edges,
symmetry
Images
8
10September 8, 2014
Vehicle detection Prototype #3
• Bottleneck #3:– Some hypotheses are really likely not cars, and the comparison with
the model is costly
• Solution:– No need to fully compare with the trained model– Run pre-analysis to discard clearly not car candidates using simpler
features: shadows, simmetry, edges, etc.
Pre-analysis / Integral
SVM 2.0 TrackingPerspective clustering
Perspective Training
HOGShadows,
edges, symmetry
Images
Pre-selected candidates9
11September 8, 2014
Results
• Profile comparison– Only CPU, without code optimization– 10x speed up from prototype #1 to #3 -> real time performance
Xilinx Zynq Dual ARM® Cortex-A9 MPCore, 866 MHz, 1 GB
10
Features (ms) HOG (ms) Adaboost Cluster SVM (ms) Av. Time (ms)
Prototype #1 135.88 252.23 18.45 0 13.67 420.23
Prototype #2 56.43 112.56 5.88 0 4.12 178.99
Prototype #3 11.23 6.19 0 10.22 4.12 31.76
25
75
125
175
225
275
325
375
425
Tim
e (m
s) p
er fr
ame
12September 8, 2014
Results
• Sample video(s)
PETS 2001 dataset
11
Frontal camera Rear camera
13September 8, 2014
Conclusions
• Co-design strategies can reduce time-to-market for computer vision systems
• It allows SW and HW teams work in parallel
• Iterative SW re-design is possible
• We have shown a vehicle detection example
• Three prototypes with x10 speed-up without code optimization nor use of special HW
12
14September 8, 2014 13
Booth 3008
Live Demo at our booth:
Thank You!
