CONCEPTS, ALGORITHMS & PRACTICAL APPLICATIONS IN 2D AND 3D COMPUTER VISION Michael Rauter, Christian Zinner, Andreas Zweng, Andreas Zoufal, Julia Simon, Daniel Steininger, Markus Hofstätter und Andreas Kriechbaum Senior Scientist Center for Vision, Automation & Control Autonomous Systems AIT Austrian Institute of Technology GmbH Vienna, Austria Csaba Beleznai
61
Embed
CONCEPTS, ALGORITHMS & PRACTICAL APPLICATIONS IN 2D …€¦ · Senior Scientist Markus Hofstätter und Andreas Kriechbaum Center for Vision, Automation & Control Autonomous Systems
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
CONCEPTS, ALGORITHMS & PRACTICAL
APPLICATIONS IN 2D AND 3D COMPUTER
VISION
Michael Rauter, Christian Zinner, Andreas Zweng,
Andreas Zoufal, Julia Simon, Daniel Steininger,
Markus Hofstätter und Andreas KriechbaumSenior Scientist
Center for Vision, Automation & Control
Autonomous Systems
AIT Austrian Institute of Technology GmbH
Vienna, Austria
Csaba Beleznai
2
GRAND
CHALLENGES
RECOGNITION
SEGMENTATION
RECONSTRUCTION
TIME GRAND
CHALLENGESTIME
▪ Research is evolution → so is your learning process
▪ Balance: becoming a domain expert vs. being a „globalist“
▪ Researchers tend to favour certain paradigms - Learn to outline trends, look upstream
▪ Revisit old problems to see them under new light
▪ Specialize the general & Generalize the specific
▪ Factorize your know-how (code, topics, …) into components → sustainable, scalable
MOTIVATION
VISUAL OBJECT RECOGNITION TRENDS
Human-level performance
2012 time
Accuracy
2019
2012 time
Computational
costs
/for real-time/
2019
CPU
DEDICATED
COMP.HW. COMPUTATIONAL
BARRIER
2012 time
Amount of image
data (for training)
2019
IMAGE DATA
BARRIER
Nuclear
Engineering
Seibersdorf
GmbH
Seibersdorf
Labor GmbH
AIT AUSTRIAN INSTITUTE OF TECHNOLOGY
AIT Austrian Institute of Technology
EnergyHealth &
Bioresources
Digital Safety &
Security
Vision, Automation &
Control
AIT Austrian Institute of Technology
Mobility SystemsLow-Emission
Transport
Technology
Experience
Innovation Systems &
Policy
4
Federal Ministry for Transport,
Innovation and Technology
50,46%
Federation of Austrian
Industries
49,54%
1300+ employees
Budget: 140 Mio €
Business Model: 40:30:30
Robust and flexible
3D vision technology
VISION, AUTOMATION & CONTROL
High-Performance
Vision
3D Vision and
Modeling
Complex Dynamical
Systems
Worldwide fastest vision
sensor technology Advanced handling and
smart production
F r o m S e n s o r T o D e c i s i o n 5
AIT AUTONOMOUS & ASSISTIVE SYSTEMS
Driver Assistance
System for Trams
Assistance Systems for
Construction Machines
Driverless Missions in Crisis
& Disaster Management
Autonomous
Local Railway
Autonomous
Bus
7
ENABLING METHODOLOGIES FOR
ASSISTED OR SELF-DRIVING
Mobile platforms: sensory signals + local context (situation) → decisions
recognition
localization
motion analysis
vehicle control
objects (type, location, pose)
environment
sensor/data fusion
positioning
mapping
ego-motion computation
sensor fusion
object tracking
state prediction
probability
based behavior
elements
dynamic model
computation
vehicle model
vehicle control
safety
compliance
Deep learning based
detection & segmentationLocalization,
map building
Sparse motion
estimation, tracking
Vision algorithms testing
RELATED
KNOW-HOW
INTELLIGENT PERCEPTION FOR MOBILE MACHINES
AUTONOMOUS OFFROAD VEHICLE
1014.07.2019
A frequently asked question
Introduction
11
Example: Crop detection
▪ Radial symmetry
▪ Near regular structure
Example for robust vision
IDEA
branch & bound
research methodology
PRODUCTAPPLICATION
RESEARCH DEVELOPMENT
Alg. A
Alg. B
Alg. C
MATLAB C++
Motivation
Introduction
▪ Challenges when developing Vision Systems:
▪ Complexity Algorithmic, Systemic, Data
▪ Non-linear search for a solution
13
Real-time optical flow based particle advection
for object detection and tracking
2D
1414.07.2019
MOTIVATION – I.
OBJECT DETECTION PIPELINES
Spatial distribution of
posterior probability
Score map (DPM, R-CNN, …)
Vote map
Occupancy map
back-projected similarity map
…
R-CNN: Region-based Convolutional Neural Networks
DPM: Deformable Part Models
Delineated objects
Bounding boxes
Instance segmentation
More complex
parametric
representations
weakly constrained structural prior
Non-maximum suppressionNeubeck & Van Gool, 2006Rothe et al., 2014
Leibe et al. 2005
Comaniciu & Meer, 2002 Bradski 1998
Dollar & Zitnick 2013 Kontschieder et al. 2011
Leibe et al. 2005
Mean Shift, CAMShift
Center-surround filter
MeanShift and CamShift iterations
Implicit Shape Model
Structured random forests
RELATED STATE-OF-THE-ART
▪ Clustering detections
▪ Detection by voting/segmentation/learningimplicit or explicit structural prior
Kontschieder et al. 2011
Markov Point Processes for object configurationsVerdie, 2014
CNN‘s for Non-Max. SuppressionHosang et al. 2016, Wan et al. 2015
Hosang et al. 2016
Optical flow driven advection
16
ti ti+1
Dense optical flow field
Advection: transport mechanism induced by a force field
Vx,i
Vy,iA particle trajectory
induced by the OF field
Particle advection with FW-BW consistency
▪ A simple but powerful test
Forward:
Backward:
Successful
Failure
< x x : mean offsetConsistency check:
Pedestrian Flow Analysis
Public dataset: Grand Central Station, NYC: 720x480 pixels, 2000 particles, runs at 35 fps