This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 731667 (MULTIDRONE) Multi-sensor Fusion for Target Tracking J. Ramiro Martínez-de Dios [email protected]Robotics, Computer Vision Group University of Seville, Spain
35
Embed
Multi-sensor Fusion for Target Tracking · Multi-sensor Fusion for Target Tracking ... •Critical for many applications in cinematography ... higher than traditional data fusion
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
This project has received funding from the European Union’s Horizon 2020 research
and innovation programme under grant agreement No 731667 (MULTIDRONE)
This project has received funding from the European Union’s Horizon 2020 research
and innovation programme under grant agreement No 731667 (MULTIDRONE)
• USE, CATEC, AIRBUS D&S
• One of the two Finalists of EUROC-Challenge 3,
out of more than 35 teams.
• Award “Best Dron-based Solution”, EU Parliament, January 2017
• Drones as co-workers in factories
• Drone indoor auton. navigation
• Massive use of sensor-fusion
techniques
• High robustness
ARCOW: Aerial Robot CO-Worker
10
This project has received funding from the European Union’s Horizon 2020 research
and innovation programme under grant agreement No 731667 (MULTIDRONE)
ARCOW: Aerial Robot CO-Worker
11
This project has received funding from the European Union’s Horizon 2020 research
and innovation programme under grant agreement No 731667 (MULTIDRONE)
ARCOW: Aerial Robot CO-Worker
12
This project has received funding from the European Union’s Horizon 2020 research
and innovation programme under grant agreement No 731667 (MULTIDRONE)
• Introduction
• The multi-sensor tracking problem
• Tools for multi-sensor tracking
• An example: fusion of camera and RSSI measurements
• Active tracking
• Conclusions
Outline
13
This project has received funding from the European Union’s Horizon 2020 research
and innovation programme under grant agreement No 731667 (MULTIDRONE)
Recursive Bayesian Filters
• Strong mathematical foundation
• Explicit consideration of uncertainty in models and sensors Good performance in presence of noise
in sensors and models
• Very high flexibility: higher than traditional data fusion methods
• Allows modeling realistic systems: observations and systems under uncertainty
• Flexible approach: can be combined with other modules such as dynamic model-learning or uncertainty-
based supervisors
• It enables reasoning in terms of INFORMATION enabling combination with Information-based methods
& tools; e.g. POMDPs
Tools for Multi-sensor Tracking
14
This project has received funding from the European Union’s Horizon 2020 research
and innovation programme under grant agreement No 731667 (MULTIDRONE)
• Optimal performance in presence of noise in sensors and models
• Very high flexibility: higher than traditional data fusion methods
• Allows modeling realistic systems: observations and systems under uncertainty
Probabilistic Bayesian Filters: RBFs
Actions
Disturbances
Estimator
Estimation
State
System Sensors
Noise
Sensor model
System model
Observations
Update
Prediction
measurements
xk+1|k
xk|k
zk
15
This project has received funding from the European Union’s Horizon 2020 research
and innovation programme under grant agreement No 731667 (MULTIDRONE)
Kalman Filter:
• Use parametric models for the system and observations:
(mean vector) (covariance matrix)
• Assumes Gaussian noise (observation and model)
• Assumes Linear models
• Extension to non-linear models: Extended Kalman Filter (EKF)
Information Filter:
• Dual to the KF. Uses the canonical representation:
• Uses same assumptions as KF
• KF and IF have similar burden complexity
• KF are efficient in the Prediction Step
• IF are efficient in the Update Step. They can be decentralized
Scales well with the number of measurements
Probabilistic Bayesian Filters: RBFs
16
This project has received funding from the European Union’s Horizon 2020 research
and innovation programme under grant agreement No 731667 (MULTIDRONE)
Particle Filter:
• Non-parametric representation: cloud of particles that represent the p.d.f. of the vector state
• Pros: no constrained in noise or system representations
• Pros: allows multi-hypothesis cases
• Cons: High computational burden (>100 particles)
Probabilistic Bayesian Filters: RBFs
17
This project has received funding from the European Union’s Horizon 2020 research
and innovation programme under grant agreement No 731667 (MULTIDRONE)
Information Filters for Multi-sensor Tracking
Kalman Filter Information Filter Particle Filter
Decentralization Complex Natural Complex
Computational efficiency + +++
Flexibility, adaptability + +++
Scalability + +++
Numerical Stability +++
18
This project has received funding from the European Union’s Horizon 2020 research
and innovation programme under grant agreement No 731667 (MULTIDRONE)
Decentralized Information Filter for Multi-sensor Tracking
non-head head
Notation: “Probabilistic Robotics”, Thrun et al.
19
This project has received funding from the European Union’s Horizon 2020 research
and innovation programme under grant agreement No 731667 (MULTIDRONE)
• Distributed Extended Information Filter (EIF):
• State: object location and velocities
• Measurements: center on image plane (pin-hole nonlinear model)
• Prediction model:
Object follows a locally rectilinear trajectory
Decentralized Information Filter for Multi-sensor Tracking
A de San Bernabe, J.R. Martinez-de Dios, A Ollero, “Efficient cluster-based tracking mechanisms for camera-based wireless sensor networks”, IEEE Transactions on Mobile Computing 14 (9), 1820-1832
20
This project has received funding from the European Union’s Horizon 2020 research
and innovation programme under grant agreement No 731667 (MULTIDRONE)
Drones perceiving the same object organize autonomously in a cluster
Advantages of cluster-based tracking: Scalable and robust
• Local processing of information: avoids transmission of normally
heavy traffic
• Tracking of several objects simultaneously, each with its cluster
The cluster head (CH) is responsible for:
• Collecting and fusing measurements from all the cluster nodes
• Managing inclusion/exclusion from the cluster
• Managing changing/rotation of cluster heads
Other Tracking Functionalities
21
This project has received funding from the European Union’s Horizon 2020 research
and innovation programme under grant agreement No 731667 (MULTIDRONE)
• Introduction
• The multi-sensor tracking problem
• Tools for multi-sensor tracking
• An example: fusion of camera and RSSI measurements
• Active tracking
• Conclusions
Outline
22
This project has received funding from the European Union’s Horizon 2020 research
and innovation programme under grant agreement No 731667 (MULTIDRONE)
Mechanisms:
A distributed EIF fuse the measurements
An entropy-based active sensing method for inclusion and exclusion of camera nodes in the cluster
A method that calibrates RSSI using camera measurements
Fusion of Cameras and RSSI for Tracking
A de San Bernabé, JR Martinez-de Dios, A Ollero, “Efficient integration of RSSI for tracking using Wireless Camera Networks” Information Fusion 36, 296-312
23
This project has received funding from the European Union’s Horizon 2020 research
and innovation programme under grant agreement No 731667 (MULTIDRONE)
• Integrates available RSSI and camera measurements
Uses RSSI-range models
Fusion of Cameras and RSSI for Tracking
24
This project has received funding from the European Union’s Horizon 2020 research
and innovation programme under grant agreement No 731667 (MULTIDRONE)
Motivation: In static scenarios RSSI has low variability
RSSI-range Training using Camera Measurements
Approach: use estimations of the object location to train in real-time RSSI-range models
The computed RSSI-model will be valid locally around the current target location
25
This project has received funding from the European Union’s Horizon 2020 research
and innovation programme under grant agreement No 731667 (MULTIDRONE)
Estimate the model: linear for simplicity (it is only local)