ForeSight: Mapping Vehicles in Visual Domain and Electronic Domain Dong Li, Zhixue Lu, Tarun Bansal, Erik Schilling and Prasun Sinha Department of Computer Science and Engineering The Ohio State University 1
Feb 18, 2016
1
ForeSight: Mapping Vehicles in Visual Domain and Electronic
Domain
Dong Li, Zhixue Lu, Tarun Bansal, Erik Schilling and Prasun SinhaDepartment of Computer Science and Engineering
The Ohio State University
2
Need for Targeted Communication
OK, but who are you?
What’s in front?
Are you talking to
me?
Hey you at the back -- Your lights are off!
I am overtaking you, don’t change
lane!
3
Today’s Solutions
Unicast: Hand Gestures, Eye ContactRequires parties to see each other
Broadcast: Honk, ShoutDisturbs othersAgitates/annoys both parties
4
Tomorrow’s Technology: One Possibility
Broadcast using Smartphone/DSRCHonking/shouting in the electronic domainWould cause sensory overload for drivers
5
Fundamental Problem in Targeted V2V Communication
Who is the sender/receiver?
Sender: What is the receiver’s unique address?Receiver: Which vehicle sent message to me?
6
To match vehicles in visual and electronic domains.
Objective
At the same time• Decrease matching time• Increase accuracy• Generate less network traffic
VID: Visual ID assigned by camera (e.g., red/yellow/blue box)
EID: Electronic ID of the vehicle (e.g., IP/MAC address)
7
Available Features
Features Accuracy & UniquenessVehicle Color Not always uniqueGPS Not accurate enoughVehicle Image Not unique, environment dependentPlate number Unique, but hard to readRelative Speed May not be accurate…. ….
A unique set of features known to both vehicles is desired.
8
Main Idea
If single feature is unreliable, can we use multiple features to do matching?
System requirement: Camera, GPS Receiver and Radio
Radio: communicationSmartphone, DSRC
Camera: identify vehiclesSmartphone, Vehicle Security Driving Recorder Camera
9
Challenges
Feature InaccuracyE.g., A blue vehicle might be observed as black.
Heterogeneous CapabilityVehicles may not have smartphone, camera, radio, or may not be running our solution.
Distributed in NatureEach vehicle only knows limited information
10
Vehicle Matching Process
Matching vehicles based on similarity
Estimate similarity between vehiclesWeight Features Cluster {VIDs, EIDs}
Obtain VIDs & EIDsGet VIDs from camera Get EIDs from radio
11
Visual Matrix(from Video-Analysis)
VID N Features
V1 f11 f12 … f1N
V2 f21 f22 … f2N
... … … …. …VM fM1 fM2 … fMN
Vehicles Observed through Camera
VID : Visual ID (camera assigns visual IDs to the observed vehicles)
VID only has local meaning (cannot be used by neighbors)
12
Electronic Matrix(from Electronic Messages)
EID N Features
E1 f11 f12 … f1N
E2 f21 f22 … f2N
... … … …. …EK fK1 fK2 … fKN
IDs received through WiFi/DSRC
EID: Electronic ID (IP address, MAC address, etc.)
13
Create Similarity Matrix
E1 E2 ... EK
V1 … 0.84 ... …
V2 … … ... ...
... ... ... ... ...
VM ... ... ... ...
VID Features
V1 f11 f12 … f1N
V2 f21 f22 … f2N
... … … …. …VM fM1 fM2 … fMN
EID Features
E1 f11 f12 … f1N
E2 f21 f22 … f2N
... … … …. …EK fK1 fK2 … fKN
Electronic Matrix E Visual Matrix V
Similarity Matrix S
S = V ET
14
Adaptive Weight (AW) Algorithm
The Problem:How to combine different features to get the similarity value between two cars?
The Intuition: Features with diversity values are important.
E.g., color provides no information if the cars have the same color
The Solution:Define Feature Distinguishability: the probability that any two observed vehicles are different based on this featureSimilarity of two vehicles: weighted mean of the feature distinguishability values.
15
different lane, different color
Eb
Ea
Matching with Similarity Matrix
V1
me
V2
Visual Domain Electronic Domain
0.99
0.50.5
0.01
Steps• Assign VIDs• Receive EIDs• Calc. Similarity• Remove low
similarity linksdifferent lane, similar color
same lane, different color
same lane, similar color
16
Matching with Similarity Matrix
Greedy Matching Maximal Matching
Weighted Bipartite Graph Matching Problem
Ea
Eb0.9
0.50.5
VIDs EIDs
V2 Ea
Eb0.9
0.50.5
VIDs EIDs
V2
Greedy matching is preferred.
V1 V1
17
Clustering Vehicles
Global distinguishability not requiredNearby cars need to be distinguished
Cluster the cars into smaller groups based on feature distance.
Apply the AW algorithm within clusters
Clustering
18
Experiment
Driving in freeway & local drive with 3 carsUsing smartphone to collect GPS, videoExperiment result
Vehicles with same color leads to low precision
19
SimulationUsing SUMO + NS3Modeled the visibility of neighboring carsModeled car detection prob., color detection accuracy, etc.
ForeSight significantly improves the matching performance!
0.230.18
20
Case Studies: Improve GPS
Each vehicle estimates its location withIts own GPS measurementNeighbors’ estimation of its location (assistance from Nbrs.) Interesting Observations:
• When a car’s GPS error low, it is more likely to be matched by more neighbors.
• The match error increases as the number of neighbors increases: dense traffic makes matching more unreliable.
High vehicle density
21
Case Studies: Reduce Disturbance
Application: Send message to vehicles that are in front but has a slower speedCompare Broadcast, GPS and ForeSight
GroundTruth
Broadcas
tGPS
ForeSig
ht0
50010001500200025003000
# of
Car
s Noti
fied
29× Schemes Notified Vehicles
Ground Truth 1021Broadcast 29 x 1021GPS 2706 (95% recall)ForeSight 1141 (95% recall)
22
Eb
E1 E3Ea
E1 E2
Future Work: Conflict Resolving
Conflicts may AppearMatching result computed by different vehiclesMatching result at different time
Possible SolutionCollaboration between neighbors
Eb
E1 E3
E1
Ea
E2
EID
VID