artificial PErception under Adverse CONditions: The Case of the Visibility Range LCPC in cooperation with INRETS, France Nicolas Hautière Young Researchers Seminar 2007 Brno, Czech Republic, 27 to 30 May 2007
Dec 18, 2015
artificial PErception under Adverse CONditions:
The Case of the Visibility Range
LCPC in cooperation with INRETS, France
Nicolas Hautière
Young Researchers Seminar 2007Brno, Czech Republic, 27 to 30 May 2007
Overview
• ADAS and adverse visibility conditions• Visibility range under daytime fog
Modelling Measurement methods Experimental validation
• Under way applications • Discussion and future works
PErception under Adverse CONditions
Nicolas Hautière, LCPC
ADAS and Adverse Visibility Conditions
PErception under Adverse CONditions
Nicolas Hautière, LCPC
Sensor aspects• To improve the operation range of exteroceptive sensors• To qualify / adapt / stop the other driving assistances
Human aspects• To switch or to adapt the operation of vehicle lights (AFS)• To adapt the speed according to the weather conditions (ISA)
To detect the visibility conditions allows:
Improve the safety !
[Hautière and Aubert, 2005a] Hautière, N. and Aubert, D. (2005). Onboard evaluation of the atmospheric visibility for driving assistance systems, Recherche Transports Sécurité, 87:89-108.
Let’s talk about daytime fog
Daylight
Visibility Range under Daytime Fog
PErception under Adverse CONditions
Nicolas Hautière, LCPC
Scattering
Atmospheric veil
• Koschmieder’s Law:
• Let express the contrast of an object against the sky:
Contrast attenuation
kkVmet
3)05.0log(
1
kd
f
f eCL
LLC
0
fLeeLL kdkd )1(0
“the greatest distance at which a black object of suitable dimensions can be recognized by aday against
the horizon sky” (CIE, 1987)
• For a black object (C0=1) and a visibility contrast threshold of 5%:
Direct transmission
Exploitation of the Atmospheric Veil
PErception under Adverse CONditions
Nicolas Hautière
Extraction of a region of interestFitting of a measurement bandwidth
VVmetmet = 50m = 50m
Estimation of the meteorological visibility distance
Measurement and derivation of intensity curveExtraction of the inflection point
B&W camera
zx f
d
SX
YZ
C
y
v
u vh
H
MRoad plane
Imageplane
hi vvk
dv
Ld
20
2
2
hvv
k
f eLLLL
100
Assuming a flat road:
hh
vvifvv
d
)(2
3
himet vv
V
vh horizon line, camera parameters
Method: Instanciation of Koschmieder’s Law
[Hautière et al., 2006a] Hautière, N., Tarel, J.-P, Lavenant, J. and Aubert, D. (2006). Automatic Fog Detection and Measurement of the visibility Distance through use of an Onboard Camera. Machine Vision Applications Journal, 17(1):8-20
Exploitation of Contrast Attenuation
PErception under Adverse CONditions
Nicolas Hautière, LCPC
• Estimation of the so-called mobilized visibility distance
• Range map obtained by “v-disparity” stereovision approach
• Visible objects are those having a local contrast above 5%
Method: computation of the range to the most distant visible object
f(x1)
f(x)
Cx,x1(s)s
x x1
F(s)
)(),(
1]255,0[
0
1
))(,())((
1max)(
sFxxssxxC
sFcardsC
)(,max
)(,
)(,max
)(min)(
1
1, 1 xfs
xfs
xfs
xfssC xx
Evaluated contrast on F(s) is equal to 2C(s0)
where
Local contrast measurement based on a binarisation method:
[Hautière et al., 2006b] Hautière, N., Labayrade, R. and Aubert, D. (2005). Real-Time Disparity Contrast Combination for Onboard Estimation of the Visibility Distance . IEEE Transactions on Intelligent Transportation Systems, 7(2):201-212.
Video samples
PErception under Adverse CONditions
Nicolas Hautière, LCPC
0
50
100
150
200
250
300
0 50 100 150
Time
Vis
ibili
ty d
ista
nce
[m]
0
50
100
150
200
250
300
0 200 400 600 800 1000Time
Vis
ibili
ty d
ista
nce
[m
]
Daytime fog Twilight fog
Experimental Validation
PErception under Adverse CONditions
Nicolas Hautière, LCPC
Development of a validation site
• Objectives : To estimate Vmet thanks to
the targets:
To obtain a ground truth, To compare it with the in-
vehicle methods
)()(
)()(log
1
11
22
12 dLdL
dLdL
ddk
NB
NB
Triangle based pattern Constant solid angle 5 fixed targets:d=65m 1mx1md=98m 1.5mx1.5md=131m 2mx2md=162m 2.5mx2.5md=195m 3mx3m 1 mobile targe: 0.5mx0.5m
kVmet
3
Experimental Validation
• Content
PErception under Adverse CONditions
Nicolas Hautière, LCPC
Sunny weather Vmet= 5000 m Light rain Vmet= 3400 m Haze Vmet= 2130 m
Snow fall Vmet= 1000 m Fog Vmet= 255 m Thick fog Vmet= 61 m
Sample images of the validation site
Experimental Validation
PErception under Adverse CONditions
Nicolas Hautière, LCPC
Meteorological visibility estimation Mobilized visibility distance estimation
Quantitative results
[Hautière et al., 2006c] Hautière, N., Aubert, D., Dumont, E. and Tarel, J.-P. (2008). Experimental Validation of Dedicated Methods to In-Vehicle Estimation of Atmospheric Visibility. IEEE Transactions on Instrumentation and Measurement, 57(10), 2218-2225.
• Principle: reversal of Koschmieder’s law
• Assuming a flat world, we have:
Under Way Applications
PErception under Adverse CONditions
Nicolas Hautière, LCPC
Improved Road Departure Prevention
)1(0kd
fkd eLLeL
i
h
hi
h
hi
vv
hiif
vv
vv
fvv
vv
dv
dLvvLL
eLLeL
2
122
0
Enhancement of road markings extraction under adverse visibility conditions
[Hautière and Aubert, 2005b] Hautière, N. and Aubert, D. (2005). Contrast Restoration of Foggy Images through use of an Onboard Camera, IEEE Conference on Intelligent Transportation Systems (ITSC’05), Vienna, Austria
d1=28m
d2=62m
• Contrast enhancement of the road scene
• Iterative contrast restoration
Under Way Applications
PErception under Adverse CONditions
Nicolas Hautière, LCPC
Improved obstacle detection
22 )()(,min
hhh vvuuvvd
[Hautière et al., 2007] Hautière, N., Tarel, J.-P., Aubert, D. (2007). Towards Fog-Free In-Vehicle Vision Systems through Contrast Restoration. IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’07), Minneapolis, USA
• Objective: to compute an adequate speed according to the weather conditions based solely on a digital map and a camera
• One of the requirements: a mapping function between the driver and the sensor visions:
PErception under Adverse CONditions
Nicolas Hautière, LCPC
Under Way ApplicationsISA and “risk mitigation” or “safety margin”
[Hautière and Aubert, 2006d] Hautière, N. and Aubert, D. (2006). Visible Edges Thresholding: a HVS based Approach, International Conference on Pattern Recognition (ICPR’06), Hong-Kong, China
)0,0(),(
*
)(
)(
jiijij
R
CTFbA
dfCTFBA
Discussion, Current and Future works
• We have presented two methods, issued from the ARCOS French project, to estimate the visibility range and some applications,
• Methods are being extended in the REACT project by the Mines de Paris to develop probe vehicles,
• Currently, we are adapting the methods to the use of fixed CCTV cameras in the SAFESPOT IP,
• In the future, we would like to apply our scientific processes to other adverse visibility conditions, like:
PErception under Adverse CONditions
Nicolas Hautière, LCPC
Rain Glare Nocturnal Fog
This is the heart of the FP7 PEACON proposal lead by LCPC/INRETS !
Acknowledgments
PErception under Adverse CONditions
Nicolas Hautière, LCPC
• I would like to acknowledge the contributions of my colleagues Didier Aubert, Jean-Philippe Tarel, Raphaël Labayrade, Benoit Lusetti, Eric Dumont from LCPC and INRETS, of Michel Jourlin from the University of Saint-Etienne, and of Clément Boussard from the Mines Paris.