-
IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS, VOL.
15, NO. 5, OCTOBER 2014 2321
An Efficient Visibility Enhancement Algorithm forRoad Scenes
Captured by Intelligent
Transportation SystemsShih-Chia Huang, Bo-Hao Chen, and Yi-Jui
Cheng
AbstractThe visibility of images of outdoor road scenes
willgenerally become degraded when captured during inclementweather
conditions. Drivers often turn on the headlights of theirvehicles
and streetlights are often activated, resulting in localizedlight
sources in images capturing road scenes in these
conditions.Additionally, sandstorms are also weather events that
are com-monly encountered when driving in some regions. In
sandstorms,atmospheric sand has a propensity to irregularly absorb
specificportions of a spectrum, thereby causing color-shift
problems inthe captured image. Traditional state-of-the-art
restoration tech-niques are unable to effectively cope with these
hazy road imagesthat feature localized light sources or color-shift
problems. In re-sponse, we present a novel and effective haze
removal approach toremedy problems caused by localized light
sources and color shifts,which thereby achieves superior
restoration results for single hazyimages. The performance of the
proposed method has been proventhrough quantitative and qualitative
evaluations. Experimentalresults demonstrate that the proposed haze
removal technique canmore effectively recover scene radiance while
demanding fewercomputational costs than traditional
state-of-the-art haze removaltechniques.
Index TermsColor shift, dark channel prior, localized light.
I. INTRODUCTION
V ISIBILITY in road images can be degraded due to
naturalatmospheric phenomena such as haze, fog, and sand-storms.
This visibility degradation is due to the absorptionand scattering
of light by atmospheric particles. Road imagedegradation can cause
problems for intelligent transportationsystems such as traveling
vehicle data recorders and trafficsurveillance systems, which must
operate under a wide rangeof weather conditions [1][13]. The amount
of absorption andscattering depends on the depth of the scene
between a trafficcamera and a scene point; therefore, scene depth
informationis important for recovering scene radiance in images of
hazyenvironments.
Manuscript received November 25, 2013; revised March 11, 2014;
acceptedMarch 20, 2014. Date of publication May 16, 2014; date of
current versionSeptember 26, 2014. The work of author S.-C. Huang
was supported by theNational Science Council (NSC) of Taiwan under
Grants NSC 100-2628-E-027-012-MY3 and NSC 102-2221-E-027-065. The
Associate Editor for thispaper was S. S. Nedevschi.
The authors are with the Department of Electronic Engineering,
Collegeof Electric Engineering and Computer Science, National
Taipei University ofTechnology, Taipei 106, Taiwan (e-mail:
[email protected]).
Color versions of one or more of the figures in this paper are
available onlineat http://ieeexplore.ieee.org.
Digital Object Identifier 10.1109/TITS.2014.2314696
In order to improve visibility in hazy images, haze
removaltechniques have been recently proposed. These can be
dividedinto two principal classifications, i.e., the given depth
[14][16] and unknown depth [17][24] approaches. Given
depthapproaches use additional information [14][16]. They relyon
the assumption that the depth is given; they then use thedepth
information to restore hazy images. Tan and Oakley [14],Narasimhan
and Nayar [15], and Kopf et al. [16] developedhaze removal
approaches based on the given depth information.This information is
acquired from additional operations orinteractions, such as
applying information pertaining to thealtitude, tilt, and position
of the camera [14], or through manualapproximation of the distance
distribution of the sky areaand vanishing point in a captured image
[15], or through anapproximate 3-D geometrical model of the
captured scene [16].However, these approaches are not suitable for
haze removal inreal-world applications because the depth
information needs tobe provided by the user, yet it is scarcely
given.
Therefore, a haze removal technique changes the given depthinto
an unknown depth. Many studies have proposed the es-timation of an
unknown depth to recover scene radiance inhazy images. These can be
divided into two major categories,i.e., multiple images [17][19] or
a single image [20][24].Schechner et al. [17] and Narasimhan and
Nayar [18], [19] pro-posed haze removal techniques that estimate
the unknown depthby using multiple images to restore a hazy image.
Specifically,the method proposed by Schechner et al. [17] uses two
or moreimages of the same scene with different polarization degrees
byrotating a polarizing filter to estimate the depth of a scene
andthen remove haze. Narasimhan and Nayar [18], [19]
presentedmethods that compute the scene depth from two or more
imagesin different weather conditions, in which the scene radiance
ofa hazy image can be restored. However, these methods
usuallyrequire either a complex computation or the use of
additionalhardware devices. This leads to high restoration
expense.
Because of this expense, recent research has focused
onsingle-image restoration. Recent investigations [20][24]
haveexamined the use of single images to estimate the unknowndepth
without using any additional information to recover sceneradiance
in hazy images. Tan [20] proposed a single-imagehaze removal
approach that removes haze by maximizing thelocal contrast of the
recovered scene radiance based on anobservation that captured hazy
images have lower contrast thanrestored images. This approach can
produce a satisfactory resultfor haze removal in single images, but
the restored results
1524-9050 2014 IEEE. Personal use is permitted, but
republication/redistribution requires IEEE permission.See
http://www.ieee.org/publications_standards/publications/rights/index.html
for more information.
-
2322 IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS,
VOL. 15, NO. 5, OCTOBER 2014
feature artifact effects along depth edges. Fattal [21]
proposeda haze removal technique for single images that estimates
thealbedo of the scene and deduces the transmission map basedon an
assumption that the transmission shading and the surfaceshading are
locally uncorrelated. This technique can generateimpressive results
when the captured image is not heavilyobscured by fog. In other
words, this technique cannot contendwith images featuring dense
fog. Li et al. [22] describe acharacteristic property in which the
smaller transmission inten-sity values possess larger coefficients
in the gradient domain,whereas the larger transmission intensity
values possess smallercoefficients. Based on this property, the
method of Li et al. canrestore the visibility of hazy images by
employing a multiscaletechnique in the regions containing small
transmission values.However, this method usually results in
excessive restorationwith regard to the sky regions of the
resultant image.
He et al. [23] proposed a haze removal algorithm via the
darkchannel prior technique that uses the observation that at
leastone color channel is composed of pixels that have lower
intensi-ties within local patches in outdoor haze-free images to
directlyestimate the amount of haze and subsequently recover
sceneradiance efficiently. Until now, the approach of He et al.
[23]has attracted the most attention due to its ability to
effectivelyremove haze formation while only using single images.
Inspiredby the dark channel prior technique [23], Xie et al.
proposedan improved haze removal algorithm by employing a
schemeconsisting of the dark channel prior and the multiscale
Retinextechnique to quickly restore hazy images [24]. However,
thescene radiance recovered via the dark-channel-prior-based
tech-niques [24], [23] is usually accompanied by the generation
ofserious artifacts when the captured hazy road image
containslocalized light sources or color-shift problems due to
sandstormconditions. This can be problematic for many common
roadscenarios. For example, in inclement weather conditions,
thedrivers generally turn on headlights when they are driving
inorder to improve visual perception, and streetlamps are lit
forsimilar reasons. The techniques based on the dark channel
prior[23], [24] cannot produce satisfactory restoration results
whenpresented with these situations.
Therefore, we propose a novel haze removal approach bywhich to
avoid the generation of serious artifacts by the con-junctive
utilization of the proposed hybrid dark channel prior(HDCP) module,
the proposed color analysis (CA) module, andthe proposed visibility
recovery (VR) module. These modulesare further discussed in Section
III. The proposed techniquecan effectively conceal localized light
sources and restrain theformation of color shifts when the captured
road image containslocalized light sources or color-shift problems.
Experimentalresults and subsequent quantitative and qualitative
evaluationsdemonstrate that the proposed technique can more
effectivelyremove haze from single images captured in real-world
condi-tions than other state-of-the-art techniques [22][24].
The remainder of this paper is organized as follows. InSection
II we briefly describe the dark channel prior technique.Section III
presents a detailed description of the proposedtechnique and its
applicability for single-image haze removaland the circumvention of
the previously mentioned problems.Section IV presents and contrasts
the experimental results
Fig. 1. Pictorial description of hazy image acquisition via the
optical model.
produced via the four methods for images representing a
widerange of weather conditions. Section V discusses and
demon-strates the improvement of traffic surveillance applications
viathe use of the proposed method. Finally, the conclusion
ispresented in Section VI.
II. BACKGROUND
A. Optical Model
In computer vision and pattern analysis, the optical modelis
widely used to describe the digital camera information of ahazy
image under realistic atmospheric conditions in the RGBcolor space
as
Ic(x, y) = Jc(x, y)t(x, y) +Ac (1 t(x, y)) (1)
where c {r, g, b}, Ic(x, y) represents the captured image,Jc(x,
y) represents the scene radiance that is the ideal haze-free image,
Ac represents the atmospheric light, and t(x, y)represents the
transmission map describing the portion of thelight that arrives at
a digital camera without scattering. The firstterm of (1), i.e.,
Jc(x, y)t(x, y), represents the direct attenua-tion describing the
decayed scene radiance in the medium. Thesecond term of (1), i.e.,
Ac(1 t(x, y)), represents the airlightthat resulted from the
scattered light and leading to the colorshifting in the scene.
Generally, the homogenous atmosphere can be assumed tobe
uniform, and the scene radiance is exponentially
attenuatedaccording to the depth of the scene. The transmission map
canbe expressed as
t(x, y) = ed(x,y) (2)
where is the atmospheric attenuation coefficient, and d(x, y)is
the scene depth that represents the distance between anobserved
object and the digital camera. Fig. 1 shows theoptical model that
describes the hazy image information ob-tained by a traveling
vehicle data recorder under atmosphericconditions.
B. Haze Removal Using Dark Channel Prior
Dark Channel Prior: The dark channel prior is a state-of-the-art
image restoration technique by which to remove haze
-
HUANG et al.: ALGORITHM FOR ROAD SCENES CAPTURED BY INTELLIGENT
TRANSPORTATION SYSTEMS 2323
from a single image [23]. In order to estimate the amount ofhaze
in an image, dark channel Jdark can be expressed as
Jdark(x, y) = min(i,j)(x,y)
(min
c{r,g,b}Jc(i, j)
)(3)
where c {r, g, b}, J represents an arbitrary color image,
Jcrepresents a channel of color image J , represents a localpatch
centered at (x, y), minc{r,g,b} Jc(i, j) is performed asthe minimum
operation on Jc, and min(i,j)(x,y) is performedas a minimum filter
on the local patch centered at (x, y).
As described in [23], dark channel Jdark has a low intensitywhen
the outdoor image lacks haze, with the exception of thesky region.
The dark channel value of a haze-free image is closeto zero and can
be represented by
Jdark 0. (4)
In other words, if the dark channel value is larger than zero,
itmeans that regions exhibit haze. As such, we can estimate
theamount of haze via (3).
Estimating the Transmission Map: In a single hazy image,these
dark channel values can provide a direct and accurateestimation of
haze transmission. First, the optical model in (1) isindependently
normalized by atmospheric light Ac in the RGBcolor space as
Ic(x, y)
Ac=
Jc(x, y)
Act(x, y) + 1 t(x, y). (5)
Then, the dark channel operation is calculated on the both
sidesof (5) as
min(i,j)(x,y)
(min
c{r,g,b}Ic(i, j)
Ac
)
= t(x, y) min(i,j)(x,y)
(min
c{r,g,b}Jc(i, j)
Ac
)+1 t(x, y). (6)
According to the work in [23], as J is a haze-free image,
darkchannel Jdark can be obtained by
min(i,j)(x,y)
(min
c{r,g,b}Jc(i, j)
Ac
)= 0. (7)
Equation (7) can be incorporated into (6), resulting in
theestimation of the transmission map as
t(x, y) = 1 min(i,j)(x,y)
(min
c{r,g,b}Ic(i, j)
Ac
). (8)
As described in [23], the image may appear somewhat un-natural
if the haze is removed thoroughly. Therefore, a constantparameter
(set to 0.95) is added into (8) in order to retain aportion of the
haze for distant objects as
t(x, y) = 1 min(i,j)(x,y)
(min
c{r,g,b}Ic(i, j)
Ac
). (9)
Moreover, He et al. suggested an optimal patch size of thedark
channel prior as 15 15 [23].
Soft Matting: The recovered images produced by (9) maycontain
some block effects in a hazy image. In order to reducethese
artifacts, He et al. adopted a soft matting [25] techniqueto refine
the transmission map in (9). The matting Laplacianmatrix L is given
as
k|(i,j)wk
(ij 1|wk|
(1+(Iik)T
(k+
|wk|U3)1
(Ijk)))
(10)
where ij is the Kronecker delta, k is the mean matrix of
thecolors in window wk, k is the covariance matrix of the colorsin
window wk, U3 is an identity matrix of size 3 3, is aregularizing
parameter, and |wk| is the total number of pixels inwindow wk.
The refined transmission map t can be obtained by thefollowing
sparse linear system:
(L+ U)t = t (11)
where L is the matting Laplacian matrix, U is an identity
matrixof the same size as L, and is 104.
Recovering the Scene Radiance: Finally, a single hazy im-age I
can be recovered as scene radiance J as
Jc(x, y) =Ic(x, y)Ac
max (t(x, y), t0)+Ac (12)
where c {r, g, b}, the value of t0 is assumed to be 0.1, andthe
value of atmospheric light A is the highest intensity pixel inthe
original input image according to its correspondence to
thebrightest 0.1% of pixels in the dark channel.
III. PROPOSED METHOD
In this section, we present an effective approach for the
hazeremoval of single images captured during different
environmen-tal conditions that not only avoids the generation of
artifacteffects but also recovers true color. Our approach involves
threeproposed modules, i.e., an HDCP module, a CA module, and aVR
module.
Initially, the proposed HDCP module designs an
effectivetransmission map to circumvent halo effects in the
recoveredimage and estimates the location of the atmospheric light
toavoid underexposure. In order to recover the true color of
scenesfeaturing a wide range of weather conditions, we propose
theCA module. This CA module determines the intensity statisticsfor
the RGB color space of a captured image in order to acquirethe
color information. As the final step of our process, theproposed VR
module recovers a high-quality haze-free image.
A. HDCP Module
As mentioned in the previous section, the dark channel
priortechnique [23] can work well for haze removal in single
imagesthat lack localized light sources. However, haze removal by
the
-
2324 IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS,
VOL. 15, NO. 5, OCTOBER 2014
Fig. 2. Framework of the HDCP module.
dark channel prior technique [23] usually results in a
seriouslyunderexposed image when the captured scene features
localizedlight sources. The proposed HDCP module can produce
arestored image that is not underexposed by using a procedurebased
on the dark channel prior technique [23]. The darkchannel prior
technique in [23] can employ large patch sizeoperation for the
captured image in order to acquire the correctatmospheric light.
However, the use of a large local patchwill result in invariable
transmission and thereby leads to thegeneration of halo effects in
the recovered image.
In contrast, when the dark channel prior technique [23]uses a
small patch size, the recovered image will not exhibithalo effects.
However, localized light will be misjudged asatmospheric light.
Hence, we present the HDCP module thatensures correct atmospheric
light estimation and the subsequentavoidance of halo effects during
the haze removal of singleimages based on the hybrid dark channel
prior technique. Thistechnique will be introduced in the
following.
To effectively estimate the density of the haze featured byan
image, we combine the advantages of small and large patchsizes via
different weights. In addition, we use the large patchsize to
acquire the correct atmospheric light during the imple-mentation of
the hybrid dark channel prior technique. Equation(3) can be
rewritten via the HDCP as
Jdark(x, y) =
+ min
(i,j)(x,y)
(min
c{r,g,b}Jc(i, j)
)
+
+ min
(i,j)(x,y)
(min
c{r,g,b}Jc(i, j)
)(13)
where J represents an arbitrary image under a wide rangeof
weather conditions, Jc represents a channel of color im-age J , and
represent local patches centered at (x, y),minc{r,g,b} Jc(i, j)
performs the minimum operation on Jc,min(i,j)(x,y) performs a
minimum filter on the local patchcentered at (x, y) using the small
patch size, and min(i,j)(x,y)performs a minimum filter on the local
patch centered at (x, y)using the large patch size. After
calculating the haze density,
the transmission map can be directly and accurately estimatedby
rewriting (8) as
th(x, y) = 1 +
min(i,j)(x,y)
(min
c{r,g,b}Ic(i, j)
Ac
)
+
min(i,j)(x,y)
(min
c{r,g,b}Ic(i, j)
Ac
). (14)
In order to retain a small amount of haze for the
naturalappearance, a constant parameter is added to (14). Thus,
thetransmission map can be expressed as
th(x, y) = 1 +
min(i,j)(x,y)
(min
c{r,g,b}Ic(i, j)
Ac
)
+
min(i,j)(x,y)
(min
c{r,g,b}Ic(i, j)
Ac
)(15)
where can be set to 0.95 experimentally, the most optimalsmall
patch size of the image can be set to 3 3 experimentally,and the
most optimal large patch size can be set to 45 45experimentally.
Moreover, and are the constant factorsfor a small patch size and a
large patch size, respectively, bywhich the optimum results for
single-image haze removal canbe acquired. Note that the values of
atmospheric light Ac are,respectively, the highest intensity pixels
in each RGB channelof the original input image according to its
correspondenceto the brightest 0.1% of pixels in the dark channel
image, asdescribed in [23]. The general framework of the HDCP
moduleis shown in Fig. 2.
B. CA Module
The particles of sand in the atmosphere caused by sand-storms
absorb specific portions of the color spectrum. This phe-nomenon
leads to color shifts in images captured during suchconditions,
resulting in different color channel distributions.The dark channel
prior method [23] uses the same formula foreach color channel when
recovering scene radiance, therebycausing serious color shifts in
restored images. In order to solve
-
HUANG et al.: ALGORITHM FOR ROAD SCENES CAPTURED BY INTELLIGENT
TRANSPORTATION SYSTEMS 2325
Fig. 3. Proposed algorithm for the visibility enhancement of
single images via the HDCP technique.
this problem, we propose the CA module that is based on thegray
world assumption [26]. The gray world assumption relieson the
notion that average intensities should be equal in eachRGB color
channel for a typical image, which is described as
Ravg =1
MN
Mx=1
Ny=1
Ir(x, y)
Gavg =1
MN
Mx=1
Ny=1
Ig(x, y)
Bavg =1
MN
Mx=1
Ny=1
Ib(x, y) (16)
where Ir(x, y), Ig(x, y), and Ib(x, y) represent the
capturedimage in the RGB color channels, respectively, and MN
rep-resents the total number of pixels in the captured image.
Basedon this assumption, color spectrum adjustment parameter canbe
produced for each RGB color channel in order to avoid colorshifts
in the restored image. This can be measured as
c =1
MN
Mx=1
Ny=1 I
r(x, y)
1MN
Mx=1
Ny=1 I
c(x, y)=
Mx=1
Ny=1 I
r(x, y)Mx=1
Ny=1 I
c(x, y).
(17)
C. VR Module
In order to produce a high-quality haze-free image capturedin
different environments, we combine the information pro-vided via
the HDCP and CA modules to effectively recover thescene radiance.
Equation (12) can be rewritten as
Jc(x, y) =c (Ic(x, y)Ac)max (th(x, y), t0)
+Ac + c(c 1) (18)
where c {r, g, b}, Jc(x, y) represents the scene radiance,Ic(x,
y) represents the image captured under different condi-tions, Ac
represents the atmospheric light, th(x, y) representsthe
transmission map using the HDCP module, t0 is assumed tohave a
typical value of 0.1, and and represent the adjustmentparameters.
Moreover, specific portions of the color spectrumcan be irregularly
absorbed by atmospheric particles underdifferent weather
conditions. Thus, we employ parameter toadjust the atmospheric
variables. First, the intensity statistics ofthe RGB color channel
of the captured image can be calculatedfor the acquisition of color
information via the probability massfunction (PMF), which is
described as
PMF (Ick) =nckMN
, for k = 0, 1, . . . , L (19)
where c {r, g, b}, nck denotes the total number of pixels
thathave intensity Ick, MN denotes the total number of pixels
forthe captured image, and a constant factor L is set equal to
themaximum intensity value of a pixel. Ultimately, parameter can be
produced by using this color information, which can beexpressed in
(20), shown at the bottom of the next page.
As mentioned, the framework of our proposed algorithm isshown in
Fig. 3.
IV. EXPERIMENTAL RESULTS
The objective of this section is to demonstrate via qualita-tive
and quantitative evaluations the advantage of our HDCPmethod in
comparison with other state-of-the-art methods, in-cluding the
method of Li et al. [22], the method of He et al.[23], and the
method of Xie et al. [24] for haze removal in singleimages captured
by traveling vehicle data recorders in realisticconditions.
Here, we supply two video sequences, which are calledStreet and
Highway, to test the efficacy of each method.Fig. 4 shows the
Street video sequence that features a road
-
2326 IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS,
VOL. 15, NO. 5, OCTOBER 2014
Fig. 4. Restoration results using the method of He et al. [23],
the method of Xie et al. [24], and the proposed method for the
Street sequence.
along which the headlights of vehicles and streetlights
areturned on due to foggy conditions. Fig. 5 shows the Highwayvideo
sequence, which shows an area where many vehicles arepassing along
a highway during a sandstorm.
In order to prove that the proposed method is an effectiveimage
restoration technique for images captured in a wide rangeof weather
conditions, its efficacy will be analyzed according tothe following
four situations: (a) visual assessment for localized
light sources; (b) visual assessment for color-shift problems;
(c)quantitative evaluation; and (d) performance results.
Part (a) describes the haze removal and recovery of thescene
radiance for the captured Street video sequence con-taining
localized light sources. Part (b) discusses the haze re-moval for
the Highway video sequence, which was capturedduring sandstorm
conditions and features subsequent color-shift problems. Part (c)
provides the quantitative evaluation
r = arg max0kL1PMF (Irk)
g =arg max0kL1PMF (Irk) + arg max0kL1PMF (I
gk )
2
b =arg max0kL1PMF (Igk ) + arg max0kL1PMF
(Ibk)
2(20)
-
HUANG et al.: ALGORITHM FOR ROAD SCENES CAPTURED BY INTELLIGENT
TRANSPORTATION SYSTEMS 2327
Fig. 5. Restoration results using the method of He et al. [23],
the method of Xie et al. [24], the method of Li et al. [22], and
the proposed method for theHighway sequence.
of haze removal by using these two video sequences and
therepresentative Foggy Road Image DAtabase (FRIDA) [27].Part (d)
details the processing speeds of the proposed methodand the
dark-channel-prior-based methods.
A. Visual Assessment for Localized Light SourcesIn this section,
we compare our HDCP technique with the
dark-channel-prior-based techniques [23], [24] and demon-strate
that the proposed method can work well for single hazyimages with
localized light sources.
As can be observed in Fig. 4, the localized light sources
arebrighter than the atmospheric light in the captured frames,
andthe haze removal performed via each method was measured byvisual
evaluation. It is obvious from the results that the pre-
vious dark-channel-prior-based techniques [23], [24]
produceunderexposed recovered scene radiance in frames 218,
3543,and 7710. This is because the localized light sources in
thoseframes are misjudged as atmospheric light.
In contrast, our HDCP technique can effectively conceal
thelocalized light sources and thus can accurately estimate
theposition of the atmospheric light, as shown in frames 218,3543,
and 7710. Hence, the proposed approach can avoid thegeneration of
artifact effects.
B. Visual Assessment for the Color-Shift ProblemsThe
experimental results in this section confirm that the
proposed approach can more effectively recover the sceneradiance
of single images captured during sandstorm conditions
-
2328 IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS,
VOL. 15, NO. 5, OCTOBER 2014
than the other state-of-the-art methods [22][24]. As can beseen
in frames 260, 503, and 729 in Fig. 5, a yellowing huepervades, and
all frames exhibit color-shift problems. This isbecause the
atmospheric particles caused by sandstorm con-ditions absorb
certain portions of the color spectrum, therebycausing
discrepancies in the distributions of each RGB colorchannel.
As can be also observed in Fig. 5, the haze removal per-formed
via each method is measured by visual evaluation. Itis apparent
that the recovered sandstorm frames produced bythe
dark-channel-prior-based techniques [23], [24] still fea-ture
serious color-shift problems, as shown in frames 260,503, and 729.
This is because the dark-channel-prior-basedtechniques [23], [24]
assume that the color spectrum chan-nels are all equally absorbed
by the atmospheric particles. Inaddition, the method of Li et al.
[22] not only contaminatesthe brightness of the sandstorm images
but also generatesserious artifacts in the recovered frames, as
shown in theirrestoration results in Fig. 5. This is because the
use of themethod proposed by Li et al. [22] is based on the
multiscaletechnique that only considers restoring the contrast of
thesandstorm frames and thus cannot effectively recover
vividcolor.
In contrast, the proposed approach can effectively recover
ahaze-free frame captured under sandstorm conditions. This isdue to
its ability to solve color-shift problems by adjusting thevalue of
each color channel based on the proposed CA module,as shown in
frames 260, 503, and 729 in Fig. 5.
C. Quantitative EvaluationQuantifying the results of image
restoration is not an easy
task because a standard real-world reference image for
quan-tifying restored perception has not been validated. In
general,the major categories of objective metrics are nonreference
andreference methods [28].
Due to the unavailability of a real-world haze-free
referenceimage by which to compare the efficacy of the proposed
methodwith that of the other state-of-the-art methods, this paper
firstemploys three well-known quantitative metrics [29], whichare
e, r, and , for the nonreference method. In addition,FRIDA supplies
the standard synthetic reference image for thereference methods.
This study also employed these ground-truth images of the synthetic
images for assessment via the peakto SNR (PSNR) and mean difference
metrics [5] between theground-truth images and the restored
images.
For the nonreference method, the e metric detects the rateof the
restored visible edges in the restored image. This can beexpressed
as
e =nr no
no(21)
where nr and no are the numbers of visible edges in the
restoredhaze-free image and the original hazy image, respectively.
Next,the r metric supplies the geometric mean ratios of the
gradientnorms after and before restoration in relation to the total
amount
TABLE IAVERAGE RESTORATION VALUES OF THE COMPARED METHODS
ATTAINED BY e, r, AND IN A WIDE RANGE OF WEATHER CONDITIONS
of visible edges within the restored haze-free image. This canbe
represented as
r = exp
1n r
Pir
log ri
(22)
where Pi is the ith element of the corresponding set r, andr
represents the set of visible edges in the restored haze-freeimage.
Note that ri is the ith rate of the gradients betweenthe restored
haze-free image and the original hazy image.Moreover, the metric
estimates the number of pixels thatmight be either saturated as
white or smeared as black in therestored haze-free image. This can
be expressed as
=ns
dimxdimy(23)
where ns represents the total amount of both the saturatedpixels
and the smeared pixels, and dimxdimy represents thehazy image
resolution.
Table I offers the average restoration rates produced by
usingthe e, r, and metrics in each video sequence for the
proposedmethod and the compared methods. It should be noted that
thehigher values produced by the e and r metrics indicate
superiorrestoration rates, whereas a higher value produced by the
metric indicates inferior restoration rates. As shown in Table
I,the results of the comparison of the average restoration
ratesclearly indicate that the visibility restoration performance
of theproposed method was significantly superior to the
performanceof previous state-of-the-art methods [22][24] under a
widerange of weather conditions.
For the reference method, we supplied a comparison via thePSNR
and mean difference metrics between the ground-truthimages and the
images restored by the method of He et al., themethod of Xie et
al., and the method of Li et al. The attainedvalues of the PSNR
metric with higher values indicate betterrestoration, whereas the
attained values of the mean differencemetric with lower values
indicate better restoration. Fig. 6shows three images of FRIDA that
consist of a scene capturedalong a road. It is apparent from the
quantitative results that theproposed method is capable of not only
effectively restoringsynthetic road images but also attaining a
substantially higherdegree of efficacy in comparison with the other
state-of-the-artmethods [22][24].
-
HUANG et al.: ALGORITHM FOR ROAD SCENES CAPTURED BY INTELLIGENT
TRANSPORTATION SYSTEMS 2329
Fig. 6. Comparison of the restoration efficacy of each compared
approach via the reference method using FRIDA.
D. Performance ResultsTable II details the overall processing
speeds achieved
by the dark-channel-prior-based methods [23], [24] and
theproposed method for each image resolution, where we im-plemented
each compared method for x86-64 with 128-bitSSE2/SSE3 extensions.
These sources are written in theC/C++/single instruction multiple
data (SIMD) assembly, com-piled with GCC 4.2.4, and run on an Intel
Xeon E5520 proces-sor and 32-GB main memory running a Windows
server 2008operating system.
According to Table II, which presents the overall
processingspeeds, the proposed method was up to 3.6654 times faster
thanthe method of He et al. [23]. This is because the method ofHe
et al. employs the soft matting technique [25] to refinethe large
patch size in the transmission map, which inevitablycauses an
enormous computational burden. Moreover, the pro-posed method was
up to 1.1717 times faster than the method of
TABLE IIPROCESSING SPEEDS (IN FRAMES PER SECOND) OF
THE COMPARED METHODS
Xie et al. [24]. This is due to the traditional methods
utilizationof the multiscale Retinex technique based on the
surroundingfunction whose form is Gaussian, which also results in a
hugecomputational burden when restoring hazy images.
-
2330 IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS,
VOL. 15, NO. 5, OCTOBER 2014
Fig. 7. Improvement of traffic surveillance applications for a
video sequence captured in a haze-filled environment.
In contrast, the proposed method uses a small patch sizeinstead
of the soft matting and multiscale Retinex techniquesto restore a
hazy image without the generation of halo effects.The result is
dramatically superior performance by the proposedmethod in
comparison with the other dark-channel-prior-basedmethods.
V. APPLICATIONS
This section demonstrates the improvement of traffic
surveil-lance applications via the use of the proposed method
throughquantitative and qualitative evaluations of intelligent
transporta-tion systems. Currently, one factor of intelligent
transportationsystems that is critical in supporting traffic
management tasksis the ability to extract information about moving
objects withinscenes captured by traffic surveillance systems. As
such, trafficsurveillance systems are important components of
intelligenttransportation systems [1][5]. Moreover, automated
motiondetection is the first essential process in the development
oftraffic surveillance systems and is also crucial in
accomplishingtasks such as vehicle classification, vehicle
recognition, vehicletracking, collision avoidance, and so on
[6][11].
A survey of the state-of-the-art approach for motion detec-tion
has been proposed in [30]. However, the use of this ap-proach often
results in the incomplete detection of the extractedshapes of
moving objects when the traffic surveillance cam-era is
contaminated by atmospheric particles during inclementweather
conditions, as shown in Fig. 7. This is because thegathered
background information based on the cerebellar modelarticulation
controller network is insufficient for moving objectdetection in a
haze-filled environment.
Due to the incorporation of the proposed method into
thisstate-of-the-art approach for traffic surveillance, the
accuracyrates obtained via Similarity and F1 for the frames
restored byusing the proposed method are up to 41.13% and 44.92%
higherthan those produced using the original frames,
respectively.Therefore, our findings prove that the proposed method
canbe effectively applied to traffic surveillance systems, which
arecommonly operated under challenging weather conditions.
VI. CONCLUSION
In this paper, we have proposed a novel approach basedon our
HDCP technique for haze removal in single images
-
HUANG et al.: ALGORITHM FOR ROAD SCENES CAPTURED BY INTELLIGENT
TRANSPORTATION SYSTEMS 2331
captured under a wide range of weather conditions. First,
theproposed HDCP module efficiently conceals localized lightsources
and, consequently, accurately estimates the positionof the
atmospheric light. In addition, our HDCP module canprovide
effective transmission map estimation and therebyavoids the
production of artifact effects in the restored image.In the second
stage, the proposed CA module uses the grayworld assumption to
effectively obtain the color informationof the captured image and
thereby circumvent the color-shiftproblems in the restored image.
In the final stage, the VRmodule combines the information obtained
by the HDCP andCA modules to avoid the generation of serious
artifact effectsand thus obtain a high-quality haze-free image
regardless ofweather conditions. The experimental results
demonstrate thatthe proposed technique produces a satisfactory
restored image,as measured by the quantitative and qualitative
evaluationsof realistic scenes, while demanding less computational
cost.Moreover, the proposed technique is significantly superiorto
other state-of-the-art methods. To the best of our knowl-edge, this
is the first study that presents an effective hazeremoval approach
that is applicable in a wide range of weatherconditions.
REFERENCES[1] N. K. Kanhere and S. T. Birchfield, A taxonomy and
analysis of cam-
era calibration methods for traffic monitoring applications,
IEEE Trans.Intell. Transp. Syst., vol. 11, no. 2, pp. 441452, Jun.
2010.
[2] J.-P. Tarel, N. Hautire, L. Caraffa, A. Cord, H. Halmaoui,
and D. Gruyer,Vision enhancement in homogeneous and heterogeneous
fog, IEEEIntell. Transp. Syst. Mag., vol. 4, no. 2, pp. 620, Summer
2012.
[3] N. Buch, S. A. Velastin, and J. Orwell, A review of computer
visiontechniques for the analysis of urban traffic, IEEE Trans.
Intell. Transp.Syst., vol. 12, no. 3, pp. 920939, Sep. 2011.
[4] L. Caraffa and J.-P. Tarel, Markov random field model for
single imagedefogging, in Proc. IEEE Intell. Veh. Symp., Jun. 2013,
pp. 994999.
[5] J.-P. Tarel and N. Hautire, Fast visibility restoration from
a single coloror gray level image, in Proc. IEEE Int. Conf. Comput.
Vis., Sep. 2009,pp. 22012208.
[6] S. C. Huang, An advanced motion detection algorithm with
video qualityanalysis for video surveillance systems, IEEE Trans.
Circuits Syst. VideoTechnol., vol. 21, no. 1, pp. 114, Jan.
2011.
[7] F. C. Cheng, S. C. Huang, and S. J. Ruan, Scene analysis for
objectdetection in advanced surveillance systems using Laplacian
distributionmodel, IEEE Trans. Syst., Man, Cybern. C, Appl. Rev.,
vol. 41, no. 5,pp. 589598, Sep. 2011.
[8] S. C. Huang and B. H. Chen, Automatic moving object
extractionthrough a real world variable-bandwidth network for
traffic monitoringsystems, IEEE Trans. Ind. Electron., vol. 61, no.
4, pp. 20992112,Apr. 2014.
[9] F. C. Cheng and S. J. Ruan, Accurate motion detection using
a self-adaptive background matching framework, IEEE Trans. Intell.
Transp.Syst., vol. 13, no. 2, pp. 671679, Jun. 2011.
[10] P. Foucher, P. Charbonnier, and H. Kebbous, Evaluation of a
road signpre-detection system by image analysis, in Proc. Int.
Conf. Comput. Vis.Theory Appl., 2009, pp. 362367.
[11] D. J. Dailey, F. W. Cathey, and S. Pumrin, An algorithm to
estimate meantraffic speed using uncalibrated cameras, IEEE Trans.
Intell. Transp.Syst., vol. 1, no. 2, pp. 98107, Jun. 2000.
[12] W. J. Wang, B. H. Chen, and S. C. Huang, A novel visibility
restorationalgorithm for single hazy images, in Proc. IEEE Int.
Conf. Syst., Man,Cybern., Oct. 2013, pp. 847851.
[13] Y. J. Cheng, B. H. Chen, S. C. Huang, S. Y. Kuo, A.
Kopylov, O. Seredin,Y. Vizilter, L. Mestetskiy, B. Vishnyakov, O.
Vygolov, C. R. Lian, andC. T. Wu, Visibility enhancement of single
hazy images using hybriddark channel prior, in Proc. IEEE Int.
Conf. Syst., Man, Cybern., Oct.2013, pp. 36273632.
[14] K. Tan and J. P. Oakley, Enhancement of color images in
poor visibilityconditions, in Proc. ICIP, Sep. 2000, vol. 2, pp.
788791.
[15] S. G. Narasimhan and S. K. Nayar, Interactive
(De)weathering of animage using physical models, in Proc. ICCV
Workshop Color Photometr.Methods Comput. Vis., Oct. 2003, pp.
13871394.
[16] J. Kopf, B. Neubert, B. Chen, M. Cohen, D. Cohen-Or, O.
Deussen,M. Uyttendaele, and D. Lischinski, Deep photo: Model-based
photo-graph enhancement and viewing, ACM Trans. Graphics, vol. 27,
no. 5,pp. 116:1116:10, Dec. 2008.
[17] Y. Y. Schechner, S. G. Narasimhan, and S. K. Nayar,
Polarization basedvision through haze, Appl. Opt., vol. 42, no. 3,
pp. 511525, Jan. 2003.
[18] S. G. Narasimhan and S. K. Nayar, Contrast restoration of
weatherdegraded images, IEEE Trans. Pattern Anal. Mach. Intell.,
vol. 25, no. 6,pp. 713724, Jun. 2003.
[19] S. K. Nayar and S. G. Narasimhan, Vision in bad weather, in
Proc. 7thIEEE Int. Conf. Comput. Vis., Jun. 1999, vol. 2, pp.
820827.
[20] R. Tan, Visibility in bad weather from a single image, in
Proc. IEEEConf. Comput. Vis. Pattern Recognit., Jun. 2008, pp.
18.
[21] R. Fattal, Single image dehazing, in Proc. ACM SIGGRAPH,
2008,p. 72.
[22] W. J. Li, B. Gu, J. T. Huang, S. Y. Wang, and M. H. Wang,
Single imagevisibility enhancement in gradient domain, IET Image
Process., vol. 6,no. 5, pp. 589595, Jul. 2012.
[23] K. He, J. Sun, and X. Tang, Single image haze removal using
darkchannel prior, IEEE Trans. Pattern Anal. Mach. Intell., vol.
33, no. 12,pp. 23412353, Dec. 2011.
[24] B. Xie, F. Guo, and Z. Cai, Improved single image dehazing
using darkchannel prior and multi-scale Retinex, in Proc. Int.
Conf. Intell. Syst.Des. Eng. Appl., Oct. 2010, pp. 848851.
[25] A. Levin, D. Lischinski, and Y. Weiss, A closed form
solution to naturalimage matting, in Proc. IEEE Conf. Comput. Vis.
Pattern Recognit.,2006, vol. 1, pp. 6168.
[26] A. C. Hurlbert, Formal connections between lightness
algorithms, J.Opt. Soc. Amer. A, Opt. Image Sci., vol. 3, no. 10,
pp. 16841693,Oct. 1986.
[27] Foggy Road Image DAtabase FRIDA. [Online]. Available:
http://www.lcpc.fr/english/products/image-databases/article/frida-foggy-road-image-database
[28] S. C. Huang, F. C. Cheng, and Y. S. Chiu, Efficient
contrast enhance-ment using adaptive gamma correction with
weighting distribution, IEEETrans. Image Process., vol. 9, no. 1,
pp. 10321041, Mar. 2013.
[29] N. Hautire, J.-P. Tarel, D. Aubert, and E. Dumont, Blind
contrast en-hancement assessment by gradient ratioing at visible
edges, Image Anal.Stereol., vol. 27, no. 2, pp. 8795, 2008.
[30] S. C. Huang and B. H. Chen, Highly accurate moving object
detectionin variable-bit-rate video-based traffic monitoring
systems, IEEE Trans.Neural Netw. Learn. Syst., vol. 24, no. 12, pp.
19201931, Dec. 2013.
Shih-Chia Huang received the Ph.D. degree in elec-trical
engineering from National Taiwan University,Taipei, Taiwan, in
2009.
He is an Associate Professor with the Depart-ment of Electronic
Engineering, College of Elec-tric Engineering and Computer Science,
NationalTaipei University of Technology and an Inter-national
Adjunct Professor with the Faculty ofBusiness and Information
Technology, Universityof Ontario Institute of Technology, Oshawa,
ON,Canada. He is the author or coauthor of more than
40 journal and conference papers, and he holds more than 30
patents inthe United States, Europe, Taiwan, and China. His
research interests includeimage and video coding, wireless video
transmission, video surveillance,error resilience and concealment
techniques, digital signal processing, cloudcomputing, mobile
applications and systems, embedded processor design, andembedded
software and hardware codesign.
Dr. Huang received the Kwoh-Ting Li Young Researcher Award in
2011from the Taipei Chapter of the Association for Computing
Machinery and theDr. Shechtman Young Researcher Award in 2012 from
the National TaipeiUniversity of Technology. He is an Associate
Editor of Journal of ArtificialIntelligence.
-
2332 IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS,
VOL. 15, NO. 5, OCTOBER 2014
Bo-Hao Chen received the B.S. degree from Na-tional Taipei
University of Technology, Taipei,Taiwan, in 2011. He is currently
working toward thePh.D. degree in the Department of Electronic
Engi-neering, National Taipei University of Technology.
His research interests include digital image pro-cessing, video
coding, particularly, moving objectdetection, contrast enhancement,
and haze removal.
Yi-Jui Cheng received the B.S. degree from Na-tional Chin-Yi
University of Technology, Taichung,Taiwan, in 2011 and the M.S.
degree from Na-tional Taipei University of Technology,
Taipei,Taiwan, in 2013.
He is with the Department of Electronic Engineer-ing, College of
Electric Engineering and ComputerScience, National Taipei
University of Technology,Taipei, Taiwan. His research interests
include digi-tal image processing, particularly contrast
enhance-ment, depth generation, and haze removal.
/ColorImageDict > /JPEG2000ColorACSImageDict >
/JPEG2000ColorImageDict > /AntiAliasGrayImages false
/CropGrayImages true /GrayImageMinResolution 300
/GrayImageMinResolutionPolicy /OK /DownsampleGrayImages true
/GrayImageDownsampleType /Bicubic /GrayImageResolution 300
/GrayImageDepth -1 /GrayImageMinDownsampleDepth 2
/GrayImageDownsampleThreshold 1.50000 /EncodeGrayImages true
/GrayImageFilter /DCTEncode /AutoFilterGrayImages false
/GrayImageAutoFilterStrategy /JPEG /GrayACSImageDict >
/GrayImageDict > /JPEG2000GrayACSImageDict >
/JPEG2000GrayImageDict > /AntiAliasMonoImages false
/CropMonoImages true /MonoImageMinResolution 1200
/MonoImageMinResolutionPolicy /OK /DownsampleMonoImages true
/MonoImageDownsampleType /Bicubic /MonoImageResolution 600
/MonoImageDepth -1 /MonoImageDownsampleThreshold 1.50000
/EncodeMonoImages true /MonoImageFilter /CCITTFaxEncode
/MonoImageDict > /AllowPSXObjects false /CheckCompliance [ /None
] /PDFX1aCheck false /PDFX3Check false /PDFXCompliantPDFOnly false
/PDFXNoTrimBoxError true /PDFXTrimBoxToMediaBoxOffset [ 0.00000
0.00000 0.00000 0.00000 ] /PDFXSetBleedBoxToMediaBox true
/PDFXBleedBoxToTrimBoxOffset [ 0.00000 0.00000 0.00000 0.00000 ]
/PDFXOutputIntentProfile (None) /PDFXOutputConditionIdentifier ()
/PDFXOutputCondition () /PDFXRegistryName () /PDFXTrapped
/False
/CreateJDFFile false /Description > /Namespace [ (Adobe)
(Common) (1.0) ] /OtherNamespaces [ > /FormElements false
/GenerateStructure false /IncludeBookmarks false /IncludeHyperlinks
false /IncludeInteractive false /IncludeLayers false
/IncludeProfiles false /MultimediaHandling /UseObjectSettings
/Namespace [ (Adobe) (CreativeSuite) (2.0) ]
/PDFXOutputIntentProfileSelector /DocumentCMYK /PreserveEditing
true /UntaggedCMYKHandling /LeaveUntagged /UntaggedRGBHandling
/UseDocumentProfile /UseDocumentBleed false >> ]>>
setdistillerparams> setpagedevice