This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Research ArticleMoving Object Detection and Shadow Removing underChanging Illumination Condition
Jinhai Xiang12 Heng Fan2 Honghong Liao1 Jun Xu3 Weiping Sun1 and Shengsheng Yu1
1 School of Computer Science and Technology Huazhong University of Science and Technology Wuhan 430074 China2 College of Science Huazhong Agricultural University Wuhan 430070 China3Department of Physics Center China Normal University Wuhan 430079 China
Correspondence should be addressed to Shengsheng Yu ssyumailhusteducn
Received 5 October 2013 Accepted 28 December 2013 Published 24 February 2014
Academic Editor Chung-Hao Chen
Copyright copy 2014 Jinhai Xiang et al This is an open access article distributed under the Creative Commons Attribution Licensewhich permits unrestricted use distribution and reproduction in any medium provided the original work is properly cited
Moving object detection is a fundamental step in video surveillance system To eliminate the influence of illumination change andshadow associated with the moving objects we proposed a local intensity ratio model (LIRM) which is robust to illuminationchange Based on the analysis of the illumination and shadowmodel we discussed the distribution of local intensity ratio And themoving objects are segmented without shadow using normalized local intensity ratio via Gaussian mixture model (GMM) Thenerosion is used to get the moving objects contours and erase the scatter shadow patches and noises After that we get the enhancedmoving objects contours by a new contour enhancement method in which foreground ratio and spatial relation are consideredAt last a new method is used to fill foreground with holes Experimental results demonstrate that the proposed approach can getmoving objects without cast shadow and shows excellent performance under various illumination change conditions
1 Introduction
Moving object detection is a fundamental step inmany imageanalysis applications including automated visual surveillancevideo indexing and human machine interaction Manyresearches on moving object detection have been proposedsuch as background subtraction optical flow and tem-poral differencing [1] However moving object detectiontechniques are often affected by factors such as shadowillumination changes and noise Generally different factorscause different consequences In this paper we focus onshadow and illumination factors
The shadow causes many problems in object localizationsegmentation object detection and tracking Furthermorethe shadowmay cause the following problems objects mergewith each other object shapesmay be altered the backgroundmay be misclassified as foreground and objects are missed[2] Shadows associated with moving objects can easilybe misinterpreted as additional objects At the same time
if illumination changes some background pixels may bedetected as foreground pixels which makes it hard to obtainthe cleanmoving objectsTherefore eliminating shadows andhandling illumination changes have a great effect on the per-formance of subsequent steps such as tracking recognitionclassification and activity analysis which need the accuratedetection of a moving object and the acquisition of its exactshape
Some reviews about shadowdetection have been reportedin the literature [2ndash4] from the views of physical andgeometrical to heuristic techniques Most of the real-timeshadow detection techniques work at pixel level and usecolor information for shadow detection directly or indirectlywholly or partially Shadowdetectionmethods can be roughlydivided into two categories based on statistic and based onthe video features
The principle of statistic-based methods is to build pixel-based statistical models of detecting cast shadows In [5]Zivkovic and van der Heijden use Gaussian mixture model
Hindawi Publishing CorporationMathematical Problems in EngineeringVolume 2014 Article ID 827461 10 pageshttpdxdoiorg1011552014827461
2 Mathematical Problems in Engineering
(GMM) to detect moving cast shadowsThis method consistsof building a GMM for moving objects identifying the dis-tribution of moving objects and shadows and modifying thelearning rates of the distributions Joshi and Papanikolopou-los [6] propose semisupervised learning technique to tacklethe static setting with human input for detecting shadowsJung [7] uses a statistical approach combined with geomet-rical constraints for detecting shadows but it is only for gray-scale video sequences The statistic-based methods identifydistribution of shadow pixel value and are robust in differentscenes Although these methods reduce false hits of propertydescriptions (ie color or intensity) of shadow it cannoteliminate them Generally these methods require trainingvideo sequences and shadowmust be extracted from trainingvideo sequences by hand Furthermore it is hard to operateonline and in real time because these methods need anadditional learning step
The video features-based methods are based on the factthat video features including geometric color gradient andbrightness and so forth are different in shadow backgroundand moving objects These methods are more general thanthose by statistic The methods based on image featurescan be further divided into the following four categories[2] chromaticity-based light physical characteristics-basedgeometric relations-based and texture-based
The chromaticity-based methods are mostly on the basisof a single pixel and combined with color information totest the shadow These methods make certain assumptionsabout the shadow properties (1) a shadow darkens thebackground area on which it falls (2) a shadows falls on thebackground plane (3) a shadow changes luminance of anarea more significantly than color [8] If these assumptionsare not satisfied the accuracy of color-based approachesfor shadow detection will decrease obviously For betterseparation between intensity and chromaticity several colorspaces such as HSV [9] c1c2c3 [10] HSL [11] and RGB [12]have been developed to detect moving cast shadow robustlyMost of these methods are computationally inexpensive andeasy to implement However they are sensitive to noise andwill fail when shadow regions are darker or moving objectshave similar color information with background
The light physical characteristics-based methods arebased on the linear attenuation model of the light intensitywhich assumes that illumination source produces pure whitelight In outdoor environment the main light source issunlight (white) and the reflected light comes from the sky(blue) Generally other light source is influenced by sunlightIf sunlight is blocked the effect of reflected light fromthe sky increases and the chromaticity of shaded region isshifted toward the blue component Nadimi and Bhanu [13]presented a dichromatic model which takes into account twolight sources to predict the color changes of the shaded regioneffectively Further works consider a variety of different lightintensity conditions and build a more general nonlinearattenuation model to adapt to the indoor and outdoor scenes[14 15] These methods are still using the chromaticitycharacteristics and if the foreground color is closer to the
cast shadow there would be some mistakes The geometricfeature-based methods mainly consider that under certainlight source shape and size of shadow position relationshipbetween objects can be ensured and segment shadow fromforeground [16ndash18] These methods do not need backgroundmodel however they need detailed shape information andother information of foreground targets These methods arelimited to detect specific objects and furthermore they needposition relationship between objects and shadow In the caseof multiple shadows and multiple objects these methods donot work well
The texture-based methods assume that the texture ofshaded region is invariant These methods generally includethe following steps (1) detect foregroundwith shadow and (2)classify foreground as either foreground or shadow based onthe texture correlation If texture of candidate area is similarto that of background it may be misclassified as the shadowLeone and Distante [19] propose a moving cast shadowsmethod based on Gabor functions and matching pursuitstrategy Zhang et al [20] employ ratio edge as the ratiobetween the intensity of one pixel and its neighboring pixelsto detect shadows Confirming the existence of shadowsXiao et al [21] reconstruct coarse object shapes and thenextract cast shadows by subtracting moving objects from onechangedmaskThe texture-basedmethods are effective with-out colors information and robust to illumination changesHowever texture-based shadow detection methods need tocompare adjacent pixels and their complexity is high
Generally we must obtain true foreground pixels beforeshadow detection by using methods above A backgroundsubtractionmethod such as Gaussianmixturemodel (GMM)and its modified versions are some representative methodsto detect moving pixels with shadow However in a dynamicscene the varying background is detected as amoving objectIn order to remove shadow directly and make it robust toillumination variations we present a local intensity ratiomodel (LIRM) which shows illumination invariance Firstwe use normalized local intensity ratio to replace the pixel todetect moving object without shadow via Gaussian mixturemodel Second erosion is used to get the moving objectscontours and erase the scatter shadow patches and noisesAfter that we get the enhanced moving object contoursby contour enhancement method from erosion image andforeground image At last we use the local foregrounddensity and contour orientation to fill enhanced movingobjects with holes Experimental results demonstrate thatthe proposed approach can get moving objects without castshadow and shows excellent performance under variouschanging illumination conditions
This paper is organized as follows Section 2 introducesillumination change model presents LIRM and analyzesits distribution The process of Gaussian mixture modelfor foreground detection and corresponding postprocessalgorithm are described in Section 3 Section 4 analyzesforeground detection results in four test videos with differentlight conditions and compares the resultswith othermethodsFinally Section 5 concludes the paper
Mathematical Problems in Engineering 3
2 Illumination Change Model
21 Local Intensity Ratio and Illumination Invariant Zhanget al [20] proved that the ratio edge is illumination invariantand use it to classify eachmoving pixel into foreground objector moving shadow Inspired by this work we define the localintensity ratio and analyze its illumination invariance
First the definition of the local intensity ratio (LIR) is
where 119901(119909 119910) is the intensity of pixel (119909 119910) |119860(119909 119910)| isthe number of pixels in a local region 119860(119909 119910) is the localregion of given pixel (119909 119910) (a rectangle region) and definedas follows
where 120588(119909 119910) is the reflectance of object surface in pixel(119909 119910) that is the reflection coefficient 119868(119909 119910) is the amountof light power per receiving object surface area in pixel (119909 119910)120579(119909 119910) is the sensor sensitivity of the camera The light inscenes can be divided into direct light of light source andscattered light of environment We assume that the lightsource is a distant light source and the light is parallel lightsuch as sunlight Light of environment is scattered light lightdirection is random and its intensity in scenes is assumed tobe constant If objects are occluded the scene can be dividedinto three cases illuminated area penumbra area and umbraarea [22 23] and the light intensity of the real target pixel(119909 119910) is
sdot cos120601 (119909 119910) penumbra area119871119886 umbra area
(4)
where 119871119886 is the intensity of ambient light 119871119901 is the intensityof light source 120572(119909 119910) is the transition inside the penumbrawhich depends on the light source and scene geometry and0 ⩽ 120572(119909 119910) ⩽ 1 120601(119909 119910) is the angle between light sourcedirection and surface normal
While using RGBmodel to express light intensity (4) canbe shown as follows
119868(119909 119910)119896=
119871119896119886 + 119871119896119901 sdot cos120601 (119909 119910) illuminated area
sdot cos120601 (119909 119910) penumbra area119871119896119886 umbra area
(5)
where 119896 is the number of color channels 119896 = 1 2 and 3 Forsimplification in this paper we consider only one light sourceand ignore the influence of different color channels
The analysis of the local intensity ratio under the threedifferent scenes is as shown in Figure 1
If all the local regions belong to one of the three areasshown in Figure 1 that is 119860(119909 119910) isin 1198601 or 119860(119909 119910) isin 1198602 or119860(119909 119910) isin 1198603 and11986011198602 and1198603 express the illuminate areapenumbra area and umbra area respectively we can obtainthe following results by formulas (4) (3) and (2) and assume|119860(119909 119910)| = 119899
Case 1 (119860(119909 119910) isin 1198601) Consider
If the pixels are in the same local region we assume that(1) pixels in the same local region belong to the same objectplane thus 120601(119909 119910) = 120601(119894 119895) (2) sensor sensitivity of camerain all local regions is the same 120579(119909 119910) = 120579(119894 119895) (3) intensityof penumbra is also the same 120572(119909 119910) = 120572(119894 119895)
If local region belongs to one of the three illuminationcases local intensity ratio is influenced only by reflectivityTherefore local intensity ratio is related to the reflectivity oftarget surface not to the illumination changes and types thatis illumination invariance
4 Mathematical Problems in Engineering
Sun
Object
Umbra
Penumbra
Sky
(a)
Illuminated(1)
Penumbra(2)
Umbra(3)
(b)
Figure 1 The illumination of the scene (three cases)
The local intensity ratio of pixels is constant underdifferent illumination conditions according to formula (12)As a result local intensity ratio model not only removes theinfluence of illumination but also eliminates shadow of theforeground targets
22 Distribution of Local Intensity Ratio Generally it isassumed that the images are corrupted by Gaussian whitenoise which can be expressed as
where 119875119903(119909 119910) represents actual pixel value 119875119887(119909 119910) denotesreal pixel value in scene 120576(119909 119910) represents noise The actuallocal intensity ratio is defined as
If pixel value in scene is constant 119875119887(119909 119910)
sum(119894119895)isin119860(119909119910) 119875119887(119894 119895) obeys Gaussian white noise distributionTherefore local intensity ratio and pixel value have the same
distribution On the other hand if sum(119894119895)isin119860(119909119910) 119875119887(119894 119895) issmall the white noise will be amplified
Theoretically the local intensity ratio ranges from 0 toinfinite In order to be consistent with the scope of a pixelvalue the normalized definition of the local intensity ratio isas follows
LR (119909 119910) = (2
1 + 119890minus119877(119909119910)minus 1) times 255 (17)
3 Moving Object Detection Based onLocal Intensity Ratio
In this section we present the detailed procedure of shadowremovingTheproposedmethod is amultistage approach andthe flow chart is shown in Figure 2
Generally the shadow detection methods utilize objectdetection or image segmentation algorithms to detect theforeground with shadow and then classify the foregroundas foreground or shadow According to the results in theprevious section the local intensity ratio obeys Gaussiandistribution This paper uses the normalized LIR to replacethe pixel to detect foreground and the process is shownin Figure 2 In this paper Gaussian mixture model [5] isused to acquire the foreground In mixture of Gaussians therecent history of each pixel is maintained using 119896 Gaussiandistributions and the value of 119896 typically chooses from 3 to5 Each distribution has its associated attributes like weight120596119896 mean 120583119896 and variance 120590119896 Each Gaussian updates itsparameters using LIR in every new frame
After detecting foreground the foreground target withlittle shadow can be obtained (called foreground image asfollows 119868fore) Meanwhile some parts of the foregroundobject may be darker than the background and easily befalse detected as background The false detected parts ofshadow are discrete the false detected parts of moving objectlike holes are surrounded by foreground pixels (such ascontour) The moving objects contour can be detected whilethe shadows contour can be removed at most time In orderto get enhanced moving objects without discrete shadowpatches erosion is used to erase the scatter and noises and toget moving objects contours image without shadow (calledcontour image as follows 119868cont) Then the contour image is
Mathematical Problems in Engineering 5
Calculate LIR Calculate LIRMoving objectsdetection via GMM
Enhanced movingobject contour
Background andshadow pixels Moving object pixels
Figure 2 Flow chart of moving object detection and shadow removing
Figure 3 Local region of the center pixel detection and its eightdirections
used to enhance moving objects by contour enhancementmethodThe enhancement procedure is as follows for a pixel119875(119909 119910) being not foreground pixel in contours image wecalculate the weighted foreground pixel ratio of a local regionin the contour image and foreground image as follows
where 119904(119909 119910) represents the rate of the enhanced movingobject contours in (119909 119910) 119873119888(119909 119910) is the number of fore-ground pixels in a neighbor region of pixel (119909 119910) in contourimage 119873119891(119909 119910) is the number of foreground pixels in aneighbor region of pixel (119909 119910) in foreground image and120596119888 and 120596119891 are the weight to contour image and foregroundimage We select 120596119888 = 3 120596119891 = 1 in our experiments |119860(119909 119910)|
is the number of pixels in a local region 119860(119909 119910) is the localregion of given pixel (119909 119910) (a rectangle region) and definedas in (2)
On the other hand we consider eight directions of a pixeldeterminewhether there is foreground pixel in each direction
of the local region and count how many times it reachesforeground pixels as shown in Figure 3
120575119894 (119909 119910) = 1 foreground pixels in the 119894th direction0 otherwise
V (119909 119910) =1
8
8
sum
119894=1
120575119894 (119909 119910)
(19)
where 120575119894 means whether there is a foreground pixel in the 119894thdirection and if it exists its value is 1 otherwise 0 V(119909 119910) isthe ratio of foreground in eight directions After calculating119904(119909 119910) and V(119909 119910) by verifying whether or not the 119904(119909 119910)
and V(119909 119910) of a pixel is in the feasible range the pixel isdetermined by
where 120574V and 120574119904 are the threshold valuesSo we get the enhanced foreground contours image
(called enhanced foreground contour image 119868enfc) Howeversome moving object pixels are false detected as backgroundpixels such as holes in the image To get integral foregroundobject we use filling method to eliminate false detected pixelfor background pixel (Figure 4) For a pixel (119909 119910) whichis detected as background pixel in enhanced foregroundcontour image we first consider eight directions of a pixeldeterminewhether there is foreground pixel in each directionof the local region and count how many times it reachesforeground pixels as expressed in (19) We also consider theratio in a local region between numbers of foreground pixelsin foreground image and in contour image and the numbersof pixels in the same region
1199041015840(119909 119910) =
119873119904 (119909 119910)
119873119903 (119909 119910) (21)
where 1199041015840(119909 119910) is the ratio of pixels detected as foreground
in local region 119873119904(119909 119910) is the number of pixels which aredetected as foreground and 119873119903(119909 119910) is the sum of pixels inthe local region
6 Mathematical Problems in Engineering
(a) (b) (c)
(d) (e) (f)
Figure 4 Illustration of the shadow removal process using Campus sequence (a) Current frame (b) Normalized LIR of current frame (c)Foreground object detected using GMM of LIR (d) Result of erosion operation (contour image) to remove false detection shadow and noise(e) Enhanced contour image by contour enhancement method (f) Final result of filling holes
Shadow detection rate before fillingShadow discrimination rate before fillingCombined score before fillingShadow detection rate after fillingShadow discrimination rate after fillingCombined score after filling
100
90
80
70
60
50
40
30
20
Shad
ow d
etec
tion
perfo
rman
ce (
)
Hallway Campus Room Highway
Figure 5 Compared detection result of LIRM via GMM andpostprocess
After calculating V(119909 119910) and 1199041015840(119909 119910) we can get the
option that the pixel 119875(119909 119910) is moving object pixel or not by
119901 (119909 119910) = 255 V (119909 119910) lt 120574
1015840V 1199041015840(119909 119910) lt 120574
1015840119904
0 otherwise(22)
where 1205741015840V and 1205741015840119904 are the threshold values 119901(119909 119910) denotes the
final value of pixel (119909 119910)
4 Experiment Results
In this section we describe the test videos used in thisstudymeasure the performance of the proposedmethod andcompare the performance to the state-of-the-art methodsIn general state-of-the-art methods use different steps toremove shadow and reduce the influence of illuminationchanges To compare with other methods in different stepswe first compare the shadow removing results and then com-pare the results under various illumination change conditionsin different data sets
41 Experiment Conditions and Parameter Setting The size ofthe local rectangle region in (2) is important to the proposedmethod If the size is too large its inner region cannot meetthe assumptions (9) On the other hand if too small thechange of its value is also small In our experiment 119903 is setas 2
42 Experiment Results To test the effect of shadow remov-ing based on local intensity ratio and the effect of foregrounddetection we use four typical videos from the ATON videoset [4] as the test videos which are Campus Hallway High-way 1 and Intelligent room Intelligent room andHallway aretypical indoor environment In the videos people are walkingin scene and their shadows are mapped to the backgroundThe Campus is a video of campus parking lot and there areshadows of people and cars in the scenes
Mathematical Problems in Engineering 7
Origin
Realresult
Proposed
(a)
Origin
Realresult
Proposed
(b)
Origin
Realresult
Proposed
(c)
Origin
Realresult
Proposed
(d)
Figure 6 Detection results in different scenes (a) is Highway I sequence (b) is intelligent room sequence (c) is Campus sequence and (d)is Hallway sequence In the image of detection result the moving object pixels are white the background is black and the shadow is gray
To compare the performance of the proposed movingobjects detectionmethod with other shadow detectionmeth-ods we assume that if shadow is detected as backgroundit is correct detection otherwise it is false detection andif moving object is detected as foreground it is correct
detection otherwise it is false detection The proposedmethod is compared with five methods introduced by [2]which are chromaticity-based method physical methodgeometry-based method Small region (SR) texture-basedmethod and large region (LR) texture-based method To
Figure 7 The comparison of moving object detection results under the changing illumination condition (three scenes) From left to right(a) is video 1 (b) is video 2 and (c) is video 3 The 1st row is an input image the remaining nine rows are detection results of each methodthe 2nd row is a detection result of GMM the 3rd row is of adaptive GMM the 4th row is of LBP the 5th row is of Xu the 6th row is of Piletthe 7th row is of Choi the 8th row is of the proposed method without postprocessing and the 9th row is the final result of proposed method
estimate the effect of these methods shadow detection rate(120578) and shadow distinguishing rate (120585) are the discriminateparameters [4] and their definitions are as follows
120578 =119879119875119878
119879119875119878 + 119865119873119878
times 100 120585 =119879119875119865
119879119875119865 + 119865119873119865
times 100
(23)
where 119878 is shadow 119865 is foreground 119879119875119878 is the number ofthe pixels which are detected correctly 119865119873119878 is the number ofshadow pixels which are mistakenly detected as foreground119879119875119865 is the number of foreground pixels detected correctlyafter removing shadows 119865119873119865 is the number of foregroundpixels which are mistakenly detected as shadow The shadowdistinguishing rate is concerned with maintaining the pixels
which belong to the moving object as foreground In thispaper we use the average of the two rates as a singleperformance measure (avg)
Table 1 is the compared results among the method basedon LIRM and other methods in [2] Table 1 shows the averageshadow detection and discrimination rates on each testsequence From the compared results the method proposedin this paper is better at most time The main cause isthat the presented method directly detects the foregroundfrom video data but other methods extract the shadowfrom foreground by true background and foreground withshadow Our method use local intensity ratio to replace theactual pixel values complexity is lower Figure 6 is the resultof proposed method in moving object detection withoutshadow in different scenes
Mathematical Problems in Engineering 9
Table 1 Comparison of moving object detection with different methods
Figure 5 represents compared results of the LIRMmovingobject detection via GMM and postprocess From Figure 5the postprocess obviously improves the shadow discrimi-nation rate and slightly reduces the shadow detection rateHowever the average of the two rates is increased obviouslySo the postprocess enhances the moving object detectionresult
Finally we show that the proposedmethod rapidly adaptsto variations in environment To test moving object detectionunder illumination changes condition we compare the resultof other techniques which are GMM [5] adaptive GMM [24]that controls the learning rate of the GMM adaptively LBP[25] that detects moving objects using texture and edges Xursquosmethods [26] that use a color chromaticity Piletrsquos methods[27] that use illumination and spatial likelihood and Choirsquosmethods [28] that develop chromaticity differencemodel andbrightness model that estimates the intensity difference andintensity ratio of false foreground pixels For the quantitativecomparison we have made three test videos [28] Figure 7represents moving object detection results of each methodFrom Figure 7 we find the postprocessing of the proposedmethod can strengthen the moving objects and remove theshadow patches and the noise but it sometimes enhances theedge of part shadows incorrectly and makes moving objectsrsquoedge fuzzy In a word the proposed method adapts to thechanged background completely and detects moving objectpixels without shadow pixels successfully Although someother methods are robust to illumination their detectionmoving pixels results show that they cannot detect movingobject pixels meanwhile removing shadow
5 Conclusions
In this paper we propose a method to detect movingobject meanwhile removing cast shadow which is robust inchanging illumination condition This paper presents a localintensity ratio model according to illumination model andproves its illumination invariance Meanwhile if the noisein video obeys the Gaussian distribution the local intensityratio also obeys the Gaussian distribution In the process ofGaussian mixture model to obtain foreground this paperreplaces actual pixel value by local intensity ratio Finallypostprocess methods are used to get pure moving objectpixels without noise such as erosion method to removeshadow pitch and noise which is false detected as foregroundforeground contour enhanced method to strengthen moving
object contour and filling methods to deal with holes inforeground object Experiment results demonstrate that themethod presented by this paper can eliminate shadow onforeground effectively and is robust in changing illuminationconditionHowever in some scenes that foreground is similarto background or foreground is similar to shadow fore-groundmay be easily detected as background If illuminationchanges severely such as turn onoff light in indoor scenethe backgroundmay be easily detected as foreground and theperformance drops
Conflict of Interests
The authors declare that there is no conflict of interestsregarding the publication of this paper
Acknowledgments
Jinhai Xiang is supported by the Fundamental ResearchFunds for the Central Universities under Grant no2013QC024 Jun Xu is supported by the National NaturalScience Foundation of China under Grant no 11204099and self-determined research funds of CCNU from thecolleges basic research and operation of MOE Weiping Sunis supported by the National Natural Science Foundation ofChina under Grant no 61300140
References
[1] M Piccardi ldquoBackground subtraction techniques a reviewrdquo inProceedings of the IEEE International Conference on SystemsMan and Cybernetics (SMC rsquo04) pp 3099ndash3104 October 2004
[2] A Sanin C Sanderson and B C Lovell ldquoShadow detection asurvey and comparative evaluation of recent methodsrdquo PatternRecognition vol 45 no 4 pp 1684ndash1695 2012
[3] N Al-Najdawi H E Bez J Singhai and E A Edirisinghe ldquoAsurvey of cast shadow detection algorithmsrdquo Pattern Recogni-tion Letters vol 33 no 6 pp 752ndash764 2012
[4] A Prati I Mikic M M Trivedi and R Cucchiara ldquoDetectingmoving shadows algorithms and evaluationrdquo IEEE Transac-tions on Pattern Analysis and Machine Intelligence vol 25 no7 pp 918ndash923 2003
[5] Z Zivkovic and F van der Heijden ldquoEfficient adaptive densityestimation per image pixel for the task of background subtrac-tionrdquo Pattern Recognition Letters vol 27 no 7 pp 773ndash7802006
10 Mathematical Problems in Engineering
[6] A J Joshi and N P Papanikolopoulos ldquoLearning to detectmoving shadows in dynamic environmentsrdquo IEEE Transactionson Pattern Analysis and Machine Intelligence vol 30 no 11 pp2055ndash2063 2008
[7] C R Jung ldquoEfficient background subtraction and shadowremoval for monochromatic video sequencesrdquo IEEE Transac-tions on Multimedia vol 11 no 3 pp 571ndash577 2009
[8] M Shoaib R Dragon and J Ostermann ldquoShadow detection formoving humans using gradient-based background subtractionrdquoin Proceedings of the IEEE International Conference onAcousticsSpeech and Signal Processing (ICASSP rsquo09) pp 773ndash776 April2009
[9] R Cucchiara C Grana M Piccardi and A Prati ldquoDetectingmoving objects ghosts and shadows in video streamsrdquo IEEETransactions on Pattern Analysis and Machine Intelligence vol25 no 10 pp 1337ndash1342 2003
[10] E Salvador A Cavallaro and T Ebrahimi ldquoCast shadowsegmentation using invariant color featuresrdquo Computer Visionand Image Understanding vol 95 no 2 pp 238ndash259 2004
[11] D Grest J M Frahm and R Koch ldquoA color similarity measurefor robust shadow removal in real timerdquo in Proceedings of theVision Modeling and Visualization (VMV rsquo03) pp 253ndash260November 2003
[12] THorprasert DHarwood and L Davis ldquoA statistical approachfor real-time robust background subtraction and shadow detec-tionrdquo in Proceedings of the IEEE International Conference onComputer Vision (ICCV rsquo99) Frame Rate Workshop 1999
[13] S Nadimi and B Bhanu ldquoPhysical models for moving shadowand object detection in videordquo IEEE Transactions on PatternAnalysis and Machine Intelligence vol 26 no 8 pp 1079ndash10872004
[14] J B Huang and C S Chen ldquoMoving cast shadow detec-tion using Physics-based featuresrdquo in Proceedings of the IEEEComputer Society Conference on Computer Vision and PatternRecognition Workshops (CVPR rsquo09) pp 2310ndash2317 June 2009
[15] N Martel-Brisson and A Zaccarin ldquoKernel-based learning ofcast shadows fromaphysicalmodel of light sources and surfacesfor low-level segmentationrdquo in Proceedings of the 26th IEEEConference on Computer Vision and Pattern Recognition (CVPRrsquo08) pp 1ndash8 June 2008
[16] C C Chen and J K Aggarwal ldquoHuman shadow removal withunknown light sourcerdquo in Proceedings of the 20th InternationalConference on Pattern Recognition (ICPR rsquo10) pp 2407ndash2410August 2010
[17] J W Hsieh W F Hu C J Chang and Y S Chen ldquoShadowelimination for effective moving object detection by Gaussianshadow modelingrdquo Image and Vision Computing vol 21 no 6pp 505ndash516 2003
[18] H Nicolas and J M Pinel ldquoJoint moving cast shadows seg-mentation and light source detection in video sequencesrdquo SignalProcessing Image Communication vol 21 no 1 pp 22ndash43 2006
[19] A Leone andCDistante ldquoShadowdetection formoving objectsbased on texture analysisrdquoPattern Recognition vol 40 no 4 pp1222ndash1233 2007
[20] W Zhang X Z Fang X K Yang and Q M J Wu ldquoMovingcast shadows detection using ratio edgerdquo IEEE Transactions onMultimedia vol 9 no 6 pp 1202ndash1213 2007
[21] M Xiao C Z Han and L Zhang ldquoMoving shadow detectionand removal for traffic sequencesrdquo International Journal ofAutomation and Computing vol 4 no 1 pp 38ndash46 2007
[22] J Stauder R Mech and J Ostermann ldquoDetection of movingcast shadows for object segmentationrdquo IEEE Transactions onMultimedia vol 1 no 1 pp 65ndash76 1999
[23] M S Andersen T Jensen and C B Madsen ldquoEstimationof dynamic light changes in outdoor scenes without the useof calibration objectsrdquo in Proceedings of the 18th InternationalConference on Pattern Recognition (ICPR rsquo06) vol 4 pp 91ndash94August 2006
[24] D S Lee ldquoEffective Gaussian mixture learning for videobackground subtractionrdquo IEEETransactions on PatternAnalysisand Machine Intelligence vol 27 no 5 pp 827ndash832 2005
[25] M Heikkila and M Pietikainen ldquoA texture-based method formodeling the background and detecting moving objectsrdquo IEEETransactions on Pattern Analysis and Machine Intelligence vol28 no 4 pp 657ndash662 2006
[26] M Xu and T Ellis ldquoColour-invariant motion detection underfast illumination changesrdquo in Proceedings of the 2nd EuropeanWorkshop on Advanced Video-Based Surveillance Systems vol4 pp 101ndash112 September 2001
[27] J Pilet C Strecha and P Fua ldquoMaking background subtractionrobust to sudden illumination changesrdquo in Proceedings of the10th European Conference on Computer Vision (ECCV rsquo08) pp567ndash580 October 2008
[28] J Choi H J Chang Y J Yoo and J Y Choi ldquoRobust movingobject detection against fast illumination changerdquo ComputerVision and Image Understanding vol 116 no 2 pp 179ndash1932012
(GMM) to detect moving cast shadowsThis method consistsof building a GMM for moving objects identifying the dis-tribution of moving objects and shadows and modifying thelearning rates of the distributions Joshi and Papanikolopou-los [6] propose semisupervised learning technique to tacklethe static setting with human input for detecting shadowsJung [7] uses a statistical approach combined with geomet-rical constraints for detecting shadows but it is only for gray-scale video sequences The statistic-based methods identifydistribution of shadow pixel value and are robust in differentscenes Although these methods reduce false hits of propertydescriptions (ie color or intensity) of shadow it cannoteliminate them Generally these methods require trainingvideo sequences and shadowmust be extracted from trainingvideo sequences by hand Furthermore it is hard to operateonline and in real time because these methods need anadditional learning step
The video features-based methods are based on the factthat video features including geometric color gradient andbrightness and so forth are different in shadow backgroundand moving objects These methods are more general thanthose by statistic The methods based on image featurescan be further divided into the following four categories[2] chromaticity-based light physical characteristics-basedgeometric relations-based and texture-based
The chromaticity-based methods are mostly on the basisof a single pixel and combined with color information totest the shadow These methods make certain assumptionsabout the shadow properties (1) a shadow darkens thebackground area on which it falls (2) a shadows falls on thebackground plane (3) a shadow changes luminance of anarea more significantly than color [8] If these assumptionsare not satisfied the accuracy of color-based approachesfor shadow detection will decrease obviously For betterseparation between intensity and chromaticity several colorspaces such as HSV [9] c1c2c3 [10] HSL [11] and RGB [12]have been developed to detect moving cast shadow robustlyMost of these methods are computationally inexpensive andeasy to implement However they are sensitive to noise andwill fail when shadow regions are darker or moving objectshave similar color information with background
The light physical characteristics-based methods arebased on the linear attenuation model of the light intensitywhich assumes that illumination source produces pure whitelight In outdoor environment the main light source issunlight (white) and the reflected light comes from the sky(blue) Generally other light source is influenced by sunlightIf sunlight is blocked the effect of reflected light fromthe sky increases and the chromaticity of shaded region isshifted toward the blue component Nadimi and Bhanu [13]presented a dichromatic model which takes into account twolight sources to predict the color changes of the shaded regioneffectively Further works consider a variety of different lightintensity conditions and build a more general nonlinearattenuation model to adapt to the indoor and outdoor scenes[14 15] These methods are still using the chromaticitycharacteristics and if the foreground color is closer to the
cast shadow there would be some mistakes The geometricfeature-based methods mainly consider that under certainlight source shape and size of shadow position relationshipbetween objects can be ensured and segment shadow fromforeground [16ndash18] These methods do not need backgroundmodel however they need detailed shape information andother information of foreground targets These methods arelimited to detect specific objects and furthermore they needposition relationship between objects and shadow In the caseof multiple shadows and multiple objects these methods donot work well
The texture-based methods assume that the texture ofshaded region is invariant These methods generally includethe following steps (1) detect foregroundwith shadow and (2)classify foreground as either foreground or shadow based onthe texture correlation If texture of candidate area is similarto that of background it may be misclassified as the shadowLeone and Distante [19] propose a moving cast shadowsmethod based on Gabor functions and matching pursuitstrategy Zhang et al [20] employ ratio edge as the ratiobetween the intensity of one pixel and its neighboring pixelsto detect shadows Confirming the existence of shadowsXiao et al [21] reconstruct coarse object shapes and thenextract cast shadows by subtracting moving objects from onechangedmaskThe texture-basedmethods are effective with-out colors information and robust to illumination changesHowever texture-based shadow detection methods need tocompare adjacent pixels and their complexity is high
Generally we must obtain true foreground pixels beforeshadow detection by using methods above A backgroundsubtractionmethod such as Gaussianmixturemodel (GMM)and its modified versions are some representative methodsto detect moving pixels with shadow However in a dynamicscene the varying background is detected as amoving objectIn order to remove shadow directly and make it robust toillumination variations we present a local intensity ratiomodel (LIRM) which shows illumination invariance Firstwe use normalized local intensity ratio to replace the pixel todetect moving object without shadow via Gaussian mixturemodel Second erosion is used to get the moving objectscontours and erase the scatter shadow patches and noisesAfter that we get the enhanced moving object contoursby contour enhancement method from erosion image andforeground image At last we use the local foregrounddensity and contour orientation to fill enhanced movingobjects with holes Experimental results demonstrate thatthe proposed approach can get moving objects without castshadow and shows excellent performance under variouschanging illumination conditions
This paper is organized as follows Section 2 introducesillumination change model presents LIRM and analyzesits distribution The process of Gaussian mixture modelfor foreground detection and corresponding postprocessalgorithm are described in Section 3 Section 4 analyzesforeground detection results in four test videos with differentlight conditions and compares the resultswith othermethodsFinally Section 5 concludes the paper
Mathematical Problems in Engineering 3
2 Illumination Change Model
21 Local Intensity Ratio and Illumination Invariant Zhanget al [20] proved that the ratio edge is illumination invariantand use it to classify eachmoving pixel into foreground objector moving shadow Inspired by this work we define the localintensity ratio and analyze its illumination invariance
First the definition of the local intensity ratio (LIR) is
where 119901(119909 119910) is the intensity of pixel (119909 119910) |119860(119909 119910)| isthe number of pixels in a local region 119860(119909 119910) is the localregion of given pixel (119909 119910) (a rectangle region) and definedas follows
where 120588(119909 119910) is the reflectance of object surface in pixel(119909 119910) that is the reflection coefficient 119868(119909 119910) is the amountof light power per receiving object surface area in pixel (119909 119910)120579(119909 119910) is the sensor sensitivity of the camera The light inscenes can be divided into direct light of light source andscattered light of environment We assume that the lightsource is a distant light source and the light is parallel lightsuch as sunlight Light of environment is scattered light lightdirection is random and its intensity in scenes is assumed tobe constant If objects are occluded the scene can be dividedinto three cases illuminated area penumbra area and umbraarea [22 23] and the light intensity of the real target pixel(119909 119910) is
sdot cos120601 (119909 119910) penumbra area119871119886 umbra area
(4)
where 119871119886 is the intensity of ambient light 119871119901 is the intensityof light source 120572(119909 119910) is the transition inside the penumbrawhich depends on the light source and scene geometry and0 ⩽ 120572(119909 119910) ⩽ 1 120601(119909 119910) is the angle between light sourcedirection and surface normal
While using RGBmodel to express light intensity (4) canbe shown as follows
119868(119909 119910)119896=
119871119896119886 + 119871119896119901 sdot cos120601 (119909 119910) illuminated area
sdot cos120601 (119909 119910) penumbra area119871119896119886 umbra area
(5)
where 119896 is the number of color channels 119896 = 1 2 and 3 Forsimplification in this paper we consider only one light sourceand ignore the influence of different color channels
The analysis of the local intensity ratio under the threedifferent scenes is as shown in Figure 1
If all the local regions belong to one of the three areasshown in Figure 1 that is 119860(119909 119910) isin 1198601 or 119860(119909 119910) isin 1198602 or119860(119909 119910) isin 1198603 and11986011198602 and1198603 express the illuminate areapenumbra area and umbra area respectively we can obtainthe following results by formulas (4) (3) and (2) and assume|119860(119909 119910)| = 119899
Case 1 (119860(119909 119910) isin 1198601) Consider
If the pixels are in the same local region we assume that(1) pixels in the same local region belong to the same objectplane thus 120601(119909 119910) = 120601(119894 119895) (2) sensor sensitivity of camerain all local regions is the same 120579(119909 119910) = 120579(119894 119895) (3) intensityof penumbra is also the same 120572(119909 119910) = 120572(119894 119895)
If local region belongs to one of the three illuminationcases local intensity ratio is influenced only by reflectivityTherefore local intensity ratio is related to the reflectivity oftarget surface not to the illumination changes and types thatis illumination invariance
4 Mathematical Problems in Engineering
Sun
Object
Umbra
Penumbra
Sky
(a)
Illuminated(1)
Penumbra(2)
Umbra(3)
(b)
Figure 1 The illumination of the scene (three cases)
The local intensity ratio of pixels is constant underdifferent illumination conditions according to formula (12)As a result local intensity ratio model not only removes theinfluence of illumination but also eliminates shadow of theforeground targets
22 Distribution of Local Intensity Ratio Generally it isassumed that the images are corrupted by Gaussian whitenoise which can be expressed as
where 119875119903(119909 119910) represents actual pixel value 119875119887(119909 119910) denotesreal pixel value in scene 120576(119909 119910) represents noise The actuallocal intensity ratio is defined as
If pixel value in scene is constant 119875119887(119909 119910)
sum(119894119895)isin119860(119909119910) 119875119887(119894 119895) obeys Gaussian white noise distributionTherefore local intensity ratio and pixel value have the same
distribution On the other hand if sum(119894119895)isin119860(119909119910) 119875119887(119894 119895) issmall the white noise will be amplified
Theoretically the local intensity ratio ranges from 0 toinfinite In order to be consistent with the scope of a pixelvalue the normalized definition of the local intensity ratio isas follows
LR (119909 119910) = (2
1 + 119890minus119877(119909119910)minus 1) times 255 (17)
3 Moving Object Detection Based onLocal Intensity Ratio
In this section we present the detailed procedure of shadowremovingTheproposedmethod is amultistage approach andthe flow chart is shown in Figure 2
Generally the shadow detection methods utilize objectdetection or image segmentation algorithms to detect theforeground with shadow and then classify the foregroundas foreground or shadow According to the results in theprevious section the local intensity ratio obeys Gaussiandistribution This paper uses the normalized LIR to replacethe pixel to detect foreground and the process is shownin Figure 2 In this paper Gaussian mixture model [5] isused to acquire the foreground In mixture of Gaussians therecent history of each pixel is maintained using 119896 Gaussiandistributions and the value of 119896 typically chooses from 3 to5 Each distribution has its associated attributes like weight120596119896 mean 120583119896 and variance 120590119896 Each Gaussian updates itsparameters using LIR in every new frame
After detecting foreground the foreground target withlittle shadow can be obtained (called foreground image asfollows 119868fore) Meanwhile some parts of the foregroundobject may be darker than the background and easily befalse detected as background The false detected parts ofshadow are discrete the false detected parts of moving objectlike holes are surrounded by foreground pixels (such ascontour) The moving objects contour can be detected whilethe shadows contour can be removed at most time In orderto get enhanced moving objects without discrete shadowpatches erosion is used to erase the scatter and noises and toget moving objects contours image without shadow (calledcontour image as follows 119868cont) Then the contour image is
Mathematical Problems in Engineering 5
Calculate LIR Calculate LIRMoving objectsdetection via GMM
Enhanced movingobject contour
Background andshadow pixels Moving object pixels
Figure 2 Flow chart of moving object detection and shadow removing
Figure 3 Local region of the center pixel detection and its eightdirections
used to enhance moving objects by contour enhancementmethodThe enhancement procedure is as follows for a pixel119875(119909 119910) being not foreground pixel in contours image wecalculate the weighted foreground pixel ratio of a local regionin the contour image and foreground image as follows
where 119904(119909 119910) represents the rate of the enhanced movingobject contours in (119909 119910) 119873119888(119909 119910) is the number of fore-ground pixels in a neighbor region of pixel (119909 119910) in contourimage 119873119891(119909 119910) is the number of foreground pixels in aneighbor region of pixel (119909 119910) in foreground image and120596119888 and 120596119891 are the weight to contour image and foregroundimage We select 120596119888 = 3 120596119891 = 1 in our experiments |119860(119909 119910)|
is the number of pixels in a local region 119860(119909 119910) is the localregion of given pixel (119909 119910) (a rectangle region) and definedas in (2)
On the other hand we consider eight directions of a pixeldeterminewhether there is foreground pixel in each direction
of the local region and count how many times it reachesforeground pixels as shown in Figure 3
120575119894 (119909 119910) = 1 foreground pixels in the 119894th direction0 otherwise
V (119909 119910) =1
8
8
sum
119894=1
120575119894 (119909 119910)
(19)
where 120575119894 means whether there is a foreground pixel in the 119894thdirection and if it exists its value is 1 otherwise 0 V(119909 119910) isthe ratio of foreground in eight directions After calculating119904(119909 119910) and V(119909 119910) by verifying whether or not the 119904(119909 119910)
and V(119909 119910) of a pixel is in the feasible range the pixel isdetermined by
where 120574V and 120574119904 are the threshold valuesSo we get the enhanced foreground contours image
(called enhanced foreground contour image 119868enfc) Howeversome moving object pixels are false detected as backgroundpixels such as holes in the image To get integral foregroundobject we use filling method to eliminate false detected pixelfor background pixel (Figure 4) For a pixel (119909 119910) whichis detected as background pixel in enhanced foregroundcontour image we first consider eight directions of a pixeldeterminewhether there is foreground pixel in each directionof the local region and count how many times it reachesforeground pixels as expressed in (19) We also consider theratio in a local region between numbers of foreground pixelsin foreground image and in contour image and the numbersof pixels in the same region
1199041015840(119909 119910) =
119873119904 (119909 119910)
119873119903 (119909 119910) (21)
where 1199041015840(119909 119910) is the ratio of pixels detected as foreground
in local region 119873119904(119909 119910) is the number of pixels which aredetected as foreground and 119873119903(119909 119910) is the sum of pixels inthe local region
6 Mathematical Problems in Engineering
(a) (b) (c)
(d) (e) (f)
Figure 4 Illustration of the shadow removal process using Campus sequence (a) Current frame (b) Normalized LIR of current frame (c)Foreground object detected using GMM of LIR (d) Result of erosion operation (contour image) to remove false detection shadow and noise(e) Enhanced contour image by contour enhancement method (f) Final result of filling holes
Shadow detection rate before fillingShadow discrimination rate before fillingCombined score before fillingShadow detection rate after fillingShadow discrimination rate after fillingCombined score after filling
100
90
80
70
60
50
40
30
20
Shad
ow d
etec
tion
perfo
rman
ce (
)
Hallway Campus Room Highway
Figure 5 Compared detection result of LIRM via GMM andpostprocess
After calculating V(119909 119910) and 1199041015840(119909 119910) we can get the
option that the pixel 119875(119909 119910) is moving object pixel or not by
119901 (119909 119910) = 255 V (119909 119910) lt 120574
1015840V 1199041015840(119909 119910) lt 120574
1015840119904
0 otherwise(22)
where 1205741015840V and 1205741015840119904 are the threshold values 119901(119909 119910) denotes the
final value of pixel (119909 119910)
4 Experiment Results
In this section we describe the test videos used in thisstudymeasure the performance of the proposedmethod andcompare the performance to the state-of-the-art methodsIn general state-of-the-art methods use different steps toremove shadow and reduce the influence of illuminationchanges To compare with other methods in different stepswe first compare the shadow removing results and then com-pare the results under various illumination change conditionsin different data sets
41 Experiment Conditions and Parameter Setting The size ofthe local rectangle region in (2) is important to the proposedmethod If the size is too large its inner region cannot meetthe assumptions (9) On the other hand if too small thechange of its value is also small In our experiment 119903 is setas 2
42 Experiment Results To test the effect of shadow remov-ing based on local intensity ratio and the effect of foregrounddetection we use four typical videos from the ATON videoset [4] as the test videos which are Campus Hallway High-way 1 and Intelligent room Intelligent room andHallway aretypical indoor environment In the videos people are walkingin scene and their shadows are mapped to the backgroundThe Campus is a video of campus parking lot and there areshadows of people and cars in the scenes
Mathematical Problems in Engineering 7
Origin
Realresult
Proposed
(a)
Origin
Realresult
Proposed
(b)
Origin
Realresult
Proposed
(c)
Origin
Realresult
Proposed
(d)
Figure 6 Detection results in different scenes (a) is Highway I sequence (b) is intelligent room sequence (c) is Campus sequence and (d)is Hallway sequence In the image of detection result the moving object pixels are white the background is black and the shadow is gray
To compare the performance of the proposed movingobjects detectionmethod with other shadow detectionmeth-ods we assume that if shadow is detected as backgroundit is correct detection otherwise it is false detection andif moving object is detected as foreground it is correct
detection otherwise it is false detection The proposedmethod is compared with five methods introduced by [2]which are chromaticity-based method physical methodgeometry-based method Small region (SR) texture-basedmethod and large region (LR) texture-based method To
Figure 7 The comparison of moving object detection results under the changing illumination condition (three scenes) From left to right(a) is video 1 (b) is video 2 and (c) is video 3 The 1st row is an input image the remaining nine rows are detection results of each methodthe 2nd row is a detection result of GMM the 3rd row is of adaptive GMM the 4th row is of LBP the 5th row is of Xu the 6th row is of Piletthe 7th row is of Choi the 8th row is of the proposed method without postprocessing and the 9th row is the final result of proposed method
estimate the effect of these methods shadow detection rate(120578) and shadow distinguishing rate (120585) are the discriminateparameters [4] and their definitions are as follows
120578 =119879119875119878
119879119875119878 + 119865119873119878
times 100 120585 =119879119875119865
119879119875119865 + 119865119873119865
times 100
(23)
where 119878 is shadow 119865 is foreground 119879119875119878 is the number ofthe pixels which are detected correctly 119865119873119878 is the number ofshadow pixels which are mistakenly detected as foreground119879119875119865 is the number of foreground pixels detected correctlyafter removing shadows 119865119873119865 is the number of foregroundpixels which are mistakenly detected as shadow The shadowdistinguishing rate is concerned with maintaining the pixels
which belong to the moving object as foreground In thispaper we use the average of the two rates as a singleperformance measure (avg)
Table 1 is the compared results among the method basedon LIRM and other methods in [2] Table 1 shows the averageshadow detection and discrimination rates on each testsequence From the compared results the method proposedin this paper is better at most time The main cause isthat the presented method directly detects the foregroundfrom video data but other methods extract the shadowfrom foreground by true background and foreground withshadow Our method use local intensity ratio to replace theactual pixel values complexity is lower Figure 6 is the resultof proposed method in moving object detection withoutshadow in different scenes
Mathematical Problems in Engineering 9
Table 1 Comparison of moving object detection with different methods
Figure 5 represents compared results of the LIRMmovingobject detection via GMM and postprocess From Figure 5the postprocess obviously improves the shadow discrimi-nation rate and slightly reduces the shadow detection rateHowever the average of the two rates is increased obviouslySo the postprocess enhances the moving object detectionresult
Finally we show that the proposedmethod rapidly adaptsto variations in environment To test moving object detectionunder illumination changes condition we compare the resultof other techniques which are GMM [5] adaptive GMM [24]that controls the learning rate of the GMM adaptively LBP[25] that detects moving objects using texture and edges Xursquosmethods [26] that use a color chromaticity Piletrsquos methods[27] that use illumination and spatial likelihood and Choirsquosmethods [28] that develop chromaticity differencemodel andbrightness model that estimates the intensity difference andintensity ratio of false foreground pixels For the quantitativecomparison we have made three test videos [28] Figure 7represents moving object detection results of each methodFrom Figure 7 we find the postprocessing of the proposedmethod can strengthen the moving objects and remove theshadow patches and the noise but it sometimes enhances theedge of part shadows incorrectly and makes moving objectsrsquoedge fuzzy In a word the proposed method adapts to thechanged background completely and detects moving objectpixels without shadow pixels successfully Although someother methods are robust to illumination their detectionmoving pixels results show that they cannot detect movingobject pixels meanwhile removing shadow
5 Conclusions
In this paper we propose a method to detect movingobject meanwhile removing cast shadow which is robust inchanging illumination condition This paper presents a localintensity ratio model according to illumination model andproves its illumination invariance Meanwhile if the noisein video obeys the Gaussian distribution the local intensityratio also obeys the Gaussian distribution In the process ofGaussian mixture model to obtain foreground this paperreplaces actual pixel value by local intensity ratio Finallypostprocess methods are used to get pure moving objectpixels without noise such as erosion method to removeshadow pitch and noise which is false detected as foregroundforeground contour enhanced method to strengthen moving
object contour and filling methods to deal with holes inforeground object Experiment results demonstrate that themethod presented by this paper can eliminate shadow onforeground effectively and is robust in changing illuminationconditionHowever in some scenes that foreground is similarto background or foreground is similar to shadow fore-groundmay be easily detected as background If illuminationchanges severely such as turn onoff light in indoor scenethe backgroundmay be easily detected as foreground and theperformance drops
Conflict of Interests
The authors declare that there is no conflict of interestsregarding the publication of this paper
Acknowledgments
Jinhai Xiang is supported by the Fundamental ResearchFunds for the Central Universities under Grant no2013QC024 Jun Xu is supported by the National NaturalScience Foundation of China under Grant no 11204099and self-determined research funds of CCNU from thecolleges basic research and operation of MOE Weiping Sunis supported by the National Natural Science Foundation ofChina under Grant no 61300140
References
[1] M Piccardi ldquoBackground subtraction techniques a reviewrdquo inProceedings of the IEEE International Conference on SystemsMan and Cybernetics (SMC rsquo04) pp 3099ndash3104 October 2004
[2] A Sanin C Sanderson and B C Lovell ldquoShadow detection asurvey and comparative evaluation of recent methodsrdquo PatternRecognition vol 45 no 4 pp 1684ndash1695 2012
[3] N Al-Najdawi H E Bez J Singhai and E A Edirisinghe ldquoAsurvey of cast shadow detection algorithmsrdquo Pattern Recogni-tion Letters vol 33 no 6 pp 752ndash764 2012
[4] A Prati I Mikic M M Trivedi and R Cucchiara ldquoDetectingmoving shadows algorithms and evaluationrdquo IEEE Transac-tions on Pattern Analysis and Machine Intelligence vol 25 no7 pp 918ndash923 2003
[5] Z Zivkovic and F van der Heijden ldquoEfficient adaptive densityestimation per image pixel for the task of background subtrac-tionrdquo Pattern Recognition Letters vol 27 no 7 pp 773ndash7802006
10 Mathematical Problems in Engineering
[6] A J Joshi and N P Papanikolopoulos ldquoLearning to detectmoving shadows in dynamic environmentsrdquo IEEE Transactionson Pattern Analysis and Machine Intelligence vol 30 no 11 pp2055ndash2063 2008
[7] C R Jung ldquoEfficient background subtraction and shadowremoval for monochromatic video sequencesrdquo IEEE Transac-tions on Multimedia vol 11 no 3 pp 571ndash577 2009
[8] M Shoaib R Dragon and J Ostermann ldquoShadow detection formoving humans using gradient-based background subtractionrdquoin Proceedings of the IEEE International Conference onAcousticsSpeech and Signal Processing (ICASSP rsquo09) pp 773ndash776 April2009
[9] R Cucchiara C Grana M Piccardi and A Prati ldquoDetectingmoving objects ghosts and shadows in video streamsrdquo IEEETransactions on Pattern Analysis and Machine Intelligence vol25 no 10 pp 1337ndash1342 2003
[10] E Salvador A Cavallaro and T Ebrahimi ldquoCast shadowsegmentation using invariant color featuresrdquo Computer Visionand Image Understanding vol 95 no 2 pp 238ndash259 2004
[11] D Grest J M Frahm and R Koch ldquoA color similarity measurefor robust shadow removal in real timerdquo in Proceedings of theVision Modeling and Visualization (VMV rsquo03) pp 253ndash260November 2003
[12] THorprasert DHarwood and L Davis ldquoA statistical approachfor real-time robust background subtraction and shadow detec-tionrdquo in Proceedings of the IEEE International Conference onComputer Vision (ICCV rsquo99) Frame Rate Workshop 1999
[13] S Nadimi and B Bhanu ldquoPhysical models for moving shadowand object detection in videordquo IEEE Transactions on PatternAnalysis and Machine Intelligence vol 26 no 8 pp 1079ndash10872004
[14] J B Huang and C S Chen ldquoMoving cast shadow detec-tion using Physics-based featuresrdquo in Proceedings of the IEEEComputer Society Conference on Computer Vision and PatternRecognition Workshops (CVPR rsquo09) pp 2310ndash2317 June 2009
[15] N Martel-Brisson and A Zaccarin ldquoKernel-based learning ofcast shadows fromaphysicalmodel of light sources and surfacesfor low-level segmentationrdquo in Proceedings of the 26th IEEEConference on Computer Vision and Pattern Recognition (CVPRrsquo08) pp 1ndash8 June 2008
[16] C C Chen and J K Aggarwal ldquoHuman shadow removal withunknown light sourcerdquo in Proceedings of the 20th InternationalConference on Pattern Recognition (ICPR rsquo10) pp 2407ndash2410August 2010
[17] J W Hsieh W F Hu C J Chang and Y S Chen ldquoShadowelimination for effective moving object detection by Gaussianshadow modelingrdquo Image and Vision Computing vol 21 no 6pp 505ndash516 2003
[18] H Nicolas and J M Pinel ldquoJoint moving cast shadows seg-mentation and light source detection in video sequencesrdquo SignalProcessing Image Communication vol 21 no 1 pp 22ndash43 2006
[19] A Leone andCDistante ldquoShadowdetection formoving objectsbased on texture analysisrdquoPattern Recognition vol 40 no 4 pp1222ndash1233 2007
[20] W Zhang X Z Fang X K Yang and Q M J Wu ldquoMovingcast shadows detection using ratio edgerdquo IEEE Transactions onMultimedia vol 9 no 6 pp 1202ndash1213 2007
[21] M Xiao C Z Han and L Zhang ldquoMoving shadow detectionand removal for traffic sequencesrdquo International Journal ofAutomation and Computing vol 4 no 1 pp 38ndash46 2007
[22] J Stauder R Mech and J Ostermann ldquoDetection of movingcast shadows for object segmentationrdquo IEEE Transactions onMultimedia vol 1 no 1 pp 65ndash76 1999
[23] M S Andersen T Jensen and C B Madsen ldquoEstimationof dynamic light changes in outdoor scenes without the useof calibration objectsrdquo in Proceedings of the 18th InternationalConference on Pattern Recognition (ICPR rsquo06) vol 4 pp 91ndash94August 2006
[24] D S Lee ldquoEffective Gaussian mixture learning for videobackground subtractionrdquo IEEETransactions on PatternAnalysisand Machine Intelligence vol 27 no 5 pp 827ndash832 2005
[25] M Heikkila and M Pietikainen ldquoA texture-based method formodeling the background and detecting moving objectsrdquo IEEETransactions on Pattern Analysis and Machine Intelligence vol28 no 4 pp 657ndash662 2006
[26] M Xu and T Ellis ldquoColour-invariant motion detection underfast illumination changesrdquo in Proceedings of the 2nd EuropeanWorkshop on Advanced Video-Based Surveillance Systems vol4 pp 101ndash112 September 2001
[27] J Pilet C Strecha and P Fua ldquoMaking background subtractionrobust to sudden illumination changesrdquo in Proceedings of the10th European Conference on Computer Vision (ECCV rsquo08) pp567ndash580 October 2008
[28] J Choi H J Chang Y J Yoo and J Y Choi ldquoRobust movingobject detection against fast illumination changerdquo ComputerVision and Image Understanding vol 116 no 2 pp 179ndash1932012
21 Local Intensity Ratio and Illumination Invariant Zhanget al [20] proved that the ratio edge is illumination invariantand use it to classify eachmoving pixel into foreground objector moving shadow Inspired by this work we define the localintensity ratio and analyze its illumination invariance
First the definition of the local intensity ratio (LIR) is
where 119901(119909 119910) is the intensity of pixel (119909 119910) |119860(119909 119910)| isthe number of pixels in a local region 119860(119909 119910) is the localregion of given pixel (119909 119910) (a rectangle region) and definedas follows
where 120588(119909 119910) is the reflectance of object surface in pixel(119909 119910) that is the reflection coefficient 119868(119909 119910) is the amountof light power per receiving object surface area in pixel (119909 119910)120579(119909 119910) is the sensor sensitivity of the camera The light inscenes can be divided into direct light of light source andscattered light of environment We assume that the lightsource is a distant light source and the light is parallel lightsuch as sunlight Light of environment is scattered light lightdirection is random and its intensity in scenes is assumed tobe constant If objects are occluded the scene can be dividedinto three cases illuminated area penumbra area and umbraarea [22 23] and the light intensity of the real target pixel(119909 119910) is
sdot cos120601 (119909 119910) penumbra area119871119886 umbra area
(4)
where 119871119886 is the intensity of ambient light 119871119901 is the intensityof light source 120572(119909 119910) is the transition inside the penumbrawhich depends on the light source and scene geometry and0 ⩽ 120572(119909 119910) ⩽ 1 120601(119909 119910) is the angle between light sourcedirection and surface normal
While using RGBmodel to express light intensity (4) canbe shown as follows
119868(119909 119910)119896=
119871119896119886 + 119871119896119901 sdot cos120601 (119909 119910) illuminated area
sdot cos120601 (119909 119910) penumbra area119871119896119886 umbra area
(5)
where 119896 is the number of color channels 119896 = 1 2 and 3 Forsimplification in this paper we consider only one light sourceand ignore the influence of different color channels
The analysis of the local intensity ratio under the threedifferent scenes is as shown in Figure 1
If all the local regions belong to one of the three areasshown in Figure 1 that is 119860(119909 119910) isin 1198601 or 119860(119909 119910) isin 1198602 or119860(119909 119910) isin 1198603 and11986011198602 and1198603 express the illuminate areapenumbra area and umbra area respectively we can obtainthe following results by formulas (4) (3) and (2) and assume|119860(119909 119910)| = 119899
Case 1 (119860(119909 119910) isin 1198601) Consider
If the pixels are in the same local region we assume that(1) pixels in the same local region belong to the same objectplane thus 120601(119909 119910) = 120601(119894 119895) (2) sensor sensitivity of camerain all local regions is the same 120579(119909 119910) = 120579(119894 119895) (3) intensityof penumbra is also the same 120572(119909 119910) = 120572(119894 119895)
If local region belongs to one of the three illuminationcases local intensity ratio is influenced only by reflectivityTherefore local intensity ratio is related to the reflectivity oftarget surface not to the illumination changes and types thatis illumination invariance
4 Mathematical Problems in Engineering
Sun
Object
Umbra
Penumbra
Sky
(a)
Illuminated(1)
Penumbra(2)
Umbra(3)
(b)
Figure 1 The illumination of the scene (three cases)
The local intensity ratio of pixels is constant underdifferent illumination conditions according to formula (12)As a result local intensity ratio model not only removes theinfluence of illumination but also eliminates shadow of theforeground targets
22 Distribution of Local Intensity Ratio Generally it isassumed that the images are corrupted by Gaussian whitenoise which can be expressed as
where 119875119903(119909 119910) represents actual pixel value 119875119887(119909 119910) denotesreal pixel value in scene 120576(119909 119910) represents noise The actuallocal intensity ratio is defined as
If pixel value in scene is constant 119875119887(119909 119910)
sum(119894119895)isin119860(119909119910) 119875119887(119894 119895) obeys Gaussian white noise distributionTherefore local intensity ratio and pixel value have the same
distribution On the other hand if sum(119894119895)isin119860(119909119910) 119875119887(119894 119895) issmall the white noise will be amplified
Theoretically the local intensity ratio ranges from 0 toinfinite In order to be consistent with the scope of a pixelvalue the normalized definition of the local intensity ratio isas follows
LR (119909 119910) = (2
1 + 119890minus119877(119909119910)minus 1) times 255 (17)
3 Moving Object Detection Based onLocal Intensity Ratio
In this section we present the detailed procedure of shadowremovingTheproposedmethod is amultistage approach andthe flow chart is shown in Figure 2
Generally the shadow detection methods utilize objectdetection or image segmentation algorithms to detect theforeground with shadow and then classify the foregroundas foreground or shadow According to the results in theprevious section the local intensity ratio obeys Gaussiandistribution This paper uses the normalized LIR to replacethe pixel to detect foreground and the process is shownin Figure 2 In this paper Gaussian mixture model [5] isused to acquire the foreground In mixture of Gaussians therecent history of each pixel is maintained using 119896 Gaussiandistributions and the value of 119896 typically chooses from 3 to5 Each distribution has its associated attributes like weight120596119896 mean 120583119896 and variance 120590119896 Each Gaussian updates itsparameters using LIR in every new frame
After detecting foreground the foreground target withlittle shadow can be obtained (called foreground image asfollows 119868fore) Meanwhile some parts of the foregroundobject may be darker than the background and easily befalse detected as background The false detected parts ofshadow are discrete the false detected parts of moving objectlike holes are surrounded by foreground pixels (such ascontour) The moving objects contour can be detected whilethe shadows contour can be removed at most time In orderto get enhanced moving objects without discrete shadowpatches erosion is used to erase the scatter and noises and toget moving objects contours image without shadow (calledcontour image as follows 119868cont) Then the contour image is
Mathematical Problems in Engineering 5
Calculate LIR Calculate LIRMoving objectsdetection via GMM
Enhanced movingobject contour
Background andshadow pixels Moving object pixels
Figure 2 Flow chart of moving object detection and shadow removing
Figure 3 Local region of the center pixel detection and its eightdirections
used to enhance moving objects by contour enhancementmethodThe enhancement procedure is as follows for a pixel119875(119909 119910) being not foreground pixel in contours image wecalculate the weighted foreground pixel ratio of a local regionin the contour image and foreground image as follows
where 119904(119909 119910) represents the rate of the enhanced movingobject contours in (119909 119910) 119873119888(119909 119910) is the number of fore-ground pixels in a neighbor region of pixel (119909 119910) in contourimage 119873119891(119909 119910) is the number of foreground pixels in aneighbor region of pixel (119909 119910) in foreground image and120596119888 and 120596119891 are the weight to contour image and foregroundimage We select 120596119888 = 3 120596119891 = 1 in our experiments |119860(119909 119910)|
is the number of pixels in a local region 119860(119909 119910) is the localregion of given pixel (119909 119910) (a rectangle region) and definedas in (2)
On the other hand we consider eight directions of a pixeldeterminewhether there is foreground pixel in each direction
of the local region and count how many times it reachesforeground pixels as shown in Figure 3
120575119894 (119909 119910) = 1 foreground pixels in the 119894th direction0 otherwise
V (119909 119910) =1
8
8
sum
119894=1
120575119894 (119909 119910)
(19)
where 120575119894 means whether there is a foreground pixel in the 119894thdirection and if it exists its value is 1 otherwise 0 V(119909 119910) isthe ratio of foreground in eight directions After calculating119904(119909 119910) and V(119909 119910) by verifying whether or not the 119904(119909 119910)
and V(119909 119910) of a pixel is in the feasible range the pixel isdetermined by
where 120574V and 120574119904 are the threshold valuesSo we get the enhanced foreground contours image
(called enhanced foreground contour image 119868enfc) Howeversome moving object pixels are false detected as backgroundpixels such as holes in the image To get integral foregroundobject we use filling method to eliminate false detected pixelfor background pixel (Figure 4) For a pixel (119909 119910) whichis detected as background pixel in enhanced foregroundcontour image we first consider eight directions of a pixeldeterminewhether there is foreground pixel in each directionof the local region and count how many times it reachesforeground pixels as expressed in (19) We also consider theratio in a local region between numbers of foreground pixelsin foreground image and in contour image and the numbersof pixels in the same region
1199041015840(119909 119910) =
119873119904 (119909 119910)
119873119903 (119909 119910) (21)
where 1199041015840(119909 119910) is the ratio of pixels detected as foreground
in local region 119873119904(119909 119910) is the number of pixels which aredetected as foreground and 119873119903(119909 119910) is the sum of pixels inthe local region
6 Mathematical Problems in Engineering
(a) (b) (c)
(d) (e) (f)
Figure 4 Illustration of the shadow removal process using Campus sequence (a) Current frame (b) Normalized LIR of current frame (c)Foreground object detected using GMM of LIR (d) Result of erosion operation (contour image) to remove false detection shadow and noise(e) Enhanced contour image by contour enhancement method (f) Final result of filling holes
Shadow detection rate before fillingShadow discrimination rate before fillingCombined score before fillingShadow detection rate after fillingShadow discrimination rate after fillingCombined score after filling
100
90
80
70
60
50
40
30
20
Shad
ow d
etec
tion
perfo
rman
ce (
)
Hallway Campus Room Highway
Figure 5 Compared detection result of LIRM via GMM andpostprocess
After calculating V(119909 119910) and 1199041015840(119909 119910) we can get the
option that the pixel 119875(119909 119910) is moving object pixel or not by
119901 (119909 119910) = 255 V (119909 119910) lt 120574
1015840V 1199041015840(119909 119910) lt 120574
1015840119904
0 otherwise(22)
where 1205741015840V and 1205741015840119904 are the threshold values 119901(119909 119910) denotes the
final value of pixel (119909 119910)
4 Experiment Results
In this section we describe the test videos used in thisstudymeasure the performance of the proposedmethod andcompare the performance to the state-of-the-art methodsIn general state-of-the-art methods use different steps toremove shadow and reduce the influence of illuminationchanges To compare with other methods in different stepswe first compare the shadow removing results and then com-pare the results under various illumination change conditionsin different data sets
41 Experiment Conditions and Parameter Setting The size ofthe local rectangle region in (2) is important to the proposedmethod If the size is too large its inner region cannot meetthe assumptions (9) On the other hand if too small thechange of its value is also small In our experiment 119903 is setas 2
42 Experiment Results To test the effect of shadow remov-ing based on local intensity ratio and the effect of foregrounddetection we use four typical videos from the ATON videoset [4] as the test videos which are Campus Hallway High-way 1 and Intelligent room Intelligent room andHallway aretypical indoor environment In the videos people are walkingin scene and their shadows are mapped to the backgroundThe Campus is a video of campus parking lot and there areshadows of people and cars in the scenes
Mathematical Problems in Engineering 7
Origin
Realresult
Proposed
(a)
Origin
Realresult
Proposed
(b)
Origin
Realresult
Proposed
(c)
Origin
Realresult
Proposed
(d)
Figure 6 Detection results in different scenes (a) is Highway I sequence (b) is intelligent room sequence (c) is Campus sequence and (d)is Hallway sequence In the image of detection result the moving object pixels are white the background is black and the shadow is gray
To compare the performance of the proposed movingobjects detectionmethod with other shadow detectionmeth-ods we assume that if shadow is detected as backgroundit is correct detection otherwise it is false detection andif moving object is detected as foreground it is correct
detection otherwise it is false detection The proposedmethod is compared with five methods introduced by [2]which are chromaticity-based method physical methodgeometry-based method Small region (SR) texture-basedmethod and large region (LR) texture-based method To
Figure 7 The comparison of moving object detection results under the changing illumination condition (three scenes) From left to right(a) is video 1 (b) is video 2 and (c) is video 3 The 1st row is an input image the remaining nine rows are detection results of each methodthe 2nd row is a detection result of GMM the 3rd row is of adaptive GMM the 4th row is of LBP the 5th row is of Xu the 6th row is of Piletthe 7th row is of Choi the 8th row is of the proposed method without postprocessing and the 9th row is the final result of proposed method
estimate the effect of these methods shadow detection rate(120578) and shadow distinguishing rate (120585) are the discriminateparameters [4] and their definitions are as follows
120578 =119879119875119878
119879119875119878 + 119865119873119878
times 100 120585 =119879119875119865
119879119875119865 + 119865119873119865
times 100
(23)
where 119878 is shadow 119865 is foreground 119879119875119878 is the number ofthe pixels which are detected correctly 119865119873119878 is the number ofshadow pixels which are mistakenly detected as foreground119879119875119865 is the number of foreground pixels detected correctlyafter removing shadows 119865119873119865 is the number of foregroundpixels which are mistakenly detected as shadow The shadowdistinguishing rate is concerned with maintaining the pixels
which belong to the moving object as foreground In thispaper we use the average of the two rates as a singleperformance measure (avg)
Table 1 is the compared results among the method basedon LIRM and other methods in [2] Table 1 shows the averageshadow detection and discrimination rates on each testsequence From the compared results the method proposedin this paper is better at most time The main cause isthat the presented method directly detects the foregroundfrom video data but other methods extract the shadowfrom foreground by true background and foreground withshadow Our method use local intensity ratio to replace theactual pixel values complexity is lower Figure 6 is the resultof proposed method in moving object detection withoutshadow in different scenes
Mathematical Problems in Engineering 9
Table 1 Comparison of moving object detection with different methods
Figure 5 represents compared results of the LIRMmovingobject detection via GMM and postprocess From Figure 5the postprocess obviously improves the shadow discrimi-nation rate and slightly reduces the shadow detection rateHowever the average of the two rates is increased obviouslySo the postprocess enhances the moving object detectionresult
Finally we show that the proposedmethod rapidly adaptsto variations in environment To test moving object detectionunder illumination changes condition we compare the resultof other techniques which are GMM [5] adaptive GMM [24]that controls the learning rate of the GMM adaptively LBP[25] that detects moving objects using texture and edges Xursquosmethods [26] that use a color chromaticity Piletrsquos methods[27] that use illumination and spatial likelihood and Choirsquosmethods [28] that develop chromaticity differencemodel andbrightness model that estimates the intensity difference andintensity ratio of false foreground pixels For the quantitativecomparison we have made three test videos [28] Figure 7represents moving object detection results of each methodFrom Figure 7 we find the postprocessing of the proposedmethod can strengthen the moving objects and remove theshadow patches and the noise but it sometimes enhances theedge of part shadows incorrectly and makes moving objectsrsquoedge fuzzy In a word the proposed method adapts to thechanged background completely and detects moving objectpixels without shadow pixels successfully Although someother methods are robust to illumination their detectionmoving pixels results show that they cannot detect movingobject pixels meanwhile removing shadow
5 Conclusions
In this paper we propose a method to detect movingobject meanwhile removing cast shadow which is robust inchanging illumination condition This paper presents a localintensity ratio model according to illumination model andproves its illumination invariance Meanwhile if the noisein video obeys the Gaussian distribution the local intensityratio also obeys the Gaussian distribution In the process ofGaussian mixture model to obtain foreground this paperreplaces actual pixel value by local intensity ratio Finallypostprocess methods are used to get pure moving objectpixels without noise such as erosion method to removeshadow pitch and noise which is false detected as foregroundforeground contour enhanced method to strengthen moving
object contour and filling methods to deal with holes inforeground object Experiment results demonstrate that themethod presented by this paper can eliminate shadow onforeground effectively and is robust in changing illuminationconditionHowever in some scenes that foreground is similarto background or foreground is similar to shadow fore-groundmay be easily detected as background If illuminationchanges severely such as turn onoff light in indoor scenethe backgroundmay be easily detected as foreground and theperformance drops
Conflict of Interests
The authors declare that there is no conflict of interestsregarding the publication of this paper
Acknowledgments
Jinhai Xiang is supported by the Fundamental ResearchFunds for the Central Universities under Grant no2013QC024 Jun Xu is supported by the National NaturalScience Foundation of China under Grant no 11204099and self-determined research funds of CCNU from thecolleges basic research and operation of MOE Weiping Sunis supported by the National Natural Science Foundation ofChina under Grant no 61300140
References
[1] M Piccardi ldquoBackground subtraction techniques a reviewrdquo inProceedings of the IEEE International Conference on SystemsMan and Cybernetics (SMC rsquo04) pp 3099ndash3104 October 2004
[2] A Sanin C Sanderson and B C Lovell ldquoShadow detection asurvey and comparative evaluation of recent methodsrdquo PatternRecognition vol 45 no 4 pp 1684ndash1695 2012
[3] N Al-Najdawi H E Bez J Singhai and E A Edirisinghe ldquoAsurvey of cast shadow detection algorithmsrdquo Pattern Recogni-tion Letters vol 33 no 6 pp 752ndash764 2012
[4] A Prati I Mikic M M Trivedi and R Cucchiara ldquoDetectingmoving shadows algorithms and evaluationrdquo IEEE Transac-tions on Pattern Analysis and Machine Intelligence vol 25 no7 pp 918ndash923 2003
[5] Z Zivkovic and F van der Heijden ldquoEfficient adaptive densityestimation per image pixel for the task of background subtrac-tionrdquo Pattern Recognition Letters vol 27 no 7 pp 773ndash7802006
10 Mathematical Problems in Engineering
[6] A J Joshi and N P Papanikolopoulos ldquoLearning to detectmoving shadows in dynamic environmentsrdquo IEEE Transactionson Pattern Analysis and Machine Intelligence vol 30 no 11 pp2055ndash2063 2008
[7] C R Jung ldquoEfficient background subtraction and shadowremoval for monochromatic video sequencesrdquo IEEE Transac-tions on Multimedia vol 11 no 3 pp 571ndash577 2009
[8] M Shoaib R Dragon and J Ostermann ldquoShadow detection formoving humans using gradient-based background subtractionrdquoin Proceedings of the IEEE International Conference onAcousticsSpeech and Signal Processing (ICASSP rsquo09) pp 773ndash776 April2009
[9] R Cucchiara C Grana M Piccardi and A Prati ldquoDetectingmoving objects ghosts and shadows in video streamsrdquo IEEETransactions on Pattern Analysis and Machine Intelligence vol25 no 10 pp 1337ndash1342 2003
[10] E Salvador A Cavallaro and T Ebrahimi ldquoCast shadowsegmentation using invariant color featuresrdquo Computer Visionand Image Understanding vol 95 no 2 pp 238ndash259 2004
[11] D Grest J M Frahm and R Koch ldquoA color similarity measurefor robust shadow removal in real timerdquo in Proceedings of theVision Modeling and Visualization (VMV rsquo03) pp 253ndash260November 2003
[12] THorprasert DHarwood and L Davis ldquoA statistical approachfor real-time robust background subtraction and shadow detec-tionrdquo in Proceedings of the IEEE International Conference onComputer Vision (ICCV rsquo99) Frame Rate Workshop 1999
[13] S Nadimi and B Bhanu ldquoPhysical models for moving shadowand object detection in videordquo IEEE Transactions on PatternAnalysis and Machine Intelligence vol 26 no 8 pp 1079ndash10872004
[14] J B Huang and C S Chen ldquoMoving cast shadow detec-tion using Physics-based featuresrdquo in Proceedings of the IEEEComputer Society Conference on Computer Vision and PatternRecognition Workshops (CVPR rsquo09) pp 2310ndash2317 June 2009
[15] N Martel-Brisson and A Zaccarin ldquoKernel-based learning ofcast shadows fromaphysicalmodel of light sources and surfacesfor low-level segmentationrdquo in Proceedings of the 26th IEEEConference on Computer Vision and Pattern Recognition (CVPRrsquo08) pp 1ndash8 June 2008
[16] C C Chen and J K Aggarwal ldquoHuman shadow removal withunknown light sourcerdquo in Proceedings of the 20th InternationalConference on Pattern Recognition (ICPR rsquo10) pp 2407ndash2410August 2010
[17] J W Hsieh W F Hu C J Chang and Y S Chen ldquoShadowelimination for effective moving object detection by Gaussianshadow modelingrdquo Image and Vision Computing vol 21 no 6pp 505ndash516 2003
[18] H Nicolas and J M Pinel ldquoJoint moving cast shadows seg-mentation and light source detection in video sequencesrdquo SignalProcessing Image Communication vol 21 no 1 pp 22ndash43 2006
[19] A Leone andCDistante ldquoShadowdetection formoving objectsbased on texture analysisrdquoPattern Recognition vol 40 no 4 pp1222ndash1233 2007
[20] W Zhang X Z Fang X K Yang and Q M J Wu ldquoMovingcast shadows detection using ratio edgerdquo IEEE Transactions onMultimedia vol 9 no 6 pp 1202ndash1213 2007
[21] M Xiao C Z Han and L Zhang ldquoMoving shadow detectionand removal for traffic sequencesrdquo International Journal ofAutomation and Computing vol 4 no 1 pp 38ndash46 2007
[22] J Stauder R Mech and J Ostermann ldquoDetection of movingcast shadows for object segmentationrdquo IEEE Transactions onMultimedia vol 1 no 1 pp 65ndash76 1999
[23] M S Andersen T Jensen and C B Madsen ldquoEstimationof dynamic light changes in outdoor scenes without the useof calibration objectsrdquo in Proceedings of the 18th InternationalConference on Pattern Recognition (ICPR rsquo06) vol 4 pp 91ndash94August 2006
[24] D S Lee ldquoEffective Gaussian mixture learning for videobackground subtractionrdquo IEEETransactions on PatternAnalysisand Machine Intelligence vol 27 no 5 pp 827ndash832 2005
[25] M Heikkila and M Pietikainen ldquoA texture-based method formodeling the background and detecting moving objectsrdquo IEEETransactions on Pattern Analysis and Machine Intelligence vol28 no 4 pp 657ndash662 2006
[26] M Xu and T Ellis ldquoColour-invariant motion detection underfast illumination changesrdquo in Proceedings of the 2nd EuropeanWorkshop on Advanced Video-Based Surveillance Systems vol4 pp 101ndash112 September 2001
[27] J Pilet C Strecha and P Fua ldquoMaking background subtractionrobust to sudden illumination changesrdquo in Proceedings of the10th European Conference on Computer Vision (ECCV rsquo08) pp567ndash580 October 2008
[28] J Choi H J Chang Y J Yoo and J Y Choi ldquoRobust movingobject detection against fast illumination changerdquo ComputerVision and Image Understanding vol 116 no 2 pp 179ndash1932012
Figure 1 The illumination of the scene (three cases)
The local intensity ratio of pixels is constant underdifferent illumination conditions according to formula (12)As a result local intensity ratio model not only removes theinfluence of illumination but also eliminates shadow of theforeground targets
22 Distribution of Local Intensity Ratio Generally it isassumed that the images are corrupted by Gaussian whitenoise which can be expressed as
where 119875119903(119909 119910) represents actual pixel value 119875119887(119909 119910) denotesreal pixel value in scene 120576(119909 119910) represents noise The actuallocal intensity ratio is defined as
If pixel value in scene is constant 119875119887(119909 119910)
sum(119894119895)isin119860(119909119910) 119875119887(119894 119895) obeys Gaussian white noise distributionTherefore local intensity ratio and pixel value have the same
distribution On the other hand if sum(119894119895)isin119860(119909119910) 119875119887(119894 119895) issmall the white noise will be amplified
Theoretically the local intensity ratio ranges from 0 toinfinite In order to be consistent with the scope of a pixelvalue the normalized definition of the local intensity ratio isas follows
LR (119909 119910) = (2
1 + 119890minus119877(119909119910)minus 1) times 255 (17)
3 Moving Object Detection Based onLocal Intensity Ratio
In this section we present the detailed procedure of shadowremovingTheproposedmethod is amultistage approach andthe flow chart is shown in Figure 2
Generally the shadow detection methods utilize objectdetection or image segmentation algorithms to detect theforeground with shadow and then classify the foregroundas foreground or shadow According to the results in theprevious section the local intensity ratio obeys Gaussiandistribution This paper uses the normalized LIR to replacethe pixel to detect foreground and the process is shownin Figure 2 In this paper Gaussian mixture model [5] isused to acquire the foreground In mixture of Gaussians therecent history of each pixel is maintained using 119896 Gaussiandistributions and the value of 119896 typically chooses from 3 to5 Each distribution has its associated attributes like weight120596119896 mean 120583119896 and variance 120590119896 Each Gaussian updates itsparameters using LIR in every new frame
After detecting foreground the foreground target withlittle shadow can be obtained (called foreground image asfollows 119868fore) Meanwhile some parts of the foregroundobject may be darker than the background and easily befalse detected as background The false detected parts ofshadow are discrete the false detected parts of moving objectlike holes are surrounded by foreground pixels (such ascontour) The moving objects contour can be detected whilethe shadows contour can be removed at most time In orderto get enhanced moving objects without discrete shadowpatches erosion is used to erase the scatter and noises and toget moving objects contours image without shadow (calledcontour image as follows 119868cont) Then the contour image is
Mathematical Problems in Engineering 5
Calculate LIR Calculate LIRMoving objectsdetection via GMM
Enhanced movingobject contour
Background andshadow pixels Moving object pixels
Figure 2 Flow chart of moving object detection and shadow removing
Figure 3 Local region of the center pixel detection and its eightdirections
used to enhance moving objects by contour enhancementmethodThe enhancement procedure is as follows for a pixel119875(119909 119910) being not foreground pixel in contours image wecalculate the weighted foreground pixel ratio of a local regionin the contour image and foreground image as follows
where 119904(119909 119910) represents the rate of the enhanced movingobject contours in (119909 119910) 119873119888(119909 119910) is the number of fore-ground pixels in a neighbor region of pixel (119909 119910) in contourimage 119873119891(119909 119910) is the number of foreground pixels in aneighbor region of pixel (119909 119910) in foreground image and120596119888 and 120596119891 are the weight to contour image and foregroundimage We select 120596119888 = 3 120596119891 = 1 in our experiments |119860(119909 119910)|
is the number of pixels in a local region 119860(119909 119910) is the localregion of given pixel (119909 119910) (a rectangle region) and definedas in (2)
On the other hand we consider eight directions of a pixeldeterminewhether there is foreground pixel in each direction
of the local region and count how many times it reachesforeground pixels as shown in Figure 3
120575119894 (119909 119910) = 1 foreground pixels in the 119894th direction0 otherwise
V (119909 119910) =1
8
8
sum
119894=1
120575119894 (119909 119910)
(19)
where 120575119894 means whether there is a foreground pixel in the 119894thdirection and if it exists its value is 1 otherwise 0 V(119909 119910) isthe ratio of foreground in eight directions After calculating119904(119909 119910) and V(119909 119910) by verifying whether or not the 119904(119909 119910)
and V(119909 119910) of a pixel is in the feasible range the pixel isdetermined by
where 120574V and 120574119904 are the threshold valuesSo we get the enhanced foreground contours image
(called enhanced foreground contour image 119868enfc) Howeversome moving object pixels are false detected as backgroundpixels such as holes in the image To get integral foregroundobject we use filling method to eliminate false detected pixelfor background pixel (Figure 4) For a pixel (119909 119910) whichis detected as background pixel in enhanced foregroundcontour image we first consider eight directions of a pixeldeterminewhether there is foreground pixel in each directionof the local region and count how many times it reachesforeground pixels as expressed in (19) We also consider theratio in a local region between numbers of foreground pixelsin foreground image and in contour image and the numbersof pixels in the same region
1199041015840(119909 119910) =
119873119904 (119909 119910)
119873119903 (119909 119910) (21)
where 1199041015840(119909 119910) is the ratio of pixels detected as foreground
in local region 119873119904(119909 119910) is the number of pixels which aredetected as foreground and 119873119903(119909 119910) is the sum of pixels inthe local region
6 Mathematical Problems in Engineering
(a) (b) (c)
(d) (e) (f)
Figure 4 Illustration of the shadow removal process using Campus sequence (a) Current frame (b) Normalized LIR of current frame (c)Foreground object detected using GMM of LIR (d) Result of erosion operation (contour image) to remove false detection shadow and noise(e) Enhanced contour image by contour enhancement method (f) Final result of filling holes
Shadow detection rate before fillingShadow discrimination rate before fillingCombined score before fillingShadow detection rate after fillingShadow discrimination rate after fillingCombined score after filling
100
90
80
70
60
50
40
30
20
Shad
ow d
etec
tion
perfo
rman
ce (
)
Hallway Campus Room Highway
Figure 5 Compared detection result of LIRM via GMM andpostprocess
After calculating V(119909 119910) and 1199041015840(119909 119910) we can get the
option that the pixel 119875(119909 119910) is moving object pixel or not by
119901 (119909 119910) = 255 V (119909 119910) lt 120574
1015840V 1199041015840(119909 119910) lt 120574
1015840119904
0 otherwise(22)
where 1205741015840V and 1205741015840119904 are the threshold values 119901(119909 119910) denotes the
final value of pixel (119909 119910)
4 Experiment Results
In this section we describe the test videos used in thisstudymeasure the performance of the proposedmethod andcompare the performance to the state-of-the-art methodsIn general state-of-the-art methods use different steps toremove shadow and reduce the influence of illuminationchanges To compare with other methods in different stepswe first compare the shadow removing results and then com-pare the results under various illumination change conditionsin different data sets
41 Experiment Conditions and Parameter Setting The size ofthe local rectangle region in (2) is important to the proposedmethod If the size is too large its inner region cannot meetthe assumptions (9) On the other hand if too small thechange of its value is also small In our experiment 119903 is setas 2
42 Experiment Results To test the effect of shadow remov-ing based on local intensity ratio and the effect of foregrounddetection we use four typical videos from the ATON videoset [4] as the test videos which are Campus Hallway High-way 1 and Intelligent room Intelligent room andHallway aretypical indoor environment In the videos people are walkingin scene and their shadows are mapped to the backgroundThe Campus is a video of campus parking lot and there areshadows of people and cars in the scenes
Mathematical Problems in Engineering 7
Origin
Realresult
Proposed
(a)
Origin
Realresult
Proposed
(b)
Origin
Realresult
Proposed
(c)
Origin
Realresult
Proposed
(d)
Figure 6 Detection results in different scenes (a) is Highway I sequence (b) is intelligent room sequence (c) is Campus sequence and (d)is Hallway sequence In the image of detection result the moving object pixels are white the background is black and the shadow is gray
To compare the performance of the proposed movingobjects detectionmethod with other shadow detectionmeth-ods we assume that if shadow is detected as backgroundit is correct detection otherwise it is false detection andif moving object is detected as foreground it is correct
detection otherwise it is false detection The proposedmethod is compared with five methods introduced by [2]which are chromaticity-based method physical methodgeometry-based method Small region (SR) texture-basedmethod and large region (LR) texture-based method To
Figure 7 The comparison of moving object detection results under the changing illumination condition (three scenes) From left to right(a) is video 1 (b) is video 2 and (c) is video 3 The 1st row is an input image the remaining nine rows are detection results of each methodthe 2nd row is a detection result of GMM the 3rd row is of adaptive GMM the 4th row is of LBP the 5th row is of Xu the 6th row is of Piletthe 7th row is of Choi the 8th row is of the proposed method without postprocessing and the 9th row is the final result of proposed method
estimate the effect of these methods shadow detection rate(120578) and shadow distinguishing rate (120585) are the discriminateparameters [4] and their definitions are as follows
120578 =119879119875119878
119879119875119878 + 119865119873119878
times 100 120585 =119879119875119865
119879119875119865 + 119865119873119865
times 100
(23)
where 119878 is shadow 119865 is foreground 119879119875119878 is the number ofthe pixels which are detected correctly 119865119873119878 is the number ofshadow pixels which are mistakenly detected as foreground119879119875119865 is the number of foreground pixels detected correctlyafter removing shadows 119865119873119865 is the number of foregroundpixels which are mistakenly detected as shadow The shadowdistinguishing rate is concerned with maintaining the pixels
which belong to the moving object as foreground In thispaper we use the average of the two rates as a singleperformance measure (avg)
Table 1 is the compared results among the method basedon LIRM and other methods in [2] Table 1 shows the averageshadow detection and discrimination rates on each testsequence From the compared results the method proposedin this paper is better at most time The main cause isthat the presented method directly detects the foregroundfrom video data but other methods extract the shadowfrom foreground by true background and foreground withshadow Our method use local intensity ratio to replace theactual pixel values complexity is lower Figure 6 is the resultof proposed method in moving object detection withoutshadow in different scenes
Mathematical Problems in Engineering 9
Table 1 Comparison of moving object detection with different methods
Figure 5 represents compared results of the LIRMmovingobject detection via GMM and postprocess From Figure 5the postprocess obviously improves the shadow discrimi-nation rate and slightly reduces the shadow detection rateHowever the average of the two rates is increased obviouslySo the postprocess enhances the moving object detectionresult
Finally we show that the proposedmethod rapidly adaptsto variations in environment To test moving object detectionunder illumination changes condition we compare the resultof other techniques which are GMM [5] adaptive GMM [24]that controls the learning rate of the GMM adaptively LBP[25] that detects moving objects using texture and edges Xursquosmethods [26] that use a color chromaticity Piletrsquos methods[27] that use illumination and spatial likelihood and Choirsquosmethods [28] that develop chromaticity differencemodel andbrightness model that estimates the intensity difference andintensity ratio of false foreground pixels For the quantitativecomparison we have made three test videos [28] Figure 7represents moving object detection results of each methodFrom Figure 7 we find the postprocessing of the proposedmethod can strengthen the moving objects and remove theshadow patches and the noise but it sometimes enhances theedge of part shadows incorrectly and makes moving objectsrsquoedge fuzzy In a word the proposed method adapts to thechanged background completely and detects moving objectpixels without shadow pixels successfully Although someother methods are robust to illumination their detectionmoving pixels results show that they cannot detect movingobject pixels meanwhile removing shadow
5 Conclusions
In this paper we propose a method to detect movingobject meanwhile removing cast shadow which is robust inchanging illumination condition This paper presents a localintensity ratio model according to illumination model andproves its illumination invariance Meanwhile if the noisein video obeys the Gaussian distribution the local intensityratio also obeys the Gaussian distribution In the process ofGaussian mixture model to obtain foreground this paperreplaces actual pixel value by local intensity ratio Finallypostprocess methods are used to get pure moving objectpixels without noise such as erosion method to removeshadow pitch and noise which is false detected as foregroundforeground contour enhanced method to strengthen moving
object contour and filling methods to deal with holes inforeground object Experiment results demonstrate that themethod presented by this paper can eliminate shadow onforeground effectively and is robust in changing illuminationconditionHowever in some scenes that foreground is similarto background or foreground is similar to shadow fore-groundmay be easily detected as background If illuminationchanges severely such as turn onoff light in indoor scenethe backgroundmay be easily detected as foreground and theperformance drops
Conflict of Interests
The authors declare that there is no conflict of interestsregarding the publication of this paper
Acknowledgments
Jinhai Xiang is supported by the Fundamental ResearchFunds for the Central Universities under Grant no2013QC024 Jun Xu is supported by the National NaturalScience Foundation of China under Grant no 11204099and self-determined research funds of CCNU from thecolleges basic research and operation of MOE Weiping Sunis supported by the National Natural Science Foundation ofChina under Grant no 61300140
References
[1] M Piccardi ldquoBackground subtraction techniques a reviewrdquo inProceedings of the IEEE International Conference on SystemsMan and Cybernetics (SMC rsquo04) pp 3099ndash3104 October 2004
[2] A Sanin C Sanderson and B C Lovell ldquoShadow detection asurvey and comparative evaluation of recent methodsrdquo PatternRecognition vol 45 no 4 pp 1684ndash1695 2012
[3] N Al-Najdawi H E Bez J Singhai and E A Edirisinghe ldquoAsurvey of cast shadow detection algorithmsrdquo Pattern Recogni-tion Letters vol 33 no 6 pp 752ndash764 2012
[4] A Prati I Mikic M M Trivedi and R Cucchiara ldquoDetectingmoving shadows algorithms and evaluationrdquo IEEE Transac-tions on Pattern Analysis and Machine Intelligence vol 25 no7 pp 918ndash923 2003
[5] Z Zivkovic and F van der Heijden ldquoEfficient adaptive densityestimation per image pixel for the task of background subtrac-tionrdquo Pattern Recognition Letters vol 27 no 7 pp 773ndash7802006
10 Mathematical Problems in Engineering
[6] A J Joshi and N P Papanikolopoulos ldquoLearning to detectmoving shadows in dynamic environmentsrdquo IEEE Transactionson Pattern Analysis and Machine Intelligence vol 30 no 11 pp2055ndash2063 2008
[7] C R Jung ldquoEfficient background subtraction and shadowremoval for monochromatic video sequencesrdquo IEEE Transac-tions on Multimedia vol 11 no 3 pp 571ndash577 2009
[8] M Shoaib R Dragon and J Ostermann ldquoShadow detection formoving humans using gradient-based background subtractionrdquoin Proceedings of the IEEE International Conference onAcousticsSpeech and Signal Processing (ICASSP rsquo09) pp 773ndash776 April2009
[9] R Cucchiara C Grana M Piccardi and A Prati ldquoDetectingmoving objects ghosts and shadows in video streamsrdquo IEEETransactions on Pattern Analysis and Machine Intelligence vol25 no 10 pp 1337ndash1342 2003
[10] E Salvador A Cavallaro and T Ebrahimi ldquoCast shadowsegmentation using invariant color featuresrdquo Computer Visionand Image Understanding vol 95 no 2 pp 238ndash259 2004
[11] D Grest J M Frahm and R Koch ldquoA color similarity measurefor robust shadow removal in real timerdquo in Proceedings of theVision Modeling and Visualization (VMV rsquo03) pp 253ndash260November 2003
[12] THorprasert DHarwood and L Davis ldquoA statistical approachfor real-time robust background subtraction and shadow detec-tionrdquo in Proceedings of the IEEE International Conference onComputer Vision (ICCV rsquo99) Frame Rate Workshop 1999
[13] S Nadimi and B Bhanu ldquoPhysical models for moving shadowand object detection in videordquo IEEE Transactions on PatternAnalysis and Machine Intelligence vol 26 no 8 pp 1079ndash10872004
[14] J B Huang and C S Chen ldquoMoving cast shadow detec-tion using Physics-based featuresrdquo in Proceedings of the IEEEComputer Society Conference on Computer Vision and PatternRecognition Workshops (CVPR rsquo09) pp 2310ndash2317 June 2009
[15] N Martel-Brisson and A Zaccarin ldquoKernel-based learning ofcast shadows fromaphysicalmodel of light sources and surfacesfor low-level segmentationrdquo in Proceedings of the 26th IEEEConference on Computer Vision and Pattern Recognition (CVPRrsquo08) pp 1ndash8 June 2008
[16] C C Chen and J K Aggarwal ldquoHuman shadow removal withunknown light sourcerdquo in Proceedings of the 20th InternationalConference on Pattern Recognition (ICPR rsquo10) pp 2407ndash2410August 2010
[17] J W Hsieh W F Hu C J Chang and Y S Chen ldquoShadowelimination for effective moving object detection by Gaussianshadow modelingrdquo Image and Vision Computing vol 21 no 6pp 505ndash516 2003
[18] H Nicolas and J M Pinel ldquoJoint moving cast shadows seg-mentation and light source detection in video sequencesrdquo SignalProcessing Image Communication vol 21 no 1 pp 22ndash43 2006
[19] A Leone andCDistante ldquoShadowdetection formoving objectsbased on texture analysisrdquoPattern Recognition vol 40 no 4 pp1222ndash1233 2007
[20] W Zhang X Z Fang X K Yang and Q M J Wu ldquoMovingcast shadows detection using ratio edgerdquo IEEE Transactions onMultimedia vol 9 no 6 pp 1202ndash1213 2007
[21] M Xiao C Z Han and L Zhang ldquoMoving shadow detectionand removal for traffic sequencesrdquo International Journal ofAutomation and Computing vol 4 no 1 pp 38ndash46 2007
[22] J Stauder R Mech and J Ostermann ldquoDetection of movingcast shadows for object segmentationrdquo IEEE Transactions onMultimedia vol 1 no 1 pp 65ndash76 1999
[23] M S Andersen T Jensen and C B Madsen ldquoEstimationof dynamic light changes in outdoor scenes without the useof calibration objectsrdquo in Proceedings of the 18th InternationalConference on Pattern Recognition (ICPR rsquo06) vol 4 pp 91ndash94August 2006
[24] D S Lee ldquoEffective Gaussian mixture learning for videobackground subtractionrdquo IEEETransactions on PatternAnalysisand Machine Intelligence vol 27 no 5 pp 827ndash832 2005
[25] M Heikkila and M Pietikainen ldquoA texture-based method formodeling the background and detecting moving objectsrdquo IEEETransactions on Pattern Analysis and Machine Intelligence vol28 no 4 pp 657ndash662 2006
[26] M Xu and T Ellis ldquoColour-invariant motion detection underfast illumination changesrdquo in Proceedings of the 2nd EuropeanWorkshop on Advanced Video-Based Surveillance Systems vol4 pp 101ndash112 September 2001
[27] J Pilet C Strecha and P Fua ldquoMaking background subtractionrobust to sudden illumination changesrdquo in Proceedings of the10th European Conference on Computer Vision (ECCV rsquo08) pp567ndash580 October 2008
[28] J Choi H J Chang Y J Yoo and J Y Choi ldquoRobust movingobject detection against fast illumination changerdquo ComputerVision and Image Understanding vol 116 no 2 pp 179ndash1932012
Calculate LIR Calculate LIRMoving objectsdetection via GMM
Enhanced movingobject contour
Background andshadow pixels Moving object pixels
Figure 2 Flow chart of moving object detection and shadow removing
Figure 3 Local region of the center pixel detection and its eightdirections
used to enhance moving objects by contour enhancementmethodThe enhancement procedure is as follows for a pixel119875(119909 119910) being not foreground pixel in contours image wecalculate the weighted foreground pixel ratio of a local regionin the contour image and foreground image as follows
where 119904(119909 119910) represents the rate of the enhanced movingobject contours in (119909 119910) 119873119888(119909 119910) is the number of fore-ground pixels in a neighbor region of pixel (119909 119910) in contourimage 119873119891(119909 119910) is the number of foreground pixels in aneighbor region of pixel (119909 119910) in foreground image and120596119888 and 120596119891 are the weight to contour image and foregroundimage We select 120596119888 = 3 120596119891 = 1 in our experiments |119860(119909 119910)|
is the number of pixels in a local region 119860(119909 119910) is the localregion of given pixel (119909 119910) (a rectangle region) and definedas in (2)
On the other hand we consider eight directions of a pixeldeterminewhether there is foreground pixel in each direction
of the local region and count how many times it reachesforeground pixels as shown in Figure 3
120575119894 (119909 119910) = 1 foreground pixels in the 119894th direction0 otherwise
V (119909 119910) =1
8
8
sum
119894=1
120575119894 (119909 119910)
(19)
where 120575119894 means whether there is a foreground pixel in the 119894thdirection and if it exists its value is 1 otherwise 0 V(119909 119910) isthe ratio of foreground in eight directions After calculating119904(119909 119910) and V(119909 119910) by verifying whether or not the 119904(119909 119910)
and V(119909 119910) of a pixel is in the feasible range the pixel isdetermined by
where 120574V and 120574119904 are the threshold valuesSo we get the enhanced foreground contours image
(called enhanced foreground contour image 119868enfc) Howeversome moving object pixels are false detected as backgroundpixels such as holes in the image To get integral foregroundobject we use filling method to eliminate false detected pixelfor background pixel (Figure 4) For a pixel (119909 119910) whichis detected as background pixel in enhanced foregroundcontour image we first consider eight directions of a pixeldeterminewhether there is foreground pixel in each directionof the local region and count how many times it reachesforeground pixels as expressed in (19) We also consider theratio in a local region between numbers of foreground pixelsin foreground image and in contour image and the numbersof pixels in the same region
1199041015840(119909 119910) =
119873119904 (119909 119910)
119873119903 (119909 119910) (21)
where 1199041015840(119909 119910) is the ratio of pixels detected as foreground
in local region 119873119904(119909 119910) is the number of pixels which aredetected as foreground and 119873119903(119909 119910) is the sum of pixels inthe local region
6 Mathematical Problems in Engineering
(a) (b) (c)
(d) (e) (f)
Figure 4 Illustration of the shadow removal process using Campus sequence (a) Current frame (b) Normalized LIR of current frame (c)Foreground object detected using GMM of LIR (d) Result of erosion operation (contour image) to remove false detection shadow and noise(e) Enhanced contour image by contour enhancement method (f) Final result of filling holes
Shadow detection rate before fillingShadow discrimination rate before fillingCombined score before fillingShadow detection rate after fillingShadow discrimination rate after fillingCombined score after filling
100
90
80
70
60
50
40
30
20
Shad
ow d
etec
tion
perfo
rman
ce (
)
Hallway Campus Room Highway
Figure 5 Compared detection result of LIRM via GMM andpostprocess
After calculating V(119909 119910) and 1199041015840(119909 119910) we can get the
option that the pixel 119875(119909 119910) is moving object pixel or not by
119901 (119909 119910) = 255 V (119909 119910) lt 120574
1015840V 1199041015840(119909 119910) lt 120574
1015840119904
0 otherwise(22)
where 1205741015840V and 1205741015840119904 are the threshold values 119901(119909 119910) denotes the
final value of pixel (119909 119910)
4 Experiment Results
In this section we describe the test videos used in thisstudymeasure the performance of the proposedmethod andcompare the performance to the state-of-the-art methodsIn general state-of-the-art methods use different steps toremove shadow and reduce the influence of illuminationchanges To compare with other methods in different stepswe first compare the shadow removing results and then com-pare the results under various illumination change conditionsin different data sets
41 Experiment Conditions and Parameter Setting The size ofthe local rectangle region in (2) is important to the proposedmethod If the size is too large its inner region cannot meetthe assumptions (9) On the other hand if too small thechange of its value is also small In our experiment 119903 is setas 2
42 Experiment Results To test the effect of shadow remov-ing based on local intensity ratio and the effect of foregrounddetection we use four typical videos from the ATON videoset [4] as the test videos which are Campus Hallway High-way 1 and Intelligent room Intelligent room andHallway aretypical indoor environment In the videos people are walkingin scene and their shadows are mapped to the backgroundThe Campus is a video of campus parking lot and there areshadows of people and cars in the scenes
Mathematical Problems in Engineering 7
Origin
Realresult
Proposed
(a)
Origin
Realresult
Proposed
(b)
Origin
Realresult
Proposed
(c)
Origin
Realresult
Proposed
(d)
Figure 6 Detection results in different scenes (a) is Highway I sequence (b) is intelligent room sequence (c) is Campus sequence and (d)is Hallway sequence In the image of detection result the moving object pixels are white the background is black and the shadow is gray
To compare the performance of the proposed movingobjects detectionmethod with other shadow detectionmeth-ods we assume that if shadow is detected as backgroundit is correct detection otherwise it is false detection andif moving object is detected as foreground it is correct
detection otherwise it is false detection The proposedmethod is compared with five methods introduced by [2]which are chromaticity-based method physical methodgeometry-based method Small region (SR) texture-basedmethod and large region (LR) texture-based method To
Figure 7 The comparison of moving object detection results under the changing illumination condition (three scenes) From left to right(a) is video 1 (b) is video 2 and (c) is video 3 The 1st row is an input image the remaining nine rows are detection results of each methodthe 2nd row is a detection result of GMM the 3rd row is of adaptive GMM the 4th row is of LBP the 5th row is of Xu the 6th row is of Piletthe 7th row is of Choi the 8th row is of the proposed method without postprocessing and the 9th row is the final result of proposed method
estimate the effect of these methods shadow detection rate(120578) and shadow distinguishing rate (120585) are the discriminateparameters [4] and their definitions are as follows
120578 =119879119875119878
119879119875119878 + 119865119873119878
times 100 120585 =119879119875119865
119879119875119865 + 119865119873119865
times 100
(23)
where 119878 is shadow 119865 is foreground 119879119875119878 is the number ofthe pixels which are detected correctly 119865119873119878 is the number ofshadow pixels which are mistakenly detected as foreground119879119875119865 is the number of foreground pixels detected correctlyafter removing shadows 119865119873119865 is the number of foregroundpixels which are mistakenly detected as shadow The shadowdistinguishing rate is concerned with maintaining the pixels
which belong to the moving object as foreground In thispaper we use the average of the two rates as a singleperformance measure (avg)
Table 1 is the compared results among the method basedon LIRM and other methods in [2] Table 1 shows the averageshadow detection and discrimination rates on each testsequence From the compared results the method proposedin this paper is better at most time The main cause isthat the presented method directly detects the foregroundfrom video data but other methods extract the shadowfrom foreground by true background and foreground withshadow Our method use local intensity ratio to replace theactual pixel values complexity is lower Figure 6 is the resultof proposed method in moving object detection withoutshadow in different scenes
Mathematical Problems in Engineering 9
Table 1 Comparison of moving object detection with different methods
Figure 5 represents compared results of the LIRMmovingobject detection via GMM and postprocess From Figure 5the postprocess obviously improves the shadow discrimi-nation rate and slightly reduces the shadow detection rateHowever the average of the two rates is increased obviouslySo the postprocess enhances the moving object detectionresult
Finally we show that the proposedmethod rapidly adaptsto variations in environment To test moving object detectionunder illumination changes condition we compare the resultof other techniques which are GMM [5] adaptive GMM [24]that controls the learning rate of the GMM adaptively LBP[25] that detects moving objects using texture and edges Xursquosmethods [26] that use a color chromaticity Piletrsquos methods[27] that use illumination and spatial likelihood and Choirsquosmethods [28] that develop chromaticity differencemodel andbrightness model that estimates the intensity difference andintensity ratio of false foreground pixels For the quantitativecomparison we have made three test videos [28] Figure 7represents moving object detection results of each methodFrom Figure 7 we find the postprocessing of the proposedmethod can strengthen the moving objects and remove theshadow patches and the noise but it sometimes enhances theedge of part shadows incorrectly and makes moving objectsrsquoedge fuzzy In a word the proposed method adapts to thechanged background completely and detects moving objectpixels without shadow pixels successfully Although someother methods are robust to illumination their detectionmoving pixels results show that they cannot detect movingobject pixels meanwhile removing shadow
5 Conclusions
In this paper we propose a method to detect movingobject meanwhile removing cast shadow which is robust inchanging illumination condition This paper presents a localintensity ratio model according to illumination model andproves its illumination invariance Meanwhile if the noisein video obeys the Gaussian distribution the local intensityratio also obeys the Gaussian distribution In the process ofGaussian mixture model to obtain foreground this paperreplaces actual pixel value by local intensity ratio Finallypostprocess methods are used to get pure moving objectpixels without noise such as erosion method to removeshadow pitch and noise which is false detected as foregroundforeground contour enhanced method to strengthen moving
object contour and filling methods to deal with holes inforeground object Experiment results demonstrate that themethod presented by this paper can eliminate shadow onforeground effectively and is robust in changing illuminationconditionHowever in some scenes that foreground is similarto background or foreground is similar to shadow fore-groundmay be easily detected as background If illuminationchanges severely such as turn onoff light in indoor scenethe backgroundmay be easily detected as foreground and theperformance drops
Conflict of Interests
The authors declare that there is no conflict of interestsregarding the publication of this paper
Acknowledgments
Jinhai Xiang is supported by the Fundamental ResearchFunds for the Central Universities under Grant no2013QC024 Jun Xu is supported by the National NaturalScience Foundation of China under Grant no 11204099and self-determined research funds of CCNU from thecolleges basic research and operation of MOE Weiping Sunis supported by the National Natural Science Foundation ofChina under Grant no 61300140
References
[1] M Piccardi ldquoBackground subtraction techniques a reviewrdquo inProceedings of the IEEE International Conference on SystemsMan and Cybernetics (SMC rsquo04) pp 3099ndash3104 October 2004
[2] A Sanin C Sanderson and B C Lovell ldquoShadow detection asurvey and comparative evaluation of recent methodsrdquo PatternRecognition vol 45 no 4 pp 1684ndash1695 2012
[3] N Al-Najdawi H E Bez J Singhai and E A Edirisinghe ldquoAsurvey of cast shadow detection algorithmsrdquo Pattern Recogni-tion Letters vol 33 no 6 pp 752ndash764 2012
[4] A Prati I Mikic M M Trivedi and R Cucchiara ldquoDetectingmoving shadows algorithms and evaluationrdquo IEEE Transac-tions on Pattern Analysis and Machine Intelligence vol 25 no7 pp 918ndash923 2003
[5] Z Zivkovic and F van der Heijden ldquoEfficient adaptive densityestimation per image pixel for the task of background subtrac-tionrdquo Pattern Recognition Letters vol 27 no 7 pp 773ndash7802006
10 Mathematical Problems in Engineering
[6] A J Joshi and N P Papanikolopoulos ldquoLearning to detectmoving shadows in dynamic environmentsrdquo IEEE Transactionson Pattern Analysis and Machine Intelligence vol 30 no 11 pp2055ndash2063 2008
[7] C R Jung ldquoEfficient background subtraction and shadowremoval for monochromatic video sequencesrdquo IEEE Transac-tions on Multimedia vol 11 no 3 pp 571ndash577 2009
[8] M Shoaib R Dragon and J Ostermann ldquoShadow detection formoving humans using gradient-based background subtractionrdquoin Proceedings of the IEEE International Conference onAcousticsSpeech and Signal Processing (ICASSP rsquo09) pp 773ndash776 April2009
[9] R Cucchiara C Grana M Piccardi and A Prati ldquoDetectingmoving objects ghosts and shadows in video streamsrdquo IEEETransactions on Pattern Analysis and Machine Intelligence vol25 no 10 pp 1337ndash1342 2003
[10] E Salvador A Cavallaro and T Ebrahimi ldquoCast shadowsegmentation using invariant color featuresrdquo Computer Visionand Image Understanding vol 95 no 2 pp 238ndash259 2004
[11] D Grest J M Frahm and R Koch ldquoA color similarity measurefor robust shadow removal in real timerdquo in Proceedings of theVision Modeling and Visualization (VMV rsquo03) pp 253ndash260November 2003
[12] THorprasert DHarwood and L Davis ldquoA statistical approachfor real-time robust background subtraction and shadow detec-tionrdquo in Proceedings of the IEEE International Conference onComputer Vision (ICCV rsquo99) Frame Rate Workshop 1999
[13] S Nadimi and B Bhanu ldquoPhysical models for moving shadowand object detection in videordquo IEEE Transactions on PatternAnalysis and Machine Intelligence vol 26 no 8 pp 1079ndash10872004
[14] J B Huang and C S Chen ldquoMoving cast shadow detec-tion using Physics-based featuresrdquo in Proceedings of the IEEEComputer Society Conference on Computer Vision and PatternRecognition Workshops (CVPR rsquo09) pp 2310ndash2317 June 2009
[15] N Martel-Brisson and A Zaccarin ldquoKernel-based learning ofcast shadows fromaphysicalmodel of light sources and surfacesfor low-level segmentationrdquo in Proceedings of the 26th IEEEConference on Computer Vision and Pattern Recognition (CVPRrsquo08) pp 1ndash8 June 2008
[16] C C Chen and J K Aggarwal ldquoHuman shadow removal withunknown light sourcerdquo in Proceedings of the 20th InternationalConference on Pattern Recognition (ICPR rsquo10) pp 2407ndash2410August 2010
[17] J W Hsieh W F Hu C J Chang and Y S Chen ldquoShadowelimination for effective moving object detection by Gaussianshadow modelingrdquo Image and Vision Computing vol 21 no 6pp 505ndash516 2003
[18] H Nicolas and J M Pinel ldquoJoint moving cast shadows seg-mentation and light source detection in video sequencesrdquo SignalProcessing Image Communication vol 21 no 1 pp 22ndash43 2006
[19] A Leone andCDistante ldquoShadowdetection formoving objectsbased on texture analysisrdquoPattern Recognition vol 40 no 4 pp1222ndash1233 2007
[20] W Zhang X Z Fang X K Yang and Q M J Wu ldquoMovingcast shadows detection using ratio edgerdquo IEEE Transactions onMultimedia vol 9 no 6 pp 1202ndash1213 2007
[21] M Xiao C Z Han and L Zhang ldquoMoving shadow detectionand removal for traffic sequencesrdquo International Journal ofAutomation and Computing vol 4 no 1 pp 38ndash46 2007
[22] J Stauder R Mech and J Ostermann ldquoDetection of movingcast shadows for object segmentationrdquo IEEE Transactions onMultimedia vol 1 no 1 pp 65ndash76 1999
[23] M S Andersen T Jensen and C B Madsen ldquoEstimationof dynamic light changes in outdoor scenes without the useof calibration objectsrdquo in Proceedings of the 18th InternationalConference on Pattern Recognition (ICPR rsquo06) vol 4 pp 91ndash94August 2006
[24] D S Lee ldquoEffective Gaussian mixture learning for videobackground subtractionrdquo IEEETransactions on PatternAnalysisand Machine Intelligence vol 27 no 5 pp 827ndash832 2005
[25] M Heikkila and M Pietikainen ldquoA texture-based method formodeling the background and detecting moving objectsrdquo IEEETransactions on Pattern Analysis and Machine Intelligence vol28 no 4 pp 657ndash662 2006
[26] M Xu and T Ellis ldquoColour-invariant motion detection underfast illumination changesrdquo in Proceedings of the 2nd EuropeanWorkshop on Advanced Video-Based Surveillance Systems vol4 pp 101ndash112 September 2001
[27] J Pilet C Strecha and P Fua ldquoMaking background subtractionrobust to sudden illumination changesrdquo in Proceedings of the10th European Conference on Computer Vision (ECCV rsquo08) pp567ndash580 October 2008
[28] J Choi H J Chang Y J Yoo and J Y Choi ldquoRobust movingobject detection against fast illumination changerdquo ComputerVision and Image Understanding vol 116 no 2 pp 179ndash1932012
Figure 4 Illustration of the shadow removal process using Campus sequence (a) Current frame (b) Normalized LIR of current frame (c)Foreground object detected using GMM of LIR (d) Result of erosion operation (contour image) to remove false detection shadow and noise(e) Enhanced contour image by contour enhancement method (f) Final result of filling holes
Shadow detection rate before fillingShadow discrimination rate before fillingCombined score before fillingShadow detection rate after fillingShadow discrimination rate after fillingCombined score after filling
100
90
80
70
60
50
40
30
20
Shad
ow d
etec
tion
perfo
rman
ce (
)
Hallway Campus Room Highway
Figure 5 Compared detection result of LIRM via GMM andpostprocess
After calculating V(119909 119910) and 1199041015840(119909 119910) we can get the
option that the pixel 119875(119909 119910) is moving object pixel or not by
119901 (119909 119910) = 255 V (119909 119910) lt 120574
1015840V 1199041015840(119909 119910) lt 120574
1015840119904
0 otherwise(22)
where 1205741015840V and 1205741015840119904 are the threshold values 119901(119909 119910) denotes the
final value of pixel (119909 119910)
4 Experiment Results
In this section we describe the test videos used in thisstudymeasure the performance of the proposedmethod andcompare the performance to the state-of-the-art methodsIn general state-of-the-art methods use different steps toremove shadow and reduce the influence of illuminationchanges To compare with other methods in different stepswe first compare the shadow removing results and then com-pare the results under various illumination change conditionsin different data sets
41 Experiment Conditions and Parameter Setting The size ofthe local rectangle region in (2) is important to the proposedmethod If the size is too large its inner region cannot meetthe assumptions (9) On the other hand if too small thechange of its value is also small In our experiment 119903 is setas 2
42 Experiment Results To test the effect of shadow remov-ing based on local intensity ratio and the effect of foregrounddetection we use four typical videos from the ATON videoset [4] as the test videos which are Campus Hallway High-way 1 and Intelligent room Intelligent room andHallway aretypical indoor environment In the videos people are walkingin scene and their shadows are mapped to the backgroundThe Campus is a video of campus parking lot and there areshadows of people and cars in the scenes
Mathematical Problems in Engineering 7
Origin
Realresult
Proposed
(a)
Origin
Realresult
Proposed
(b)
Origin
Realresult
Proposed
(c)
Origin
Realresult
Proposed
(d)
Figure 6 Detection results in different scenes (a) is Highway I sequence (b) is intelligent room sequence (c) is Campus sequence and (d)is Hallway sequence In the image of detection result the moving object pixels are white the background is black and the shadow is gray
To compare the performance of the proposed movingobjects detectionmethod with other shadow detectionmeth-ods we assume that if shadow is detected as backgroundit is correct detection otherwise it is false detection andif moving object is detected as foreground it is correct
detection otherwise it is false detection The proposedmethod is compared with five methods introduced by [2]which are chromaticity-based method physical methodgeometry-based method Small region (SR) texture-basedmethod and large region (LR) texture-based method To
Figure 7 The comparison of moving object detection results under the changing illumination condition (three scenes) From left to right(a) is video 1 (b) is video 2 and (c) is video 3 The 1st row is an input image the remaining nine rows are detection results of each methodthe 2nd row is a detection result of GMM the 3rd row is of adaptive GMM the 4th row is of LBP the 5th row is of Xu the 6th row is of Piletthe 7th row is of Choi the 8th row is of the proposed method without postprocessing and the 9th row is the final result of proposed method
estimate the effect of these methods shadow detection rate(120578) and shadow distinguishing rate (120585) are the discriminateparameters [4] and their definitions are as follows
120578 =119879119875119878
119879119875119878 + 119865119873119878
times 100 120585 =119879119875119865
119879119875119865 + 119865119873119865
times 100
(23)
where 119878 is shadow 119865 is foreground 119879119875119878 is the number ofthe pixels which are detected correctly 119865119873119878 is the number ofshadow pixels which are mistakenly detected as foreground119879119875119865 is the number of foreground pixels detected correctlyafter removing shadows 119865119873119865 is the number of foregroundpixels which are mistakenly detected as shadow The shadowdistinguishing rate is concerned with maintaining the pixels
which belong to the moving object as foreground In thispaper we use the average of the two rates as a singleperformance measure (avg)
Table 1 is the compared results among the method basedon LIRM and other methods in [2] Table 1 shows the averageshadow detection and discrimination rates on each testsequence From the compared results the method proposedin this paper is better at most time The main cause isthat the presented method directly detects the foregroundfrom video data but other methods extract the shadowfrom foreground by true background and foreground withshadow Our method use local intensity ratio to replace theactual pixel values complexity is lower Figure 6 is the resultof proposed method in moving object detection withoutshadow in different scenes
Mathematical Problems in Engineering 9
Table 1 Comparison of moving object detection with different methods
Figure 5 represents compared results of the LIRMmovingobject detection via GMM and postprocess From Figure 5the postprocess obviously improves the shadow discrimi-nation rate and slightly reduces the shadow detection rateHowever the average of the two rates is increased obviouslySo the postprocess enhances the moving object detectionresult
Finally we show that the proposedmethod rapidly adaptsto variations in environment To test moving object detectionunder illumination changes condition we compare the resultof other techniques which are GMM [5] adaptive GMM [24]that controls the learning rate of the GMM adaptively LBP[25] that detects moving objects using texture and edges Xursquosmethods [26] that use a color chromaticity Piletrsquos methods[27] that use illumination and spatial likelihood and Choirsquosmethods [28] that develop chromaticity differencemodel andbrightness model that estimates the intensity difference andintensity ratio of false foreground pixels For the quantitativecomparison we have made three test videos [28] Figure 7represents moving object detection results of each methodFrom Figure 7 we find the postprocessing of the proposedmethod can strengthen the moving objects and remove theshadow patches and the noise but it sometimes enhances theedge of part shadows incorrectly and makes moving objectsrsquoedge fuzzy In a word the proposed method adapts to thechanged background completely and detects moving objectpixels without shadow pixels successfully Although someother methods are robust to illumination their detectionmoving pixels results show that they cannot detect movingobject pixels meanwhile removing shadow
5 Conclusions
In this paper we propose a method to detect movingobject meanwhile removing cast shadow which is robust inchanging illumination condition This paper presents a localintensity ratio model according to illumination model andproves its illumination invariance Meanwhile if the noisein video obeys the Gaussian distribution the local intensityratio also obeys the Gaussian distribution In the process ofGaussian mixture model to obtain foreground this paperreplaces actual pixel value by local intensity ratio Finallypostprocess methods are used to get pure moving objectpixels without noise such as erosion method to removeshadow pitch and noise which is false detected as foregroundforeground contour enhanced method to strengthen moving
object contour and filling methods to deal with holes inforeground object Experiment results demonstrate that themethod presented by this paper can eliminate shadow onforeground effectively and is robust in changing illuminationconditionHowever in some scenes that foreground is similarto background or foreground is similar to shadow fore-groundmay be easily detected as background If illuminationchanges severely such as turn onoff light in indoor scenethe backgroundmay be easily detected as foreground and theperformance drops
Conflict of Interests
The authors declare that there is no conflict of interestsregarding the publication of this paper
Acknowledgments
Jinhai Xiang is supported by the Fundamental ResearchFunds for the Central Universities under Grant no2013QC024 Jun Xu is supported by the National NaturalScience Foundation of China under Grant no 11204099and self-determined research funds of CCNU from thecolleges basic research and operation of MOE Weiping Sunis supported by the National Natural Science Foundation ofChina under Grant no 61300140
References
[1] M Piccardi ldquoBackground subtraction techniques a reviewrdquo inProceedings of the IEEE International Conference on SystemsMan and Cybernetics (SMC rsquo04) pp 3099ndash3104 October 2004
[2] A Sanin C Sanderson and B C Lovell ldquoShadow detection asurvey and comparative evaluation of recent methodsrdquo PatternRecognition vol 45 no 4 pp 1684ndash1695 2012
[3] N Al-Najdawi H E Bez J Singhai and E A Edirisinghe ldquoAsurvey of cast shadow detection algorithmsrdquo Pattern Recogni-tion Letters vol 33 no 6 pp 752ndash764 2012
[4] A Prati I Mikic M M Trivedi and R Cucchiara ldquoDetectingmoving shadows algorithms and evaluationrdquo IEEE Transac-tions on Pattern Analysis and Machine Intelligence vol 25 no7 pp 918ndash923 2003
[5] Z Zivkovic and F van der Heijden ldquoEfficient adaptive densityestimation per image pixel for the task of background subtrac-tionrdquo Pattern Recognition Letters vol 27 no 7 pp 773ndash7802006
10 Mathematical Problems in Engineering
[6] A J Joshi and N P Papanikolopoulos ldquoLearning to detectmoving shadows in dynamic environmentsrdquo IEEE Transactionson Pattern Analysis and Machine Intelligence vol 30 no 11 pp2055ndash2063 2008
[7] C R Jung ldquoEfficient background subtraction and shadowremoval for monochromatic video sequencesrdquo IEEE Transac-tions on Multimedia vol 11 no 3 pp 571ndash577 2009
[8] M Shoaib R Dragon and J Ostermann ldquoShadow detection formoving humans using gradient-based background subtractionrdquoin Proceedings of the IEEE International Conference onAcousticsSpeech and Signal Processing (ICASSP rsquo09) pp 773ndash776 April2009
[9] R Cucchiara C Grana M Piccardi and A Prati ldquoDetectingmoving objects ghosts and shadows in video streamsrdquo IEEETransactions on Pattern Analysis and Machine Intelligence vol25 no 10 pp 1337ndash1342 2003
[10] E Salvador A Cavallaro and T Ebrahimi ldquoCast shadowsegmentation using invariant color featuresrdquo Computer Visionand Image Understanding vol 95 no 2 pp 238ndash259 2004
[11] D Grest J M Frahm and R Koch ldquoA color similarity measurefor robust shadow removal in real timerdquo in Proceedings of theVision Modeling and Visualization (VMV rsquo03) pp 253ndash260November 2003
[12] THorprasert DHarwood and L Davis ldquoA statistical approachfor real-time robust background subtraction and shadow detec-tionrdquo in Proceedings of the IEEE International Conference onComputer Vision (ICCV rsquo99) Frame Rate Workshop 1999
[13] S Nadimi and B Bhanu ldquoPhysical models for moving shadowand object detection in videordquo IEEE Transactions on PatternAnalysis and Machine Intelligence vol 26 no 8 pp 1079ndash10872004
[14] J B Huang and C S Chen ldquoMoving cast shadow detec-tion using Physics-based featuresrdquo in Proceedings of the IEEEComputer Society Conference on Computer Vision and PatternRecognition Workshops (CVPR rsquo09) pp 2310ndash2317 June 2009
[15] N Martel-Brisson and A Zaccarin ldquoKernel-based learning ofcast shadows fromaphysicalmodel of light sources and surfacesfor low-level segmentationrdquo in Proceedings of the 26th IEEEConference on Computer Vision and Pattern Recognition (CVPRrsquo08) pp 1ndash8 June 2008
[16] C C Chen and J K Aggarwal ldquoHuman shadow removal withunknown light sourcerdquo in Proceedings of the 20th InternationalConference on Pattern Recognition (ICPR rsquo10) pp 2407ndash2410August 2010
[17] J W Hsieh W F Hu C J Chang and Y S Chen ldquoShadowelimination for effective moving object detection by Gaussianshadow modelingrdquo Image and Vision Computing vol 21 no 6pp 505ndash516 2003
[18] H Nicolas and J M Pinel ldquoJoint moving cast shadows seg-mentation and light source detection in video sequencesrdquo SignalProcessing Image Communication vol 21 no 1 pp 22ndash43 2006
[19] A Leone andCDistante ldquoShadowdetection formoving objectsbased on texture analysisrdquoPattern Recognition vol 40 no 4 pp1222ndash1233 2007
[20] W Zhang X Z Fang X K Yang and Q M J Wu ldquoMovingcast shadows detection using ratio edgerdquo IEEE Transactions onMultimedia vol 9 no 6 pp 1202ndash1213 2007
[21] M Xiao C Z Han and L Zhang ldquoMoving shadow detectionand removal for traffic sequencesrdquo International Journal ofAutomation and Computing vol 4 no 1 pp 38ndash46 2007
[22] J Stauder R Mech and J Ostermann ldquoDetection of movingcast shadows for object segmentationrdquo IEEE Transactions onMultimedia vol 1 no 1 pp 65ndash76 1999
[23] M S Andersen T Jensen and C B Madsen ldquoEstimationof dynamic light changes in outdoor scenes without the useof calibration objectsrdquo in Proceedings of the 18th InternationalConference on Pattern Recognition (ICPR rsquo06) vol 4 pp 91ndash94August 2006
[24] D S Lee ldquoEffective Gaussian mixture learning for videobackground subtractionrdquo IEEETransactions on PatternAnalysisand Machine Intelligence vol 27 no 5 pp 827ndash832 2005
[25] M Heikkila and M Pietikainen ldquoA texture-based method formodeling the background and detecting moving objectsrdquo IEEETransactions on Pattern Analysis and Machine Intelligence vol28 no 4 pp 657ndash662 2006
[26] M Xu and T Ellis ldquoColour-invariant motion detection underfast illumination changesrdquo in Proceedings of the 2nd EuropeanWorkshop on Advanced Video-Based Surveillance Systems vol4 pp 101ndash112 September 2001
[27] J Pilet C Strecha and P Fua ldquoMaking background subtractionrobust to sudden illumination changesrdquo in Proceedings of the10th European Conference on Computer Vision (ECCV rsquo08) pp567ndash580 October 2008
[28] J Choi H J Chang Y J Yoo and J Y Choi ldquoRobust movingobject detection against fast illumination changerdquo ComputerVision and Image Understanding vol 116 no 2 pp 179ndash1932012
Figure 6 Detection results in different scenes (a) is Highway I sequence (b) is intelligent room sequence (c) is Campus sequence and (d)is Hallway sequence In the image of detection result the moving object pixels are white the background is black and the shadow is gray
To compare the performance of the proposed movingobjects detectionmethod with other shadow detectionmeth-ods we assume that if shadow is detected as backgroundit is correct detection otherwise it is false detection andif moving object is detected as foreground it is correct
detection otherwise it is false detection The proposedmethod is compared with five methods introduced by [2]which are chromaticity-based method physical methodgeometry-based method Small region (SR) texture-basedmethod and large region (LR) texture-based method To
Figure 7 The comparison of moving object detection results under the changing illumination condition (three scenes) From left to right(a) is video 1 (b) is video 2 and (c) is video 3 The 1st row is an input image the remaining nine rows are detection results of each methodthe 2nd row is a detection result of GMM the 3rd row is of adaptive GMM the 4th row is of LBP the 5th row is of Xu the 6th row is of Piletthe 7th row is of Choi the 8th row is of the proposed method without postprocessing and the 9th row is the final result of proposed method
estimate the effect of these methods shadow detection rate(120578) and shadow distinguishing rate (120585) are the discriminateparameters [4] and their definitions are as follows
120578 =119879119875119878
119879119875119878 + 119865119873119878
times 100 120585 =119879119875119865
119879119875119865 + 119865119873119865
times 100
(23)
where 119878 is shadow 119865 is foreground 119879119875119878 is the number ofthe pixels which are detected correctly 119865119873119878 is the number ofshadow pixels which are mistakenly detected as foreground119879119875119865 is the number of foreground pixels detected correctlyafter removing shadows 119865119873119865 is the number of foregroundpixels which are mistakenly detected as shadow The shadowdistinguishing rate is concerned with maintaining the pixels
which belong to the moving object as foreground In thispaper we use the average of the two rates as a singleperformance measure (avg)
Table 1 is the compared results among the method basedon LIRM and other methods in [2] Table 1 shows the averageshadow detection and discrimination rates on each testsequence From the compared results the method proposedin this paper is better at most time The main cause isthat the presented method directly detects the foregroundfrom video data but other methods extract the shadowfrom foreground by true background and foreground withshadow Our method use local intensity ratio to replace theactual pixel values complexity is lower Figure 6 is the resultof proposed method in moving object detection withoutshadow in different scenes
Mathematical Problems in Engineering 9
Table 1 Comparison of moving object detection with different methods
Figure 5 represents compared results of the LIRMmovingobject detection via GMM and postprocess From Figure 5the postprocess obviously improves the shadow discrimi-nation rate and slightly reduces the shadow detection rateHowever the average of the two rates is increased obviouslySo the postprocess enhances the moving object detectionresult
Finally we show that the proposedmethod rapidly adaptsto variations in environment To test moving object detectionunder illumination changes condition we compare the resultof other techniques which are GMM [5] adaptive GMM [24]that controls the learning rate of the GMM adaptively LBP[25] that detects moving objects using texture and edges Xursquosmethods [26] that use a color chromaticity Piletrsquos methods[27] that use illumination and spatial likelihood and Choirsquosmethods [28] that develop chromaticity differencemodel andbrightness model that estimates the intensity difference andintensity ratio of false foreground pixels For the quantitativecomparison we have made three test videos [28] Figure 7represents moving object detection results of each methodFrom Figure 7 we find the postprocessing of the proposedmethod can strengthen the moving objects and remove theshadow patches and the noise but it sometimes enhances theedge of part shadows incorrectly and makes moving objectsrsquoedge fuzzy In a word the proposed method adapts to thechanged background completely and detects moving objectpixels without shadow pixels successfully Although someother methods are robust to illumination their detectionmoving pixels results show that they cannot detect movingobject pixels meanwhile removing shadow
5 Conclusions
In this paper we propose a method to detect movingobject meanwhile removing cast shadow which is robust inchanging illumination condition This paper presents a localintensity ratio model according to illumination model andproves its illumination invariance Meanwhile if the noisein video obeys the Gaussian distribution the local intensityratio also obeys the Gaussian distribution In the process ofGaussian mixture model to obtain foreground this paperreplaces actual pixel value by local intensity ratio Finallypostprocess methods are used to get pure moving objectpixels without noise such as erosion method to removeshadow pitch and noise which is false detected as foregroundforeground contour enhanced method to strengthen moving
object contour and filling methods to deal with holes inforeground object Experiment results demonstrate that themethod presented by this paper can eliminate shadow onforeground effectively and is robust in changing illuminationconditionHowever in some scenes that foreground is similarto background or foreground is similar to shadow fore-groundmay be easily detected as background If illuminationchanges severely such as turn onoff light in indoor scenethe backgroundmay be easily detected as foreground and theperformance drops
Conflict of Interests
The authors declare that there is no conflict of interestsregarding the publication of this paper
Acknowledgments
Jinhai Xiang is supported by the Fundamental ResearchFunds for the Central Universities under Grant no2013QC024 Jun Xu is supported by the National NaturalScience Foundation of China under Grant no 11204099and self-determined research funds of CCNU from thecolleges basic research and operation of MOE Weiping Sunis supported by the National Natural Science Foundation ofChina under Grant no 61300140
References
[1] M Piccardi ldquoBackground subtraction techniques a reviewrdquo inProceedings of the IEEE International Conference on SystemsMan and Cybernetics (SMC rsquo04) pp 3099ndash3104 October 2004
[2] A Sanin C Sanderson and B C Lovell ldquoShadow detection asurvey and comparative evaluation of recent methodsrdquo PatternRecognition vol 45 no 4 pp 1684ndash1695 2012
[3] N Al-Najdawi H E Bez J Singhai and E A Edirisinghe ldquoAsurvey of cast shadow detection algorithmsrdquo Pattern Recogni-tion Letters vol 33 no 6 pp 752ndash764 2012
[4] A Prati I Mikic M M Trivedi and R Cucchiara ldquoDetectingmoving shadows algorithms and evaluationrdquo IEEE Transac-tions on Pattern Analysis and Machine Intelligence vol 25 no7 pp 918ndash923 2003
[5] Z Zivkovic and F van der Heijden ldquoEfficient adaptive densityestimation per image pixel for the task of background subtrac-tionrdquo Pattern Recognition Letters vol 27 no 7 pp 773ndash7802006
10 Mathematical Problems in Engineering
[6] A J Joshi and N P Papanikolopoulos ldquoLearning to detectmoving shadows in dynamic environmentsrdquo IEEE Transactionson Pattern Analysis and Machine Intelligence vol 30 no 11 pp2055ndash2063 2008
[7] C R Jung ldquoEfficient background subtraction and shadowremoval for monochromatic video sequencesrdquo IEEE Transac-tions on Multimedia vol 11 no 3 pp 571ndash577 2009
[8] M Shoaib R Dragon and J Ostermann ldquoShadow detection formoving humans using gradient-based background subtractionrdquoin Proceedings of the IEEE International Conference onAcousticsSpeech and Signal Processing (ICASSP rsquo09) pp 773ndash776 April2009
[9] R Cucchiara C Grana M Piccardi and A Prati ldquoDetectingmoving objects ghosts and shadows in video streamsrdquo IEEETransactions on Pattern Analysis and Machine Intelligence vol25 no 10 pp 1337ndash1342 2003
[10] E Salvador A Cavallaro and T Ebrahimi ldquoCast shadowsegmentation using invariant color featuresrdquo Computer Visionand Image Understanding vol 95 no 2 pp 238ndash259 2004
[11] D Grest J M Frahm and R Koch ldquoA color similarity measurefor robust shadow removal in real timerdquo in Proceedings of theVision Modeling and Visualization (VMV rsquo03) pp 253ndash260November 2003
[12] THorprasert DHarwood and L Davis ldquoA statistical approachfor real-time robust background subtraction and shadow detec-tionrdquo in Proceedings of the IEEE International Conference onComputer Vision (ICCV rsquo99) Frame Rate Workshop 1999
[13] S Nadimi and B Bhanu ldquoPhysical models for moving shadowand object detection in videordquo IEEE Transactions on PatternAnalysis and Machine Intelligence vol 26 no 8 pp 1079ndash10872004
[14] J B Huang and C S Chen ldquoMoving cast shadow detec-tion using Physics-based featuresrdquo in Proceedings of the IEEEComputer Society Conference on Computer Vision and PatternRecognition Workshops (CVPR rsquo09) pp 2310ndash2317 June 2009
[15] N Martel-Brisson and A Zaccarin ldquoKernel-based learning ofcast shadows fromaphysicalmodel of light sources and surfacesfor low-level segmentationrdquo in Proceedings of the 26th IEEEConference on Computer Vision and Pattern Recognition (CVPRrsquo08) pp 1ndash8 June 2008
[16] C C Chen and J K Aggarwal ldquoHuman shadow removal withunknown light sourcerdquo in Proceedings of the 20th InternationalConference on Pattern Recognition (ICPR rsquo10) pp 2407ndash2410August 2010
[17] J W Hsieh W F Hu C J Chang and Y S Chen ldquoShadowelimination for effective moving object detection by Gaussianshadow modelingrdquo Image and Vision Computing vol 21 no 6pp 505ndash516 2003
[18] H Nicolas and J M Pinel ldquoJoint moving cast shadows seg-mentation and light source detection in video sequencesrdquo SignalProcessing Image Communication vol 21 no 1 pp 22ndash43 2006
[19] A Leone andCDistante ldquoShadowdetection formoving objectsbased on texture analysisrdquoPattern Recognition vol 40 no 4 pp1222ndash1233 2007
[20] W Zhang X Z Fang X K Yang and Q M J Wu ldquoMovingcast shadows detection using ratio edgerdquo IEEE Transactions onMultimedia vol 9 no 6 pp 1202ndash1213 2007
[21] M Xiao C Z Han and L Zhang ldquoMoving shadow detectionand removal for traffic sequencesrdquo International Journal ofAutomation and Computing vol 4 no 1 pp 38ndash46 2007
[22] J Stauder R Mech and J Ostermann ldquoDetection of movingcast shadows for object segmentationrdquo IEEE Transactions onMultimedia vol 1 no 1 pp 65ndash76 1999
[23] M S Andersen T Jensen and C B Madsen ldquoEstimationof dynamic light changes in outdoor scenes without the useof calibration objectsrdquo in Proceedings of the 18th InternationalConference on Pattern Recognition (ICPR rsquo06) vol 4 pp 91ndash94August 2006
[24] D S Lee ldquoEffective Gaussian mixture learning for videobackground subtractionrdquo IEEETransactions on PatternAnalysisand Machine Intelligence vol 27 no 5 pp 827ndash832 2005
[25] M Heikkila and M Pietikainen ldquoA texture-based method formodeling the background and detecting moving objectsrdquo IEEETransactions on Pattern Analysis and Machine Intelligence vol28 no 4 pp 657ndash662 2006
[26] M Xu and T Ellis ldquoColour-invariant motion detection underfast illumination changesrdquo in Proceedings of the 2nd EuropeanWorkshop on Advanced Video-Based Surveillance Systems vol4 pp 101ndash112 September 2001
[27] J Pilet C Strecha and P Fua ldquoMaking background subtractionrobust to sudden illumination changesrdquo in Proceedings of the10th European Conference on Computer Vision (ECCV rsquo08) pp567ndash580 October 2008
[28] J Choi H J Chang Y J Yoo and J Y Choi ldquoRobust movingobject detection against fast illumination changerdquo ComputerVision and Image Understanding vol 116 no 2 pp 179ndash1932012
Figure 7 The comparison of moving object detection results under the changing illumination condition (three scenes) From left to right(a) is video 1 (b) is video 2 and (c) is video 3 The 1st row is an input image the remaining nine rows are detection results of each methodthe 2nd row is a detection result of GMM the 3rd row is of adaptive GMM the 4th row is of LBP the 5th row is of Xu the 6th row is of Piletthe 7th row is of Choi the 8th row is of the proposed method without postprocessing and the 9th row is the final result of proposed method
estimate the effect of these methods shadow detection rate(120578) and shadow distinguishing rate (120585) are the discriminateparameters [4] and their definitions are as follows
120578 =119879119875119878
119879119875119878 + 119865119873119878
times 100 120585 =119879119875119865
119879119875119865 + 119865119873119865
times 100
(23)
where 119878 is shadow 119865 is foreground 119879119875119878 is the number ofthe pixels which are detected correctly 119865119873119878 is the number ofshadow pixels which are mistakenly detected as foreground119879119875119865 is the number of foreground pixels detected correctlyafter removing shadows 119865119873119865 is the number of foregroundpixels which are mistakenly detected as shadow The shadowdistinguishing rate is concerned with maintaining the pixels
which belong to the moving object as foreground In thispaper we use the average of the two rates as a singleperformance measure (avg)
Table 1 is the compared results among the method basedon LIRM and other methods in [2] Table 1 shows the averageshadow detection and discrimination rates on each testsequence From the compared results the method proposedin this paper is better at most time The main cause isthat the presented method directly detects the foregroundfrom video data but other methods extract the shadowfrom foreground by true background and foreground withshadow Our method use local intensity ratio to replace theactual pixel values complexity is lower Figure 6 is the resultof proposed method in moving object detection withoutshadow in different scenes
Mathematical Problems in Engineering 9
Table 1 Comparison of moving object detection with different methods
Figure 5 represents compared results of the LIRMmovingobject detection via GMM and postprocess From Figure 5the postprocess obviously improves the shadow discrimi-nation rate and slightly reduces the shadow detection rateHowever the average of the two rates is increased obviouslySo the postprocess enhances the moving object detectionresult
Finally we show that the proposedmethod rapidly adaptsto variations in environment To test moving object detectionunder illumination changes condition we compare the resultof other techniques which are GMM [5] adaptive GMM [24]that controls the learning rate of the GMM adaptively LBP[25] that detects moving objects using texture and edges Xursquosmethods [26] that use a color chromaticity Piletrsquos methods[27] that use illumination and spatial likelihood and Choirsquosmethods [28] that develop chromaticity differencemodel andbrightness model that estimates the intensity difference andintensity ratio of false foreground pixels For the quantitativecomparison we have made three test videos [28] Figure 7represents moving object detection results of each methodFrom Figure 7 we find the postprocessing of the proposedmethod can strengthen the moving objects and remove theshadow patches and the noise but it sometimes enhances theedge of part shadows incorrectly and makes moving objectsrsquoedge fuzzy In a word the proposed method adapts to thechanged background completely and detects moving objectpixels without shadow pixels successfully Although someother methods are robust to illumination their detectionmoving pixels results show that they cannot detect movingobject pixels meanwhile removing shadow
5 Conclusions
In this paper we propose a method to detect movingobject meanwhile removing cast shadow which is robust inchanging illumination condition This paper presents a localintensity ratio model according to illumination model andproves its illumination invariance Meanwhile if the noisein video obeys the Gaussian distribution the local intensityratio also obeys the Gaussian distribution In the process ofGaussian mixture model to obtain foreground this paperreplaces actual pixel value by local intensity ratio Finallypostprocess methods are used to get pure moving objectpixels without noise such as erosion method to removeshadow pitch and noise which is false detected as foregroundforeground contour enhanced method to strengthen moving
object contour and filling methods to deal with holes inforeground object Experiment results demonstrate that themethod presented by this paper can eliminate shadow onforeground effectively and is robust in changing illuminationconditionHowever in some scenes that foreground is similarto background or foreground is similar to shadow fore-groundmay be easily detected as background If illuminationchanges severely such as turn onoff light in indoor scenethe backgroundmay be easily detected as foreground and theperformance drops
Conflict of Interests
The authors declare that there is no conflict of interestsregarding the publication of this paper
Acknowledgments
Jinhai Xiang is supported by the Fundamental ResearchFunds for the Central Universities under Grant no2013QC024 Jun Xu is supported by the National NaturalScience Foundation of China under Grant no 11204099and self-determined research funds of CCNU from thecolleges basic research and operation of MOE Weiping Sunis supported by the National Natural Science Foundation ofChina under Grant no 61300140
References
[1] M Piccardi ldquoBackground subtraction techniques a reviewrdquo inProceedings of the IEEE International Conference on SystemsMan and Cybernetics (SMC rsquo04) pp 3099ndash3104 October 2004
[2] A Sanin C Sanderson and B C Lovell ldquoShadow detection asurvey and comparative evaluation of recent methodsrdquo PatternRecognition vol 45 no 4 pp 1684ndash1695 2012
[3] N Al-Najdawi H E Bez J Singhai and E A Edirisinghe ldquoAsurvey of cast shadow detection algorithmsrdquo Pattern Recogni-tion Letters vol 33 no 6 pp 752ndash764 2012
[4] A Prati I Mikic M M Trivedi and R Cucchiara ldquoDetectingmoving shadows algorithms and evaluationrdquo IEEE Transac-tions on Pattern Analysis and Machine Intelligence vol 25 no7 pp 918ndash923 2003
[5] Z Zivkovic and F van der Heijden ldquoEfficient adaptive densityestimation per image pixel for the task of background subtrac-tionrdquo Pattern Recognition Letters vol 27 no 7 pp 773ndash7802006
10 Mathematical Problems in Engineering
[6] A J Joshi and N P Papanikolopoulos ldquoLearning to detectmoving shadows in dynamic environmentsrdquo IEEE Transactionson Pattern Analysis and Machine Intelligence vol 30 no 11 pp2055ndash2063 2008
[7] C R Jung ldquoEfficient background subtraction and shadowremoval for monochromatic video sequencesrdquo IEEE Transac-tions on Multimedia vol 11 no 3 pp 571ndash577 2009
[8] M Shoaib R Dragon and J Ostermann ldquoShadow detection formoving humans using gradient-based background subtractionrdquoin Proceedings of the IEEE International Conference onAcousticsSpeech and Signal Processing (ICASSP rsquo09) pp 773ndash776 April2009
[9] R Cucchiara C Grana M Piccardi and A Prati ldquoDetectingmoving objects ghosts and shadows in video streamsrdquo IEEETransactions on Pattern Analysis and Machine Intelligence vol25 no 10 pp 1337ndash1342 2003
[10] E Salvador A Cavallaro and T Ebrahimi ldquoCast shadowsegmentation using invariant color featuresrdquo Computer Visionand Image Understanding vol 95 no 2 pp 238ndash259 2004
[11] D Grest J M Frahm and R Koch ldquoA color similarity measurefor robust shadow removal in real timerdquo in Proceedings of theVision Modeling and Visualization (VMV rsquo03) pp 253ndash260November 2003
[12] THorprasert DHarwood and L Davis ldquoA statistical approachfor real-time robust background subtraction and shadow detec-tionrdquo in Proceedings of the IEEE International Conference onComputer Vision (ICCV rsquo99) Frame Rate Workshop 1999
[13] S Nadimi and B Bhanu ldquoPhysical models for moving shadowand object detection in videordquo IEEE Transactions on PatternAnalysis and Machine Intelligence vol 26 no 8 pp 1079ndash10872004
[14] J B Huang and C S Chen ldquoMoving cast shadow detec-tion using Physics-based featuresrdquo in Proceedings of the IEEEComputer Society Conference on Computer Vision and PatternRecognition Workshops (CVPR rsquo09) pp 2310ndash2317 June 2009
[15] N Martel-Brisson and A Zaccarin ldquoKernel-based learning ofcast shadows fromaphysicalmodel of light sources and surfacesfor low-level segmentationrdquo in Proceedings of the 26th IEEEConference on Computer Vision and Pattern Recognition (CVPRrsquo08) pp 1ndash8 June 2008
[16] C C Chen and J K Aggarwal ldquoHuman shadow removal withunknown light sourcerdquo in Proceedings of the 20th InternationalConference on Pattern Recognition (ICPR rsquo10) pp 2407ndash2410August 2010
[17] J W Hsieh W F Hu C J Chang and Y S Chen ldquoShadowelimination for effective moving object detection by Gaussianshadow modelingrdquo Image and Vision Computing vol 21 no 6pp 505ndash516 2003
[18] H Nicolas and J M Pinel ldquoJoint moving cast shadows seg-mentation and light source detection in video sequencesrdquo SignalProcessing Image Communication vol 21 no 1 pp 22ndash43 2006
[19] A Leone andCDistante ldquoShadowdetection formoving objectsbased on texture analysisrdquoPattern Recognition vol 40 no 4 pp1222ndash1233 2007
[20] W Zhang X Z Fang X K Yang and Q M J Wu ldquoMovingcast shadows detection using ratio edgerdquo IEEE Transactions onMultimedia vol 9 no 6 pp 1202ndash1213 2007
[21] M Xiao C Z Han and L Zhang ldquoMoving shadow detectionand removal for traffic sequencesrdquo International Journal ofAutomation and Computing vol 4 no 1 pp 38ndash46 2007
[22] J Stauder R Mech and J Ostermann ldquoDetection of movingcast shadows for object segmentationrdquo IEEE Transactions onMultimedia vol 1 no 1 pp 65ndash76 1999
[23] M S Andersen T Jensen and C B Madsen ldquoEstimationof dynamic light changes in outdoor scenes without the useof calibration objectsrdquo in Proceedings of the 18th InternationalConference on Pattern Recognition (ICPR rsquo06) vol 4 pp 91ndash94August 2006
[24] D S Lee ldquoEffective Gaussian mixture learning for videobackground subtractionrdquo IEEETransactions on PatternAnalysisand Machine Intelligence vol 27 no 5 pp 827ndash832 2005
[25] M Heikkila and M Pietikainen ldquoA texture-based method formodeling the background and detecting moving objectsrdquo IEEETransactions on Pattern Analysis and Machine Intelligence vol28 no 4 pp 657ndash662 2006
[26] M Xu and T Ellis ldquoColour-invariant motion detection underfast illumination changesrdquo in Proceedings of the 2nd EuropeanWorkshop on Advanced Video-Based Surveillance Systems vol4 pp 101ndash112 September 2001
[27] J Pilet C Strecha and P Fua ldquoMaking background subtractionrobust to sudden illumination changesrdquo in Proceedings of the10th European Conference on Computer Vision (ECCV rsquo08) pp567ndash580 October 2008
[28] J Choi H J Chang Y J Yoo and J Y Choi ldquoRobust movingobject detection against fast illumination changerdquo ComputerVision and Image Understanding vol 116 no 2 pp 179ndash1932012
Figure 5 represents compared results of the LIRMmovingobject detection via GMM and postprocess From Figure 5the postprocess obviously improves the shadow discrimi-nation rate and slightly reduces the shadow detection rateHowever the average of the two rates is increased obviouslySo the postprocess enhances the moving object detectionresult
Finally we show that the proposedmethod rapidly adaptsto variations in environment To test moving object detectionunder illumination changes condition we compare the resultof other techniques which are GMM [5] adaptive GMM [24]that controls the learning rate of the GMM adaptively LBP[25] that detects moving objects using texture and edges Xursquosmethods [26] that use a color chromaticity Piletrsquos methods[27] that use illumination and spatial likelihood and Choirsquosmethods [28] that develop chromaticity differencemodel andbrightness model that estimates the intensity difference andintensity ratio of false foreground pixels For the quantitativecomparison we have made three test videos [28] Figure 7represents moving object detection results of each methodFrom Figure 7 we find the postprocessing of the proposedmethod can strengthen the moving objects and remove theshadow patches and the noise but it sometimes enhances theedge of part shadows incorrectly and makes moving objectsrsquoedge fuzzy In a word the proposed method adapts to thechanged background completely and detects moving objectpixels without shadow pixels successfully Although someother methods are robust to illumination their detectionmoving pixels results show that they cannot detect movingobject pixels meanwhile removing shadow
5 Conclusions
In this paper we propose a method to detect movingobject meanwhile removing cast shadow which is robust inchanging illumination condition This paper presents a localintensity ratio model according to illumination model andproves its illumination invariance Meanwhile if the noisein video obeys the Gaussian distribution the local intensityratio also obeys the Gaussian distribution In the process ofGaussian mixture model to obtain foreground this paperreplaces actual pixel value by local intensity ratio Finallypostprocess methods are used to get pure moving objectpixels without noise such as erosion method to removeshadow pitch and noise which is false detected as foregroundforeground contour enhanced method to strengthen moving
object contour and filling methods to deal with holes inforeground object Experiment results demonstrate that themethod presented by this paper can eliminate shadow onforeground effectively and is robust in changing illuminationconditionHowever in some scenes that foreground is similarto background or foreground is similar to shadow fore-groundmay be easily detected as background If illuminationchanges severely such as turn onoff light in indoor scenethe backgroundmay be easily detected as foreground and theperformance drops
Conflict of Interests
The authors declare that there is no conflict of interestsregarding the publication of this paper
Acknowledgments
Jinhai Xiang is supported by the Fundamental ResearchFunds for the Central Universities under Grant no2013QC024 Jun Xu is supported by the National NaturalScience Foundation of China under Grant no 11204099and self-determined research funds of CCNU from thecolleges basic research and operation of MOE Weiping Sunis supported by the National Natural Science Foundation ofChina under Grant no 61300140
References
[1] M Piccardi ldquoBackground subtraction techniques a reviewrdquo inProceedings of the IEEE International Conference on SystemsMan and Cybernetics (SMC rsquo04) pp 3099ndash3104 October 2004
[2] A Sanin C Sanderson and B C Lovell ldquoShadow detection asurvey and comparative evaluation of recent methodsrdquo PatternRecognition vol 45 no 4 pp 1684ndash1695 2012
[3] N Al-Najdawi H E Bez J Singhai and E A Edirisinghe ldquoAsurvey of cast shadow detection algorithmsrdquo Pattern Recogni-tion Letters vol 33 no 6 pp 752ndash764 2012
[4] A Prati I Mikic M M Trivedi and R Cucchiara ldquoDetectingmoving shadows algorithms and evaluationrdquo IEEE Transac-tions on Pattern Analysis and Machine Intelligence vol 25 no7 pp 918ndash923 2003
[5] Z Zivkovic and F van der Heijden ldquoEfficient adaptive densityestimation per image pixel for the task of background subtrac-tionrdquo Pattern Recognition Letters vol 27 no 7 pp 773ndash7802006
10 Mathematical Problems in Engineering
[6] A J Joshi and N P Papanikolopoulos ldquoLearning to detectmoving shadows in dynamic environmentsrdquo IEEE Transactionson Pattern Analysis and Machine Intelligence vol 30 no 11 pp2055ndash2063 2008
[7] C R Jung ldquoEfficient background subtraction and shadowremoval for monochromatic video sequencesrdquo IEEE Transac-tions on Multimedia vol 11 no 3 pp 571ndash577 2009
[8] M Shoaib R Dragon and J Ostermann ldquoShadow detection formoving humans using gradient-based background subtractionrdquoin Proceedings of the IEEE International Conference onAcousticsSpeech and Signal Processing (ICASSP rsquo09) pp 773ndash776 April2009
[9] R Cucchiara C Grana M Piccardi and A Prati ldquoDetectingmoving objects ghosts and shadows in video streamsrdquo IEEETransactions on Pattern Analysis and Machine Intelligence vol25 no 10 pp 1337ndash1342 2003
[10] E Salvador A Cavallaro and T Ebrahimi ldquoCast shadowsegmentation using invariant color featuresrdquo Computer Visionand Image Understanding vol 95 no 2 pp 238ndash259 2004
[11] D Grest J M Frahm and R Koch ldquoA color similarity measurefor robust shadow removal in real timerdquo in Proceedings of theVision Modeling and Visualization (VMV rsquo03) pp 253ndash260November 2003
[12] THorprasert DHarwood and L Davis ldquoA statistical approachfor real-time robust background subtraction and shadow detec-tionrdquo in Proceedings of the IEEE International Conference onComputer Vision (ICCV rsquo99) Frame Rate Workshop 1999
[13] S Nadimi and B Bhanu ldquoPhysical models for moving shadowand object detection in videordquo IEEE Transactions on PatternAnalysis and Machine Intelligence vol 26 no 8 pp 1079ndash10872004
[14] J B Huang and C S Chen ldquoMoving cast shadow detec-tion using Physics-based featuresrdquo in Proceedings of the IEEEComputer Society Conference on Computer Vision and PatternRecognition Workshops (CVPR rsquo09) pp 2310ndash2317 June 2009
[15] N Martel-Brisson and A Zaccarin ldquoKernel-based learning ofcast shadows fromaphysicalmodel of light sources and surfacesfor low-level segmentationrdquo in Proceedings of the 26th IEEEConference on Computer Vision and Pattern Recognition (CVPRrsquo08) pp 1ndash8 June 2008
[16] C C Chen and J K Aggarwal ldquoHuman shadow removal withunknown light sourcerdquo in Proceedings of the 20th InternationalConference on Pattern Recognition (ICPR rsquo10) pp 2407ndash2410August 2010
[17] J W Hsieh W F Hu C J Chang and Y S Chen ldquoShadowelimination for effective moving object detection by Gaussianshadow modelingrdquo Image and Vision Computing vol 21 no 6pp 505ndash516 2003
[18] H Nicolas and J M Pinel ldquoJoint moving cast shadows seg-mentation and light source detection in video sequencesrdquo SignalProcessing Image Communication vol 21 no 1 pp 22ndash43 2006
[19] A Leone andCDistante ldquoShadowdetection formoving objectsbased on texture analysisrdquoPattern Recognition vol 40 no 4 pp1222ndash1233 2007
[20] W Zhang X Z Fang X K Yang and Q M J Wu ldquoMovingcast shadows detection using ratio edgerdquo IEEE Transactions onMultimedia vol 9 no 6 pp 1202ndash1213 2007
[21] M Xiao C Z Han and L Zhang ldquoMoving shadow detectionand removal for traffic sequencesrdquo International Journal ofAutomation and Computing vol 4 no 1 pp 38ndash46 2007
[22] J Stauder R Mech and J Ostermann ldquoDetection of movingcast shadows for object segmentationrdquo IEEE Transactions onMultimedia vol 1 no 1 pp 65ndash76 1999
[23] M S Andersen T Jensen and C B Madsen ldquoEstimationof dynamic light changes in outdoor scenes without the useof calibration objectsrdquo in Proceedings of the 18th InternationalConference on Pattern Recognition (ICPR rsquo06) vol 4 pp 91ndash94August 2006
[24] D S Lee ldquoEffective Gaussian mixture learning for videobackground subtractionrdquo IEEETransactions on PatternAnalysisand Machine Intelligence vol 27 no 5 pp 827ndash832 2005
[25] M Heikkila and M Pietikainen ldquoA texture-based method formodeling the background and detecting moving objectsrdquo IEEETransactions on Pattern Analysis and Machine Intelligence vol28 no 4 pp 657ndash662 2006
[26] M Xu and T Ellis ldquoColour-invariant motion detection underfast illumination changesrdquo in Proceedings of the 2nd EuropeanWorkshop on Advanced Video-Based Surveillance Systems vol4 pp 101ndash112 September 2001
[27] J Pilet C Strecha and P Fua ldquoMaking background subtractionrobust to sudden illumination changesrdquo in Proceedings of the10th European Conference on Computer Vision (ECCV rsquo08) pp567ndash580 October 2008
[28] J Choi H J Chang Y J Yoo and J Y Choi ldquoRobust movingobject detection against fast illumination changerdquo ComputerVision and Image Understanding vol 116 no 2 pp 179ndash1932012
[6] A J Joshi and N P Papanikolopoulos ldquoLearning to detectmoving shadows in dynamic environmentsrdquo IEEE Transactionson Pattern Analysis and Machine Intelligence vol 30 no 11 pp2055ndash2063 2008
[7] C R Jung ldquoEfficient background subtraction and shadowremoval for monochromatic video sequencesrdquo IEEE Transac-tions on Multimedia vol 11 no 3 pp 571ndash577 2009
[8] M Shoaib R Dragon and J Ostermann ldquoShadow detection formoving humans using gradient-based background subtractionrdquoin Proceedings of the IEEE International Conference onAcousticsSpeech and Signal Processing (ICASSP rsquo09) pp 773ndash776 April2009
[9] R Cucchiara C Grana M Piccardi and A Prati ldquoDetectingmoving objects ghosts and shadows in video streamsrdquo IEEETransactions on Pattern Analysis and Machine Intelligence vol25 no 10 pp 1337ndash1342 2003
[10] E Salvador A Cavallaro and T Ebrahimi ldquoCast shadowsegmentation using invariant color featuresrdquo Computer Visionand Image Understanding vol 95 no 2 pp 238ndash259 2004
[11] D Grest J M Frahm and R Koch ldquoA color similarity measurefor robust shadow removal in real timerdquo in Proceedings of theVision Modeling and Visualization (VMV rsquo03) pp 253ndash260November 2003
[12] THorprasert DHarwood and L Davis ldquoA statistical approachfor real-time robust background subtraction and shadow detec-tionrdquo in Proceedings of the IEEE International Conference onComputer Vision (ICCV rsquo99) Frame Rate Workshop 1999
[13] S Nadimi and B Bhanu ldquoPhysical models for moving shadowand object detection in videordquo IEEE Transactions on PatternAnalysis and Machine Intelligence vol 26 no 8 pp 1079ndash10872004
[14] J B Huang and C S Chen ldquoMoving cast shadow detec-tion using Physics-based featuresrdquo in Proceedings of the IEEEComputer Society Conference on Computer Vision and PatternRecognition Workshops (CVPR rsquo09) pp 2310ndash2317 June 2009
[15] N Martel-Brisson and A Zaccarin ldquoKernel-based learning ofcast shadows fromaphysicalmodel of light sources and surfacesfor low-level segmentationrdquo in Proceedings of the 26th IEEEConference on Computer Vision and Pattern Recognition (CVPRrsquo08) pp 1ndash8 June 2008
[16] C C Chen and J K Aggarwal ldquoHuman shadow removal withunknown light sourcerdquo in Proceedings of the 20th InternationalConference on Pattern Recognition (ICPR rsquo10) pp 2407ndash2410August 2010
[17] J W Hsieh W F Hu C J Chang and Y S Chen ldquoShadowelimination for effective moving object detection by Gaussianshadow modelingrdquo Image and Vision Computing vol 21 no 6pp 505ndash516 2003
[18] H Nicolas and J M Pinel ldquoJoint moving cast shadows seg-mentation and light source detection in video sequencesrdquo SignalProcessing Image Communication vol 21 no 1 pp 22ndash43 2006
[19] A Leone andCDistante ldquoShadowdetection formoving objectsbased on texture analysisrdquoPattern Recognition vol 40 no 4 pp1222ndash1233 2007
[20] W Zhang X Z Fang X K Yang and Q M J Wu ldquoMovingcast shadows detection using ratio edgerdquo IEEE Transactions onMultimedia vol 9 no 6 pp 1202ndash1213 2007
[21] M Xiao C Z Han and L Zhang ldquoMoving shadow detectionand removal for traffic sequencesrdquo International Journal ofAutomation and Computing vol 4 no 1 pp 38ndash46 2007
[22] J Stauder R Mech and J Ostermann ldquoDetection of movingcast shadows for object segmentationrdquo IEEE Transactions onMultimedia vol 1 no 1 pp 65ndash76 1999
[23] M S Andersen T Jensen and C B Madsen ldquoEstimationof dynamic light changes in outdoor scenes without the useof calibration objectsrdquo in Proceedings of the 18th InternationalConference on Pattern Recognition (ICPR rsquo06) vol 4 pp 91ndash94August 2006
[24] D S Lee ldquoEffective Gaussian mixture learning for videobackground subtractionrdquo IEEETransactions on PatternAnalysisand Machine Intelligence vol 27 no 5 pp 827ndash832 2005
[25] M Heikkila and M Pietikainen ldquoA texture-based method formodeling the background and detecting moving objectsrdquo IEEETransactions on Pattern Analysis and Machine Intelligence vol28 no 4 pp 657ndash662 2006
[26] M Xu and T Ellis ldquoColour-invariant motion detection underfast illumination changesrdquo in Proceedings of the 2nd EuropeanWorkshop on Advanced Video-Based Surveillance Systems vol4 pp 101ndash112 September 2001
[27] J Pilet C Strecha and P Fua ldquoMaking background subtractionrobust to sudden illumination changesrdquo in Proceedings of the10th European Conference on Computer Vision (ECCV rsquo08) pp567ndash580 October 2008
[28] J Choi H J Chang Y J Yoo and J Y Choi ldquoRobust movingobject detection against fast illumination changerdquo ComputerVision and Image Understanding vol 116 no 2 pp 179ndash1932012