A Physical Approach to Moving Cast Shadow Detection Jia-Bin Huang and Chu-Song Chen [email protected], [email protected] Institute of Information Science Academia Sinica, Taipei, Taiwan April 23, 2009 1 / 26
Nov 10, 2014
A Physical Approach to Moving Cast ShadowDetection
Jia-Bin Huang and Chu-Song Chen
[email protected], [email protected]
Institute of Information ScienceAcademia Sinica, Taipei, Taiwan
April 23, 2009
1 / 26
Outline
1 Introduction
2 Related Works
3 Physical Model for Cast Shadows
4 Learning and Detecting Cast Shadows
5 Experimental Results
6 Conclusion and Future Work
2 / 26
Outline
1 Introduction
2 Related Works
3 Physical Model for Cast Shadows
4 Learning and Detecting Cast Shadows
5 Experimental Results
6 Conclusion and Future Work
3 / 26
Introduction
Motivation
Moving object detection is one of the most important taskin low-level vision.
Detecting moving cast shadows is one of the mostchallenging problems for accurate object detection in videostreams since shadow points are often misclassified asobject points.
Without careful consideration, cast shadows may introducesignificant error in segmentation, tracking, and recognition.
4 / 26
Introduction
The Cause of Cast Shadows
Light sources are partially or totally blocked by the foregroundobjects.
Why Detecting Cast Shadows Is Difficult?
1 Shadow points are detectable as foreground points andtypically differ significantly from the background.
2 Cast shadows have the same motion as the objectscasting them.
3 Shaded regions are usually connected with the foregroundobjects.
5 / 26
Outline
1 Introduction
2 Related Works
3 Physical Model for Cast Shadows
4 Learning and Detecting Cast Shadows
5 Experimental Results
6 Conclusion and Future Work
6 / 26
Related Works (1/2)
Previous Works (before 2003)
A Survey paper: [Prati et al. PAMI 2003]
Statistical parametric: [Mikic et al. ICPR 2000]
Statistical nonparametric: [Horprasert et al. ICCVWorkshop 1999]
Deterministic model-based: [Onoguchi ICPR 1998]
Deterministic nonmodel-based: [Cucchiara et al. PAMI2001]
Major Drawbacks
Need to explicitly tune the parameters for each scene.
Hard to adapt to the illumination conditions andenvironment changes.
7 / 26
Related Works (2/2)
Learning-based Approaches
Basic Idea: Learn cast shadow model from video sequences.
Shadow Flow, [Porikli et al. ICCV 2005]
Gaussian Mixture Shadow Modeling, [Martel-Brisson et al.PAMI 2007]
Combining Local and Global Features, [Liu et al. CVPR 07]
Learning Physical Model of Light Sources and Surfaces[Martel-Brisson et al. CVPR 2008]
Drawbacks
Most of them assume shadow values will attenuate linearlyalong the line between the value of the correspondingbackground and the origin.
Pixel-based models may suffer from slow learning due tothe lack of sufficient samples.
8 / 26
Outline
1 Introduction
2 Related Works
3 Physical Model for Cast Shadows
4 Learning and Detecting Cast Shadows
5 Experimental Results
6 Conclusion and Future Work
9 / 26
Main Idea
A general physics-based shadow modelDecompose light incident at the background surface intotwo classes
Direct light sources (e.g., sun)Ambient illumination (e.g., light scattered by the sky,colored light from nearby surfaces (color bleeding))
Suppose we have N light sources and M ambientillumination, then the intensity function of light:
E(λ) =
N∑n
Eincident,n(λ) +
M∑m
Eambient,m(λ).
10 / 26
Ambient Illuminations and Direct Light Sources
Lambertian model: camerasensor response gk(p) at point p
gk(p) =
∫E(λ, p)ρ(λ, p)Sk(λ)dλ.
E(λ, p) Intensity function of lightsources
ρ(λ, p) The reflectance of anobject surface
Sk(λ) Sensor spectral sensitivityfunction
11 / 26
Appearance Variation Under Cast Shadow
Part or total light sources are blocked by foregroundobjectsAmbient illumination may be slightly changed (from BGA toBG ′
A)
12 / 26
Color Feature Vector
We encode the difference vector between background andshadow value as our color feature.
xs,t(p) = [αt(p), θt(p), φt(p)]T (in spherical coordinate
system)
Illumination attenuation
αt(p) =||vt(p)||
||BGt(p)||
Angle information
θt(p) = arctan(vG
t (p)
vRt (p)
)
φt(p) = arccos(vB
t (p)
||vt(p)||)
13 / 26
Outline
1 Introduction
2 Related Works
3 Physical Model for Cast Shadows
4 Learning and Detecting Cast Shadows
5 Experimental Results
6 Conclusion and Future Work
14 / 26
Overview
1 Perform background subtraction to obtain foregroundcandidates (i.e., including real foreground and castshadows)
2 Apply weak shadow detector as a pre-filter to obtainshadow candidates (e.g., filter out those pixels whoseillumination values are larger than the correspondingbackground values)
3 For these shadow candidates, learn the color featurevector xs,t(p) using GMM over time
4 Detecting cast shadow using the learned cast shadowmodel
15 / 26
Weak Shadow Detector
Criterion for shadow candidates
rl(p) =‖BGt(p)‖
‖xt(p)‖ cos(ψ(p))
ψ(p) = arccos( 〈xt(p), BGt(p)〉
‖xt(p)‖‖BGt(p)‖
)
rmin < rl(p) < rmax, ψ(p) < ψmax
16 / 26
Incorporating Spatial Information
Prior Knowledge of Cast Shadows
Cast shadows would not enhance the spatial gradient intensity
Introduce ωt(p) as a confidence value of cast shadows
ωt(p) =ε + |∇(Bt(p))|
ε+ max{|∇(It(p))|, |∇(Bt(p))|},
where ε is a smooth term.
To accelerate the learning speed of the pixel-basedshadow model, take ωt(p) as confidence value to updateshadow model at pixel p
Penalize samples having larger gradient intensity thanbackground by lessening the learning rate
17 / 26
Detecting Shadows at Light/Shadow Border
Shadows at light/shadow border show different behaviorfrom shadows inside the shaded region
Solution: Detecting cast shadows only with angleinformation
(a)αt(p)
(b)θt(p)
(c)φt(p)
18 / 26
Outline
1 Introduction
2 Related Works
3 Physical Model for Cast Shadows
4 Learning and Detecting Cast Shadows
5 Experimental Results
6 Conclusion and Future Work
19 / 26
Qualitative Evaluation
(a) (b) (c) (d)
Figure: (a) Original images, (b) Background posterior probability, (c)Shadow posterior probability, and (d) Forground posterior probability
20 / 26
Quantitative Evaluation
Performance Evaluation Metrics [Prati et al. PAMI 2003]
Shadow Detection Rate η
η =TPS
TPS + FNS
Shadow Discriminative Rate ξ
ξ =TPF
TPF + FNF
Sequence Highway I Highway II HallwayMethod η% ξ% η% ξ% η% ξ%
Proposed 72.34 84.98 72.70 79.89 71.69 88.25Kernel 70.50 84.40 68.40 71.20 72.40 86.70
LGf 72.10 79.70 - - - -GMSM 63.30 71.30 58.51 44.40 60.50 87.00
21 / 26
Effect of Shadows at Shadow/Light border
(a) (b) (c)
Figure: Effect of shadows at shadow/light border (a) Original frame ofsequence “Highway I". (b)(c) Foreground posterior without/withconsidering shadows at shadow/light border.
22 / 26
Outline
1 Introduction
2 Related Works
3 Physical Model for Cast Shadows
4 Learning and Detecting Cast Shadows
5 Experimental Results
6 Conclusion and Future Work
23 / 26
Conclusion
Provide a better description for background surface valuevariation under cast shadow
Incorporate spatial information to accelerate the learning ofpixel-based shadow model
Take shadows at light/shadow border into consideration
24 / 26
Future Work
Derive physics-based features for building a global shadowmodel in a scene
Jia-Bin Huang and Chu-Song Chen, “Moving Cast ShadowDetection using Physics-based Features", CVPR 2009
Extend the physical model to handle more general cases(e.g., surface with specular reflection, spatial varingambient illumination, etc.)
25 / 26
The End
Thank you
26 / 26