Dec 21, 2015
04/18/23 CIC 10, 2002 2
Color constancy at a pixel [Finlayson et al. CIC8, 2000]
Idea: plot log(R/G) vs. log(B/G):
14 daylights14 daylights
24 p
atch
es24
pat
ches
04/18/23 CIC 10, 2002 3
Log(R/G)
Log(R/G)
Log(B
/G)
Log
(B/G
)
For every patch, the direction from light color change is about the same!
04/18/23 CIC 10, 2002 4
Why all linear and same direction?
color shading
intensitylight SPD
reflectancesensor
k=1..3
Now let’s make some assumptions:
The image formation equation:The image formation equation:
04/18/23 CIC 10, 2002 5
Assumption 1Assumption 1: Light is ~ Planckian: Light is ~ Planckian (or some other 1D assumption)
Wien’s approximation of a Planckian source:
Note: 1D parameter: T == temperature == light color.
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
r/(r+g+b)
g/(
r+g
+b
)
Illuminant Chromaticities
P100
6
Assumption 2: Narrow band sensorsAssumption 2: Narrow band sensors
)()( kkk qq
SONY DXC-930
The Sony Camera has fairly narrow band sensitivities
Using spectral sharpening, we can make almost all sensor sets have this property.
[Finlayson, Drew, Funt]
T
ccSI
kkkk
251 ))(ln(
~lnln
Modified Image FormationModified Image Formation
T
c
kkkkecSI 2
51)(
~
dqSEI kk )()()(~
The kth response
Substituting Narrow-band and Planckian Assumptions
Take logs
Response = light intensity + surface + light color
II )(~
, na
8
ImplicationsImplications
T
ccSI
kkkk
251 ))(ln(
~lnln
We have k equations of the form:
I~
ln is common to all equations and can be removed bysimple differencing at this pixel
This results in k-1 independent equations of the form
T
kjkjkj
,,/ln
reflectanceterm
light colorterm
04/18/23 CIC 10, 2002 9
ImplicationsImplications
The log chromaticities of 7 surfacesviewed under 10 lights
T
kjkjkj
,,/ln
(1) If there are 3 sensors wehave two independent equations
of this form:
(2) For a single surface viewed underdifferent colored lights the log
chromaticities must fall on a line:
(3) Different surfaces induce lines withthe *same* orientation
04/18/23 CIC 10, 2002 10
Luminance
1D invariant
Gray
One degree of freedom is invariantOne degree of freedom is invariantto light changeto light change
11
More formally:
and define
form ratios
define vectors
line in 2D
12
What is this good for?
With certain restrictions, from a 3-band color image we can derive a 1-D
grayscale image which is:
- illuminant invariant
- and so, shadow free
13
Then use edge info. to integrate back without shadows [ECCV02 Finlayson, Hordley, and Drew]
These are approximately the same, except that the invariant edge map has
no shadow edges
14
Other tasks: Tracking, etc.
Tracking result for moving hand under lamp light.[Jiang and Drew, 2003]
04/18/23 CIC 10, 2002 15
But problem: doesn’t always remove all shadows:
Depends on camera sensors
16
How do we find light color change direction?
Sony DXC-930 camera
Mean-subtracted log-chromaticity
(Use robust line-finder)
04/18/23 CIC 10, 2002 17
Problem: invariant image isn’t invariant across illuminants
04/18/23 CIC 10, 2002 18
Gets worse: Kodak DCS420 camera is much less sharp
04/18/23 CIC 10, 2002 19
How to proceed? Try spectral sharpening, since wish to make sensors more narrowband….Or just optimize directly, making invariant image more invariant.
E.g. optimize color-matching functions :
)(),(),( zyx
04/18/23 20
Invariant image for patches apply optimized sensors to any image
Before optimization of sensors After optimization of sensors
21
How to optimize?Firstly, let’s use a linear matrixing transform, taking 31 x 3 sensor matrix Q to a new sensor set:
Should we sharpen to get M?
sensors colors
3 x 3
22
Should we sharpen to get M?
There’s a problem: If we made sensors that were all the same, the definition
makes the invariant go to zero… The more the sensors are alike, the “better”
Sharpening & flattening both work…
04/18/23 CIC 10, 2002 23
So need to use a term to steer away from a rank-reduced M
Optimize on the (correlation coefficient)2 R 2
and encourage high effective rank
are singular values of M
Initialize with data-driven spectral sharpening matrix.
04/18/23 CIC 10, 2002 24
So optimize M:
E.g., color-matching functions: R2 goes from 0.43 to 0.94
04/18/23 CIC 10, 2002 25
HP912 camera:
R2 : 0.86 0.93
entropy : 5.856 5.590 bits/pixel
26
Real image:
entropy 5.295 4.939 bits/pixel with an M
27
Than
ks!