A Standardized Workflow for Illumination-Invariant Image Extraction Mark S. Drew Muntaseer Salahuddin Alireza Fathi Simon Fraser University, Vancouver,

A Standardized Workflow A Standardized Workflow for Illumination-Invariant for Illumination-Invariant

Image ExtractionImage Extraction

Mark S. DrewMark S. Drew

Muntaseer SalahuddinMuntaseer Salahuddin

Alireza FathiAlireza Fathi

Simon Fraser University, Vancouver, CanadaSimon Fraser University, Vancouver, Canada{mark,msalahud,alirezaf}@cs.sfu.ca{mark,msalahud,alirezaf}@cs.sfu.ca

www.cs.sfu.ca/~markwww.cs.sfu.ca/~mark

22

IntroductionIntroduction

Illumination-invariant image extraction is Illumination-invariant image extraction is an interesting and open problem in vision.an interesting and open problem in vision.

illustration shows the objective:illustration shows the objective:

(the “intrinsic image”)

33

Introduction (cont.)Introduction (cont.)

To obtain (b) from (a), we take the To obtain (b) from (a), we take the logarithm of band-ratio chromaticity colour logarithm of band-ratio chromaticity colour coordinates, and then project in a coordinates, and then project in a special special directiondirection [Finlayson and Hordley, 2001].[Finlayson and Hordley, 2001].

The resultant grey-scale image is The resultant grey-scale image is illumination invariant.illumination invariant.

44

Introduction (cont.)Introduction (cont.)

Objective: we argue that Objective: we argue that sharpening sRGBsharpening sRGB allows us to allows us to findfind the invariant image as a generic the invariant image as a generic workflow for images, from workflow for images, from unknown camerasunknown cameras unknown actual special directionunknown actual special direction no complex algorithm using evidence in each imageno complex algorithm using evidence in each image

Works well (but not as well as knowing the Works well (but not as well as knowing the camera or using internal evidence in image!)camera or using internal evidence in image!)

55

Illumination invariant is crucial step!Illumination invariant is crucial step!

Shadow RemovalShadow Removal

66

Finding directionFinding direction

The direction of projection is crucial.The direction of projection is crucial.

77

Finding direction…Finding direction… Could calibrate the camera to find the Could calibrate the camera to find the

invariant directioninvariant direction [Finlayson et al. (2002)]:[Finlayson et al. (2002)]:

HP912 Digital Still Camera: Log-chromaticities of 24 patches;6 patches, imaged under 9 illuminants.

88

Finding direction…Finding direction… Without calibrating the camera, can use entropy of Without calibrating the camera, can use entropy of

projection to find the invariant direction projection to find the invariant direction [Finlayson et al. (2004)][Finlayson et al. (2004)]::

Cor

rect

dire

ctio

n –

smal

ler e

ntro

py

Wro

ng d

irect

ion

– hi

gher

ent

ropy

Uses internal evidence in image.

99

This paper: This paper: Sharpening HelpsSharpening Helps

Argument at AIC05 Argument at AIC05 [Finlayson et al. 2005] [Finlayson et al. 2005] : : recommended that if we recommended that if we sharpen the sharpen the values in XYZ spacevalues in XYZ space, get better invariant., get better invariant.

HOWEVER…HOWEVER…

1010

Proposed Approach: Proposed Approach: Sharpen Sharpen sRGBsRGB

However, going from sRGB to XYZ is However, going from sRGB to XYZ is a broadening transform: a counter-a broadening transform: a counter-intuitive approach.intuitive approach.

Therefore we propose to sharpen the Therefore we propose to sharpen the sRGB space itself.sRGB space itself.

Works better!Works better!

1111

Old: Sharpen XYZ;Old: Sharpen XYZ;new: new: Sharpen sRGBSharpen sRGB

Old: Assume input is in nonlinear sRGB Old: Assume input is in nonlinear sRGB space; linearize; transform to XYZ; space; linearize; transform to XYZ; sharpen XYZ; chromaticity; project sharpen XYZ; chromaticity; project lighting-change direction.lighting-change direction.

NewNew: : Assume input is in nonlinear sRGB space; linearize; Assume input is in nonlinear sRGB space; linearize;

sharpen sRGBsharpen sRGBlinear linear using synthetic data, using synthetic data,

producing standardized transform for all imagesproducing standardized transform for all images ; ; chromaticity; project chromaticity; project lighting-change direction. lighting-change direction.

1212

sRGB to XYZ is a Broadening sRGB to XYZ is a Broadening TransformTransform

350 400 450 500 550 600 650 700 750 800 850-0.5

0

0.5

1

1.5

2

2.5

XYZ

RGB

350 400 450 500 550 600 650 700 750 800 850-0.5

0

0.5

1

1.5

2

2.5

3

3.5

XYZ

RGB

350 400 450 500 550 600 650 700 750 800 850-0.2

0

0.2

0.4

0.6

0.8

1

1.2

XYZ

RGB

1313

Comparing XYZ to sRGB: Comparing XYZ to sRGB: no sharpeningno sharpening

Log-chromaticity coordinates for Macbeth patches, as light changes.

XYZ sRGB

1414

Comparing Comparing sharpeningsharpening XYZ to XYZ to sharpeningsharpening RGB RGB

After Sharpening

XYZR = 0.764 (with Mean Subtraction)

sRGBR = 0.920 √

Synthetic: Macbeth + Planckians

1515

Standard colour transformStandard colour transform

Colour transform is “data-based” Colour transform is “data-based” sharpening, optimizing lighting-invariance sharpening, optimizing lighting-invariance of output (with positivity enforced) of output (with positivity enforced) [Drew et al. [Drew et al. 2002]2002]

The transformation matrix The transformation matrix TT that does so that does so is the following,is the following,

1616

Standard colour transform…Standard colour transform…

Sharpen, form chromaticities, then project Sharpen, form chromaticities, then project in in standard directionstandard direction

1717

The Standardized WorkflowThe Standardized Workflow

Apply Standardized method to Apply Standardized method to measured chart datameasured chart data

best fit - - - -

standardized sRGB sharpening ───

105 illuminants, Nikon D70

Macbeth chart, under 14 different daylights. HP912 camera.

Apply Standardized method: invariantApply Standardized method: invariant

Invariant image, formed by calibrating camera.

Av. of Std. Dev. across illuminants= 4.42%

Best possible:Best possible:

Invariant image, formed by sharpening sRGB.

Av. of Std. Dev. across illuminants= 6.11% not as good as calibrated version, of course! but usable.

Standardized method:Standardized method:

Standardized method:Standardized method:input chromaticity, segmented for display

output, segmented

shadow gone√

2323

ConclusionConclusion

The sharpening transform does a good The sharpening transform does a good enough job finding an invariant, given that enough job finding an invariant, given that it does not depend on any information it does not depend on any information specific to the camera or even the image.specific to the camera or even the image.

It can serve as a preprocessing step to It can serve as a preprocessing step to many different vision problems.many different vision problems.

2424

Thanks!To Natural Sciences and

Engineering Research Council of Canada