Classification-Based Color Constancy Cusano-Visual-08

8/14/2019 Classification-Based Color Constancy Cusano-Visual-08

1/19

Classification-basedColor Constancy

S. Bianco, G. Ciocca,C. Cusano, R. Schettini

www.ivl.disco.unimib.it


2/19

Color constancy on digital devices

The Human Visual System is(almost) able to compensate for

illuminants (color constancy)

People expect Digital ImagingAcquisition systems to do the same

Computational color constancytries to emulate this HVS feature ondigital devices


3/19

Illuminant estimation

Color correction

Computational color constancy

A popular approach adopts a two stageprocedure: the scene illuminant is estimated from the image data image colors are then corrected on the basis of this

estimate

R,G,B

IR,IG,IBR = R / IR

G = G / IGB = B / IB


4/19

Illuminant estimation

An ill-posed problem

Algorithms usually exploit some assumptions aboutstatistical properties of expected illuminants or of theobjects reflectances

Original image Gray world White Point


5/19

Our approach

Improve illuminant estimation by taking intoaccount automatically extracted informationabout the content of the images

We considered indoor/outdoor classification because

Indoor/outdoor images present different content

Are usually taken under different illumination conditions


6/19

Our approach

We have designed different strategies for theselection of the most appropriate algorithm (orcombination of algorithms) for each class

Image

Classifier Outdoor

Indoor Best Indoor Algorithm

Best Outdoor Algorithm

Image

Classifier

Outdoor

Indoor Best Indoor Algorithm

Best Outdoor Algorithm

Best Global Algorithm

PT1

P

T2

yes

yes


7/19

Dataset

The Ciurea and Funt image database

15 video clips at 15 fps

More than 11000 frames

A gray sphere is used to estimate the illuminant color

Video summarization techniques are used to select 1135uncorrelated images

F. Ciurea, B. Funt, A LargeImage Database for Color

Constancy Research, Proc.IS&T/SID 11th Color ImagingConf., pp. 160-164, 2003.


8/19

44444444 21

... ...

O I O O

O

Images to be

classified

CART

trees

Majority vote

Feature extraction

Image classification

I


9/19

Image description

Images are described by a set of low-level visualfeatures

General purpose (i.e. not specifically related toindoor/outdoor classification)

Easy (and fast) to compute

Feature vectors of 107 components are used

Color moments in the YCbCr color space (7 x 2 x 3) RGB Histogram (27 bins)

Edge direction histogram (18 bins)

Wavelets statistics (10 x 2)


10/19

Image classification

A forest of 50 trees has been trained usingbootstrap replicates of a training set composedby 6785 images downloaded from the web (2105indoor and 4680 outdoor)

No enhancement procedure (such as white balancing)has been applied to the images

We obtained a classification accuracy of 93.1% on an independent validation set

85.1% on the Ciurea and Funt dataset


11/19

Some misclassified images


12/19

Color constancy algorithms

We considered the framework proposed by Vande Weijer et al.

We selected six, widely used, algorithms

Gray World (GW): n = 0, p = 1, = 0 White Point (WP): n = 0, p = , = 0

Shades of Gray (SG): n = 0, = 0

General Gray World (gGW): n = 0

Gray Edge (GE1): n = 1 Second Order Gray Edge (GE2): n = 2

Some algorithms require a tuning for theparameters p and

J. van de Weijer, T. Gevers, A. Gijsenij,

Edge-based Color Constancy, IEEE Trans.on Image Processing, 16(9), pp. 22072214,2007


13/19

Color constancy algorithms

We considered also two combining algorithms:

Least Mean Squares (LMS): the output of the sixalgorithms are linearly combined (weights need to be

estimated)

No2Max: the two estimations with the highest distancefrom the others are discared. The remaining four areaveraged (S. Bianco, F. Gasparini, R. Schettini, A Consensus Based

Framework For Illuminant Chromaticity Estimation, J. of Electronic Imaging, 17,pp. 023013-19, 2008)


14/19

Experimental results

A training set of 300 images has been used to determine theoptimal parameters of the algorithms (via pattern search)

On the two classes

On the whole training set

15.5722.8164.32gGW

74.5022.7154.58LMS

65.0222.8335.13N2M

15.4772.4815.57GE2

15.4513.7215.40GE1

15.5622.8164.31SG

07.7622.81011.83WP

15.6207.8634.91GW

WSTMedianWSTMedianWSTMedian

Whole TrainingOutdoor ImagesIndoor Images


15/19

On the 835 images of the test set (331 indoor 504 outdoor)

Content Independent Strategy (CI): algorithms tuned on thewhole training set

Content Dependent Parameterization (CDP): class specificparameters, chosen on the basis of the output of the classifier


44.0574.18LMS

44.0144.79N2M

43.9454.65GE2

34.3254.47GE115.3905.80gGW

44.0805.80SG

35.4835.48WP

05.9505.95GWWSTMedianWSTMedian

CDP StrategyCI strategy


16/19


Algorithm selection

Class-Dependent Algorithms (CDA): for each class the bestalgorithm (and its corresponding parameters) is selected

Class-Dependent Algorithms with Uncertainty Class (CDAUC):introduction of the uncertainty class. Images falling in thatclass are processed by the algorithm that has proved to be thebest class-independent algorithm.

33.54SG ind., GE2 out., LMS uncertainCDAUC13.78SG for indoor and GE2 for outdoorCDA

13.94GE2, indoor and outdoor parametersCDP

04.18LMS, general purpose parametersCI

WSTMedianUnderlying algorithmsStrategy


17/19


Image misclassification approximately doublesangular errors

4.85 vs. 9.79 on indoor images

2.31 vs. 5.07 on outdoor images

How much improvement is expected using abetter classifier?

We obtained a median angular error of 3.48 degreesusing an optimal classifier

An error of 5.63 has been obtained using a randomclassifier


18/19

Summary of the experiments

If no knowledge of the image content is exploited(CI strategy), combining methods perform betterthan the single ones

The algorithms that can be tuned on the basis ofimage contents benefit by the classificationprocess

For the specific classes, combining methods donot seem to be the best choice

Illuminant estimation can be improved by usingdifferent algorithms (or a different set of

parameters) for indoor and outdoor images (CDPand CDA strategies)

The introduction of an uncertainty class (CDAUCstrategy), further improves the results


19/19

Future work

Collection of a large dataset with high contentand illuminant variability

The results can be improved using additionalclasses?

And using additional algorithms?

How knowledge about the acquisition device canbe incorporated in the framework?

Classification-Based Color Constancy Cusano-Visual-08

Documents