8/14/2019 Classification-Based Color Constancy Cusano-Visual-08
1/19
Classification-basedColor Constancy
S. Bianco, G. Ciocca,C. Cusano, R. Schettini
www.ivl.disco.unimib.it
8/14/2019 Classification-Based Color Constancy Cusano-Visual-08
2/19
Color constancy on digital devices
The Human Visual System is(almost) able to compensate for
illuminants (color constancy)
People expect Digital ImagingAcquisition systems to do the same
Computational color constancytries to emulate this HVS feature ondigital devices
8/14/2019 Classification-Based Color Constancy Cusano-Visual-08
3/19
Illuminant estimation
Color correction
Computational color constancy
A popular approach adopts a two stageprocedure: the scene illuminant is estimated from the image data image colors are then corrected on the basis of this
estimate
R,G,B
IR,IG,IBR = R / IR
G = G / IGB = B / IB
8/14/2019 Classification-Based Color Constancy Cusano-Visual-08
4/19
Illuminant estimation
An ill-posed problem
Algorithms usually exploit some assumptions aboutstatistical properties of expected illuminants or of theobjects reflectances
Original image Gray world White Point
8/14/2019 Classification-Based Color Constancy Cusano-Visual-08
5/19
Our approach
Improve illuminant estimation by taking intoaccount automatically extracted informationabout the content of the images
We considered indoor/outdoor classification because
Indoor/outdoor images present different content
Are usually taken under different illumination conditions
8/14/2019 Classification-Based Color Constancy Cusano-Visual-08
6/19
Our approach
We have designed different strategies for theselection of the most appropriate algorithm (orcombination of algorithms) for each class
Image
Classifier Outdoor
Indoor Best Indoor Algorithm
Best Outdoor Algorithm
Image
Classifier
Outdoor
Indoor Best Indoor Algorithm
Best Outdoor Algorithm
Best Global Algorithm
PT1
P
T2
yes
yes
8/14/2019 Classification-Based Color Constancy Cusano-Visual-08
7/19
Dataset
The Ciurea and Funt image database
15 video clips at 15 fps
More than 11000 frames
A gray sphere is used to estimate the illuminant color
Video summarization techniques are used to select 1135uncorrelated images
F. Ciurea, B. Funt, A LargeImage Database for Color
Constancy Research, Proc.IS&T/SID 11th Color ImagingConf., pp. 160-164, 2003.
8/14/2019 Classification-Based Color Constancy Cusano-Visual-08
8/19
44444444 21
... ...
O I O O
O
Images to be
classified
CART
trees
Majority vote
Feature extraction
Image classification
I
8/14/2019 Classification-Based Color Constancy Cusano-Visual-08
9/19
Image description
Images are described by a set of low-level visualfeatures
General purpose (i.e. not specifically related toindoor/outdoor classification)
Easy (and fast) to compute
Feature vectors of 107 components are used
Color moments in the YCbCr color space (7 x 2 x 3) RGB Histogram (27 bins)
Edge direction histogram (18 bins)
Wavelets statistics (10 x 2)
8/14/2019 Classification-Based Color Constancy Cusano-Visual-08
10/19
Image classification
A forest of 50 trees has been trained usingbootstrap replicates of a training set composedby 6785 images downloaded from the web (2105indoor and 4680 outdoor)
No enhancement procedure (such as white balancing)has been applied to the images
We obtained a classification accuracy of 93.1% on an independent validation set
85.1% on the Ciurea and Funt dataset
8/14/2019 Classification-Based Color Constancy Cusano-Visual-08
11/19
Some misclassified images
8/14/2019 Classification-Based Color Constancy Cusano-Visual-08
12/19
Color constancy algorithms
We considered the framework proposed by Vande Weijer et al.
We selected six, widely used, algorithms
Gray World (GW): n = 0, p = 1, = 0 White Point (WP): n = 0, p = , = 0
Shades of Gray (SG): n = 0, = 0
General Gray World (gGW): n = 0
Gray Edge (GE1): n = 1 Second Order Gray Edge (GE2): n = 2
Some algorithms require a tuning for theparameters p and
J. van de Weijer, T. Gevers, A. Gijsenij,
Edge-based Color Constancy, IEEE Trans.on Image Processing, 16(9), pp. 22072214,2007
8/14/2019 Classification-Based Color Constancy Cusano-Visual-08
13/19
Color constancy algorithms
We considered also two combining algorithms:
Least Mean Squares (LMS): the output of the sixalgorithms are linearly combined (weights need to be
estimated)
No2Max: the two estimations with the highest distancefrom the others are discared. The remaining four areaveraged (S. Bianco, F. Gasparini, R. Schettini, A Consensus Based
Framework For Illuminant Chromaticity Estimation, J. of Electronic Imaging, 17,pp. 023013-19, 2008)
8/14/2019 Classification-Based Color Constancy Cusano-Visual-08
14/19
Experimental results
A training set of 300 images has been used to determine theoptimal parameters of the algorithms (via pattern search)
On the two classes
On the whole training set
15.5722.8164.32gGW
74.5022.7154.58LMS
65.0222.8335.13N2M
15.4772.4815.57GE2
15.4513.7215.40GE1
15.5622.8164.31SG
07.7622.81011.83WP
15.6207.8634.91GW
WSTMedianWSTMedianWSTMedian
Whole TrainingOutdoor ImagesIndoor Images
8/14/2019 Classification-Based Color Constancy Cusano-Visual-08
15/19
On the 835 images of the test set (331 indoor 504 outdoor)
Content Independent Strategy (CI): algorithms tuned on thewhole training set
Content Dependent Parameterization (CDP): class specificparameters, chosen on the basis of the output of the classifier
Experimental results
44.0574.18LMS
44.0144.79N2M
43.9454.65GE2
34.3254.47GE115.3905.80gGW
44.0805.80SG
35.4835.48WP
05.9505.95GWWSTMedianWSTMedian
CDP StrategyCI strategy
8/14/2019 Classification-Based Color Constancy Cusano-Visual-08
16/19
Experimental results
Algorithm selection
Class-Dependent Algorithms (CDA): for each class the bestalgorithm (and its corresponding parameters) is selected
Class-Dependent Algorithms with Uncertainty Class (CDAUC):introduction of the uncertainty class. Images falling in thatclass are processed by the algorithm that has proved to be thebest class-independent algorithm.
33.54SG ind., GE2 out., LMS uncertainCDAUC13.78SG for indoor and GE2 for outdoorCDA
13.94GE2, indoor and outdoor parametersCDP
04.18LMS, general purpose parametersCI
WSTMedianUnderlying algorithmsStrategy
8/14/2019 Classification-Based Color Constancy Cusano-Visual-08
17/19
Experimental results
Image misclassification approximately doublesangular errors
4.85 vs. 9.79 on indoor images
2.31 vs. 5.07 on outdoor images
How much improvement is expected using abetter classifier?
We obtained a median angular error of 3.48 degreesusing an optimal classifier
An error of 5.63 has been obtained using a randomclassifier
8/14/2019 Classification-Based Color Constancy Cusano-Visual-08
18/19
Summary of the experiments
If no knowledge of the image content is exploited(CI strategy), combining methods perform betterthan the single ones
The algorithms that can be tuned on the basis ofimage contents benefit by the classificationprocess
For the specific classes, combining methods donot seem to be the best choice
Illuminant estimation can be improved by usingdifferent algorithms (or a different set of
parameters) for indoor and outdoor images (CDPand CDA strategies)
The introduction of an uncertainty class (CDAUCstrategy), further improves the results
8/14/2019 Classification-Based Color Constancy Cusano-Visual-08
19/19
Future work
Collection of a large dataset with high contentand illuminant variability
The results can be improved using additionalclasses?
And using additional algorithms?
How knowledge about the acquisition device canbe incorporated in the framework?