Hashim Mir, Peter Xu, and Peter van Beek University of Waterloo.

An Extensive Empirical Evaluation of Focus Measures for Digital Photography

Hashim Mir, Peter Xu, and Peter van BeekUniversity of Waterloo

• Digital cameras are equipped with contrast-based autofocus

• Contrast-detection autofocus makes use of a focus measure• maps an image to a value that represents degree of focus

of image

• Speed and accuracy of autofocus crucial to user satisfaction• search for an image with maximal focus according to focus

measure• choice of focus measure is an important factor in

performance of autofocus

February 4, 2014 Focus Measures for Digital Photography 2

Introduction

• Previous studies on focus measures:• small number of benchmarks images in their evaluation,• directed at microscopy and not digital cameras, or• based on ad hoc evaluation criteria

• Here: • extensive empirical evaluation of focus measures for digital

photography• advocate using three standard statistical measures of

performance—precision, recall, and mean absolute error—as evaluation criteria


Introduction

• Introduction• Focus Measures• Related Work• Experimental Methodology• Experimental Results• Conclusion


Outline

• Many focus measures have been proposed in the literature

• Let f (x, y) be the luminance or grayscale at pixel (x, y) in an image of size M × N pixels

• Example: squared gradient focus measure for an image acquired when the lens is at position p is given by,


Focus measures

1 22

=0 =0

( ) = ( ( , 1) ( , ))M N

x y

p f x y f x y

(a) Focus measures of images at each lens position; (b) flower in focus


(a) (b)

(a) Focus measures of images at each lens position; (b) fern & grasses in focus


(a) (b)

• We evaluated more than 30 focus measures in our study

• The focus measures can be classified under six types:1. first-order differentiation2. second-order differentiation3. image histogram4. image statistics5. correlation6. data compression


Focus measures: Categorizing

• Example: squared gradient• Example: focus measure based on 3×3 Sobel

operators,


Focus measures: First-order differentiation

1 0 1 1 2 1

2 0 2 0 0 0

1 0 1 1 2 1

(horizontal) (vertical)

• Example: focus measure based on 3×3 Laplacian operator,


Focus measures: Second-order differentiation

1 1 1

1 8 1

1 1 1

• Example: variance focus measure is given by,


Focus measures: Image statistics

1 12

=0 =0

1( ) = ( ( , ) )

M N

x y

p f x yMN

where µ is the mean luminance of the image

• Example: Vollath’s F4 focus measure is given by,


Focus measures: Correlation

2 1

=0 =0

31 1

=0 =0

( ) = ( , ) ( 1, )

( , ) ( 2, )

M N

x y

M N

x y

p f x y f x y

f x y f x y


Related work: Evaluating focus measures

setting citation conclusions

Microscopy applications

Groen, Young & Ligthart, 1985

squared gradient

Firestone et al. 1991 vertical Brenner, variance

Santos et al. 1997 Vollath's F4, F5

Liu, Wang & Sun 2007 variance

Digital photography

Subbarao and Tyan 1998 Laplacian

Ng, Neow & Ang 2001 first-order 3×3 Sobel, variance

Shih 2007 first-order 3×3 Sobel & Prewitt

Yousefi et al. 2011 Vollath's F4

• Step 1: Implemented a camera remote control application

• Step 2: Using app, gathered 25 sets of benchmark images • cover a range of common photography settings • 4,303 jpeg images (23 × 167 + 2 × 231, one for each

focus position)• Canon EOS 550D/Rebel T2i camera


Experimental methodology: Comparative evaluation of focus measures for digital photography

• Step 3: Determined ground truth• true lens position (or image) where an object is in best

focus

• Step 4: Implemented more than 30 focus measures• extensive survey of the literature

• Step 5: Given a benchmark set of images and a focus measure• applied focus measure to each jpeg image in benchmark • focus measure predicts where an object will be in best

focus• given predictions of focus measure and ground truth,

calculated precision, recall, and mean absolute error


Experimental methodology: Comparative evaluation of focus measures for digital photography

ground truthtrue false

predicted

truetp

(true positive)

fp(false

positive)

falsefn

(false negative)

tn(true

negative)


Experimental methodology: Precision and recall

=tp

precisiontp fp

=tp

recalltp fn


squared gradient Laplacian


squared gradient Laplacian

3=

3 0precision

3=

3 1recall

4=

4 0precision

4=

4 0recall

• MAE chosen over mean squared error • MAE prefers errors that are occasionally large but small on

average• mean square error prefers errors that avoid large errors

while still possibly being quite unsatisfactory overall


Experimental methodology: Mean absolute error (MAE)

p p absolute error =

where p is lens position of image in maximal focus

is lens position that focus measure predictsp

Focus measure precision

recall MAE

Brenner 100.00 99.00 0.00squared gradient 100.00 99.00 0.003 × 3 difference 98.00 97.00 0.023 × 3 Sobel 98.00 97.00 0.023 × 3 Scharr 98.00 97.00 0.023 × 3 Roberts 98.00 97.00 0.023 × 3 Prewitt 98.00 97.00 0.02Gaussian 95.00 92.00 0.05vertical sq. gradient

91.33 92.00 0.55

vertical Brenner 91.33 90.00 0.23


Experimental results: First-order differentiation


recall MAE

Laplacian of Gaussian

98.00 99.00 0.32

5 × 5 vertical Sobel

96.67 99.00 0.65

5 × 5 Laplacian 96.67 99.00 0.465 × 5 cross Sobel 90.33 93.00 0.663 × 3 cross Sobel 89.67 93.00 1.183 × 3 vertical Sobel

88.73 96.00 3.94

3 × 3 Laplacian 85.00 91.00 1.415 × 5 horizontal Sobel

83.33 88.00 1.08

3 × 3 horizontal Sobel

73.27 78.00 2.95February 4, 2014 Focus Measures for Digital Photography 21

Experimental results : Second-order differentiation


recall MAE

normalized variance

43.57 53.00 26.36

variance 32.33 35.00 20.80threshold pixel count

4.24 18.00 52.38

threshold content 2.13 4.00 36.31power 0.53 2.00 43.80


Experimental results: Image statistics


recall MAE

autocorrelation 80.40 99.00 26.87Vollath's F4 61.74 87.00 24.60

Vollath's F5 33.00 35.00 19.00


Experimental results: Correlation

• Results:• some popular focus measures perform poorly when applied

to autofocusing in digital photography or to more representative sets of images• simple focus measures based on taking the first derivative

of an image perform exceedingly well in digital photography, although care must be taken in which direction derivative is taken

April 8th 2013Combinatorial Problems in Compiler

Optimization24

Experimental results: Discussion

• The speed and accuracy of a digital camera's contrast-based autofocus algorithm are crucial to user satisfaction

• We advocate using three standard statistical measures of performance—precision, recall, and mean absolute error—as accuracy criteria

• Our experimental results indicate that two simple focus measures based on taking first derivative of an image perform exceedingly well in digital photography


Conclusion

Hashim Mir, Peter Xu, and Peter van Beek University of Waterloo.

Documents

studythe focus measures

maximal focus

degree of focus

focus measuremaps

variance focus measure

squared gradient focus

vollaths f4 focus measure

digital photographyadvocate