A New Free Reference Image Quality Index Based on Perceptual Blur Estimation

P. Muneesawang et al. (Eds.): PCM 2009, LNCS 5879, pp. 1185–1196, 2009. © Springer-Verlag Berlin Heidelberg 2009

A New Free Reference Image Quality Index Based on Perceptual Blur Estimation

Aladine Chetouani, Ghiles Mostafaoui, and Azeddine Beghdadi

Laboratoire de Traitement et de Transport de l’Information (L2TI) Institut Galilée – Université Paris 13

99, avenue Jean-Baptiste Clément, 93430 Villetaneuse [email protected]

Abstract. A new free reference image quality index based on the perceptual blur estimation is proposed. Here, we limit the study to isotropic blurring deg-radation although the principle could be extended to other distortions. The main idea developed here is to exploit the limitation of the blurring discriminability of the Human Visual System (HVS). The proposed method consists of adding a small amount of blur to the image and measuring its impact on the image qual-ity level. From the two images, a perceptual map is then obtained using some HVS characteristics. A quality index is finally derived by extracting some geo-metrical features from the blurring map visibility. The obtained results are compared with some known methods.

Keywords: Blur Estimation, Perceptual Analysis, HVS models, Segmentation, Quality Index.

1 Introduction

Image quality assessment plays nowadays an important role in various multimedia applications. This topic is still progressing and has reached a certain level of maturity in the multimedia communications community. Several image quality metrics have been proposed in the literature [1]. Unfortunately, in the absence of a universal image quality index, people still continue to use the classical PSNR in many applications. We believe that a universal image quality index able to evaluate various distortions is still not on the horizon and people continue to propose heuristic metrics established on psychophysical tests and some known image databases. The intend of this work is not to propose a quality index for all the known distortions but rather to focus on a particular artifact, namely blurring which one of the most studied degradation. This artefact mostly affects salient features such as contours. These fine details correspond to the high frequencies in image which is essentially due to the fact that generally, the high frequencies are the first attenuated components by the compression process.

Blurring effect has been widely studied and many ad hoc methods for measuring it have been proposed in literature. In [2] a no reference blur estimation method based on edge detection is proposed. After applying a Sobel edge detector, the length of each edge is measured. Finally, a blur measure is obtained by averaging all edge

1186 A. Chetouani, G. Mostafaoui, and A. Beghdadi

lengths. A reduced reference method is also proposed in [3] where the edge detector is applied to the original image. In [4] a blur estimation method based on wavelets is proposed. An edge map is obtained after combining the coefficients of high frequen-cies of each decomposition level. The blur measure is then obtained by analyzing the types of the edges contained in the image. In [5], a no reference blur estimation method based on DCT is proposed. First a block-wise DCT transform is applied to the image. By comparing the DCT coefficients inside each block 8x8 to some thresholds a global blur measure is then derived. In [6] the DCT is used to estimate the blur without a reference image. The idea is to measure the peakedness of each block 8x8 around each edge point by computing the kurtosis coefficient. Then the kurtosis coef-ficient is computed in each block. The mean value of the kurtosis coefficients is then used as a global blur measure.

In this study, we propose a new blur estimation inspired by the idea developed in [7]. It is worth to notice that since the blurring affect mostly object contours, blur estimation methods are generally based on edge detection. However, in the case of highly blurred image, the edge detection process fails and consequently bad estima-tion of the blurring would result. Here, we adopt another strategy without referring to edge detection. The idea is to add blur to the distorted image and analyze the impact of this amount of blur on the image quality. From these two images a blur visibility map using some Human Visual System (HVS) characteristics is derived. A Perceptual Blur Index (PBI) is then computed from this map by exploiting some geometrical information.

The paper is organized as follows. Section 2 presents the relevant steps of the pro-posed method. Section 3 is dedicated to the results and the performance evaluation of the proposed method. Finally, the last section contains conclusion and perspectives.

2 The Proposed Method

The main idea developed here is to exploit the limitation of our HVS in discriminat-ing blurring effect. Let us consider an image and its blurred version. If now we blur the degraded image again and look to the three images we could make the following observations. The first and the second image are perceptually different (see Fig. 2.a and Fig. 2.b). The second image and the third are nearly indistinguishable (see Fig. 2.c). The reasoning behind this simple experiment is that the perceptual impact of the blur depends on the original level of blurriness. In other words, the blur effect on a blurred image has less perceptual impact than on a contrasted image. This subjective result serves as the starting point of our method. Indeed, to estimate the level of blur-riness of a given image, we add a small amount of blur by applying a low-pass filter. By analyzing the perceptual impact of this blur using some characteristics of the HVS, a visibility map is computed. Then, an index of quality is derived from this map. In this section, we first explain the visibility map process. Then, we present the strategy to obtain the perceptual blur index. The flowchart of the proposed method is shown in Fig.1.

A New Free Reference Image Quality Index Based on Perceptual Blur Estimation 1187

Fig. 1. Flowchart of the proposed method

2.1 Perceptual Map Procedure

As explained above, we propose to estimate the blur effect by adding blur to the test image. It is argued that the perception of the blur depends on the original image blur-riness level as can seen by comparing the images in Fig. 2. The image under test is shown in Fig. 2.a. The application of a first blur yields image displayed in fig. 2.b whereas Fig. 2.c is the application of a second blur to image Fig. 2b, using the same filter. It could be noticed that the perceptual difference is more visible between the images in Fig. 2.a and Fig. 2.b than the images Fig. 2.b and Fig. 2.c even though the same blurring strength is applied.

a) b) c)

Fig. 2. a) Original image, b) Filtered image with a 3x3 binomial filter and c) Result of applying the same filter to image (2b)

In order to better visualize this effect, a random test signal is used. We analyze the energy spectrum of this signal and the result of applying a 3x3 binomial filter repeat-edly, first to the original signal and then to this filtered version. The energy spectrum of the original and the filtered signals are displayed on Fig. 3. It could be noticed that the filter impact is effectively less visible on the energy spectrum as shown in Fig. 3b and Fig. 3c.

Test image

Binomial Filtering (3x3)

Visibility Map Computation

PBI computation


a) b) c)

Fig. 3. a) Energy spectrum of the original signal, b) Energy spectrum of the filtered signal c) Energy spectrum of the refiltered signal

The filtering effect could be also analyzed by using the radial spectrum energy [8]. From these observations, some remarks could be drawn. First, it is argued that the

perceptual estimation of blur is not monotonic. Second, the impact of the blurring for a given image depends on the original image quality level, i.e. sharpness/blur strength. Therefore, the idea here is to apply a blurring operation to the image and then analyze the impact of this degradation in order to evaluate the original image quality level. Here, we use the binomial filter to simulate the isotropic blurring. Fig. 4 shows a real image and its degraded version. To better point out the blurring effect a zoomed zone taken from the original and filtered images is displayed in Fig. 4.c and Fig. 4.d, respectively.

a) b)

c) d)

Fig. 4. a) Test Image, b) Result of applying with a 3x3 binomial filter, c) Zoomed zone d) the corresponding zoomed zone in c


From these two images, a perceptual blur visibility map is then computed using some HVS characteristics. Before describing the process of computing the visibility map, we present briefly the most relevant HVS characteristics used in our method.

The first characteristic is the Contrast Sensitivity Function (CSF) which represents the limitation of the HVS to discern fine details from a certain distance and under certain illumination conditions [9]. This HVS characteristic is modelled as a band-pass filter whose bandwidth depends on the viewing distance and also the type of image (luminance or chrominance). Here, we use the CSF luminance model of Daly [10]. Fig. 5 presents the CSF filter.

Fig. 5. Contrast Sensitivity Function (CSF)

The cortex transform is then used to model the frequency and directional selectiv-ity of the HVS. It is a multi channel decomposition using both Dom filter for the spa-tial frequency selectivity and Fan filter for the orientation selectivity [11]. Fig. 6 pre-sents the frequency decomposition.

Fig. 6. Cortex transform (multi channel decomposition)

Finally, Visual masking refers to the reduction of the visibility of a signal (the tar-get) in the presence of another (masked). Masking model attempts to explain how the presence of one signal affects the detectability of another signal in an image [12]. Different models exist. In this work, we use the masking model proposed by Daly [10]. The visibility threshold elevation is defined as follows:


1/, 1 2 ,( , ) (1 ( *( *| ( , ) |) ) ) (1)s b bE x y k k Isb x yρ θ ρ θ= +

where k1 = 0.0153, k2 = 392.5, Isbρ,θ (x,y) is the signal value at the position (x,y) in the subband (ρ,θ). s and b are parameters depending on the frequency subband.

To compute the visibility map the two images undergo different processes. First, the CSF filter is applied to each image. Then, for each sub image obtained after Cor-tex transform, a differential visibility threshold elevation is calculated by applying Daly’s masking model. The result is used as input to a symmetric sigmoid function in the decision step in order to weight the masking effect as shown in Fig. 7.

a) b)

Fig. 7. a) Weighted masking model, b) The visibility weight function

The obtained visibility weights w are given by the following equation:

( ( , , , ) ( , , , ))

1( ( , , , ), ( , , , )) (2)

1 DVT x y E x yw E x y DVT x y

e ρ θ ρ θρ θ ρ θ − −=+

where E(ρ,θ,x,y) and DVT(ρ,θ,x,y) are respectively, the elevation obtained for the masked signal and the differential visibility threshold elevation obtained at the posi-tion (x,y) in the subband (ρ,θ).

The global visibility map is then obtained by combining these sub images. The obtained results are presented in Fig. 8. Fig. 8.a-c are test images with different blur distortions, respectively. Figs. 8.d-f show the blurred versions of the test images (Fig. 8.a-c). Figs. 8.g-i illustrate the corresponding visibility maps. The obtained re-sults clearly confirm that the impact of the filtering process is more visible when the image is less blurred.

Differential visibility threshold elevation (DVT)

Decision step Masked signal con-trast


a) b) c)

d) e) f)

g) h) i)

Fig. 8. a-c) Test images, d-f) The corresponding blurred versions using a 3x3 binomial filter and g-i) the corresponding visibility maps respectively

2.2 The Perceptual Blur Index

Once the visibility map computed, the next step consists in exploiting this map to obtain a Perceptual Blur Index (PBI). Some geometrical characteristics are used to estimate the index of blurring. Indeed, many observations confirm that the morpho-logical parameters and the spatial distribution of the degradation have a great influence on the image quality evaluation. Other parameters such as the conspicuity, proximity and other Gestalt theory tokens [13] are to be taken into account. Here, we content ourselves with the pattern area distribution of the distortions extracted from the segmented visibility map. Here we choose the Expectation Maximization based segmentation method [14]. The basic idea is to model the histogram as a mixture of N Gaussians (i.e. N classes) and affects each pixel of the image to its nearest class.


Fig. 9 presents a segmented result. Fig. 9.a and b represent respectively a test image and the corresponding visibility map. Fig. 9.c is the segmented image of the visibility map presented in Fig. 9.b using 9 Gaussians. Fig. 9.d and 9.e present the pixel af-fected to one class and its labeled version, respectively. The final labeled image is illustrated in Fig. 9.f.

a) b) c)

d) e) f)

Fig. 9. a) Test image, b) the visibility map, c) the segmented visibility map, d) Regions affected to the 1st class, e) labeling of image d) and e) labeling of image c

Once the segmented visibility map computed, the PBI index is then obtained as follows:

1

1log ( , )* (3)

*

Rk

k k kk

SurfacePBI VisibMap x y W with W

R x y=

⎛ ⎞= =⎜ ⎟⎜ ⎟

⎝ ⎠∑

where x, y and R represent respectively the number of rows, columns and regions in the

image. ( , )kVisibMap x y is the visibility average of the region k. kW corresponds to the

percentage of the region k in the image with kSurface is the surface of the region k.

3 Experimental Results

To evaluate the efficiency of the proposed method, we use both objective and subjec-tive assessment. In the following, we describe the two strategies.


3.1 Objective Tests

To test the efficiency of the proposed measure, we apply the procedure to different images containing different levels of blur. Fig. 10 shows an example of PBI obtained for natural test images. Fig. 10.a is the reference image. Fig. 10.b and 10.c are its two distorted versions. The obtained PBIs are 6.55, 5.15 and 2.19 respectively which cor-respond well to perceptual quality ranking.

a) b)

c)

Fig. 10. a) Reference image where PBI=6.55 and DMOS = 30.95, b) Distorted version of the reference image where PBI=5.15 and DMOS = 55.06 and b) Distorted version of the reference image where PBI=2.19 and DMOS = 82.28

3.2 Subjective Tests

In our experiments, the efficiency of the proposed method was tested using two image databases: LIVE database release 2 [15] and IVC [16]. The performance is evaluated in terms of correlation between the proposed PBI and subjective scores.

The LIVE database provides the Difference Mean Opinion Scores (DMOS). DMOS equal to zero corresponds to a high image quality and a high DMOS value corresponds to a poor image quality. The database used contains different types of degradations. Here, the performance of our method is tested using only the Gaussian blur images. Some original images are presented in Fig. 11.


Fig. 11. Some reference images of the LIVE database

The subjective approach to obtain the IVC database is described by the authors as follows: “Subjective evaluations were made at viewing distance of 6 times the screen height using a DSIS (Double Stimulus Impairment Scale) method with 5 categories and 15 observers”. The IVC database provides the Mean Opinion Scores (MOS) where MOS equal zero corresponds to a poor quality of the image and higher the MOS is, high the quality of the image is. The database used contains different types of degradations. Here, the performance of our method is tested using only the Gaussian blur images. Fig. 12 presents some reference images of the IVC database.

Fig. 12. Some reference images of the IVC database

To test the efficiency of the proposed method, we compare our method with some no reference blur methods [4, 7]. We compare also the proposed method by integrat-ing a full reference metric instead of the visibility map and PBI (i.e. we use a full reference quality metric to measure the distortion between the test image and its blurred version instead the proposed approach). We choose the SSIM metric which is one of the most used metrics [17], named here Blur-SSIM.


Table 1 shows the obtained results. The best correlation is obtained for the pro-posed method excepted for the LIVE database, the BlurM obtained sensibly a higher correlation than the proposed method. Also, we can notice that the Blur-SSIM ap-proach provides the bad results than the others methods. Therefore, the full reference cannot be used directly instead of the visibility map and PBI.

Table 1. Correlation obtained using the LIVE and the IVC databases

Method LIVE database IVC database BlurWavelet [4] 0.722 0.848

BlurM [7] 0.885 0.874 Blur-SSIM [17] 0.64 0.67

PBI 0.854 0.91

To observe the distribution of the PBI compared with subjective scores, we plot also the PBI versus DMOS of the LIVE database curve in figure 13. Note that scatters of these distributions are smaller.

Fig. 13. PBI vs DMOS of the proposed method

4 Conclusions and Perspectives

An efficient blur measure based on some HVS models is proposed. An original ap-proach is also proposed to obtain a quality index by exploiting the geometrical infor-mation. The obtained results demonstrate that the geometrical information plays an important role in the perception of the degradation.


The efficiency of the proposed method could be improved by taking into account other geometrical information as the form or the compactness of the regions and their combination. Another point to be considered is to introduce a deblurring process us-ing the PBI as an index of improvement. An adaptive deblurring could be then devel-oped using the visibility map. These issues will be considered in a near future.

References

1. Avcibas, I., Sankur, B., Sayood, K.: Statistical Evaluation of Image Quality Measures. Journal of Electronic Imaging 11, 206–223 (2002)

2. Marziliano, P., Dufaux, F., Winkler, S., Ebrahimi, T.: A no-reference perceptual blur met-ric. IEEE International Conference on Image Processing 3, 57–60 (2002)

3. Marziliano, P., Dufaux, F., Winkler, S., Ebrahimi, T.: Perceptual blur and ringing metrics: application to JPEG2000. Signal Processing: Image Communication 19, 163–172 (2004)

4. Tong, H., Li, M., Zhang, H., Zhang, C.: Blur detection for digital images using wavelet transform. IEEE ICME 1, 17–20 (2004)

5. Marichal, X., Wei-Ying, M., Hong Jiang, Z.: Blur determination in the compressed domain using DCT information. In: International Conference, vol. 2, pp. 386–390 (1999)

6. Caviedes, J., Oberti, F.: A new sharpness metric based on local kurtosis, edge and energy information. Signal Processing: Image Communication 19, 147–163 (2004)

7. Crête, F.: Estimer, mesurer et corriger les artefacts de compression pour la télévision. Uni-versité Joseph Fourier (2007)

8. Beghdadi, A., Deriche, M.: Features extraction from fingerprints using frequency analysis. In: Proceedings of IEEE Workshop on Signal Processing and Applications, pp. 14–15 (2000)

9. Watson, A.B., Barlow, H.B., Robson, J.G.: What does the eye see best? Nature, 419–422 (1983)

10. Daly, S.: The Visible Differences Predictor: An Algorithm for the Assessment of Image Fidelity. In: Watson, A.B. (ed.) Digital Images and Human Vision, ch. 14, pp. 179–206. MIT Press, Cambridge (1993)

11. Watson, A.B.: The cortex transform: Rapid computation of simulated neural images. In: Computer Vision Graphics and Image Processing, pp. 311–327 (1987)

12. Legge, G., Foley, J.: Contrast masking in human vision, vol. 70, pp. 1458–4471 (1980) 13. Bruce, V., Green, P., Georgeson, M.: Visual perception: Physiology, psychology and ecol-

ogy. In: LEA, p. 110 (1996) 14. http://www.mathworks.com/matlabcentral/fileexchange/10956 15. Sheikh, H.R., Wang, Z., Cormack, L., Bovik, A.C.: LIVE Image Quality Assessment Da-

tabase, http://live.ece.utexas.eduesearch/quality 16. Le Callet, P., Autrusseau, F.: Subjective quality assessment IRCCyN/IVC database,

http://www.irccyn.ec-nantes.fr/ivcdb/ 17. Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: From

error visibility to structural similarity. IEEE Transactions on Image Processing 13(4), 600–612 (2004)

A New Free Reference Image Quality Index Based on Perceptual Blur Estimation

Documents