Signal Processing: Image Communication 21 (2006) 280–292 A simulated annealing restoration algorithm for restoring halftoned color-quantized images Yik-Hing Fung, Yuk-Hee Chan Centre for Multimedia Signal Processing, Department of Electronic and Information Engineering, The Hong Kong Polytechnic University, Hung Hom, Kowloon, Hong Kong Received 15 June 2004; received in revised form 17 September 2005; accepted 18 November 2005 Abstract Restoration of color-quantized images is rarely addressed in the literature especially when the images are color- quantized with halftoning. Most existing restoration algorithms are generally inadequate to deal with this problem as they were proposed for restoring noisy blurred images. In this paper, a restoration algorithm based on simulated annealing is proposed to solve the problem. This algorithm makes a good use of the available color palette and the mechanism of a halftoning process to derive useful a priori information for restoration. Simulation results show that it can improve the quality of a halftoned color-quantized image remarkably in terms of both SNR and CIELAB color difference metric. The subjective quality of the restored images can also be improved. r 2005 Elsevier B.V. All rights reserved. Keywords: Image restoration; Color-quantization; Halftoning; Simulated annealing 1. Introduction Color quantization is a process of reducing the number of colors in a digital image by replacing them with some representative colors selected from a palette [1]. It is widely used because it can save transmission bandwidth and data storage require- ment in many multimedia applications. When color quantization is performed, certain types of degrada- tion are introduced due to the limited number of colors used to produce the output image. The most common artifact is false contour. False contour occurs when the available palette colors are not enough to represent a gradually changing region. Another common artifact is color shift. In general, the smaller the color palette size, the more severe the defects are. Digital halftoning [2–6] would be helpful to eliminate these defects by making use of the fact that human eyes act as spatial low-pass filters. During color quantization, the quantization error of a pixel is diffused to neighboring pixels so as to hide the defects and to achieve a more faithful reproduc- tion of colors. At the moment, the most popular halftoning method is error diffusion and several well-known error diffusion filters such as Floyd– Steinberg, Stucki, and Jarvice–Judice–Ninke [2] are generally used to achieve the goal. Sometimes it is necessary to recover the original image from its color-quantized output. However, ARTICLE IN PRESS www.elsevier.com/locate/image 0923-5965/$ - see front matter r 2005 Elsevier B.V. All rights reserved. doi:10.1016/j.image.2005.11.003 Corresponding author. Tel.: +852 27666264; fax: +852 23628439. E-mail address: [email protected] (Y.-H. Chan).
13
Embed
A simulated annealing restoration algorithm for restoring ...
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
ARTICLE IN PRESS
0923-5965/$ - se
doi:10.1016/j.im
�Correspondfax: +8522362
E-mail addr
Signal Processing: Image Communication 21 (2006) 280–292
www.elsevier.com/locate/image
A simulated annealing restoration algorithm for restoringhalftoned color-quantized images
Yik-Hing Fung, Yuk-Hee Chan�
Centre for Multimedia Signal Processing, Department of Electronic and Information Engineering,
The Hong Kong Polytechnic University, Hung Hom, Kowloon, Hong Kong
Received 15 June 2004; received in revised form 17 September 2005; accepted 18 November 2005
Abstract
Restoration of color-quantized images is rarely addressed in the literature especially when the images are color-
quantized with halftoning. Most existing restoration algorithms are generally inadequate to deal with this problem as they
were proposed for restoring noisy blurred images. In this paper, a restoration algorithm based on simulated annealing is
proposed to solve the problem. This algorithm makes a good use of the available color palette and the mechanism of a
halftoning process to derive useful a priori information for restoration. Simulation results show that it can improve the
quality of a halftoned color-quantized image remarkably in terms of both SNR and CIELAB color difference metric. The
subjective quality of the restored images can also be improved.
Color quantization is a process of reducing thenumber of colors in a digital image by replacingthem with some representative colors selected froma palette [1]. It is widely used because it can savetransmission bandwidth and data storage require-ment in many multimedia applications. When colorquantization is performed, certain types of degrada-tion are introduced due to the limited number ofcolors used to produce the output image. The mostcommon artifact is false contour. False contouroccurs when the available palette colors are not
e front matter r 2005 Elsevier B.V. All rights reserved
enough to represent a gradually changing region.Another common artifact is color shift. In general,the smaller the color palette size, the more severe thedefects are.
Digital halftoning [2–6] would be helpful toeliminate these defects by making use of the factthat human eyes act as spatial low-pass filters.During color quantization, the quantization error ofa pixel is diffused to neighboring pixels so as to hidethe defects and to achieve a more faithful reproduc-tion of colors. At the moment, the most popularhalftoning method is error diffusion and severalwell-known error diffusion filters such as Floyd–Steinberg, Stucki, and Jarvice–Judice–Ninke [2] aregenerally used to achieve the goal.
Sometimes it is necessary to recover the originalimage from its color-quantized output. However,
Y.-H. Fung, Y.-H. Chan / Signal Processing: Image Communication 21 (2006) 280–292 281
though there are a lot of reported works on therestoration of noisy and blurred color images [7–14],little effort has been seen in the literature forrestoring halftoned color-quantized images. Ob-viously, the degradation models of the two casesare completely different and hence direct adoptionof conventional restoration algorithms does notwork effectively.
Error diffusion is a nonlinear process. Conse-quently, conventional gradient-oriented optimiza-tion algorithms are not suitable to solve theaddressed problem. Simulated annealing (SA) [15]is an adaptive search algorithm that works very wellon discrete optimization problems. The basic idea ofSA is to simulate an annealing process in a system.As compared with some other conventional meth-ods, it accepts solution with deteriorated cost to alimited extent. This feature gives SA the capabilityto escape from the local minima. This paper isdevoted to formulating the process of colorquantization when error diffusion is involved anddeveloping a SA restoration algorithm to restorecorresponding degraded images.
The organization of this paper is as follows.Section 2 describes the basic concept of colorquantization with halftoning and its correspondingdegradation model. Section 3 presents the derivationof a restoration algorithm for handling halftonedcolor-quantized images. In Section 4, simulationresults and comparative study are provided toevaluate the performance of the proposed algorithm.Finally, conclusions are given in Section 5.
2. Image degradation in color quantization with
halftoning
A color image X generally consists of three colorplanes, say, Xr, Xg and Xb, which represents the red,the green and the blue color planes of the image,respectively. Accordingly, the (i, j)th color pixel of a24-bit full color image of size N�N consists ofthree color components. The intensity valuesof these three components form a 3D vector
X!
ði; jÞ ¼ Xði; jÞr;Xði; jÞg;Xði; jÞb
� �, where Xði; jÞc 2 ½0; 1� is
the intensity value of the cth color component of the(i,j)th pixel. Here, we assume that the maximum andthe minimum intensity values of a pixel are,respectively, 1 and 0.
Fig. 1 shows the system which performs colorquantization with error diffusion. The input imageX is scanned in a row-by-row fashion from top to
bottom and left to right. The relationship betweenthe original image X and the encoded image Y isdescribed by
Uði; jÞc¼Xði; jÞc �Xðk;lÞ2O
Hðk;lÞcEði�k; j�lÞc for c 2 fr; g; bg,
(1)
Y!
ði; jÞ ¼ Qc½U!
ði; jÞ� (2)
and
E!
ði; jÞ ¼ Y!
ði; jÞ �U!
ði; jÞ ¼ Qc½U!
ði; jÞ� �U!
ði; jÞ, (3)
where Uði; jÞ ¼ ðUði; jÞr;Uði; jÞg; Uði; jÞbÞ is a state vector
of the system, E!
ði; jÞ is the quantization error of the
pixel at position (i, j) andHðk;lÞc is a coefficient of the
error diffusion filter for the cth color component. Ois the corresponding causal support region of Hðk;lÞc.
The operator Qc½�� performs a 3D vectorquantization. Specifically, the 3D vector U
!
ði; jÞ iscompared with a set of representative color vectorsstored in a previously generated color paletteC ¼ fv̂i : i ¼ 1; 2; . . . Ncg. The best-matched vectorin the palette is selected based on the minimumEuclidean distance criterion. In other words, a state
vector U!
ði; jÞ is represented by color v̂k if and only if
jjU!
ði; jÞ � v̂kjjpjjU!
ði; jÞ � v̂jjj for all j ¼ 1; 2 . . .Nc.
Once the best-matched vector is selected from thecolor palette, its index is recorded and the quantiza-
tion error E!
ði; jÞ ¼ v̂k �U!
ði; jÞ is diffused to pixel
(i, j)’s neighborhood as described in Eq. (1). Note
that, to handle the boundary pixels, E!
ði; jÞ is defined
to be zero when (i, j) falls outside the image.Without loss of generality, in this paper, we use atypical Floyd–Steinberg error diffusion kernel asHðk;lÞc to perform halftoning. The recorded indices
will be used in the future to reconstruct the color-quantized image with the same color palette.
ARTICLE IN PRESSY.-H. Fung, Y.-H. Chan / Signal Processing: Image Communication 21 (2006) 280–292282
3. Formulation of restoration algorithm
Let S be the output image of the restoration.Obviously, when the restored image S is color-quantized with error diffusion, the output should beequal to Y. In formulation, we should have
Y ¼ Qch½S�, (4)
where Qch½�� denotes the operator which performscolor quantization with halftoning as shown inFig. 1. Based on this criterion, the cost function of arestored image is defined as
E ¼ S Y�Qch½S�� �
, (5)
where S½I� denotes the total number of nonzeroelements in image I. Note that E ¼ 0 implies thatboth the restoration result S and the original imageX provide the same color quantization results.Without additional priori knowledge about X, costfunction (5) would be a good measure to judge if Sis a good estimate of X based on the fact thatY ¼ Qch½X�.
Since an image usually contains a number ofpixels and color quantization is a non-linearoperation, finding an image to minimize (5) is alarge combinational optimization problem. Exhaus-tive search is impractical and it is easy to be tappedin a local minimum when a gradient-orientedoptimization algorithm is exploited. In our ap-proach, S is searched with SA to minimize costfunction E. Without loss of generality, SA is adouble-loop iterative algorithm that simulates anannealing process at a given temperature T. DuringSA, temperature T is reduced in a controlledmanner as given by
Tkþ1 ¼ aTk, (6)
where Tk is the temperature at stage k and a is aconstant used to achieve cooling. At a particulartemperature Tk, the amount of time spent inannealing is gradually adjusted by
Mkþ1 ¼ bMk, (7)
where Mk is the number of iterations performed attemperature Tk and b is a constant used to do theadjustment. The algorithm is terminated at tem-perature Tm when
Pmk¼0Mk is larger than a
predefined threshold tmax. Here, we assume thatthe SA process starts at its initial stage k ¼ 0.
Let Scur be the current estimate of the restoredimage at a particular iteration at temperature Tk
and Ecur be its corresponding cost. The new estimate
of the restored image is given by
Snew ¼ Scur þ g Y�Qch Scur½ �� �
, (8)
where g is a controlling parameter used to controlthe amount of perturbation applied to the Scur.
The cost of Snew, say, Enew, is then evaluated withEq. (5). When EnewoEcur happens, Scur is updatedto be Snew. Furthermore, if EnewoEbest happens,where Ebest is the cost of the best estimate sofar (Sbest), then Sbest will be replaced by Snew.In formulation, we have
Scur ¼Snew if EnewoEcur;
Scur otherwise
((9)
and
Sbest ¼Snew if EnewoEbest;
Sbest otherwise:
((10)
When EnewXEcur happens, Scur will be updated to beSnew only if roeðEcur�EnewÞ=KBT , where r is a randomlygenerated value which is uniformly distributedbetween 0 and 1, T denotes the current temperatureand KB is the Boltzmann constant. This criterion foraccepting the new solution is known as the Metro-polis criterion. At the beginning, temperature T ishigh. This permits many uphill moves and provideschances for the solution to leave a local minimum. Astemperature T is reduced gradually, fewer and feweruphill moves are permitted and only downhill movesare allowed eventually.
4. Performance evaluation and comparative study
Simulation has been carried out to evaluate theperformance of the proposed algorithm. In oursimulation, a number of de facto standard 24-bitfull-color images of size 256� 256 each were used.These test images were color-quantized to produceY’s. Color palettes of different size were used forquantization and they were generated with differentpalette generation algorithms such as median-cutalgorithm [3] and octree algorithm [20]. In colorquantization, halftoning was performed with errordiffusion and Floyd–Steinberg diffusion filter [2] wasused. The proposed restoration algorithm was usedto restore the halftoned color-quantized images (Y’s).
In the realization of the proposed algorithm, boththe initial estimate of S and Sbestwere initialized tobe the filtered output of the observed image Y.Specifically, a 3� 3 Gaussian filter was used togenerate the initial estimate of S and Sbest. Initial
ARTICLE IN PRESS
Fig. 2. SNR Improvements achieved with various combinations
of g and M in restoring halftoned color-quantized (a) ‘‘Lenna’’,
(b) ‘‘Peppers’’ and (c) ‘‘Couple’’.
Y.-H. Fung, Y.-H. Chan / Signal Processing: Image Communication 21 (2006) 280–292 283
temperture T0 was selected to be jES0� ES00 j=
ðKB logeð0:95ÞÞ, where ES0and ES00 were, respec-
tively, the cost of S0 and S00. Here, S0 denotes theinitial estimate of S and S00 is the first estimateobtained with Eq. (8) based on S0. In formulation,we have S00 ¼ S0 þ g Y�Qch½S0�
� �. This allows
reasonable amount of uphill move at the beginning.Parameter a was selected to be 0.9, the middle valueof the selection range suggested in [15]. To simplifythe algorithm, we selected b to be 1 such that we hadMk ¼M for all kX0.
Different combinations of M and g were evalu-ated to study their impact to the restorationperformance. Fig. 2 shows some typical results ofthe study. Specifically, it shows the Signal-to-NoiseRatio Improvement (SNRI) that was achieved whenthe proposed algorithm was used to restore color-quantized ‘‘Lenna’’, ‘‘Peppers’’ and ‘‘Couple’’. Thecolor quantization was realized with a 128-colorpalette generated with median-cut algorithm [3].Here, SNRI is defined as
SNRI ¼ 10 log
Pði; jÞjjX
!
ði; jÞ � Y!
ði; jÞjj2
Pði; jÞjjX
!
ði; jÞ � S!
bestði; jÞjj2, (11)
where X!
ði; jÞ, Y!
ði; jÞ and S!
bestði; jÞ are, respectively, the(i, j)th pixels of the original, the halftoned color-quantized and the restored images. In Fig. 2, thebroken lines correspond to the cases of M ¼ 10while the solid lines correspond to the cases ofM ¼ 50 and 100. Note the lines for M ¼ 50 and 100overlap with each other for any particular g. Forany g the value of which is larger than 0.002, it isdifficult to discriminate the line for M ¼ 10 and thelines for M ¼ 50 and 100. It was also found that,when MX50, the smaller the value of g, the moreiterations were required for the estimate to convergebut the better the converged output could beachieved in terms of SNRI.
Based on the evaluation results of the study, itcan be found that, the combination of M ¼ 50 andg ¼ 0:002 is a reasonable choice for restoringhalftoned color-quantized images. With this combi-nation, the best SNRI performance can be achievedin around 1000 iterations. It is good enough for theestimates of the iterative algorithm to converge toits restoration result. If complexity is a criticalconcern, one may select a combination of M ¼ 50and g ¼ 0:05, which can achieve a SNRI close to theone obtained with the previous selection in lessthan 100 iterations. In this paper, the presented
ARTICLE IN PRESS
Fig. 3. Training images used to realize Mese’s algorithm [17].
Y.-H. Fung, Y.-H. Chan / Signal Processing: Image Communication 21 (2006) 280–292284
results of the proposed algorithm were ob-tained with a ¼ 0:9, b ¼ 1, T0 ¼ jES0
� ES00 j=ðKB logeð0:95ÞÞ, tmax ¼ 1000, M ¼ 50 and g ¼ 0:002.
Some other restoration algorithms were alsoevaluated for comparison. Among them, Galatsa-nos’s algorithm [9] is based on the constrained leastsquare approach and Hunt’s algorithm [10] is basedon Wiener filtering. There are two algorithmspresented in Altunbasak’s work [11]. The first onemakes use of the correlation among the colorcomponents of a pixel while the other one doesnot. They are, respectively, referred to as KL andIND in [11]. All the aforementioned algorithmswere originally proposed for handling noisy blurredcolor images. Fung’s algorithm [16] adopts a POCS-based approach. Unlike [9–11], it was proposed forrestoring color-quantized images. However, in [16],it is assumed that there is no halftoning involved incolor quantization. Mese’s algorithm [17] was
originally proposed for inverse halftoning. It makesuse of some training images to pre-train a linearprediction filter for filtering binary halftones. It wasmodified here to handle halftoned color-quantizedimages.
In realizing Galatsanos’s algorithm [9], the noisepower of each channel was estimated with theoriginal full-color image. In realizing Hunt’s algo-rithm [10], three separate Wiener filters were used inthree different channels and, during the design of thefilters, the noise spectrum of each channel wasestimated with the original full-color image. Simi-larly, the original full-image was used to estimate thepower spectra of different channels in realizingAltunbasak’s algorithms [11]. In a practical situa-tion, no original image is available and hence allmust be estimated from the degraded image. In otherwords, in practice, the restoration results of [9–11]may not be as good as those presented in this paper.
ARTICLE IN PRESS
Table 1
SNR improvements of various algorithms in restoring halftoned color-quantized images
Average 7.86 24.94 14.75 15.85 15.16 18.10 10.77 14.04
Y.-H. Fung, Y.-H. Chan / Signal Processing: Image Communication 21 (2006) 280–292 287
ARTICLE IN PRESS
Fig. 4. Restoration results of color-quantized ‘‘Fruits’’ (palette was generated with median-cut algorithm [3] and is of size 128).
Y.-H. Fung, Y.-H. Chan / Signal Processing: Image Communication 21 (2006) 280–292288
Mese’s algorithm trains the linear prediction filterwith best linear estimator [17]. In the training phase,the four standard images shown in Fig. 3 and theircorresponding halftoned color-quantized outputswere used to train the filter. As for Fung’s algorithm[16] and the proposed algorithm, no original full-color image is required to extract information andno training images are required to pre-train a linearprediction filter. Hence, additional credit should beadded to the simulation results of these twoalgorithms.
Table 1 shows the SNRI performance achieved byvarious algorithms when the involved palettes wereobtained with median-cut algorithm [3]. One can seethat the proposed algorithm is superior to any otheralgorithms whatever color palette size is concerned.On average, the proposed algorithm achieved,respectively, a SNRI of 6.16, 7.89, 9.21, 9.86 and
8.86dB when the involved palette is of size 256, 128,64, 32 and 16.
Tables 2 and 3 show the performance of theevaluated algorithms in terms of the CIELAB colordifference ðDEÞ metric. The CIELAB color differenceðDEÞ metric is defined as the Euclidean distancebetween the original color of a pixel and itsreproduction in CIELAB color metric space [18].It is well accepted that color error is visuallydetectable when DE43 [11,19]. Table 2 shows theaverage of the DE values of all pixels in a restorationoutput and Table 3 shows the percentage of pixelswhose color error is visually undetectable in arestoration output. Again, one can see that theproposed algorithm is superior to the others.
Figs. 4 and 5 show the restoration results ofdifferent algorithms for visual evaluation. Figs. 4band 5b were, respectively, obtained with palettes of
ARTICLE IN PRESS
Fig. 4. (Continued)
Y.-H. Fung, Y.-H. Chan / Signal Processing: Image Communication 21 (2006) 280–292 289
sizes 128 and 16. Though halftoning can removefalse contour and color shift to a certain extent,these artifacts still appear in the color-quantizedoutputs as the palette size is too small. Besides,
pepper noise was introduced by halftoning. Afterrestoration, one can see that the proposed algorithmcan remove most of the artifacts while the others arecomparatively inferior in this aspect.
ARTICLE IN PRESS
Fig. 5. Restoration results of color-quantized ‘‘Lenna’’ (palette was generated with median-cut algorithm [3] and is of size 16).
Y.-H. Fung, Y.-H. Chan / Signal Processing: Image Communication 21 (2006) 280–292290
Table 4 shows the average performance ofvarious algorithms when the palettes were generatedwith octree algorithm [20]. The same set of standardtest images used to obtain the results presented inTables 1–3 were used for evaluation. The figures inthe table are the average values obtained with therestoration results of the color-quantized images.One can see that the proposed algorithm is stillsuperior to the others. This simulation result showsthat the proposed algorithm works with differentpalettes generated with different palette generationalgorithms.
5. Conclusions
Very little research has been carried out toaddress the restoration of halftoned color-quantizedimages. Although there are some restoration algo-rithms for restoring blurred and noisy color images,
they are generally not adequate to handle halftonedcolor-quantized images. The noise introduced bycolor quantization and halftoning is basically signaldependent and is not white, which violates theassumptions adopted in most current multichannelrestoration algorithms.
In this paper, we proposed a dedicated restorationalgorithm for restoring halftoned color-quantizedimages. This algorithm makes a good use of theavailable color palette and the halftoning process toderive useful a priori information for restoration.Simulation results and comparative study demon-strates that the proposed algorithm can achieve aremarkable improvement in the quality of a half-toned color-quantized image in terms of both SNRand CIELAB color difference DE metric. Theproposed algorithm can remove most of the artifactsintroduced by color quantization and halftoning andhence can improve the restoration result subjectively.
ARTICLE IN PRESS
Fig. 5. (Continued)
Y.-H. Fung, Y.-H. Chan / Signal Processing: Image Communication 21 (2006) 280–292 291
Acknowledgements
This work was supported by a grant fromthe Research Grants Council of the Hong Kong
Special Adminstrative Region, China (PolyU 5217/03E) and a grant from Centre for MultimediaSignal Processing, The Hong Kong PolytechnicUniversity.
ARTICLE IN PRESS
Table 4
Average performance of various algorithms in restoring halftoned color-quantized images in various aspects (palette was generated with