A simulated annealing restoration algorithm for restoring ...
Post on 16-Oct-2021
4 Views
Preview:
Transcript
ARTICLE IN PRESS
0923-5965/$ - se
doi:10.1016/j.im
�Correspondfax: +8522362
E-mail addr
Signal Processing: Image Communication 21 (2006) 280–292
www.elsevier.com/locate/image
A simulated annealing restoration algorithm for restoringhalftoned color-quantized images
Yik-Hing Fung, Yuk-Hee Chan�
Centre for Multimedia Signal Processing, Department of Electronic and Information Engineering,
The Hong Kong Polytechnic University, Hung Hom, Kowloon, Hong Kong
Received 15 June 2004; received in revised form 17 September 2005; accepted 18 November 2005
Abstract
Restoration of color-quantized images is rarely addressed in the literature especially when the images are color-
quantized with halftoning. Most existing restoration algorithms are generally inadequate to deal with this problem as they
were proposed for restoring noisy blurred images. In this paper, a restoration algorithm based on simulated annealing is
proposed to solve the problem. This algorithm makes a good use of the available color palette and the mechanism of a
halftoning process to derive useful a priori information for restoration. Simulation results show that it can improve the
quality of a halftoned color-quantized image remarkably in terms of both SNR and CIELAB color difference metric. The
subjective quality of the restored images can also be improved.
r 2005 Elsevier B.V. All rights reserved.
Keywords: Image restoration; Color-quantization; Halftoning; Simulated annealing
1. Introduction
Color quantization is a process of reducing thenumber of colors in a digital image by replacingthem with some representative colors selected froma palette [1]. It is widely used because it can savetransmission bandwidth and data storage require-ment in many multimedia applications. When colorquantization is performed, certain types of degrada-tion are introduced due to the limited number ofcolors used to produce the output image. The mostcommon artifact is false contour. False contouroccurs when the available palette colors are not
e front matter r 2005 Elsevier B.V. All rights reserved
age.2005.11.003
ing author. Tel.: +852 27666264;
8439.
ess: enyhchan@polyu.edu.hk (Y.-H. Chan).
enough to represent a gradually changing region.Another common artifact is color shift. In general,the smaller the color palette size, the more severe thedefects are.
Digital halftoning [2–6] would be helpful toeliminate these defects by making use of the factthat human eyes act as spatial low-pass filters.During color quantization, the quantization error ofa pixel is diffused to neighboring pixels so as to hidethe defects and to achieve a more faithful reproduc-tion of colors. At the moment, the most popularhalftoning method is error diffusion and severalwell-known error diffusion filters such as Floyd–Steinberg, Stucki, and Jarvice–Judice–Ninke [2] aregenerally used to achieve the goal.
Sometimes it is necessary to recover the originalimage from its color-quantized output. However,
.
ARTICLE IN PRESS
Fig. 1. Color quantization with halftoning.
Y.-H. Fung, Y.-H. Chan / Signal Processing: Image Communication 21 (2006) 280–292 281
though there are a lot of reported works on therestoration of noisy and blurred color images [7–14],little effort has been seen in the literature forrestoring halftoned color-quantized images. Ob-viously, the degradation models of the two casesare completely different and hence direct adoptionof conventional restoration algorithms does notwork effectively.
Error diffusion is a nonlinear process. Conse-quently, conventional gradient-oriented optimiza-tion algorithms are not suitable to solve theaddressed problem. Simulated annealing (SA) [15]is an adaptive search algorithm that works very wellon discrete optimization problems. The basic idea ofSA is to simulate an annealing process in a system.As compared with some other conventional meth-ods, it accepts solution with deteriorated cost to alimited extent. This feature gives SA the capabilityto escape from the local minima. This paper isdevoted to formulating the process of colorquantization when error diffusion is involved anddeveloping a SA restoration algorithm to restorecorresponding degraded images.
The organization of this paper is as follows.Section 2 describes the basic concept of colorquantization with halftoning and its correspondingdegradation model. Section 3 presents the derivationof a restoration algorithm for handling halftonedcolor-quantized images. In Section 4, simulationresults and comparative study are provided toevaluate the performance of the proposed algorithm.Finally, conclusions are given in Section 5.
2. Image degradation in color quantization with
halftoning
A color image X generally consists of three colorplanes, say, Xr, Xg and Xb, which represents the red,the green and the blue color planes of the image,respectively. Accordingly, the (i, j)th color pixel of a24-bit full color image of size N�N consists ofthree color components. The intensity valuesof these three components form a 3D vector
X!
ði; jÞ ¼ Xði; jÞr;Xði; jÞg;Xði; jÞb
� �, where Xði; jÞc 2 ½0; 1� is
the intensity value of the cth color component of the(i,j)th pixel. Here, we assume that the maximum andthe minimum intensity values of a pixel are,respectively, 1 and 0.
Fig. 1 shows the system which performs colorquantization with error diffusion. The input imageX is scanned in a row-by-row fashion from top to
bottom and left to right. The relationship betweenthe original image X and the encoded image Y isdescribed by
Uði; jÞc¼Xði; jÞc �Xðk;lÞ2O
Hðk;lÞcEði�k; j�lÞc for c 2 fr; g; bg,
(1)
Y!
ði; jÞ ¼ Qc½U!
ði; jÞ� (2)
and
E!
ði; jÞ ¼ Y!
ði; jÞ �U!
ði; jÞ ¼ Qc½U!
ði; jÞ� �U!
ði; jÞ, (3)
where Uði; jÞ ¼ ðUði; jÞr;Uði; jÞg; Uði; jÞbÞ is a state vector
of the system, E!
ði; jÞ is the quantization error of the
pixel at position (i, j) andHðk;lÞc is a coefficient of the
error diffusion filter for the cth color component. Ois the corresponding causal support region of Hðk;lÞc.
The operator Qc½�� performs a 3D vectorquantization. Specifically, the 3D vector U
!
ði; jÞ iscompared with a set of representative color vectorsstored in a previously generated color paletteC ¼ fv̂i : i ¼ 1; 2; . . . Ncg. The best-matched vectorin the palette is selected based on the minimumEuclidean distance criterion. In other words, a state
vector U!
ði; jÞ is represented by color v̂k if and only if
jjU!
ði; jÞ � v̂kjjpjjU!
ði; jÞ � v̂jjj for all j ¼ 1; 2 . . .Nc.
Once the best-matched vector is selected from thecolor palette, its index is recorded and the quantiza-
tion error E!
ði; jÞ ¼ v̂k �U!
ði; jÞ is diffused to pixel
(i, j)’s neighborhood as described in Eq. (1). Note
that, to handle the boundary pixels, E!
ði; jÞ is defined
to be zero when (i, j) falls outside the image.Without loss of generality, in this paper, we use atypical Floyd–Steinberg error diffusion kernel asHðk;lÞc to perform halftoning. The recorded indices
will be used in the future to reconstruct the color-quantized image with the same color palette.
ARTICLE IN PRESSY.-H. Fung, Y.-H. Chan / Signal Processing: Image Communication 21 (2006) 280–292282
3. Formulation of restoration algorithm
Let S be the output image of the restoration.Obviously, when the restored image S is color-quantized with error diffusion, the output should beequal to Y. In formulation, we should have
Y ¼ Qch½S�, (4)
where Qch½�� denotes the operator which performscolor quantization with halftoning as shown inFig. 1. Based on this criterion, the cost function of arestored image is defined as
E ¼ S Y�Qch½S�� �
, (5)
where S½I� denotes the total number of nonzeroelements in image I. Note that E ¼ 0 implies thatboth the restoration result S and the original imageX provide the same color quantization results.Without additional priori knowledge about X, costfunction (5) would be a good measure to judge if Sis a good estimate of X based on the fact thatY ¼ Qch½X�.
Since an image usually contains a number ofpixels and color quantization is a non-linearoperation, finding an image to minimize (5) is alarge combinational optimization problem. Exhaus-tive search is impractical and it is easy to be tappedin a local minimum when a gradient-orientedoptimization algorithm is exploited. In our ap-proach, S is searched with SA to minimize costfunction E. Without loss of generality, SA is adouble-loop iterative algorithm that simulates anannealing process at a given temperature T. DuringSA, temperature T is reduced in a controlledmanner as given by
Tkþ1 ¼ aTk, (6)
where Tk is the temperature at stage k and a is aconstant used to achieve cooling. At a particulartemperature Tk, the amount of time spent inannealing is gradually adjusted by
Mkþ1 ¼ bMk, (7)
where Mk is the number of iterations performed attemperature Tk and b is a constant used to do theadjustment. The algorithm is terminated at tem-perature Tm when
Pmk¼0Mk is larger than a
predefined threshold tmax. Here, we assume thatthe SA process starts at its initial stage k ¼ 0.
Let Scur be the current estimate of the restoredimage at a particular iteration at temperature Tk
and Ecur be its corresponding cost. The new estimate
of the restored image is given by
Snew ¼ Scur þ g Y�Qch Scur½ �� �
, (8)
where g is a controlling parameter used to controlthe amount of perturbation applied to the Scur.
The cost of Snew, say, Enew, is then evaluated withEq. (5). When EnewoEcur happens, Scur is updatedto be Snew. Furthermore, if EnewoEbest happens,where Ebest is the cost of the best estimate sofar (Sbest), then Sbest will be replaced by Snew.In formulation, we have
Scur ¼Snew if EnewoEcur;
Scur otherwise
((9)
and
Sbest ¼Snew if EnewoEbest;
Sbest otherwise:
((10)
When EnewXEcur happens, Scur will be updated to beSnew only if roeðEcur�EnewÞ=KBT , where r is a randomlygenerated value which is uniformly distributedbetween 0 and 1, T denotes the current temperatureand KB is the Boltzmann constant. This criterion foraccepting the new solution is known as the Metro-polis criterion. At the beginning, temperature T ishigh. This permits many uphill moves and provideschances for the solution to leave a local minimum. Astemperature T is reduced gradually, fewer and feweruphill moves are permitted and only downhill movesare allowed eventually.
4. Performance evaluation and comparative study
Simulation has been carried out to evaluate theperformance of the proposed algorithm. In oursimulation, a number of de facto standard 24-bitfull-color images of size 256� 256 each were used.These test images were color-quantized to produceY’s. Color palettes of different size were used forquantization and they were generated with differentpalette generation algorithms such as median-cutalgorithm [3] and octree algorithm [20]. In colorquantization, halftoning was performed with errordiffusion and Floyd–Steinberg diffusion filter [2] wasused. The proposed restoration algorithm was usedto restore the halftoned color-quantized images (Y’s).
In the realization of the proposed algorithm, boththe initial estimate of S and Sbestwere initialized tobe the filtered output of the observed image Y.Specifically, a 3� 3 Gaussian filter was used togenerate the initial estimate of S and Sbest. Initial
ARTICLE IN PRESS
Fig. 2. SNR Improvements achieved with various combinations
of g and M in restoring halftoned color-quantized (a) ‘‘Lenna’’,
(b) ‘‘Peppers’’ and (c) ‘‘Couple’’.
Y.-H. Fung, Y.-H. Chan / Signal Processing: Image Communication 21 (2006) 280–292 283
temperture T0 was selected to be jES0� ES00 j=
ðKB logeð0:95ÞÞ, where ES0and ES00 were, respec-
tively, the cost of S0 and S00. Here, S0 denotes theinitial estimate of S and S00 is the first estimateobtained with Eq. (8) based on S0. In formulation,we have S00 ¼ S0 þ g Y�Qch½S0�
� �. This allows
reasonable amount of uphill move at the beginning.Parameter a was selected to be 0.9, the middle valueof the selection range suggested in [15]. To simplifythe algorithm, we selected b to be 1 such that we hadMk ¼M for all kX0.
Different combinations of M and g were evalu-ated to study their impact to the restorationperformance. Fig. 2 shows some typical results ofthe study. Specifically, it shows the Signal-to-NoiseRatio Improvement (SNRI) that was achieved whenthe proposed algorithm was used to restore color-quantized ‘‘Lenna’’, ‘‘Peppers’’ and ‘‘Couple’’. Thecolor quantization was realized with a 128-colorpalette generated with median-cut algorithm [3].Here, SNRI is defined as
SNRI ¼ 10 log
Pði; jÞjjX
!
ði; jÞ � Y!
ði; jÞjj2
Pði; jÞjjX
!
ði; jÞ � S!
bestði; jÞjj2, (11)
where X!
ði; jÞ, Y!
ði; jÞ and S!
bestði; jÞ are, respectively, the(i, j)th pixels of the original, the halftoned color-quantized and the restored images. In Fig. 2, thebroken lines correspond to the cases of M ¼ 10while the solid lines correspond to the cases ofM ¼ 50 and 100. Note the lines for M ¼ 50 and 100overlap with each other for any particular g. Forany g the value of which is larger than 0.002, it isdifficult to discriminate the line for M ¼ 10 and thelines for M ¼ 50 and 100. It was also found that,when MX50, the smaller the value of g, the moreiterations were required for the estimate to convergebut the better the converged output could beachieved in terms of SNRI.
Based on the evaluation results of the study, itcan be found that, the combination of M ¼ 50 andg ¼ 0:002 is a reasonable choice for restoringhalftoned color-quantized images. With this combi-nation, the best SNRI performance can be achievedin around 1000 iterations. It is good enough for theestimates of the iterative algorithm to converge toits restoration result. If complexity is a criticalconcern, one may select a combination of M ¼ 50and g ¼ 0:05, which can achieve a SNRI close to theone obtained with the previous selection in lessthan 100 iterations. In this paper, the presented
ARTICLE IN PRESS
Fig. 3. Training images used to realize Mese’s algorithm [17].
Y.-H. Fung, Y.-H. Chan / Signal Processing: Image Communication 21 (2006) 280–292284
results of the proposed algorithm were ob-tained with a ¼ 0:9, b ¼ 1, T0 ¼ jES0
� ES00 j=ðKB logeð0:95ÞÞ, tmax ¼ 1000, M ¼ 50 and g ¼ 0:002.
Some other restoration algorithms were alsoevaluated for comparison. Among them, Galatsa-nos’s algorithm [9] is based on the constrained leastsquare approach and Hunt’s algorithm [10] is basedon Wiener filtering. There are two algorithmspresented in Altunbasak’s work [11]. The first onemakes use of the correlation among the colorcomponents of a pixel while the other one doesnot. They are, respectively, referred to as KL andIND in [11]. All the aforementioned algorithmswere originally proposed for handling noisy blurredcolor images. Fung’s algorithm [16] adopts a POCS-based approach. Unlike [9–11], it was proposed forrestoring color-quantized images. However, in [16],it is assumed that there is no halftoning involved incolor quantization. Mese’s algorithm [17] was
originally proposed for inverse halftoning. It makesuse of some training images to pre-train a linearprediction filter for filtering binary halftones. It wasmodified here to handle halftoned color-quantizedimages.
In realizing Galatsanos’s algorithm [9], the noisepower of each channel was estimated with theoriginal full-color image. In realizing Hunt’s algo-rithm [10], three separate Wiener filters were used inthree different channels and, during the design of thefilters, the noise spectrum of each channel wasestimated with the original full-color image. Simi-larly, the original full-image was used to estimate thepower spectra of different channels in realizingAltunbasak’s algorithms [11]. In a practical situa-tion, no original image is available and hence allmust be estimated from the degraded image. In otherwords, in practice, the restoration results of [9–11]may not be as good as those presented in this paper.
ARTICLE IN PRESS
Table 1
SNR improvements of various algorithms in restoring halftoned color-quantized images
SNR improvement (dB)
Proposed Fung
[16]
Altunbasak-IND
[11]
Altunbasak-KL
[11]
Mese
[17]
Galatsanos
[9]
Hunt
[10]
Palette size: 256
Lenna 7.2292 5.4038 4.0755 3.9394 4.1646 1.6259 3.1495
Baboon 2.7570 2.6836 1.9198 1.7546 2.5360 0.5088 1.2670
Boat 3.9082 3.4746 3.2372 3.1287 3.4953 0.8375 2.0733
Peppers 8.5542 5.0088 5.3847 5.2902 4.1263 1.1409 3.8154
Fruits 7.9054 4.4207 4.0514 3.6858 3.9884 1.4248 3.0521
Couple 5.9121 4.6150 4.0953 3.9070 3.7473 2.1920 2.8439
Girl 6.8524 4.4647 4.4029 4.2232 3.8023 2.0865 2.8951
Average 6.1598 4.2959 3.8810 3.7041 3.6943 1.4023 2.7280
Palette size: 128
Lenna 8.5110 5.8254 4.9392 4.5433 5.7123 2.4357 3.8346
Baboon 4.3958 3.7212 2.6872 2.4976 3.6064 0.8323 1.9535
Boat 5.6739 3.7710 4.0782 3.8174 4.9277 1.4246 2.7164
Peppers 10.2060 6.1409 6.2363 6.1193 5.6587 1.7904 4.5946
Fruits 10.1360 3.8925 5.1414 4.6937 5.6356 2.2694 3.8296
Couple 7.6694 5.2696 4.9311 4.5735 5.3854 3.4066 3.8429
Girl 8.6239 4.4373 5.3435 5.0748 5.1745 3.2783 3.8128
Average 7.8880 4.7226 4.7653 4.4742 5.1572 2.2053 3.5121
Palette size: 64
Lenna 9.8515 6.0192 5.6204 5.2254 7.0742 3.2720 4.5439
Baboon 5.6862 4.2475 3.4326 3.1365 4.7706 0.9225 2.9029
Boat 6.6932 4.4319 4.8928 4.5457 6.0329 2.3850 3.4323
Peppers 11.2850 5.6258 6.7896 6.6543 7.0382 2.5847 5.2137
Fruits 11.0170 3.9601 5.6140 5.1302 6.8376 2.9100 4.3907
Couple 9.8891 4.7026 5.8604 5.4856 7.0296 4.9702 4.5849
Girl 10.0790 4.9431 5.9710 5.7091 6.4191 4.1484 4.5656
Average 9.2144 4.8472 5.4544 5.1267 6.4575 3.0275 4.2334
Palette size: 32
Lenna 11.7410 5.8347 5.9938 5.7719 8.9195 5.1395 4.9865
Baboon 5.8750 3.5332 3.9324 3.5293 5.5499 1.2983 3.1871
Boat 8.3006 4.4603 5.7094 5.3401 7.6022 3.3472 4.0502
Peppers 10.9850 4.7372 6.6413 6.5218 7.9197 3.4330 5.2593
Fruits 11.9420 3.0951 5.9704 5.6623 8.4185 4.2259 4.9070
Couple 9.6600 4.8138 5.5600 5.2710 8.1417 6.2369 4.7835
Girl 10.5450 4.7718 6.2691 6.0062 7.3840 5.6861 5.0606
Average 9.8641 4.4637 5.7252 5.4432 7.7051 4.1953 4.6049
Palette size: 16
Lenna 12.1150 6.4483 6.4987 6.3982 10.059 4.6921 5.2783
Baboon 7.8493 3.3234 4.9719 4.5367 7.1757 1.2976 4.3003
Boat 9.6222 4.2405 5.7226 5.5335 8.3951 2.5125 4.4839
Peppers 7.5616 3.5723 4.9149 4.8189 6.8783 1.7858 4.1941
Fruits 5.5229 2.3384 3.0841 2.9029 4.8328 2.6197 2.8355
Couple 10.7470 4.1525 5.8154 5.7268 9.2980 6.6020 4.9114
Girl 8.5940 3.3463 5.4382 5.3473 7.0026 4.4366 4.6745
Average 8.8589 3.9174 5.2065 5.0378 7.6631 3.4209 4.3826
Y.-H. Fung, Y.-H. Chan / Signal Processing: Image Communication 21 (2006) 280–292 285
ARTICLE IN PRESS
Table 2
CIELAB difference DE measurement of the outputs of various algorithms in restoring halftoned color-quantized images
Average of CIELAB difference DE
Observed Y Proposed Fung
[16]
Altunbasak-IND
[11]
Altunbasak-KL
[11]
Mese
[17]
Galatsanos
[9]
Hunt
[10]
Palette size: 256
Lenna 3.2825 2.0043 2.2942 2.4992 2.5408 2.6592 2.9921 2.6074
Baboon 4.5157 3.4646 3.7660 3.8570 3.8893 3.8170 4.4015 4.0216
Boat 4.3246 2.9612 3.3129 3.4187 3.4453 3.5574 4.1808 3.6966
Peppers 4.8757 3.0295 3.7024 3.9070 3.9413 4.0404 4.7539 4.2931
Fruits 3.4039 2.1027 2.6086 2.6524 2.7085 2.8116 3.2263 2.8190
Couple 9.3832 4.7604 5.5639 6.8130 6.9349 7.5452 7.7575 7.0757
Girl 7.1182 3.9498 4.8083 5.1771 5.2650 5.6876 6.6614 5.7108
Average 5.2720 3.1818 3.7223 4.0463 4.1036 4.3026 4.8534 4.3177
Palette size: 128
Lenna 3.9581 2.3189 2.7375 2.8777 3.0124 2.9825 3.4767 3.0619
Baboon 5.7313 3.9406 4.3301 4.5950 4.6499 4.4859 5.4871 4.7857
Boat 5.1483 3.3552 4.0071 3.9664 4.0272 3.9847 4.8933 4.3653
Peppers 6.2261 3.7152 4.5155 4.7408 4.8001 4.8953 5.8101 5.2532
Fruits 4.4127 2.5316 3.4507 3.2907 3.3936 3.3813 4.0053 3.5722
Couple 10.0050 5.0681 6.3255 7.3146 7.5623 7.4200 7.8051 7.7401
Girl 8.4566 4.5556 6.2587 6.0465 6.2002 6.3329 7.5471 6.8269
Average 6.2769 3.6407 4.5179 4.6902 4.8065 4.7832 5.5750 5.0865
Palette size: 64
Lenna 4.6383 2.7927 3.3420 3.3728 3.5928 3.3759 4.0210 3.6044
Baboon 7.2079 4.5986 5.3470 5.4557 5.5802 5.2686 6.8561 5.6664
Boat 6.3857 3.9120 4.8146 4.6909 4.7850 4.6646 5.7469 5.2097
Peppers 7.2352 4.2868 5.3879 5.5308 5.6176 5.4084 6.5681 6.0995
Fruits 5.4861 3.1426 4.3304 4.0054 4.1481 4.0384 4.8940 4.3324
Couple 11.8820 5.6279 8.3154 8.5876 9.0331 8.0450 8.4878 9.3496
Girl 11.1570 5.6124 7.8426 7.4886 7.7466 7.7247 9.4510 8.5264
Average 7.7132 4.2819 5.6257 5.5903 5.7862 5.5037 6.5750 6.1126
Palette size: 32
Lenna 5.7174 3.2258 4.2656 4.1548 4.3888 3.8653 4.5761 4.4620
Baboon 8.1388 5.5644 6.4859 6.1560 6.3694 6.0041 7.7539 6.4359
Boat 8.0848 4.6100 6.0734 5.6439 5.8113 5.4462 7.0438 6.1993
Peppers 10.1800 6.3679 8.0440 7.7957 7.9312 7.5899 8.9396 8.4388
Fruits 7.5845 4.1855 6.2373 5.3853 5.5915 5.2303 6.4358 5.8271
Couple 10.9610 6.3518 7.8506 8.6510 9.1970 7.5988 8.5580 8.9274
Girl 13.3290 6.8493 9.4531 8.9461 9.3961 8.8682 10.728 10.122
Average 9.1422 5.3078 6.9157 6.6761 6.9550 6.3718 7.7193 7.2018
Palette size: 16
Lenna 7.3126 3.7922 5.1302 5.0374 5.1990 4.4485 5.9601 5.4103
Baboon 10.0690 6.6696 8.5034 7.4306 7.7479 7.1575 9.7593 7.7710
Boat 9.8613 6.3887 7.8429 7.4625 7.6573 7.0723 9.9600 8.0856
Peppers 13.2580 9.8104 11.2280 10.6720 10.7180 10.3110 12.4990 11.3620
Fruits 10.8120 8.4798 9.6897 8.9921 9.0418 8.9801 10.1900 9.1836
Couple 13.0560 7.5965 10.4450 10.4730 11.0520 8.6742 10.8320 10.5120
Girl 14.2730 9.5449 11.8480 10.6720 11.0010 10.6960 12.6920 11.3500
Average 11.2350 7.4689 9.2410 8.6771 8.9167 8.1914 10.2700 9.0964
Y.-H. Fung, Y.-H. Chan / Signal Processing: Image Communication 21 (2006) 280–292286
ARTICLE IN PRESS
Table 3
Percentage of pixels whose CIELAB difference is not detectable (DEo3) after restoration
% of pixels whose CIELAB DE o3
Observed Y Proposed Fung
[16]
Altunbasak-IND
[11]
Altunbasak-KL
[11]
Mese
[17]
Galatsanos
[9]
Hunt
[10]
Palette size: 256
Lenna 53.71 81.50 74.58 69.98 69.05 65.76 58.15 68.02
Baboon 36.55 51.01 45.92 43.96 43.44 44.90 37.07 42.19
Boat 45.43 60.81 56.17 55.08 54.75 53.26 46.84 52.29
Peppers 37.36 68.47 54.90 53.51 52.94 47.67 38.28 45.78
Fruits 63.48 81.08 73.81 73.60 72.72 70.84 65.67 71.34
Couple 14.79 35.54 29.21 24.33 23.78 21.01 19.26 20.65
Girl 21.72 45.58 36.69 33.02 32.30 29.14 26.45 28.25
Average 39.01 60.57 53.04 50.50 49.85 47.51 41.67 46.93
Palette size: 128
Lenna 41.76 76.32 65.04 62.18 59.46 58.85 48.48 58.32
Baboon 24.37 42.48 37.18 32.81 32.29 34.77 25.58 31.75
Boat 37.33 56.05 48.12 48.33 47.57 47.86 39.72 44.61
Peppers 26.59 60.97 46.38 43.92 43.11 39.97 30.13 35.93
Fruits 50.42 76.20 63.20 64.79 63.17 63.38 55.27 60.46
Couple 8.57 30.74 22.22 17.38 16.41 16.57 14.51 14.60
Girl 13.39 37.71 24.62 24.72 23.72 22.40 19.35 19.81
Average 28.92 54.35 43.82 42.02 40.82 40.54 33.29 37.93
Palette size: 64
Lenna 31.41 69.27 53.55 51.52 48.11 50.32 38.41 47.00
Baboon 15.15 34.89 27.17 23.50 22.89 26.36 16.40 22.90
Boat 29.25 50.01 40.69 41.18 40.22 41.50 34.55 37.03
Peppers 16.78 52.18 34.07 33.82 32.74 32.23 20.72 26.77
Fruits 38.88 70.80 53.87 56.28 54.02 56.94 43.95 51.01
Couple 3.89 24.28 10.10 10.10 9.21 10.55 8.58 7.78
Girl 4.95 30.32 13.95 15.26 14.52 13.44 9.60 12.13
Average 20.04 47.39 33.34 33.10 31.67 33.05 24.60 29.23
Palette size: 32
Lenna 19.26 60.10 39.65 38.51 35.06 41.94 28.42 32.62
Baboon 12.30 26.16 19.00 18.79 17.66 21.11 14.06 17.24
Boat 23.37 44.30 33.06 34.89 33.80 38.34 29.65 30.28
Peppers 8.51 37.56 20.04 21.42 20.30 21.88 11.90 16.11
Fruits 24.26 58.35 35.00 39.43 37.80 43.74 31.28 33.97
Couple 4.72 20.69 11.92 8.89 7.86 12.34 9.04 7.46
Girl 3.51 24.07 9.64 11.03 10.01 10.82 6.81 8.60
Average 13.70 38.75 24.04 24.71 23.21 27.17 18.74 20.90
Palette size: 16
Lenna 9.73 48.90 27.62 27.74 25.36 33.06 15.97 22.93
Baboon 6.09 18.78 9.60 11.25 10.45 14.62 6.83 10.58
Boat 14.68 26.87 20.25 18.78 18.77 22.15 18.02 17.05
Peppers 3.03 10.02 5.84 7.11 7.15 7.67 3.84 6.02
Fruits 15.16 40.91 26.71 31.59 30.84 30.76 18.38 29.37
Couple 1.96 12.56 4.53 4.86 4.31 7.54 4.95 4.30
Girl 4.40 16.51 8.72 9.60 9.23 10.92 7.38 8.05
Average 7.86 24.94 14.75 15.85 15.16 18.10 10.77 14.04
Y.-H. Fung, Y.-H. Chan / Signal Processing: Image Communication 21 (2006) 280–292 287
ARTICLE IN PRESS
Fig. 4. Restoration results of color-quantized ‘‘Fruits’’ (palette was generated with median-cut algorithm [3] and is of size 128).
Y.-H. Fung, Y.-H. Chan / Signal Processing: Image Communication 21 (2006) 280–292288
Mese’s algorithm trains the linear prediction filterwith best linear estimator [17]. In the training phase,the four standard images shown in Fig. 3 and theircorresponding halftoned color-quantized outputswere used to train the filter. As for Fung’s algorithm[16] and the proposed algorithm, no original full-color image is required to extract information andno training images are required to pre-train a linearprediction filter. Hence, additional credit should beadded to the simulation results of these twoalgorithms.
Table 1 shows the SNRI performance achieved byvarious algorithms when the involved palettes wereobtained with median-cut algorithm [3]. One can seethat the proposed algorithm is superior to any otheralgorithms whatever color palette size is concerned.On average, the proposed algorithm achieved,respectively, a SNRI of 6.16, 7.89, 9.21, 9.86 and
8.86dB when the involved palette is of size 256, 128,64, 32 and 16.
Tables 2 and 3 show the performance of theevaluated algorithms in terms of the CIELAB colordifference ðDEÞ metric. The CIELAB color differenceðDEÞ metric is defined as the Euclidean distancebetween the original color of a pixel and itsreproduction in CIELAB color metric space [18].It is well accepted that color error is visuallydetectable when DE43 [11,19]. Table 2 shows theaverage of the DE values of all pixels in a restorationoutput and Table 3 shows the percentage of pixelswhose color error is visually undetectable in arestoration output. Again, one can see that theproposed algorithm is superior to the others.
Figs. 4 and 5 show the restoration results ofdifferent algorithms for visual evaluation. Figs. 4band 5b were, respectively, obtained with palettes of
ARTICLE IN PRESS
Fig. 4. (Continued)
Y.-H. Fung, Y.-H. Chan / Signal Processing: Image Communication 21 (2006) 280–292 289
sizes 128 and 16. Though halftoning can removefalse contour and color shift to a certain extent,these artifacts still appear in the color-quantizedoutputs as the palette size is too small. Besides,
pepper noise was introduced by halftoning. Afterrestoration, one can see that the proposed algorithmcan remove most of the artifacts while the others arecomparatively inferior in this aspect.
ARTICLE IN PRESS
Fig. 5. Restoration results of color-quantized ‘‘Lenna’’ (palette was generated with median-cut algorithm [3] and is of size 16).
Y.-H. Fung, Y.-H. Chan / Signal Processing: Image Communication 21 (2006) 280–292290
Table 4 shows the average performance ofvarious algorithms when the palettes were generatedwith octree algorithm [20]. The same set of standardtest images used to obtain the results presented inTables 1–3 were used for evaluation. The figures inthe table are the average values obtained with therestoration results of the color-quantized images.One can see that the proposed algorithm is stillsuperior to the others. This simulation result showsthat the proposed algorithm works with differentpalettes generated with different palette generationalgorithms.
5. Conclusions
Very little research has been carried out toaddress the restoration of halftoned color-quantizedimages. Although there are some restoration algo-rithms for restoring blurred and noisy color images,
they are generally not adequate to handle halftonedcolor-quantized images. The noise introduced bycolor quantization and halftoning is basically signaldependent and is not white, which violates theassumptions adopted in most current multichannelrestoration algorithms.
In this paper, we proposed a dedicated restorationalgorithm for restoring halftoned color-quantizedimages. This algorithm makes a good use of theavailable color palette and the halftoning process toderive useful a priori information for restoration.Simulation results and comparative study demon-strates that the proposed algorithm can achieve aremarkable improvement in the quality of a half-toned color-quantized image in terms of both SNRand CIELAB color difference DE metric. Theproposed algorithm can remove most of the artifactsintroduced by color quantization and halftoning andhence can improve the restoration result subjectively.
ARTICLE IN PRESS
Fig. 5. (Continued)
Y.-H. Fung, Y.-H. Chan / Signal Processing: Image Communication 21 (2006) 280–292 291
Acknowledgements
This work was supported by a grant fromthe Research Grants Council of the Hong Kong
Special Adminstrative Region, China (PolyU 5217/03E) and a grant from Centre for MultimediaSignal Processing, The Hong Kong PolytechnicUniversity.
ARTICLE IN PRESS
Table 4
Average performance of various algorithms in restoring halftoned color-quantized images in various aspects (palette was generated with
octree algorithm [20])
Palette size Observed (Y) Restored (X0)
Proposed Fung
[16]
Altunbasak-IND
[11]
Altunbasak-KL
[11]
Mese
[17]
Galatsanos
[9]
Hunt
[10]
(a) Average of (SNR improvement (dB))
256 — 6.5653 4.0159 4.0097 3.6977 3.7508 1.5941 2.8789
128 — 7.3868 4.0430 4.7119 4.3549 4.8697 2.2585 3.5493
64 — 8.8699 3.6709 5.4454 5.0258 6.6426 2.9054 4.2723
32 — 8.7839 3.0231 5.4257 5.0174 7.5855 3.4035 4.4434
16 — 6.1545 2.1742 4.0367 3.7613 5.8446 3.2980 3.5478
(b) Average of (average of CIELAB difference DE)
256 4.7305 3.3756 3.8534 3.9139 3.9739 4.0520 4.4882 4.2346
128 5.8678 4.1237 4.8119 4.7375 4.8371 4.8130 5.4207 5.1096
64 7.3602 5.1048 6.2701 5.8456 5.9957 5.7979 6.7306 6.2729
32 9.3714 6.5337 8.1267 7.3447 7.5776 7.1194 8.5231 7.8172
16 11.6090 8.9606 10.5540 9.4940 9.7033 9.3656 10.6690 9.8516
(c) Average of (% of pixels whose CIELAB DEo3)
256 39.46 57.95 50.75 49.68 48.88 47.06 41.56 46.55
128 29.91 49.97 41.08 40.93 39.96 40.02 33.10 37.40
64 22.77 42.94 31.53 32.42 31.44 33.60 25.94 28.72
32 12.79 32.06 19.04 22.15 20.99 24.54 15.60 18.67
16 7.72 19.93 11.63 15.43 14.50 15.68 9.79 13.28
Y.-H. Fung, Y.-H. Chan / Signal Processing: Image Communication 21 (2006) 280–292292
References
[1] M.T. Orchard, C.A. Bouman, Color quantization of images,
IEEE Trans. Signal Process. 39 (12) (1991) 2677–2690.
[2] R. Ulichney, Digital Halftoning, MIT Press, Cambridge,
MA, 1987.
[3] P. Heckbert, Color image quantization for frame buffer
displays, Comput. Graph. 16 (4) (1982) 297–307.
[4] R.S. Gentile, E. Walowit, J.P. Allebach, Quantization and
multi-level halftoning of color images for near original image
quality, Proc. SPIE 1249 (1990) 249–259.
[5] S.S. Dixit, Quantization of color images for display/printing
on limited color output devices, Comput. Graph. 15 (4)
(1991) 561–567.
[6] X. Wu, Color quantization by dynamic programming and
principal analysis, ACM Trans. Graph. 11 (4) (1992)
372–384.
[7] M. Barni, V. Cappellini, L. Mirri, Multichannel m-filtering
for color image restoration, ICIP’2000, Vancouver, Canada,
September 2000, pp. 529–532.
[8] G. Angelopoulosnd I. Pitas, Multichannel wiener filters in
color image restoration, IEEE Trans. CASVT 4 (1) (1994)
83–87.
[9] N.P. Galatsanos, A.K. Katsaggelos, R.T. Chin, A.D.
Hillery, Least squares restoration of multichannel images,
IEEE Trans. Signal Process. 39 (10) (1991) 2222–2236.
[10] B.R. Hunt, O. Kubler, Karhunen–Loeve multispectral image
restoration, Part 1: theory, IEEE Trans. Acoust. Speech
Signal Process. 32 (3) (1984) 592–600.
[11] H. Altunbasak, H.J. Trussell, Colorimetric restoration of
digital images, IEEE Trans. Image Process. 10 (3) (2001)
393–402.
[12] K.J. Boo, N.K. Bose, Multispectral image restoration with
multisensor, IEEE Trans. Geosci. Remote Sensing 35 (5)
(1997) 1160–1170.
[13] N.P. Galatsanos, R.T. Chin, Digital restoration of multi-
channel images, IEEE Trans. Acoust. Speech Signal Process.
37 (1989) 415–421.
[14] N.P. Galatsanos, R.T. Chin, Restoration of color images by
multichannel Kalman filtering, IEEE Trans. Signal Process.
39 (10) (1991) 2237–2252.
[15] S. Kirkpatrick, C.D. Gelatt, M.P. Vecchi, Optimation by
Simulated Annealing, Science 220 (1983) 671–680.
[16] Y.H. Fung, Y.H. Chan, POCS-based algorithm for restoring
colour-quantised images, IEE Proc.—Vision Image Signal
Process. 151 (2) (2004) 119–127.
[17] M. Mese, P.P. Vaidyanathan, Look-Up Table (LUT)
method for inverse halftoning, IEEE Trans. Image Process.
10 (10) (2001) 1566–1578.
[18] C.I.E., Recommendations on uniform color spaces, color
difference equations, psychometric color terms, Supplement
No. 2 to CIE publication No. 15 (E.-1.3.1) 1971/(TC-1.3.), 1978.
[19] C. Connolly, T.W.W. Leung, J. Nobbs, Colour measure-
ment by video camera, J. Soc. Dyers Colour 111 (1995)
373–375.
[20] M. Gervautz, W. Purgathofer, A simple method for color
quantization: Octree quantization, Graphics Gems, Aca-
demic Press, New York, 1990, pp. 287–293.
top related