Page 1
INTERPOLATED HALFTONING,REHALFTONING, AND
HALFTONE COMPRESSION
Prof. Brian L. [email protected]
http://www.ece.utexas.edu/~bevans
Collaboration with Dr. Thomas D. Kite andMr. Niranjan Damera-Venkata
Laboratory for Image and Video EngineeringThe University of Texas at Austin
http://anchovy.ece.utexas.edu/
Page 2
2
OUTLINE
n Introduction to halftoning
n Halftoning by error diffusion4Linear gain model4Modified error diffusion
n Interpolated halftoning
n Rehalftoning
n JBIG2 halftone compression
n Conclusions
Page 3
3
INTRODUCTION: HALFTONING
n Was analog, now digital processing
n Wordlength reduction for images48-bit to 1-bit for grayscale424-bit RGB to 8-bit for color displays424-bit RGB to CMYK for color printers
n Applications4Printers4Digital copiers4Liquid crystal displays4Video cards
n Halftoning methods4Screening4Error diffusion4Direct binary search4Hybrid schemes
Page 4
4
EXAMPLE HALFTONES
Original image
Clustered dot screen
Dispersed dot screen
Direct binary search
Floyd Steinberg
Modified Diffusion
Page 5
5
FOURIER TRANSFORMS
Original image
Clustered dot screen
Dispersed dot screen
Direct binary search
Floyd Steinberg
Modified Diffusion
Page 6
6
ERROR DIFFUSION
n 2-D delta-sigma modulatorn Noise shaping feedback coder
x0(i; j)
e(i; j)H(z)
�
x(i; j) y(i; j)+
�
+
7
16
5
16
3
16
1
16
P P P P P P P
P P P P P P P
P P P F
F F F F F F F
F F F F F F F
F F
n Error filter
n Raster scan order
P = PastF = Future
n Serpentine scan also used
Page 7
7
ERROR DIFFUSION (cont.)
n Quantizer
y(i; j) =
�0; x0(i; j) < 0:51; x0(i; j) � 0:5
e(i; j) = y(i; j)� x0(i; j)
x0(i; j) = x(i; j)� h(i; j) � e(i; j)
Kn Knn0(i; j) + n(i; j)
n(i; j)
n0(i; j)
Ksx0(i; j) Ksx0(i; j)
n Governing equations
n Non-linearity difficult to analyzen Linearize quantizer [Kite, Evans, Bovik & Sculley 1997]
n Separate signal and noise paths[Ardalan & Paulos 1987]
Noise Path
Signal Path
Page 8
8
LINEAR GAIN MODEL
n Quantization error correlated withinput [Knox 1992]
Floyd-Steinberg Jarvis, Judice & Ninke
Ks =E[jx0(i; j)j]
2E[x0(i; j)2]
n Least squares fit of quantizer inputto output defines signal gain
n Signal gain:n Noise gain:
Ks � constant
Kn = 1
Page 9
9
GAIN MODEL PREDICTIONS
n Noise transfer function (NTF)
0
0.5
1
0
0.5
10
0.5
1
1.51.7
Frequency fx / f
NFrequency f
y / f
N
Mag
nitu
de
0
0.5
1
0
0.5
10
0.5
1
1.51.7
Frequency fx / f
NFrequency f
y / f
N
Mag
nitu
de
Predicted Measured
NTF = 1�H(z)
STF =Ks
1 + (Ks � 1)H(z)
0
0.5
1
0
0.5
11
2
3
4
Frequency fx / f
NFrequency f
y / f
N
Mag
nitu
de
0
0.5
1
0
0.5
1
2
4
6
8
Frequency fx / f
NFrequency f
y / f
N
Mag
nitu
de
Floyd-Steinberg Jarvis et al.
n Signal transfer function (STF)
Page 10
10
Predicting Signal Gain Ks
n Predict Ks from error filter as:
2.017.1 −= RKs
∫
∫
−
−= π
π
π
π
ωωωωωω
ωωωω
21
2
21
2
21
21
2
21
),(),(
),(
ddHX
ddX
R
X(ω1,ω2) = Fourier Transform of input to error filter
H(ω1,ω2) = Fourier Transform of error filter
),(1),( 2121 ωωωω HX −=
But X(ω1,ω2) is the error spectrum, so
Page 11
11
MODIFIED ERROR DIFFUSION
n Efficient method of adjustingsharpness [Eschbach & Knox 1991]
L
+
�
x(i; j)
H(z)e(i; j)
+
�
x00(i; j)x0(i; j)y(i; j)
x(i; j) G(z)+
�
H(z)e(i; j)
�
+
y(i; j)x00(i; j)
G(z) = 1 + L (1�H(z))
n Equivalent circuit: pre-filter
n L can be chosen to compensate forfrequency distortion
Page 12
12
UNSHARPENED HALFTONES
n If then (flat)
n Accounts for frequency distortion
L =1�Ks
Ks
Original image Jarvis halftone
Unsharpened halftone Residual
STF = 1
Page 13
13
INTERPOLATION
n Image resizing
n Different methods (increasing cost)4Nearest neighbor (NN)4Bilinear (BL)
n Nearest neighbor, bilinear methods4Low computational cost4Artifacts masked by quantization noise in
halftone4Correct blurring by using modified error
diffusion
Halftoning
Halftone
Interpolation
Original
F(z)I(z)
Page 14
14
INTERPOLATION
n Design L for flat transfer functionusing linear gain model (L is constantfor a given interpolator)
n Compute transfer function ofinterpolation by M
n Compute signal transfer function
n Compute L to flatten the end-to-endtransfer function of the system
)()1(1
)))(1(1()(
zz
zHK
HLKF
s
s
−+−+=
−
−
−
−=−−
y
Mx
x
Mx
NN z
z
z
zI
1
1
1
1)(z
22
1
1
1
1)(
−
−
−
−=−−
y
Mx
x
Mx
BL z
z
z
zI z
Page 15
15
INTERPOLATION RESULTS
00.2
0.40.6
0.81 0
0.20.4
0.60.8
1
0
0.2
0.4
0.6
0.8
1
1.2
Frequency fy / f
NFrequency fx / f
N
Mag
nitu
de
00.2
0.40.6
0.81 0
0.20.4
0.60.8
1
0
0.2
0.4
0.6
0.8
1
1.2
Frequency fy / f
NFrequency fx / f
N
Mag
nitu
de
Nearest neighbor ×2 Bilinear ×2
Transfer functionL = –0.0105
Transfer functionL = 0.340
Page 16
16
REHALFTONING
n Halftone conversion, manipulationn Assume input and output are error
diffused halftones4Blurring corrected by using modified
error diffusion4Noise leakage masked by halftoning464 operations per pixel
n For a 512 x 512 image416 RISC MIPS40.4 s on a 167 MHz Ultra-2 workstation
Original
Halftoning Inversehalftoning Re-halftoning
Halftone
Page 17
17
REHALFTONING (cont.)
n Halftone conversion, manipulation
n Error diffused halftones
n Fixed lowpass inverse halftoningfilter, compromise cut-off frequency4Noise leakage masked by halftoning4Correct blur by modified error diffusion4Computationally efficient
Halftoneimage
Original Inverse
halftone
G(z)
halftone
Ks(1 + L (1�H(z)))
1 + (Ks � 1)H(z)
Ks
1 + (Ks � 1)H(z)
Modi�ed
Page 18
18
REHALFTONING (cont.)
n Use linear gain model to design L forflat response
n Use approximation for digitalfrequency:
n Inverse halftoning filter is a simpleseparable FIR filter
n L is computed to flatten the end toend transfer function of the system
n We need to know halftoning filtercoefficients for this scheme
n Improve halftoning results usingknowledge of type of halftone beingrehalftoned
ej!� 1 + j! � !
2=2
Page 19
19
REHALFTONING RESULTS
00.2
0.40.6
0.81 0
0.20.4
0.60.8
1
0
0.2
0.4
0.6
0.8
1
1.2
Frequency fy / f
NFrequency fx / f
N
Mag
nitu
de
Original image Rehalftone
Signal transfer function
Page 20
20
THE JBIG2 STANDARD
n Lossy/lossless coding of bi-level textand halftone data
Document
Symbolregion
decoder
Halftone region
decoder
Genericrefinement
Symbol dictionary
decoding
Halftonedictionarydecoding
Memory
Memory
n Scan vs. random mode
Page 21
21
THE JBIG2 STANDARD (cont.)
n Bi-level text coding4Hard pattern matching (lossy)4Soft pattern matching (lossless or near
lossless) may be context based
n Halftone coding4Direct halftone compression4Context based halftone coding4 Inverse halftoning and compression of
grayscale image
n Implications4Printers, fax machines and scanners,
will need to decode JBIG2 bitstreams4Fast decoding may require dedicated
hardware and embedded software4Need for low complexity, low memory
solutions
Page 22
22
PROBLEMS TO BE SOLVED
n JBIG2 compression of halftones4Compress halftone directly, using a
dictionary of patterns, or4Convert halftone to grayscale (inverse
halftoning) and compress grayscaleimage
n Efficient coding of halftone data4Fax machines4Digital archiving, scanning, and
copying
n Fast algorithms for JBIG2 codec
4 Interpolated halftoning in decoder
4Rehalftoning in codec
Page 23
23
PROBLEMS TO BE SOLVED
n JBIG2 embedded decoders4Low memory requirements4Low computational complexity4High parallelism
n Inverse halftoning: a robust solutionfor lossy coding of halftones4Rendering device can use a different
halftoning scheme than encoder4Multiresolution halftone rendering
(archive browsing)4High halftone compression ratios (6:1)4Quality enhancement if the encoder
halftoning method is transmitted
n Low-cost embedded implementations
Page 24
24
CONCLUSIONS
n Linear gain model of error diffusion4Validate accuracy of quantizer model4Link between filter gain and signal gain
n Rehalftoning and interpolation4Efficient algorithms4 Impact on emerging JBIG2 standard
n Web site for software and papers4 http://www.ece.utexas.edu/~bevans/
projects/inverseHalftoning.html