EE5965 Advanced Image Processing Copyright Xin Li 2009-2012 1 Post-processing: Fighting Against Coding Artifacts Deblocking of DCT coded images – Image.

EE5965 Advanced Image Processing Copyright Xin Li 2009-

20121

Post-processing: Fighting Against Coding Artifacts

• Deblocking of DCT coded images– Image smoothing based approach– Wavelet-thresholding based approach

• Deringing of wavelet coded images– Re-compression approach– PDE-based approach

• Post-processing by alternating projections: a unified approach


20122

Block Artifacts

Smooth areas become blocky JPEG decoded image at 0.23bpp

zoom

Why block artifacts occur?


20123

x

f(x)

original

x

f(x)^

B 2B 3B

JPEGdecodedat lowbit rate

Only DC component is preserved


20124

Deblocking as Denoising

Standard image

denoisingalgorithm

JPEG compressed

imagepostprocessed

image

th 1000

Manually tune the threshold parameter!


20125

Experiment Results

before deblocking after deblocking


20126

Wavelet-based Deblocking

Behavior of block artifacts in wavelet domain

Fundamental Issues behind Deblocking

• Motivation - modeling uncertainty– Location of block artifacts is known (block

boundaries)– How to distinguish significant coefficients

generated by artifacts from those associated with true edges

• Strategy– Recall how JPEG2000 is free from block artifacts


20127


20128

Deblocking via Wavelet Thresholding

WaveletTransform

BlockWavelet

Transform reorder

X

X

Y1

Y2

X: JPEG decoded image

• Apply both WT and block-based WT to X to get Y1,Y2;

• Locate the coefficients at block boundaries;• If |Y1(i,j)|>T and |Y2(i,j)|<T, apply soft

thresholding to Y1(i,j);• Apply IWT to processed Y1 to obtain

deblocked image


20129

Deblocking Algorithm


201210

Example

Before deblocking (PSNR=27.39dB)After deblocking (PSNR=28.07dB)


201211

Ringing Artifacts

Sharp edges become unnatural JPEG2000 decoded image at 0.125bpp

zoom

Why ringing artifacts occur?


201212

x(n)

H1

Key observation: wavelet transform lacks translation invariance

2

x(n-1)

H1

2

origin origin


201213

Deringing by Re-compression

JPEG: JPEG2000 encoder JPEG-1: JPEG2000 decoder

Example


201214

before processing after processing

PDE-based Deringing

• The power of anisotropic diffusion– Nonlinear diffusion can handle a variety of

noise– Which PDE is suitable for deringing?– Implication into wavelet coding


201215


201216

Perona-Malik Filtering

PSNR=30.86dBPSNR=31.09dB


201217

Mean Curvature Filtering

PSNR=31.09dB PSNR=30.27dB


201218

Post-processing: Fighting Against Coding Artifacts

• Deblocking of DCT coded images– Image smoothing based approach– Wavelet-thresholding based approach

• Deringing of wavelet coded images– Re-compression approach– PDE-based approach

• Post-processing by alternating projections: a unified approach

Recall: Alternating Projection


201219

X0

X1

X2

X∞

Projection-Onto-Convex-Set (POCS) Theorem: If C1,…,Ck areconvex sets, then alternating projection P1,…,Pk will convergeto the intersection of C1,…,Ck if it is not empty Alternating projection does not always converge in the caseof non-convex set. Can you think of any counter-example?

C1

C2


201220

Projection Operators

● Constraint set QTC

otherwisexTyxTyTyxTy

xPTfPTfP QQQT 2/2/2/2/

)()},({1

y y+T/2y-T/2

● Constraint set sC fffPs )(

xB xB

f Ps(f)


201221

Projection-based Deblocking● DCT quantization set

}][|{},|{, gfQfCgTffCCCC QTQTQT

DCT Quantization● Smoothness constraint set

Cs={f|f is smooth in the block boundaries}

}||||{ 2 DffCs at block boundaries

Linear edge detection operatorYongyi Yang; Galatsanos, N.P.; Katsaggelos, A.K.; , "Projection-based spatially adaptive reconstruction of block-transform compressed images,“ IEEE Trans. on Image Proc., vol.4, no.7, pp.896-908, Jul 1995


201222

Projection-based Deringing

● WT quantization set

}][|{},|{, gfQfCgTffCCCC QTQTQT

WT Quantization● Smoothness constraint set

][,1, IcIcIcIcII WWEESSNN

tji

tji

Perona-Malik diffusion as a nonlinear projection operator

Xin Li; , "Improved wavelet decoding via set theoretic estimation," IEEE Trans. on CSVT, vol.15, no.1, pp. 108- 112, Jan. 2005

Algorithm Flowchart


201223

C1 : observation constraint set

C2 : regularization constraint set

Summary

• Connection with models (PDE-based, wavelet-based, patch-based)– They serve as image prior/regularization

constraint set– Jointly work with quantization (observation data)

constraint set• Convergence is NOT always guaranteed but

can be terminated strategically.


201224

EE5965 Advanced Image Processing Copyright Xin Li 2009-2012 1 Post-processing: Fighting Against Coding Artifacts Deblocking of DCT coded images – Image.

Documents