M. Wu: ENEE631 Digital Image Processing (Spring'09) Subband and Wavelet Coding Spring ’09 Instructor: Min Wu Electrical and Computer Engineering Department,

M. Wu: ENEE631 Digital Image Processing (Spring'09)

Subband and Wavelet CodingSubband and Wavelet Coding

Spring ’09 Instructor: Min Wu

Electrical and Computer Engineering Department,

University of Maryland, College Park

bb.eng.umd.edu (select ENEE631 S’09) [email protected]

ENEE631 Spring’09ENEE631 Spring’09Lecture 12 (3/9/2009)Lecture 12 (3/9/2009)

M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec12 – Subband/Wavelet Coding [2]

Overview and LogisticsOverview and Logistics

Last Time:

– Basis images for 2-D separable unitary transform– Basics on transform coding– JPEG compression standard: Baseline block-DCT based algorithm

lossy part: quantization with different step size for each coeff. band

lossless part: run-length coding, Huffman coding, differential coding

Today

– Wrap up JPEG compression– Subband and Wavelet based compression

Subband decomposition Exploit the structures between coefficients for removing

redundancy

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

04

)


Review and Exercise on Basis Images Review and Exercise on Basis Images

Exercise:

– A is unitary transform or not?

– Find basis images

– Represent an image X with basis images

(Jain’s e.g.5.1, pp137: A’ [5, –1; – 2, 0] A; outer product of columns of AH : [1,1]’[1 1]/2, …)

2

1

2

12

1

2

1

0

2

1

2

12

1

2

1

)2(

2

1

2

12

1

2

1

)1(

2

1

2

12

1

2

1

5*AYAX H

43

21

11

11

2

1XA

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

)

2

1

2

12

1

2

1

__

__

2

1

2

12

1

2

1

*AYAH


Review: Baseline JPEG AlgorithmReview: Baseline JPEG Algorithm

“Baseline”: simple, lossy compression Subset of various DCT-based modes of JPEG standard

A few basics: 8x8 block-DCT based coding– Shift to zero-mean by subtracting 128 [-128, 127]

Allows using signed integer to represent both DC and AC coefficients

– Color representation: YCbCr / YUV

Color components can have lower spatial resolution than luminance

Interleaving color components:4 Y blocks, 1 U block, 1 V block

=> Flash demo on Baseline JPEG algorithm

by Dr. Ken Lam (HK PolyTech Univ.)

B

G

R

C

C

Y

r

b

100.0515.0615.0

436.0289.0147.0

114.0587.0299.0

(Based on Wang’s video book Chapt.1)

UM

CP

EN

EE

40

8G

Slid

es

(cre

ate

d b

y M

.Wu

& R

.Liu

© 2

00

2)


Recap: JPEG Still Image CodingRecap: JPEG Still Image Coding From B. Liu PU EE488 F’06

Lossy, block based, transform coding


Lossy Part in JPEGLossy Part in JPEG

Important tradeoff between bit rate and visual quality

Quantization (adaptive bit allocation)– Different quantization step size for different coefficient bands– Use same quantization matrix for all blocks in one image– Choose quantization matrix to best suit a specific image– Different quantization matrices for luminance and color components

Default quantization table– “Generic” over a variety of images

Quality factor “Q” [1, 100]– Scale the quantization table– Medium quality Q = 50 ~ no scaling– High quality Q = 100 ~ quantization step is 1– Low quality ~ small Q, larger quantization step

visible artifacts like ringing and blockiness

UM

CP

EN

EE

40

8G

Slid

es

(cre

ate

d b

y M

.Wu

& R

.Liu

© 2

00

2)

M. Wu: ENEE631 Digital Image Processing (Spring'09)Lec12 – Subband/Wavelet Coding [

10]

Quantization Table Recommended in JPEGQuantization Table Recommended in JPEG

– Take account of human visual properties and statistics from representative natural images

– “Optimal” quantization tables vary, depending on image content, desired bit rate, and distortion criterion => need to send decoder the tables used

8x8 Quantization Table for Luminance

16 11 10 16 24 40 51 61 12 12 14 19 26 58 60 55 14 13 16 24 40 57 69 56 14 17 22 29 51 87 80 62 18 22 37 56 68 109 103 77 24 35 55 64 81 104 113 92 49 64 78 87 103 121 120 101 72 92 95 98 112 100 103 99

8x8 Quantization Table for Chrominance

17 18 24 47 99 99 99 99 18 21 26 66 99 99 99 99 24 26 56 99 99 99 99 99 47 66 99 99 99 99 99 99 99 99 99 99 99 99 99 99 99 99 99 99 99 99 99 99 99 99 99 99 99 99 99 99 99 99 99 99 99 99 99 99


11]

Achieving Different Coding Bit Rate vs Distortion Achieving Different Coding Bit Rate vs Distortion

Adjust quantization by multiplying scale factor to base quantization tables (below is a commonly used scaling)

– A convenient way to achieve different encoding bit rate vs distortion Medium quality Q = 50 ~ no scaling High quality Q = 100 ~ quantization step is 1 (i.e. just round coeff. to

integer) Low quality ~ larger quantization step

Determine MSE introduced– Energy preservation by unitary transf. => MSE in DCT coefficients equal MSE

in image signal samples– Artifacts contributed by DCT basis images of strongly quantized freq. bands

“Optimal” quantization tables– Depend on image content, desired bit rate, and distortion criterion– Need to inform decoder what the quantization tables were used

Encode the customized table, or quality factor if standard table is used

1001

9950*2200

5015000

(%)

Q

QQ

QQfactorscale


12]

Lossless Coding Part in JPEGLossless Coding Part in JPEG

Differentially encode DC

– (lossy part: DC differences are then quantized.)

AC coefficients in one block

– Zig-zag scan after quantization for better run-length save bits in coding consecutive zeros

– Represent each AC run-length using entropy coding use shorter codes for more likely AC run-length symbols

UM

CP

EN

EE

40

8G

Slid

es

(cre

ate

d b

y M

.Wu

& R

.Liu

© 2

00

2)


13]

Lossless Coding Part in Baseline JPEG: DetailsLossless Coding Part in Baseline JPEG: Details

Differentially encode DC

– ( SIZE, AMPLITUDE ), with amplitude range in [-2048, 2047]

AC coefficients in one block

– Zig-zag scan for better run-length– Represent each AC with a pair of symbols

Symbol-1: ( RUNLENGTH, SIZE ) Huffman coded

Symbol-2: AMPLITUDE Variable length coded

RUNLENGTH [0,15] # of consecutive zero-valued AC coefficientspreceding the nonzero AC coefficient [0,15]

SIZE [0 to 10 in unit of bits] # of bits used to encode AMPLITUDE

AMPLITUDE in range of [-1023, 1024]

UM

CP

EN

EE

40

8G

Slid

es

(cre

ate

d b

y M

.Wu

& R

.Liu

© 2

00

2)


14]

Encoding Non-zero AC CoefficientsEncoding Non-zero AC Coefficients

DCT AC coefficients of natural images follow Laplacian-like distribution – use shorter codewords for coeff. with small magnitude, and vice versa

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

04

)

Table is from slides at Gonzalez/ Woods DIP 2/e book website (Chapter 8);

Baseline JPEG use a smaller part of the coefficient range.


16]

Subband/Wavelet CodingSubband/Wavelet Coding

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

04

)


17]

Wavelet Transform for Image CompressionWavelet Transform for Image Compression

ENEE631 emphasis– Focus on conceptual aspects related to image compression– Multiresolution analysis through wavelet transforms also useful for

image modeling, denoising, enhancement, and content analysis– Build upon filterbank and subband coding from ENEE630’s

multirate signal processing

(For more in-depth info. on wavelet: wavelet course offered in Math Dept.)

K-level 1-D wavelet/subband decomposition– Successive lowpass/highpass filtering and downsampling

on different level: capture transitions of different frequency bands

on the same level: capture transitions at different locations

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

/20

04

)


18]

Review: Subband Coding ConceptReview: Subband Coding Concept


19]

Successive Wavelet/Subband DecompositionSuccessive Wavelet/Subband Decomposition

Successive lowpass/highpass filtering and downsampling on different level: capture transitions of different frequency bands on the same level: capture transitions at different locations

Figure from Matlab Wavelet Toolbox Documentation

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

/20

04

)


20]

Examples of 1-D Wavelet TransformExamples of 1-D Wavelet Transform

From Matlab Wavelet Toolbox Documentation

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

)

Figures are from slides at Gonzalez/ Woods DIP book 2/e website (Chapter 7)


21]

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

04

)

Review: Filterbank & Multiresolution AnalysisReview: Filterbank & Multiresolution Analysis


24]

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

04

)

2-D Example2-D Example

From Usevitch’s article in IEEE Sig.Proc. Mag. 9/01

Separable transform by successively applying 1-D DWT to rows and columns


25]

Subband Coding TechniquesSubband Coding Techniques

General coding approach – Allocate different bits for coefficients in different frequency bands– Encode different bands separately– Example: DCT-based JPEG and early wavelet coding

Some difference between subband coding and early wavelet coding ~ Choices of filters

– Subband filters aims at (approx.) non-overlapping frequent response

– Wavelet filters has interpretations in terms of basis and typically designed for certain smoothness constraints

=> will discuss more

Shortcomings of subband coding– Difficult to determine optimal bit allocation for low bit rate applications– Not easy to accommodate different bit rates with a single coded stream– Difficult to encode at an exact target rate

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

)


26]

Recall: Filterbank Perspective of Block ProcessingRecall: Filterbank Perspective of Block Processing

From Assignment#1


28]

Embedded Zero-Tree Wavelet Coding (EZW)Embedded Zero-Tree Wavelet Coding (EZW)

“Modern” lossy wavelet coding exploits multi-resolution and self-similar nature of wavelet decomposition

– Energy is compacted into a small number of coefficients– Significant coeff. tend to cluster at the same spatial location in

each frequency subband time-freq. or space–freq.

analysis

Two set of info. to code:

– Where are the significant coefficients?

– What values are the significant coefficients?

From Usevitch (IEEE Sig.Proc. Mag. 9/01)

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

)


29]

Key Concepts in EZW Key Concepts in EZW Parent-children relation among coeff.

– Each parent coeff at level k spatially correlates with 4 coeff at level (k-1) of same orientation

– A coeff at lowest band correlates with 3 coeff.

Coding significance map via zero-tree– Encode only high energy coefficients

Need to send location info. large overhead

– Encode “insignificance map” w/ zero-trees

Successive approximation quantization– Send most-significant-bits first and

gradually refine coefficient value– “Embedded” nature of coded bit-stream

get higher fidelity image by adding extra refining bits


UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

)


30]

EZW Algorithm and ExampleEZW Algorithm and Example Initial threshold ~ 2 ^ floor(log2 xmax)

– Put all coeff. in dominant list Dominant Pass (“zig-zag” across bands)

– Assign symbol to each coeff. and entropy encode symbols

ps – positive significance ns – negative significance iz – isolated zero ztr – zero-tree root

– Significant coefficients Move to subordinate list Set them to zero in dominant list

Subordinate Pass– Output one bit for subordinate list

According to position in up/low half of quantization interval

Repeat with half threshold– Until bit budget achieved


UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

)


31]

After 1After 1stst Pass PassFrom Usevitch (IEEE Sig.Proc. Mag. 9/01)

Divide quantization interval of [32,64) by half: [32,48) => 40 and [48, 64) => 56

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

)


32]

After 2After 2ndnd Pass Pass


[16,24) => 20, [24, 32) => 28; [32,40) => 36, [40,48) => 44; [48, 56) => 52, [56, 64) => 60

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

)


33]

EZW and BeyondEZW and Beyond

Can apply DWT to entire images or larger blocks than 8x8

Symbol sequence can be entropy encoded

– e.g. arithmetic coding

Cons of EZW– Poor error resilience; Difficult for selective spatial decoding

SPIHT (Set Partitioning in Hierarchal Trees)– Further improvement over EZW to remove redundancy

EBCOT (Embedded Block Coding with Optimal Truncation)

– Used in JPEG 2000– Address the shortcomings of EZW (random access, error resilience, …)– Embedded wavelet coding in each block + bit-allocations among blocks

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

/20

04

)


34]

A Close Look at Wavelet TransformA Close Look at Wavelet Transform

Exercise:Exercise: Show Haar Transform is Show Haar Transform is unitary and orthogonalunitary and orthogonal

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

04

)


35]

Construction of Haar FunctionsConstruction of Haar Functions

Unique decomposition of integer k (p, q)– k = 0, …, N-1 with N = 2n, 0 <= p <= n-1– q = 0, 1 (for p=0); 1 <= q <= 2p (for p>0)

e.g., k=0 k=1 k=2 k=3 k=4 … (0,0) (0,1) (1,1) (1,2) (2,1) …

hk(x) = h p,q(x) for x [0,1]

k = 2p + q – 1“remainder”

power of 2

]1,0[other for 0

22for 2

1

22

1for 2

1

)()(

]1,0[for 1

)()(

21

2/

21

2/

,

0,00

x

qx

q-

N

q-x

q-

N

xhxh

xN

xhxh

ppp

ppp

qpk

1

x

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

/20

04

)


36]

Haar TransformHaar Transform

Haar transform H

– Sample hk(x) at {m/N} m = 0, …, N-1

– Real and orthogonal– Transition at each scale p is

localized according to q

Basis images of 2-D (separable) Haar transform– Outer product of two basis vectors

2200

0022

0000

0000

0000

0000

2 2 00

0 0 2 22222

0000

1111

1111

0000

2222

1111

1111

8

1

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

)


37]

Summary of Today’s LectureSummary of Today’s Lecture

Compression via subband/wavelet coding– Subband decomposition– EZW: exploit structures between coefficients for removing redundancy

Next Lecture: – More on wavelet transforms

Readings– Gonzalez’s 3/e book Section 8.2.8; 8.2.10, 7.1, 7.4-7.5

Wallace’s paper on JPEG compression standard Usevitch’s SPM Sept.2001 tutorial paper on wavelet compression

To explore further: Gonzalez’ 3/e book 7.2-7.3 (wavelet)IEEE Sig. Proc. Magazine Special Issue on transform coding (Sept.2001);Bovik’s Handbook 5.4 (wavelet compression) & 5.5 (JPEG lossy)

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

04

)


39]

A Close Look at Wavelet TransformA Close Look at Wavelet Transform

Haar Transform – unitaryHaar Transform – unitary

Orthonormal Wavelet FiltersOrthonormal Wavelet Filters

Biorthogonal Wavelet FiltersBiorthogonal Wavelet Filters

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

04

)


40]

Compare Basis Images of DCT and HaarCompare Basis Images of DCT and Haar

See also: Jain’s Fig.5.2 pp136UMCP ENEE631 Slides (created by M.Wu © 2001)


41]

Summary on Haar TransformSummary on Haar Transform Two major sub-operations

– Scaling captures info. at different frequencies– Translation captures info. at different locations

Can be represented by filtering and downsampling

Relatively poor energy compaction– Equiv. filter response doesn’t have good cutoff & stopband attenuation

1

x

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

)

M. Wu: ENEE631 Digital Image Processing (Spring'09) Subband and Wavelet Coding Spring ’09 Instructor: Min Wu Electrical and Computer Engineering Department,

Documents

image x

yobtain basis image

image codingfrom

redundancyumcp enee631

basis images exercise

basis imagesrepresent

basis imagesjains

nxn basis images