M. Wu: ENEE631 Digital Image Processing (Spring'09) Unitary Transform Spring ’09 Instructor: Min Wu Electrical and Computer Engineering Department, University.

M. Wu: ENEE631 Digital Image Processing (Spring'09)

Unitary Transform Unitary Transform

Spring ’09 Instructor: Min Wu

Electrical and Computer Engineering Department,

University of Maryland, College Park

bb.eng.umd.edu (select ENEE631 S’09) [email protected]

ENEE631 Spring’09ENEE631 Spring’09Lecture 10 (2/25/2009)Lecture 10 (2/25/2009)

M. Wu: ENEE631 Digital Image Processing (Spring'09) Lec10 – Unitary Transform [2]

OverviewOverview

Last Time:– MMSE Quantizer for non-uniform and uniform source– Companding: Quantizer with pre- and post- nonlinear transformation– Quantizer in predictive coding

Today:– Vector vs. Scalar Quantizer – Revisit image transform from a coding and basis perspective

=> Unitary transform

– DCT transform

Logistics: (1) mid-term exam

(2) Assign#3 to be posted

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

04

)


Recap:Recap:Scalar QuantizerScalar Quantizer

Quantize one sample at a time

Quantizer

…

quantization error

…

…

…

…Input/Output

response

Input x

Output Q(x)


Recap: MMSE Quantizer Recap: MMSE Quantizer Example – Gaussian SourceExample – Gaussian Source

Start with uniform quantizer Use iterative algorithm. optimum thresholds (red) and reconstruction values (blue)

From B. Liu PU EE488 F’06

Truncated Gaussian N(0,1) with L=16 quantizer


Vector QuantizationVector Quantization

Encode a set of values together– Find the representative combinations– Encode the indices of combinations

Scalar vs. Vector quantization– SQ is simpler in implementation – VQ allows flexible partition of coding cells– VQ could naturally explore the correlation

between elements

Stages to build vector quantizer– Codebook design– Encoder– Decoder

From Bovik’s Handbook Sec.5.3

vector quantization of 2 elements

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

)

scalar quantization of 2 elements

Signal Sample-1

Sig

nal S

ampl

e-2


Outline of Core Parts in VQOutline of Core Parts in VQ

Design codebook– Optimization formulation is similar to MMSE scalar quantizer– Given a set of representative points

“Nearest neighbor” rule to determine partition boundaries

– Given a set of partition boundaries “Probability centroid” rule to determine representative

points that minimizes mean distortion in each cell

Search for codeword at encoder– Tedious exhaustive search– Design codebook with special structures to

speed up encoding E.g., tree-structured VQ

Reference:A. Gersho and R. M. Gray, Vector Quantization and Signal Compression, Kluwer Publisher. R. M. Gray, ``Vector Quantization,'' IEEE ASSP Magazine, pp. 4--29, April 1984.

vector quantization of 2 elements

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

/20

04

)


Recap: List of Compression ToolsRecap: List of Compression Tools

Lossless encoding tools– Entropy coding: Huffman, Arithmetic coding, Lemple-Ziv, …– Run-length coding

Lossy tools for reducing bit rate– Quantization: scalar quantizer vs. vector quantizer– Truncations: discard unimportant parts of data

Facilitating compression via Prediction– Convert the full signal to prediction residue with smaller dynamic range– Encode prediction parameters and residues with less bits– Be careful: use quantized version available to decoder when designing encoder

Facilitating compression via Transforms– Transform into a domain with improved energy compaction

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

04

)


Image Transform: A RevisitImage Transform: A Revisit

With A Coding PerspectiveWith A Coding Perspective

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

04

)


Why Do Transforms?Why Do Transforms?

Fast computation– E.g., convolution vs. multiplication for filter with wide support

Conceptual insights for various image processing– E.g., spatial frequency info. (smooth, moderate change, fast change, etc.)

Obtain transformed data from measurement– E.g., blurred images, radiology images (medical and astrophysics)– Often need to perform an inverse transform to obtain the actual data

For efficient storage and transmission– Pick a few “representatives” (basis) – Just store/send the major “contribution” from some basis image/vector

=> Examine a segment of signal samples together

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

)


Basic Process of Transform CodingBasic Process of Transform Coding

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

04

)

Figure is from slides at Gonzalez/ Woods DIP book website (Chapter 8)


Basis Vectors and Basis ImagesBasis Vectors and Basis Images

A basis for a vector space ~ a set of vectors that is– Linearly independent ~ ai vi = 0 if and only if all ai=0– Uniquely represent every vector in the space by their linear combination

~ bi vi ( “spanning set” {vi} )

Orthonormal basis– Orthogonality ~ inner product <x, y> = y*T x= 0 – Normalized length ~ || x ||2 = <x, x> = x*T x= 1

Inner product for 2-D arrays– <F, G> = m n f(m,n) g*(m,n) = G1

*T F1 (rewrite matrix into vector) !! Don’t do FG ~ may not even be a valid operation for MxN matrices!

2D Basis Matrices (Basis Images)– Represent any images of the same size as a linear combination of basis

images

A vector space consists of a set of vectors, a field of scalars, a vector addition operation, and a scalar multiplication operation.

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

)


Standard / “Trivial” BasisStandard / “Trivial” Basis

Standard basis vectors

Standard basis images

1

0

0

1

0

1

0

3

0

0

1

6

1

3

6

10

000

01

003

00

102

00

012

03

22


Matrix/Vector Form of 1-D DFT Matrix/Vector Form of 1-D DFT

{ z(n) } { Z(k) }

n, k = 0, 1, …, N-1, WN = exp{ - j2 / N } ~ complex conjugate of primitive Nth root of

unity

)1(

)1(

)0(

)1(

)2(

)1(

)0(

1

1

1

1 1 11

)1(

)2(

)1(

)0(

110

/)1(2/)1(22/)1(2

/)1(22/42/22

/)1(2/22/2

2 NZ

Z

Z

aaa

NZ

Z

Z

Z

eee

eee

eee

Nz

z

z

z

N

NNjNNjNNj

NNjNjNj

NNjNjNj

1

0

1

0

)(1

)(

)(1

)(

N

k

nkN

N

n

nkN

WkZN

nz

WnzN

kZ

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

/20

04

)

z

a

a

a

Nz

z

z

z

eee

eee

eee

NZ

Z

Z

Z

TN

T

T

NNjNNjNNj

NNjNjNj

NNjNjNj

*1

*1

*0

/)1(2/)1(22/)1(2

/)1(22/42/22

/)1(2/22/2

)1(

)2(

)1(

)0(

1

1

1

1 1 11

)1(

)2(

)1(

)0(

2


Matrix/Vector Form of 1-D DFT (cont’d)Matrix/Vector Form of 1-D DFT (cont’d)

{ z(n) } { Z(k) }

n, k = 0, 1, …, N-1, WN = exp{ - j2 / N }

“transform kernels” are complex exponentials

Vector form and interpretation for inverse transform

z = k Z(k) ak

where ak = [ 1, WN-k , WN

-2k , … WN-(N-1)k ]T /

N

Basis vectors – ak

H = ak* T = [ 1, WN

k , WN2k , … WN

(N-1)k ] / N

Use akH as row vectors to construct a matrix F

Z = F z z = F*T Z = F* Z

– F is symmetric (FT=F) and unitary (F-1 = FH where FH = F*T )

1

0

1

0

)(1

)(

)(1

)(

N

k

nkN

N

n

nkN

WkZN

nz

WnzN

kZ

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

/20

04

)


1-D Unitary Transform1-D Unitary Transform

Consider linear invertible transform– 1-D sequence { x(0), x(1), …, x(N-1) } as a vector – y = A x and A is invertible

Unitary matrix: A is unitary if A-1 = A*T = AH – Denote A*T as AH ~ “Hermitian”

– x = A-1 y = A*T y = ai*T y(i)

– Hermitian of row vectors of unitary matrix A form a set of orthonormal basis vectors {ai

*T }

Think: how about column vectors of A?

Orthogonal matrix ~ A-1 = AT – Real-valued unitary matrix is also an orthogonal matrix– Row vectors of real orthogonal matrix A form orthonormal basis vectors

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

)


Exercise:Exercise:

Is each matrix here unitary or orthogonal?– If yes, what are the basis vectors?

cossin

sincos5A

2

1

2

12

1

2

1

4A

21

323A

1: n; 2: y. inv(A3) = [2, –3; -1, 2]; Check A A’ = I ?

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

/20

04

)

1

1

2

12 j

jA

2

21

j

jA


Properties of 1-D Unitary Transform Properties of 1-D Unitary Transform y = A xy = A x

Energy Conservation: || y ||2 = || x ||2

Proof: || y ||2 = || Ax ||2= (Ax)*T (Ax)= x*T A*T A x = x*T x = || x ||2

Interpretation:– The angles between vectors are preserved

– A unitary transformation is a rotation of a vector in an N-dimension space, i.e., a rotation of basis coordinates

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

/20

04

)


Properties of 1-D Unitary Transform (cont’d)Properties of 1-D Unitary Transform (cont’d)

Energy Compaction and Decorrelation– Many commonly used unitary transforms tend to pack a large fraction of

signal energy into just a few transform coefficients

– Highly correlated input elements quite uncorrelated output coefficients Covariance matrix E[ ( y – E(y) ) ( y – E(y) )*T ]

small correlation implies small off-diagonal terms

Example: recall the effect of DFT

Question: What unitary transform gives the best compaction and decorrelation?

=> Will revisit this issue

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

/20

04

)


1-D Discrete Cosine Transform (DCT)1-D Discrete Cosine Transform (DCT)

Linear transform matrix C

c(k,n) = (0) for k=0

c(k,n) = (k) cos[(2n+1)/2N] for k>0=> Compare “transform kernels” of DCT vs. DFT

C is real and orthogonal

– rows of C form an orthonormal basis

– C is not symmetric!– DCT is not the real part of unitary DFT! See Assignment#3

related to DFT of a symmetrically extended signal

1..., 1,kfor 2

)( ;1

)0(

2

)12(cos)()()(

2

)12(cos)()()(

1

0

1

0

NN

kN

N

knkkZnz

N

knknzkZ

N

k

N

n

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

/20

04

)


Periodicity Implied by DFT and DCTPeriodicity Implied by DFT and DCT

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

04

)

Figure is from slides at Gonzalez/ Woods DIP book website (Chapter 8)


Example of 1-D DCT Example of 1-D DCT

100

50

0

-50

-100

0 1 2 3 4 5 6 7n

z(n)

100

50

0

-50

-100

0 1 2 3 4 5 6 7k

Z(k)

DCT

From Ken Lam’s DCT talk 2001 (HK Polytech)U

MC

P E

NE

E6

31

Slid

es

(cre

ate

d b

y M

.Wu

© 2

00

1)


UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

)

Example of 1-D DCT (cont’d): N = 8Example of 1-D DCT (cont’d): N = 8

k

Z(k)

Transform coeff.

1.0

0.0

-1.0

1.0

0.0

-1.0

1.0

0.0

-1.0

1.0

0.0

-1.0

1.0

0.0

-1.0

1.0

0.0

-1.0

1.0

0.0

-1.0

1.0

0.0

-1.0

Basis vectors

100

0

-100

100

0

-100

100

0

-100

100

0

-100

100

0

-100

100

0

-100

100

0

-100

100

0

-100u=0

u=0 to 1

u=0 to 4

u=0 to 5

u=0 to 2

u=0 to 3

u=0 to 6

u=0 to 7

Reconstructions

n

z(n)

Original signal

From Ken Lam’s DCT talk 2001 (HK Polytech)


2-D DCT2-D DCT Separable orthogonal transform

– Apply 1-D DCT to each row, then to each column

Y = C X CT X = CT Y C = mn y(m,n) Bm,n

– Set y(m,n)=1 and rest as zeros to obtain basis image Bm,n ~ outer product of C’s mth & nth rows

DCT basis images:– Equivalent to represent

an NxN image with a set of orthonormal NxN “basis images”

– Each DCT coefficient indicates the contribution from (or similarity to) the corresponding basis image

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

)


2-D Transform: General Case2-D Transform: General Case

A general 2-D linear transform with kernel {ak,l(m,n)}

– y(k,l) is a transform coefficient for Image {x(m,n)} – {y(k,l)} is “Transformed Image”– Equiv to rewriting all from 2-D to 1-D and applying 1-D transform

Computational complexity– N2 values to compute– N2 terms in summation per output coefficient– O(N4) for transforming an NxN image!

),(),(),(

),(),(),(

1

0

1

0,

1

0

1

0,

nmhlkynmx

nmanmxlky

N

k

N

llk

N

m

N

nlk

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

/20

04

)


2-D Separable Unitary Transforms2-D Separable Unitary Transforms

Focus our attention to separable transform– ak,l(m,n) = ak(m) bl(n) , denote this as a(k,m) b(l,n)

Use 1-D unitary transform as building block– {ak(m)}k and {bl(n)}l are 1-D complete orthonormal sets of basis vectors

use as row vectors to obtain unitary matrices A={a(k,m)} & B={b(l,n)}

– Apply to columns and rows Y = A X BT

often choose same unitary matrix as A and B (e.g., 2-D DFT)

For square NxN image A: Y = A X AT X = AH Y A* – For rectangular MxN image A: Y = AM X AN T X = AM

H Y AN*

Complexity ~ O(N3)– May further reduce complexity if unitary transf. has fast implementation

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

)


Basis Images for Separable TransformBasis Images for Separable Transform

X = AH Y A* => x(m,n) = k l a*(k,m)a*(l,n) y(k,l)– Represent X with NxN basis images weighted by coeff. Y– Obtain basis image by setting Y={(k-k0, l-l0)} and getting X

{ a*(k0 ,m)a*(l0 ,n) }m,n

Basis image in matrix form A*k,l = a*k al*T

~ a*k is kth column vector of AH transf. coeff. y(k,l) is the inner product of the basis A*k,l with

image X

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

)


Exercise on Basis Images Exercise on Basis Images

For 2-D separable unitary transform: Y = A X AT => X = AH Y A*

– Represent X with NxN basis images weighted by coeff. Y– Obtain basis image by setting Y={(k-k0, l-l0)} & getting X

In matrix form A*k,l = a*k al*T ~ a*k is kth column vector of AH

Exercise:

– A is Unitary transform or not?– If so, find basis images– Represent an image X with basis images

(Jain’s e.g.5.1, pp137: A’ [5, –1; – 2, 0] A; outer product of columns of AH : [1,1]’[1 1]/2, …)

2

1

2

12

1

2

1

__

__

2

1

2

12

1

2

1

*AYAX H

43

21

11

11

2

1XA

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

)


Review: 2-D DFTReview: 2-D DFT

Recall: 2-D DFT is Separable

Y = F X F X = F* Y F*

– Basis images Bk,l = (ak ) (al

)T ~ outer product of two vectorswhere ak = [ 1 WN

-k WN-2k … WN

-(N-1)k ]T / N

1

0

1

0

1

0

1

0

),(1

),(

),(1

),(

N

k

N

l

mkN

nlN

N

m

N

n

mkN

nlN

WWlkYN

nmX

WWnmXN

lkY

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

/20

04

)


Visualizing Fourier Basis ImagesVisualizing Fourier Basis Images

Fourier basis uses complex exponentials– Their real and imaginary parts give smoothly varying sinusoidal patterns

in different frequencies and orientations

exp[ j 2 (ux + vy) ] = cos[2 (ux + vy)] + j sin[2 (ux + vy)]

v

Real(cos) part

Imaginary(sin) part

(u, v) (1, 0) (0, 5)(1, 1)

Figures from Mani Thomas U.Del CISC489/689 2D FThttp://vims.cis.udel.edu/~mani/TACourses/Spring06/cv/index.html


8x8 DFT Basis Images8x8 DFT Basis Images

Figures from John Woods’ book.


Summary and ReviewSummary and Review

1-D transform of a vector– Represent an N-sample sequence as a vector in N-dimension vector space– Transform

Different representation of this vector in the space via different basis e.g., 1-D DFT from time domain to frequency domain

– Forward transform In the form of inner product Project a vector onto a new set of basis to obtain N “coefficients”

– Inverse transform Use linear combination of basis vectors weighted by transform

coefficents to represent the original signal

2-D transform of a matrix– Generally can rewrite the matrix into a long vector & apply 1-D transform– Separable transform allows applying transform to rows then columns

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

)


Summary and Review (cont’d)Summary and Review (cont’d) Vector/matrix representation of 1-D & 2-D sampled signal

– Representing an image as a matrix or sometimes as a long vector

Basis functions/vectors and orthonormal basis– Used for representing the space via their linear combinations– Many possible sets of basis and orthonormal basis

Unitary transform on input x ~ A-1 = A*T – y = A x x = A-1 y = A*T y = ai

*T y(i) ~ represented by basis vectors {ai*T}

– Rows (and columns) of a unitary matrix form an orthonormal basis

General 2-D transform and separable unitary 2-D transform– 2-D transform involves O(N4) computation– Separable: Y = A X AT = (A X) AT ~ O(N3) computation

Apply 1-D transform to all columns, then apply 1-D transform to rows

– For non-square image of size MxN: Y = AMxM X ATNxN ; basis images MxN

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

01

)


Summary of Today’s LectureSummary of Today’s Lecture

Unitary transform and properties– Basis vectors and Basis images

DCT transform

Next time: putting together …– Transform coding and JPEG compression standard

Readings– Gonzalez’s 3/e book 8.1; 8.2.1, 8.2.3-8.2.7, 8.2.9 (till before motion);

2.6.6-2.6.7; 8.2.8; – Jain’s book 5.1-5.7 (on unitary transform)

– To read more: Woods’ book 4.2, 4.3, 4.5

UM

CP

EN

EE

63

1 S

lide

s (c

rea

ted

by

M.W

u ©

20

04

)

M. Wu: ENEE631 Digital Image Processing (Spring'09) Unitary Transform Spring ’09 Instructor: Min Wu Electrical and Computer Engineering Department, University.

Documents

image transform

quantizer slide

enee631 spring09 lecture

enee631 s09

unitary transform dct

wu electrical

elements umcp enee631

various image processing