Digital Image Processing Chapter 8: Image Compression 11 August 2006

Digital Image ProcessingChapter 8:

Image Compression

11 August 2006

Digital Image ProcessingChapter 8:

Image Compression

11 August 2006

Data vs Information Data vs Information

Information = Matter (สาระ)

Data = The means by which information is conveyed

Reducing the amount of data required to represent a digital image while keeping information as much aspossible

Image Compression Image Compression

Relative Data Redundancy and Compression Ratio Relative Data Redundancy and Compression Ratio

RD C

R1

1

Relative Data Redundancy

Compression Ratio

2

1

n

nCR

Types of data redundancy

1. Coding redundancy2. Interpixel redundancy3. Psychovisual redundancy

Coding Redundancy Coding Redundancy

Different coding methods yield different amount of data needed to represent the same information.

Example of Coding Redundancy : Example of Coding Redundancy : Variable Length Coding vs. Fixed Length CodingVariable Length Coding vs. Fixed Length Coding

Lavg 3 bits/symbol Lavg 2.7 bits/symbol(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.

Variable Length Coding Variable Length Coding

Concept: assign the longest code word to the symbol with the least probability of occurrence.

(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.

Interpixel Redundancy Interpixel Redundancy

Interpixel redundancy:Parts of an image arehighly correlated.

In other words,we canpredict a given pixelfrom its neighbor.


Run Length CodingRun Length Coding

The gray scale imageof size 343x1024 pixels

Binary image= 343x1024x1 = 351232 bits

Line No. 100

Run length coding

Line 100: (1,63) (0,87) (1,37) (0,5) (1,4) (0,556) (1,62) (0,210)

Total 12166 runs, each run use 11 bits Total = 133826 Bits(Images from Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2nd Edition.

Psychovisual Redundancy Psychovisual Redundancy

The eye does not response with equal sensitivity to all visual information.

8-bit gray scale image

4-bit gray scale image

4-bit IGS image

False contours


Improved Gray Scale Quantization Improved Gray Scale Quantization

Pixeli-1i

i+1i+2i+3

Gray levelN/A0110 1100

1000 10111000 01111111 0100

Sum0000 0000

0110 11001001 01111000 11101111 0100

IGS CodeN/A0110

100110001111

+

Algorithm1. Add the least significant 4 bits of the previous value of Sum to the 8-bit current pixel. If the most significant 4 bit of the pixel is 1111 then add 0000 instead. Keep the result in Sum

2. Keep only the most significant 4 bits of Sum for IGS code.

Fidelity Criteria: Fidelity Criteria: how good is the compression algorithm how good is the compression algorithm

-Objective Fidelity Criterion- RMSE, PSNR

-Subjective Fidelity Criterion:-Human Rating


Image Compression Models Image Compression Models

Source encoder Channel encoder

Source decoder Channel decoder

Channel

),(ˆ yxf

),( yxf

Reduce data redundancy

Increase noiseimmunity

Noise

Source Encoder and Decoder Models Source Encoder and Decoder Models

Mapper Quantizer Symbol encoder),( yxf

Source encoder

Inverse mapper Symbol decoder

Source decoder

),(ˆ yxf

Reduceinterpixel

redundancy

Reducepsychovisual redundancy

Reducecoding

redundancy

Channel Encoder and Decoder Channel Encoder and Decoder

- Hamming code, Turbo code, …

Information Theory Information Theory

Measuring information

)(log)(

1log)( EP

EPEI

Entropy or Uncertainty: Average information per symbol

j

jj aPaPH ))(log()(

Simple Information System Simple Information System

Binary Symmetric Channel Binary Symmetric Channel

A = {a1, a2} ={0, 1}z = [P(a1), P(a2)]

B = {b1,b2} ={0, 1}v = [P(b1), P(b2)]

Source Destination


0

1

(1-Pe)

Pe

0

1Pe

(1-Pe)

Source Destination

Pe= probability of error

P(a1)

1-P(a1)

P(a1)(1-Pe)+(1-P(a1))Pe

(1-P(a1))(1-Pe)+P(a1)Pe


A = {a1, a2} ={0, 1}z = [P(a1), P(a2)]

B = {b1,b2} ={0, 1}v = [P(b1), P(b2)]

H(z) = - P(a1)log2P(a1) - P(a2)log2P(a2)

Source Destination

H(z|b1) = - P(a1|b1)log2P(a1|b1) - P(a2|b1)log2P(a2|b1)

H(z|b2) = - P(a1|b2)log2P(a1|b2) - P(a2|b2)log2P(a2|b2)

H(z|v) = H(z|b1) + H(z|b2)

Mutual information I(z,v)=H(z) - H(z|v)

Capacity ),(max vzz

IC


Let pe = probability of error

bs

bs

ee

ee

p

p

pp

pp

1

1v

bs

bs

p

p

ap

ap

)(1

)(

1

1z

)1(log)1()(log)( 22 bsbsbsbs ppppH z

)(log))1)(1((log)1)(1(

))1((log)1())1((log)1()|(

22

22

ebsebsebsebs

ebsebsebsebs

pppppppp

ppppppppH

vz

)()(),( ebsebsebsbs pHppppHvzI

)(1 ebs pHC



Communication System ModelCommunication System Model


2 Cases to be considered: Noiseless and noisy

Noiseless Coding TheoremNoiseless Coding Theorem

Problem: How to code data as compact as possible?

Shannon’s first theorem: defines the minimum average code word length per source that can be achieved.

Let source be {A, z} which is zero memory source with J symbols. (zero memory = each outcome is independent from other outcomes)

then a set of source output of n element be

},...,,{ 321 nJA

}111,110,101,100,011,010,001,000{A

Example:

}1,0{A

for n = 3,

Noiseless Coding Theorem (cont.)Noiseless Coding Theorem (cont.)

Probability of each j is

)()()()( 21 jnjjj aPaPaPP

Entropy of source :

)())(log()()(1

zz nHPPHnj

iii

Each code word length l(i) can be

1)(

1log)(

)(

1log

ii

i Pl

P

Then average code word length can be

1)(

1log)()()(

)(

1log)(

111

i

J

iii

J

ii

i

J

ii P

PlPP

Pnnn

Noiseless Coding Theorem (cont.)Noiseless Coding Theorem (cont.)

nH

n

LH avg 1

)()(

zz

or

)(lim zHn

Lavg

n

The minimum average code word length per source symbol cannotlower than the entropy.

Coding efficiency

avgL

zHn

)(

We get 1)()( zz HLH avg

from )()( zz nHH

then

Extension Coding Example Extension Coding Example


H = 1.83Lavg = 1.89

H = 0.918Lavg = 1

918.01

918.011 97.0

89.1

83.12

Noisy Coding TheoremNoisy Coding Theorem

Problem: How to code data as reliable as possible?

Example: Repeat each code 3 times:

Source data = {1,0,0,1,1}

Data to be sent = {111,000,000,111,111}

Shannon’s second theorem: the maximum rate of coded information is

rR

log

r = Block length = code size

Rate Distortion Function for BSC Rate Distortion Function for BSC


Error-Free Compression: Huffman Coding Error-Free Compression: Huffman Coding


Step 1: Source reduction

Huffman coding: give the smallest possible number of code symbols per source symbols.

Error-Free Compression: Huffman Coding Error-Free Compression: Huffman Coding


Step 2: Code assignment procedure

The code is instantaneous uniquely decodable without referencing succeeding symbols.

Near Optimal Variable Length Codes Near Optimal Variable Length Codes


Arithmetic Coding Arithmetic Coding


Nonblock code: one-to-one correspondence between source symbolsAnd code words does not exist.Concept: The entire sequences of source symbols is assigned a singlearithmetic code word in the form of a number in an interval of real number between 0 and 1.

Arithmetic Coding Example Arithmetic Coding Example


0.2x0.4 0.04+0.8x0.04 0.056+0.8x0.016

0.2x0.2 0.04+0.4x0.04 0.056+0.4x0.016

The number between 0.0688and 0.06752can be used torepresent thesequence a1 a2 a3 a3 a4

LZW Coding LZW Coding


Lempel-Ziv-Welch coding : assign fixed length code words to variable length sequences of source symbols.

24 Bits

9 Bits

LZW Coding Algorithm LZW Coding Algorithm

0. Initialize a dictionary by all possible gray values (0-255)1. Input current pixel2. If the current pixel combined with previous pixels form one of existing dictionary entries Then 2.1 Move to the next pixel and repeat Step 1 Else

2.2 Output the dictionary location of the currently recognized sequence (which is not include the current pixel)

2.3 Create a new dictionary entry by appending the currently recognized sequence in 2.2 with the current pixel

2.4 Move to the next pixel and repeat Step 1

LZW Coding Example LZW Coding Example

Input pixel

3939

1261263939

1261263939

126126

Dictionary Location Entry 0 0 1 1 … … 255 255 256 39-39 257 39-126 258 126-126 259 126-39 260 39-39-126 261 126-126-39 262 39-39-126-126

EncodedOutput (9 bits)

3939126126

256

258

260

CurrentlyrecognizedSequences

393912612639

39-39126

126-12639

39-3939-39-126

126

Bit-Plane Coding Bit-Plane Coding


Original imageBit 7

Bit 6

Bit 0

…

Bit planeimages

Binary image compression



Example of binary image compression: Run length coding

Bit Planes Bit Planes


Original grayscale image

Bit 7

Bit 6

Bit 5

Bit 4

Bit 3

Bit 2

Bit 1

Bit 0

Gray-coded Bit Planes Gray-coded Bit Planes


a7

a6

a5

a4

g7

g6

g5

g4

Gray code:

60for 1

i

aag iii

77 ag

and

ai= Original bit planes

= XOR

Originalbit planes

Gray-coded Bit Planes (cont.) Gray-coded Bit Planes (cont.)

a3

a2

a1

a0

g3

g2

g1

g0

There are less 0-1 and 1-0 transitions in grayed codebit planes. Hence gray coded bit planes are more efficient for coding.

Relative Address Coding (RAC) Relative Address Coding (RAC)


Concept: Tracking binary transitions that begin and end eack black and white run

Contour tracing and CodingContour tracing and Coding


Represent each contour by a set of boundary points and directionals.

Error-Free Bit-Plane Coding Error-Free Bit-Plane Coding


Lossless VS Lossy Coding Lossless VS Lossy Coding


Mapper Quantizer Symbol encoder),( yxfSource encoder

Reduceinterpixel

redundancy

Reducepsychovisual redundancy

Reducecoding

redundancy

Mapper Symbol encoder),( yxf

Source encoder

Reduceinterpixel

redundancy

Reducecoding

redundancy

Lossless coding

Lossy coding

Transform Coding (for fixed resolution transforms) Transform Coding (for fixed resolution transforms)


Construct nxn

subimages

Forward transform

QuantizerSymbolencoder

Input image(NxN)

Compressedimage

Construct nxn

subimages

Inverse transform

Symboldecoder

Decompressedimage

Decoder

Encoder

Examples of transformations used for image compression: DFT and DCT

Quantization process causesThe transform coding “lossy”

Transform Coding (for fixed resolution transforms) Transform Coding (for fixed resolution transforms)

3 Parameters that effect transform coding performance:

1. Type of transformation

2. Size of subimage

3. Quantization algorithm

2D Discrete Transformation 2D Discrete Transformation

1

0

1

0

),,,(),(),(N

x

N

y

vuyxgyxfvuT

Forward transform:

1

0

1

0

),,,(),(),(N

u

N

v

vuyxhvuTyxf

Inverse transform:

where g(x,y,u,v) = forward transformation kernel or basis function

where h(x,y,u,v) = inverse transformation kernel or inverse basis function

T(u,v) is called the transform coefficient image.

Transform Example: Transform Example: Walsh-Hadamard Basis Functions Walsh-Hadamard Basis Functions


1

0

)()()()(

)1(1

),,,(),,,(

m

iiiii vpybupxb

Nyxvuhvuyxg

N = 4

N = 2m

bk(z) = the kth bit of z

)()()(

)()()(

)()()(

)()(

011

322

211

10

ububup

ububup

ububup

ubup

m

mm

mm

m

Advantage: simple, easy to implementDisadvantage: not good packing ability

Transform Example: Transform Example: Discrete Cosine Basis Functions Discrete Cosine Basis Functions


DCT is one of the most frequently used transform for image compression.For example, DCT is used in JPG files.

N = 4

N

vy

N

uxvuyxvuhvuyxg

2

)12(cos

2

)12(cos)()(),,,(),,,(

1,,1for 2

0for 1

)(

NuN

uNu

N = 4Advantage: good packing ability,modulate computational complexity

Transform Coding Examples Transform Coding Examples


Original image512x512 pixels

Fourier

Hadamard

DCT

Error

Subimage size:8x8 pixels = 64 pixels

Quatization by truncating 50% of coefficients (only32 max cofficients are kept.)

RMS Error = 1.28

RMS Error = 0.86

RMS Error = 0.68

DCT vs DFT Coding DCT vs DFT Coding


Advantage of DCT over DFT is that the DCT coefficients aremore continuous at boundaries of blocks.

DFT coefficientshave abruptchanges atboundariesof blocks

1 Block

Subimage Size and Transform Coding PerformanceSubimage Size and Transform Coding Performance


This experiment:Quatization is made bytruncating 75% of transform coefficients

DCT is the best

Size 8x8 is enough


Subimage Size and Transform Coding PerformanceSubimage Size and Transform Coding Performance

Zoomed detailOriginal

Reconstructedby using 25%of coefficients(CR = 4:1)with 8x8 sub-images

DCT Coefficients

Zoomed detailSubimage size:

8x8 pixels


2x2 pixels


4x4 pixels

Quantization Process: Bit Allocation Quantization Process: Bit Allocation

To assign different numbers of bits to represent transform coefficients based on importance of each coefficient:

- More importance coefficeitns assign a large number of bits

- Less importance coefficients assign a small number of bits or not assign at all

2 Popular bit allocation methods1. Zonal coding : allocate bits based on the basis of

maximum variance, using fixed mask for all subimages

2. Threshold coding : allocate bits based on maximum magnitudes of coefficients

Example: Example: Results with Different Bit Allocation MethodsResults with Different Bit Allocation Methods


Reconstructedby using 12.5%of coefficients(8 coefficientswith largestmagnitude areused)

Threshold codingError

Zoom details

Reconstructedby using 12.5%of coefficients(8 coefficients with largest variance are used)

Zonal codingError

Zonal Coding Example Zonal Coding Example


Zonal mask Zonal bit allocation

Threshold Coding Example Threshold Coding Example


Threshold mask Thresholded coefficientordering

Thresholding Coding Quantization Thresholding Coding Quantization


3 Popular Thresholding MethodsMethod 1: Global thresholding : Use a single global threshold

value for all subimages

Method 2: N-largest coding: Keep only N largest coefficients

Method 3: Normalized thresholding: each subimage is normalized by a normalization matrix before rounding

Example of Normalization Matrix Z(u,v)

),(

),(),(ˆ

vuZ

vuTroundvuT

),(),(ˆ),(~

vuZvuTvuT

Bit allocation

Restoration before decompressing

DCT Coding Example DCT Coding Example


(CR = 38:1) (CR = 67:1)

Zoom details

Method: - Normalized Thresholding,- Subimage size: 8x8 pixels

BlockingArtifact at Subimageboundaries

Error imageRMS Error = 3.42

Wavelet Transform Coding: Multiresolution approachWavelet Transform Coding: Multiresolution approach

Wavelet transform


Input image(NxN)

Compressedimage

Inverse wavelet

transform

Symboldecoder

Decompressedimage

Decoder

Encoder

Unlike DFT and DCT, Wavelet transform is a multiresolution transform.

What is a Wavelet TransformWhat is a Wavelet TransformOne up on a time, human uses a line to represent a number. For example

= 25With this numerical system, we need a lot of space to representa number 1,000,000.Then, after an Arabic number system is invented, life is much easier.We can represent a number by a “digit number”:

X,XXX,XXX

The 1st digit = 1x

The 2nd digit = 10xThe 3rd digit = 100x

…

An Arabic number is onekind of multiresolutionRepresentation.

Like a number, any signal can also be represented by a multiresolutiondata structure, the wavelet transform.

What is a Wavelet TransformWhat is a Wavelet Transform

Wavelet transform has its background from multiresolutionanalysis and subband coding.

- Nyquist theorem: The minimun sampling rate needed for samplinga signal without loss of information is twice the maximum frequencyof the signal.

-We can perform frequency shift by multiplying a complex sinusiodalsignal in time domain.

Other important background:

),(),( 00)(2 00 vvuuFeyxf yvxuj

Wavelet History: Image PyramidWavelet History: Image Pyramid

Pyramidal structured image

Coarser, decrease resolution

Finer, increase resolution

If we smooth and then down sample an image repeatedly, we willget a pyramidal image:


Image Pyramid and Multiscale DecompositionImage Pyramid and Multiscale Decomposition

ImageNxN

SmoothDown

SamplingBy 2

ImageN/2xN/2

Question: What Information is loss after downSampling?

Answer: Loss Information isA prediction error image:

UpSampling

By 2

Interpolate

PredictedImageNxN

Prediction Error

(loss details)NxN

-

+

Image Pyramid and Multiscale Decomposition (cont.)Image Pyramid and Multiscale Decomposition (cont.)

Hence we can decompose an image using the following process

ImageNxN

Smooth and down sampling

by 2

Approxi--mationImage

N/2xN/2

Upsamplingby 2 and

interpolate

Prediction ErrorNxN

Smooth and down sampling

by 2

Approxi--mationImage

N/4xN/4

Upsamplingby 2 and

interpolate

Prediction Error

N/2xN/2

+-

+-

….

Image Pyramid and Multiscale Decomposition (cont.)Image Pyramid and Multiscale Decomposition (cont.)

Original ImageNxN

Prediction error(residue)

NxN

Predictionerror

N/2xN2

Prediction error N/4xN/4

Approximation image N/8xN/8

=

Multiresolution Representation

Multiresolution Decomposition Process Multiresolution Decomposition Process

Note that this process is not a wavelet decomposition process !


Example of Pyramid Images Example of Pyramid Images

ApproximationImages (usingGaussian Smoothing)

Predictionresidues


Subband Coding Subband Coding

x(n)N points

LPF

HPF

DownSampling

by 2

Freq.shift by

N/2

DownSampling

by 2

Subband decomposition process

h1(n)

h0(n)

a(n)N/2 points

d(n)N/2 points

All information of x(n) is completely preserved in a(n) and d(n).

Approximation

Detail

Subband Coding (cont.) Subband Coding (cont.)

x(n)N points

UpSampling

by 2

UpSampling

by 2

Subband reconstruction process

Interpolation

g0(n)

a(n)N/2 points

d(n)N/2 points Interpolation

Freq.shift by

N/2

g1(n)

Subband Coding (cont.) Subband Coding (cont.)


2D Subband Coding 2D Subband Coding


Example of 2D Subband CodingExample of 2D Subband Coding


Diagonal detail:filtering in bothx and y directionsusing h1(n)

Horizontal detail:filtering in x-direction using h1(n) and in y-direction usingh0(n)

Approximation:filtering in bothx and y directionsusing h0(n)

Vertical detail: filtering in x-direction using h0(n) and in y-direction using h1(n)

1D Discrete Wavelet Transformation 1D Discrete Wavelet Transformation

x(n)N points

h(n)

h(n) 2

2

h (n)

h (n) 2

2

h (n)

h (n) 2

2

d1(n) N/2 points

d2(n) N/4 points

d3(n) N/8 points

a3(n) N/8 points

Wavelet coefficients(N points)

Note that the numberof points of x(n) andwavelet coefficients are equal.

(n) = a wavelet function(n) = a scaling function



d2 h2

v2 a2

a1

h1d1

v1

Original imageNxN

a3d3 h3v3

Level 1

Level 2

Level 3

d = diagonal detailh = horizontal detailv = vertical detaila = approximation

2D Discrete Wavelet Transformation (cont.) 2D Discrete Wavelet Transformation (cont.)

d2

h2

v2h1

d1v1

a3d3h3

v3

Original imageNxN

Wavelet coefficientsNxN

d = diagonal detail: filtering in both x and y directions using h (n)h = horizontal detail: filtering in x-direction using h (n) and in y direction using h (n)v = vertical detail: filtering in x-direction using h (n) and in y

direction using h (n)a = approximation: filtering in both x and y directions using h (n)

OriginalImage

Example of 2D Wavelet Transformation Example of 2D Wavelet Transformation

Original image

LH1

HL1

HH1

LL1LL1

Example of 2D Wavelet Transformation (cont.) Example of 2D Wavelet Transformation (cont.)

The first level wavelet decomposition

LL2

LH1

HL1

HH1

LH2 HH2

HL2LL2


The second level wavelet decomposition

LH1 HH1

LH2 HH2

HL2

HL3

HH3LH3

LL3

HL1

The third level wavelet decomposition


Example of 2D Wavelet TransformationExample of 2D Wavelet Transformation


Examples: Types of Wavelet TransformExamples: Types of Wavelet Transform


Haarwavelets

Symlets

Daubechieswavelets

Biorthogonalwavelets

Wavelet Transform Coding for Image CompressionWavelet Transform Coding for Image Compression

Wavelet transform


Input image(NxN)

Compressedimage

Inverse wavelet

transform

Symboldecoder

Decompressedimage

Decoder

Encoder

Unlike DFT and DCT, Wavelet transform is a multiresolution transform.

Wavelet Transform Coding Example Wavelet Transform Coding Example


(CR = 38:1) (CR = 67:1)

Error ImageRMS Error = 2.29

Zoom details

No blockingArtifact

Error ImageRMS Error = 2.96

Wavelet Transform Coding Example (cont.) Wavelet Transform Coding Example (cont.)


(CR = 108:1) (CR = 167:1)



Zoom details

Wavelet Transform Coding vs. DCT Coding Wavelet Transform Coding vs. DCT Coding


(CR = 67:1) (CR = 67:1)

Zoom details


Wavelet DCT 8x8


Type of Wavelet Transform and PerformanceType of Wavelet Transform and Performance


No. of Wavelet Transform Level and PerformanceNo. of Wavelet Transform Level and Performance


Threshold Level and PerformanceThreshold Level and Performance


Table 8.14 (Cont’)









Table 8.19 (Con’t)





Lossless Predictive Coding Model Lossless Predictive Coding Model


Lossless Predictive Coding Example Lossless Predictive Coding Example


Lossy Predictive Coding Model Lossy Predictive Coding Model


Delta Modulation Delta Modulation


Linear Prediction Techniques: Examples Linear Prediction Techniques: Examples


Quantization Function Quantization Function


Lloyd-Max Quantizers Lloyd-Max Quantizers


Lossy DCPM Lossy DCPM


DCPM Result Images DCPM Result Images


Error Images of DCPM Error Images of DCPM


Digital Image Processing Chapter 8: Image Compression 11 August 2006

Documents

image compression11

average information

visual information

matter data

example of coding redundancy

gray scale image4

variable length coding

run length codingline