Jpeg and Mpeg

8/12/2019 Jpeg and Mpeg

1/68

Introduction to JPEG andMPEG

Ingemar J. Cox

University College London


2/68

UCLAdastra

lParkPostgraduateCampus

Nov 27th 2006 Ingemar J. Cox 2

Outline Elementary information theory

Lossless compression

Quantization

Fundamentals of images

Discrete Cosine Transform (DCT)

JPEG

MPEG-1, MPEG-2


3/68

UCLAdastra



Bibliography D. MacKay, Information Theory, Inference and learning

Algorithms, Cambridge University Press, 2003.

http://www.inference.phy.cam.ac.uk/itprnn/book.html

W. B. Pennebaker and J. L. Mitchell, JPEG Still Image

Data Compression Standard, Chapman Hall, 1993

(ISBN 0-442-01272-1).

G. K. Wallace, The JPEG Still-Picture CompressionStandard, IEEE Trans. On Consumer Electronics, 38,

1, 18-34, 1992.

http://en.wikipedia.org/wiki/JPEG


4/68

UCLAdastra



Bibliography http://en.wikipedia.org/wiki/MPEG-2

T. Sikora, MPEG Digital Video-Coding

Standards, IEEE Signal Processing Magazine,82-100, September 1997
http://en.wikipedia.org/wiki/MPEG-2http://en.wikipedia.org/wiki/MPEG-2http://en.wikipedia.org/wiki/MPEG-2http://en.wikipedia.org/wiki/MPEG-2


5/68

Elementary Information Theory


6/68

UCLAdastra



Elementary Information Theory How much information does a symbol convey?

Intuitively, the more unpredictable or surprising

it is, the more information is conveyed.

Conversely, if we strongly expected something,

and it occurs, we have not learnt very much


7/68

UCLAdastra



Elementary Information Theory If p is the probability that a symbol will occur

Then the amount of information, I, conveyed is:

The information, I, is measured in bits

It is the optimum code length for the symbol

pI

1log 2


8/68

UCLAdastra



Elementary Information Theory The entropy, H, is the average information per

symbol

Provides a lower bound on the compression

that can be achieved

))(

1(log)( 2

spspH

s


9/68

UCLAdastralParkPostgraduateCampus


Elementary Information theory A simple example. Suppose we need to

transmit four possible weather conditions:

1. Sunny2. Cloudy

3. Rainy

4. Snowy

If all conditions are equally likely, p(s)=0.25,

and H=2

i.e. we need a minimum of 2 bits per symbol


10/68



Elementary information theory Suppose instead that it is:

1. Sunny 0.5 of the time

2. Cloudy 0.25 of the time3. Rainy 0.125 of the time, and

4. Snowy 0.125 of the time

Then the entropy is

75.175.05.05.0

3125.02225.015.0

125.0

1log125.02

25.0

1log25.0

5.0

1log5.0 222

H

H

H


11/68



Elementary Information Theory Variable length codewords

Huffman codeinteger code lengths

Arithmetic codesnon-integer code lengths


12/68


13/68



Elementary Information Theory Previous illustration is an example of a lossless

code

I.e. we are able to recover the information exactly


14/68



Elementary Information Theory Note that we have assumed that each symbol

is independent of the other symbols

I.e. the current symbol provides no information

regarding the next symbol


15/68



Quantization Quantization is the process of approximating a

continuous (or range of values) by a (much)

smaller range of values

Where Round(y) rounds y to the nearest integer

is the quantization stepsize

5.0Round),( x

xQ


16/68



Quantization Example: =2

0 1-3 -2 -1 2 3 4 5-5 -4

0-1 1 2-2

0-2 2 4-4


17/68



Quantization Quantization plays an important role in lossy

compression

This is where the loss happens


18/68

Fundamentals of Images


19/68

UCLAdastr

alParkPostgraduateCampus


Fundamentals of images An image consists of pixels (picture elements)

Each pixel represents luminance (and colour)

Typically, 8-bits per pixel


20/68

UCLAdastr



Fundamentals of images Colour

Colour spaces (representations)

RGB (red-green-blue)

CMY (cyan-magenta-yellow)

YUV

Y = 0.3R+0.6G+0.1B (luminance)

U=R-Y

V=B-Y

Greyscale

Binary


21/68

UCLAdastr



Fundamentals of images A TV frame is about 640x480 pixels

If each pixels is represented by 8-bits for each

colour, then the total image size is 640480*3=921,600 bytes or 7.4Mbits

At 30 frames per second, this would be

220Mbits/second


22/68

UCLAdastr



Fundamentals of images Do we need all these bits?


23/68

UCLAdastr



Fundamentals of images Here is an image represented with 8-bits per

pixel


24/68

UCLAdastr



Fundamentals of images Here is the same image at 7-bits per pixel


25/68

UCLAdastr



Fundamentals of images And at 6-bits per pixel


26/68





27/68





28/68



Fundamentals of images Do we need all these bits?

No!

The previous example illustrated the eyessensitivity to luminance

We can build a perceptual model

Only code what is important to the human visualsystem (HVS)

Usually a function of spatial frequency


29/68



Fundamentals of Images Just as audio has temporal frequencies

Images have spatial frequencies

Transforms

Fourier transform

Discrete cosine transform

Wavelet transform Hadamard transform


30/68



Discrete cosine transform Forward DCT

Inverse DCT

1

0

)5.0(8

cos)(2)()(

N

n

nunsuCuS

)5.0(8

cos)(2

)()(

1

0

nu

uSuC

nsN

u


31/68



Basis functions DC term


32/68



Basis functions First term


33/68



Basis functions Second term


34/68



Basis functions Third term


35/68

UCLAdast

ralParkPostgraduateCampus


Basis functions Fourth term


36/68

UCLAdast



Basis functions Fifth term


37/68

UCLAdast



Basis functions Sixth term


38/68

UCLAdast



Basis functions Seventh term


39/68

DCT Example


40/68



Example Signal


41/68



Example DCT coefficients are:

4.2426

0

-3.1543

0

0

0 -0.2242

0


42/68



Example: DCT decomposition DC term


43/68



Example: DCT decomposition 2ndAC term


44/68



Example: DCT decomposition 6thAC term


45/68



Example: summation of DCT terms First two non-zero coefficients


46/68



Example: summation of DCT terms All 3 non-zero coefficients


47/68



Example

What if we quantize DCT coefficients?

=1

Quantized DCT coefficients are:

4

0

-3

0

0

0

0

0


48/68



Example

Approximate reconstruction


49/68

UCLAdas

tralParkPostgraduateCampus


Example

Exact reconstruction


50/68

UCLAdas



2-D DCT Transform

Let i(x,y) represent an image with N rows and

M columns

Its DCT I(u,v) is given by

where

M

x

N

y

vyuxyxivCuCvuI

1 1 16

)12(cos

16

)12(cos),()()(

4

1),(

2

1)0( C 1)( uC


51/68

UCLAdas




Discrete cosine transform

Coefficients are approximately uncorrelated

Except DC term

C.f. original 88 pixel block

Concentrates more power in the low frequency

coefficients

Computationally efficient

Block-based DCT

Compute DCT on 88 blocks of pixels


52/68

UCLAdas




Basis functions for the 88 DCT (courtesy

Wikipedia)


53/68

Fundamentals of JPEG


54/68

UCLAdas




DCT Quantizer Entropy coder

IDCT Dequantizer Entropy

decoder

Compressed

image data

Encoder

Decoder


55/68

UCLAdas




JPEG works on 88 blocks

Extract 88 block of pixels

Convert to DCT domain

Quantize each coefficient Different stepsize for each coefficient

Based on sensitivity of human visual system

Order coefficients in zig-zag order

Entropy code the quantized values


56/68

UCLAdas




A common quantization table is

16 11 10 16 24 40 51 61

12 12 14 19 26 58 60 5514 13 16 24 40 57 69 56

14 17 22 29 51 87 80 62

18 22 37 56 68 109 103 7724 35 55 64 81 104 113 92

49 64 78 87 103 121 120 101

72 92 95 98 112 100 103 99


57/68

UCLAdas




Zig-zag ordering

0 1 5 6 14 15 27 28

2 4 7 13 16 26 29 42

3 8 12 17 25 30 41 43

9 11 18 24 31 40 44 53

10 19 23 32 39 45 52 5420 22 33 38 46 51 55 60

21 34 37 47 50 56 59 61

35 36 48 49 57 58 62 63


58/68

UCLAdas




Entropy coding

Run length encoding followed by

Huffman

Arithmetic

DC term treated separately

Differential Pulse Code Modulation (DPCM)

2-step process

1. Convert zig-zag sequence to a symbol sequence

2. Convert symbols to a data stream


59/68

UCLAdas




Modes

Sequential

Progressive

Spectral selection

Send lower frequency coefficients first

Successive approximation

Send lower precision first, and subsequently refine

Lossless Hierarchical

Send low resolution image first


60/68

Fundamentals of MPEG-1/2


61/68

UCLAdas



Fundamentals of MPEG

A sequence of 2D images

Temporal correlation as well as spatial

correlation

TV broadcast

Frame-based

Field-based


62/68

UCLAdas



MPEG

Moving Picture Experts Group

Standard for video compression

Similarities with JPEG


63/68

UCLAdas



MPEG

Design is a compromise between

Bit rate

Encoder/decoder complexity

Random access capability


64/68

UCLAdas



MPEG

Images

Spatial redundancy

Perceptual redundancy

Video

Spatial redundancy

Intraframe coding

Temporal redundancy Interframe coding

Perceptual redundancy


65/68

UCLAdas



MPEG

Consider a sequence of n frames of video.

It consists of:

I-frames P-frames

B-frames

A sequence of one I-frame followed by P- and

B-frames is known as a GOP Group of Pictures

E.g. IBBPBBPBBPBBP


66/68



MPEG

I-frames

Intraframe coded

No motion compensation

P-frames Interframe coded

Motion compensation

Based on past frames only

B-frames Interframe coded

Motion compensation

Based on past and future frames


67/68



MPEG

Motion-compensated prediction

Divide current frame, i, into disjoint 1616

macroblocks

Search a window in previous frame, i-1, for closestmatch

Calculate the prediction error

For each of the four 88 blocks in the macroblock,

perform DCT-based coding

Transmit motion vector + entropy coded prediction

error (lossy coding)


68/68


MPEG

Like JPEG, the DC term is treated separately

DPCM

B-frame compression high Need buffer and delay

Jpeg and Mpeg

Documents