Transcript
8/2/2019 Video Coding Introduction
http://slidepdf.com/reader/full/video-coding-introduction 1/35
Introduction to Image and Video Coding
© Iain Richardson / Vcodex Ltd 2001-2002
8/2/2019 Video Coding Introduction
http://slidepdf.com/reader/full/video-coding-introduction 2/35
2
Outline
• The need for compression
• The coding model
• Image coding
• Video coding
• Standards
8/2/2019 Video Coding Introduction
http://slidepdf.com/reader/full/video-coding-introduction 3/35
3
The need for compression
• Colour image, 352x288 pixels
• “Full” colour depth: 24 bits
per pixel (8 bits red, green,
blue)
– 304128 bytes
• Reduced colour depth: 12 bits
per pixel
– 152064 bytes
8/2/2019 Video Coding Introduction
http://slidepdf.com/reader/full/video-coding-introduction 4/35
4
The need for compression
• Video signal: 25 frames per second
• “VHS video” quality: 352x288 pixels per frame, 12 bits
per pixel
– 30.4 Mbits per second
• “Television” quality: 704x576 pixels per frame, 12 bits per
pixel
– 121.7 Mbits per second
• too much data for cost-effective transmission or storage
• need compression
8/2/2019 Video Coding Introduction
http://slidepdf.com/reader/full/video-coding-introduction 5/35
5
Image or Video CODEC
• Encode (compress) and decode (decompress) still images or moving
video
• Key issues:
– compression efficiency and image quality
– computational complexity – frame rate
Encoder Channel Decoder Source
video
Source
videoOutput
video
Output
video
8/2/2019 Video Coding Introduction
http://slidepdf.com/reader/full/video-coding-introduction 6/35
6
The coding model
• General-purpose compression: entropy encoding
– remove statistical redundancy from data
– e.g. encode common values with short codes, uncommon values
with longer codes
• Good for text files, poor for images / video
Entropy
Encoder Channel Entropy
Decoder
source
data
decoded
data
enCOder / DECoder
“CODEC”
8/2/2019 Video Coding Introduction
http://slidepdf.com/reader/full/video-coding-introduction 7/35
7
Outline
• The need for compression
• The coding model
• Image coding
• Video coding
• Standards
8/2/2019 Video Coding Introduction
http://slidepdf.com/reader/full/video-coding-introduction 8/35
8
The coding model
• Solution: add a model that attempts to represent theimage/video signal in a form that can be easily compressed
by the entropy encoder
• model exploits the subjective redundancy of images and
video• decoded image may not be identical to original image
EntropyEncoder
Channel EntropyDecoder ImageModel
ImageModelSource
videoSourcevideo Output
videoOutputvideo
8/2/2019 Video Coding Introduction
http://slidepdf.com/reader/full/video-coding-introduction 9/35
9
The coding model
• Image properties that are useful for compression
– many of the pixels of a typical photographic image contain little or
no “useful” detail (e.g. “flat” areas)
– the eye is insensitive to “high frequency” image information
8/2/2019 Video Coding Introduction
http://slidepdf.com/reader/full/video-coding-introduction 10/35
10
Outline
• The need for compression
• The coding model
• Image coding
• Video coding
• Standards
8/2/2019 Video Coding Introduction
http://slidepdf.com/reader/full/video-coding-introduction 11/35
11
Image CODEC (e.g. JPEG)
DCT Quantize Zigzag RLE VLCBlock
IDCT Inverse
Quantize
Inverse
Zigzag
RLD VLDRecon
-struct
TRANSMIT
/ STORE
image model entropy coder
8/2/2019 Video Coding Introduction
http://slidepdf.com/reader/full/video-coding-introduction 12/35
12
RGB
16
16
Blocks
• Process the data in blocks of 8x8 samples• Convert Red-Green-Blue into Luminance (greyscale) and
Chrominance (Blue colour difference and Red colour difference)
• Use half resolution for Chrominance (because eye is more sensitive to
greyscale than to colour)
Cr
8
8
Cb
8
8
Luminance Y
16
16
8/2/2019 Video Coding Introduction
http://slidepdf.com/reader/full/video-coding-introduction 13/35
13
Discrete Cosine Transform
0
2
4
6
8
02
46
8
50
100
150
200
Intensity map
0
2
4
6
81 2 3 4 5
6 7 8
-20
0
20
40
60
80
100
120
DCT coefficients
• Transform each block of 8x8 samples into a block of 8x8
spatial frequency coefficients – energy tends to be concentrated into a few significant coefficients
– other coefficients are close to zero / insignificant
8/2/2019 Video Coding Introduction
http://slidepdf.com/reader/full/video-coding-introduction 14/35
14
Discrete Cosine Transform
• Any 8x8 block of pixelscan be represented as a
sum of 64 basis patterns
(black and white patterns)
•Output of the DCT is theset of weights for these
basis patterns (the DCT
coefficients)
– multiply each basis pattern
by its weight and add themtogether
– result is the original image
block
8/2/2019 Video Coding Introduction
http://slidepdf.com/reader/full/video-coding-introduction 15/35
15
DCT• Most image blocks only contain a few significant coefficients (usually
the lowest “frequencies”)
1 Top-left
coefficient
per block
3 top-left
coefficients
All
coefficients
6 top-left
coefficients
8/2/2019 Video Coding Introduction
http://slidepdf.com/reader/full/video-coding-introduction 16/35
16
Quantize
x
Divide each DCT coefficient by an integer, discard remainder x Result: loss of precision
x Typically, a few non-zero coefficients are left
8/2/2019 Video Coding Introduction
http://slidepdf.com/reader/full/video-coding-introduction 17/35
17
Zigzag Scanning
x “Scan” quantized coefficients in a zig-zag order
x Non-zero coefficients tend to be grouped together
….
8/2/2019 Video Coding Introduction
http://slidepdf.com/reader/full/video-coding-introduction 18/35
18
Run-Level Encoding
x Encode each coefficient value as a (run,level) pair
• run = number of zeros preceding value
• level = non-zero value
• Usually, the block data is reduced to a short sequence of
(run,level) pairs.
• This is now easy to compress using an Entropy Encoder.
Example:
Original data 14,3,4,0,0,-3,0,0,0,0,0,14,…
(Run,level) (0,14)(0,3)(0,4)(2,-3)(5,14)...
8/2/2019 Video Coding Introduction
http://slidepdf.com/reader/full/video-coding-introduction 19/35
19
Variable-Length Coding
x Encode each (run,level) pair using a variable-length codex Frequently occurring groups
• assign a short code
x Infrequently occurring groups
• assign a long code
• Result: compressed version of image.
8/2/2019 Video Coding Introduction
http://slidepdf.com/reader/full/video-coding-introduction 20/35
20
Image CODEC
DCT Quantize Zigzag RLE VLCBlock
IDCT Inverse
Quantize
Inverse
Zigzag
RLD VLDRecon
-struct
TRANSMIT
/ STORE
Source
video
Source
video
Output
video
Output
video
8/2/2019 Video Coding Introduction
http://slidepdf.com/reader/full/video-coding-introduction 21/35
21
Image Decoding
• Reverse the stages to recover the image• Information was thrown away during Quantization
– decoded image will not be identical to the original
• In general:
– more compression = more quality loss• Too much compression:
– block edges start to show (“blockiness”)
– high-frequency patterns start to appear (“mosquito noise”)
8/2/2019 Video Coding Introduction
http://slidepdf.com/reader/full/video-coding-introduction 22/35
22
JPEG: examples
Original
9x
compression
15x
compression
8/2/2019 Video Coding Introduction
http://slidepdf.com/reader/full/video-coding-introduction 23/35
23
Outline
• The need for compression
• The coding model
• Image coding
• Video coding
• Standards
8/2/2019 Video Coding Introduction
http://slidepdf.com/reader/full/video-coding-introduction 24/35
24
Video Coding
• Moving images contain significant temporal redundancy
– successive frames are very similar
• Add an extra “motion model” at the “front end” of the
image encoder.
8/2/2019 Video Coding Introduction
http://slidepdf.com/reader/full/video-coding-introduction 25/35
25
Video Encoder
Motion
Comp.
Motion
Est.
DCT Quant Zigzag RLE VLC
Headers
Buffer
Video frames
Motion
Vectors
Recon. IDCT Rescale
Motion
Vectors
motion model
CurrentCurrent
PreviousPrevious
ReconstructedReconstructed
8/2/2019 Video Coding Introduction
http://slidepdf.com/reader/full/video-coding-introduction 26/35
26
Motion Estimation and Compensation
• The amount of data to be coded can be reduced significantly if the previous frame is subtracted from the current frame:
Residual Image
8/2/2019 Video Coding Introduction
http://slidepdf.com/reader/full/video-coding-introduction 27/35
27
Difference frames
Frame n Frame n - Frame (n-1)
8/2/2019 Video Coding Introduction
http://slidepdf.com/reader/full/video-coding-introduction 28/35
28
Motion Estimation
• Process 16x16 luminance samples at a time (“macroblock”)
• Compare with neighbouring areas in previous frame
• Find closest matching area
– prediction reference
• Calculate offset between current macroblock and prediction reference
area – motion vector
Frame 1 Frame 2
8/2/2019 Video Coding Introduction
http://slidepdf.com/reader/full/video-coding-introduction 29/35
29
Motion Estimation
frame 1 frame 2
motion vectors
8/2/2019 Video Coding Introduction
http://slidepdf.com/reader/full/video-coding-introduction 30/35
30
Motion Compensation
• Subtract the reference area from the current macroblock – difference macroblock
• Encode the difference macroblock with an image encoder
• If motion estimation was effective
– little data left in difference macroblock – more efficient compression.
8/2/2019 Video Coding Introduction
http://slidepdf.com/reader/full/video-coding-introduction 31/35
31
Video Decoder
IZigzagRLDVLD Recon.IDCTRescale
Headers
Buffer Output
video
Output
video
Previous
frames
Previous
frames
8/2/2019 Video Coding Introduction
http://slidepdf.com/reader/full/video-coding-introduction 32/35
32
Outline
• The need for compression
• The coding model
• Image coding
• Video coding
• Standards
8/2/2019 Video Coding Introduction
http://slidepdf.com/reader/full/video-coding-introduction 33/35
33
Coding Standards
• JPEG
– Joint Photographic Experts Group
– Still image compression
• MPEG1
– Moving Picture Experts Group
– Video compression for CD storage / Internet
• MPEG2
– Video compression for digital TV
• MPEG4
– General purpose video compression
• H.261, H.263
– Video compression for video conferencing
8/2/2019 Video Coding Introduction
http://slidepdf.com/reader/full/video-coding-introduction 34/35
34
Video coding examples
MPEG1, 13x compression
30x compression
56x compression
8/2/2019 Video Coding Introduction
http://slidepdf.com/reader/full/video-coding-introduction 35/35
35
References
• “Video Codec Design”, Iain Richardson, John Wiley & Sons 2002
• “Digital Video Communications”, Riley and Richardson, pub. Artech
House, 1997.
• Bhaskaran, V, Konstantinides, K, “Image and video compression
standards - algorithms and architectures”, Kluwer academic publishers, 1996
• Netravali, A N and Haskell, B G, “Digital Pictures: Representation,
Compression and Standards”, 2nd Edition, Plenum Press, 1995.
• Ghanbari, “Video coding: an introduction to standard codecs”, IEE
Press, 1999.
top related