Standards Video Mpeg English

8/12/2019 Standards Video Mpeg English

1/30

Video coding standardsVideo coding standards -- MPEGMPEG Introduction

ISO video coding standards MPEG-x

I, P, B pictures

MPEG structure

MPEG syntax

MPEG bit allocation and rate control

Introduction to DVB

Thanks for material provided

Inald Lagendijk, Delft University of Technology

Thomas Wiegand, Heinrich-Hertz-Institut

Video coding standardsVideo coding standards -- historyhistory

1984 1986 1988 1990 1992 1994

H.261 H.263H.263+, H.26L

MPEG- 2

MPEG- 1ISO

ITU

MPEG- 4

1996 1998

Frame- based

Object- based

2000

H.264/AVC


2/30

General video coding standardGeneral video coding standardControl

DCT Q VLC MUX Buffer

Overhead information

Q-1

IDCT

+

+

+-

PredictorMemory

Motion

estimationVLC

H.261: the basis of videoH.261: the basis of videocompressioncompression

ITU-T (ex-CCITT) Rec. H.261: The first widespread practical success

First design (late 80s) embodying typical structure that dominatestoday: 16x16 macroblock motion compensation, 8x8 DCT, scalar

quantization, and variable-length coding

Other key aspects: loop filter, integer-pel motion compensationaccuracy, 2-D VLC for coefficients

Operated at 64 - 2048 kbits/s ( n x 64 kbits/s, n = 1,,30)352 x 288 x 29.97 4:2:0

QCIF Progressive

176 x 144 x 29.97 4:2:0

Still in use, although mostly as a backward-compatibility feature overtaken by H.263


3/30

The big improvementThe big improvement

IIPP -- B picturesB pictures

Time

I BB B P B BPB

I: Intraframe pictures (random access)

P: Prediction pictures

B: Bidireccional prediction pictures

I

Bidireccional predictionBidireccional prediction -- B picturesB pictures

B = A; Forward prediction

B = C; Backward prediction

B = (A + C)/2; Bidireccional prediction

A

B

C

Frame I or P

Frame I or P

Frame B


4/30

I, P, B picturesI, P, B pictures

I - Picturescoded independent of any other pictureallows decoder for random accessused as reference for coding of other pictures

P - Picturescoded like in hybrid coderused as reference for coding of other pictures

B - Picturesextension of hybrid codernever used as reference for coding other pictures

GOP structuresGOP structures Typical GOP structures:

I I I I I I I I I I I I Only intra frames Much like Motion-JPEG

I P P P P P P P P P P

Pure hybrid coder Problem to start decoding process

(B B) I B B P B B P B B P Common practice Has I picture twice per second Presence/way of coding the first 2 B pictures determines open or

closed GOP

GOP structure not prescribed and can be changed on-the-fly:Scene changes, rapid transition, slow motion, freeze frame, ...


5/30

Transmission orderTransmission order

I IB B B P BBB

1 42 3 65 987

1 42 3 65 987Temporal order:

1 35 2 94 876Transmission order:

There is a delay of B + 1 frames

ITU (H.26x)ITU (H.26x) -- ISO (MPEG) standardsISO (MPEG) standards

MPEG could be used for communicative services

Often round trip delays are important, which has not been

an issues of much importance in MPEG

ITU has standards similar to MPEG but with

less delay

less overhead in terms of selectable options

more error robustness

optimized for lower rates

H.261 (~1988) and H.263 (~1995)

but the basics are the same as MPEG


6/30

MPEG standardizationMPEG standardization

Joint standardization effort of ISO and IECISO-IEC JTC1/SC29/WG11

Nickname: MPEG - Moving Pictures Expert Group

Standard for the compression of video and associatedaudio for digital video applications

MPEG-1 (~1992): ISO/IEC 11172

Digital storage media and PC; up to 1.5 Mbit/sec

MPEG-2 (~1994): ISO/IEC 13818

Digital TV and HDTV broadcasting; up to 15 Mbit/sec

MPEG-4 (~1998)

Multimedia/Internet standard; Compression is less important

MPEG-7 (~2001)

Multimedia meta-data standard; Not a compression standard

MPEG structureMPEG structure

MPEG standards deals with:Video: describes coded representation of digital video

Audio: similar for digital audio

System: describes multiplexing structure, means to

represent timing information, and possibilities toenclose other information streams

Conformance: describes procedures for testingcompliance with the system requirements

Simulation software

Digital storage media command and control

Multi-channel audio coding

Real-time interface


7/30

MPEG syntax standardization (1)MPEG syntax standardization (1)

MPEG does not standardize the encoder nor the decoder implicitly the structure of the encoder and decoder is definednamely, the hybrid video coder

The MPEG standard definesthe bit stream syntaxthe way this syntax needs to be interpreted

Within the syntax, many degrees of freedom existMPEG encoders come in all kinds of qualities

MPEG syntax standardization (2)MPEG syntax standardization (2)

MPEG encoder

A

MPEG encoder

B

MPEG decoder

I

MPEG decoder

II

MPEG decoder

III

Bit stream A

Bit stream B

Encoded at same bit rate

Output video 2

Video input

Output video 1

Output video 3

identica

quality

(very)different

quality


8/30

Standard MPEGStandard MPEG -- 1 (1993)1 (1993)

Formally ISO/IEC 11172-2 (93), developed by ISO/IEC JTC1SC29 WG11 (MPEG) use is fairly widespread, but mostly

overtaken by MPEG-2

Can provide approximately VHS quality between 1-2 Mbps

Superior quality to H.261 when operated at higher bit rates ( 1Mbps for CIF 352x288 resolution)

Other video formats and bit rates are allowed

Internet: QSIF (176 pixels per line x 144 lines per frame) Technical features: Adds bi-directional motion prediction and

half-pixel motion to H.261 design

MPEGMPEG -- 1 preprocessing1 preprocessing

Most video sources are not SIF: preprocessing needed

Spatial resolution conversion, e.g. 720x576 to352x288

Interlaced to progressive picture conversion to getrid of the fields

Color conversion, e.g. RGB to YUV 4:2:0 Noise filtering Low pass filtering (smoothing) Frame skipping


9/30


Formally ISO/IEC 13818-2 & ITU-T H.262, developed (94)jointly by ISO/IEC SC29 WG11 (MPEG) and ITU-T

Now in wide use for DVD standard and DTV

(the most commonly used video coding standard)

Primary new technical features: support for interlaced-scanpictures and scalability

Essentially the same as MPEG-1 for progressive-scan

pictures, and MPEG-1 forward compatibility required Not especially useful below 2 Mbps (range of use normally

2-20 Mbps)


Generic coding of video for TV broadcast and HDTV

I, P, B pictures

Profiles and Levels

HDTV: 18 - 36 Mbits/s

PAL quality: 4 Mbits/s

Transparent quality: 9 Mbits/s


10/30

MPEG-1 MPEG-2

Standard 1992 1994Main application Digital video for

CD-ROMDigital TV(HDTV)

Spatial resolution CIF (1/4 TV)apr. 288 x 360 pixels

TV(4 x TV)576 x 720 pixels(1152 x 1440 pixels)

Temporal resolution 25 - 30 frames/s 50-60 fields/s(100-120 field/s)

Bit Rate 1.5 Mbit/s 15 Mbit/s

(60 Mbit/s)Quality equivalent to VHS equivalent to

NTSC/PALCompressionover PCM

apr. 20 - 30 apr. 30-40(apr. 30-40)

MPEG 1MPEG 1 -- MPEG 2 parametersMPEG 2 parameters

MPEG progressive formatsMPEG progressive formats

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

4:2:2 progressive

X

X

X

4:2:0 progressive MPEG 1

X

X

X

X

X

X

X

X

4:2:0 progressive MPEG 2

X

X

X X X

X

X

Luminance sample Y

X Chrominance sample Cb, Cr 5061

502

..

.

+

=

+

=

YRC

YBC

r

b


11/30

MPEG 2 interlaced formatMPEG 2 interlaced format

XX X

XX X

XX X

4:2:0 interlaced

top field

4:2:0 interlaced

bottom field

Luminance sample Y

X Chrominance sample Cb, Cr (defined in the top field)

Macroblock 4:2:0Macroblock 4:2:0

16 samples

Y0 Y1

Y3Y2

16 samples Cb

8 samples 8 samples

8 samples

Motion estimation: Macroblocks of Y

Interlaced systems: Field interpolation before motion estimation

DCT: On blocks of 8 x 8

Cr


12/30

Macroblock 4:2:2Macroblock 4:2:2

16 samples

Y0 Y1

Y3Y2

16 samples

Cb1

8 samples 8 samples

8 samplesCr1

Cb1 Cr1 8 samples

Motion estimation: Macroblocks of Y

Interlaced systems: Field interpolation before motion estimation

DCT: On blocks of 8 x 8

Structure of layered syntaxStructure of layered syntaxVideo sequence

Group of pictures

Picture

Slice

Slice

Macroblock

Block

8 pixels

8 pixels


13/30

Example of syntaxExample of syntax Each layer in the syntax contains

header with relevant parameters for lower layers picture size, bit rate, coding method, motion vectors, ...

unique start code for resynchronization (high layers only) hex start code: 00 00 01 xx ~ 23 zeroes+1+8 bit parameter

Pic tur e header CD #bits Mnic

picture_header(){picture_sc 32 bslbf temporal reference 10 uimsbf picture_coding_type 3 uimsbf vbv_delay 16 uimsbf if (picture_coding_type==2 ||

picture_coding_type==3){full_pel_forward_vector 1forward_f_code 3 uimsbf

}

if (picture_coding_type==3){full_pell_backward_vector 1backward_f_code 3 uimsbf

}while (next_bits()=="1"){

extra_bit_picture 1 "1"extra_information_picture 8

}extra_bit_picture 1 "0"next_start_code()

}

Scalable syntaxScalable syntax

Spatial scalability Hierarchy of compatible resolutions, e.g. HDTV, SDTV, SIF

SNR scalability Hierarchy of qualities, e.g. graceful degradations

Temporal scalability Hierarchy of compatible frame/field rates (120-100-60-50 Hz)

Data partitioning scalability Low complexity solution to getting hierarchical bit streams

with different importance for decoding process


14/30

Example of spatial scalabilityExample of spatial scalability

DCT

Q

VLC

1:2

IDCT

Q

-1

VLCDCT

+-

2:1

Q

supplement bit rate stream

for normal resolution video

low bit rate stream

for low resolution video

(e.g. MPEG-1)

high-end receiver

low-end receiver

Example of PSNR scalabilityExample of PSNR scalability

DCT

Q

VLC

IDCT

Q-1

VLCDCT

+-

Q

supplement bit rate streamfor normal quality MPEG-2

video

low bit rate stream

for low quality MPEG-2 video


15/30

Coding of IntraCoding of Intra--pictures (1)pictures (1)

$

( , ),

,

,

,

k l

k l

k l

k l

QWk l

=

= =

nint

nint other

1 1

80

=

8369564638352927

6956463834292726

5848403532292726

4840353229272622

40373429272622223834342927262219

3734292724221616

34292726221916.8

,lkW

Quantization procedure

Coarseness of quantization is controlled and adapted by

the parameter Q

can change per slice or permacroblock

the intra quantization matrix

can change per sequence (MPEG-1)

or per picture (MPEG-2)

Coding of IntraCoding of Intra--pictures (2)pictures (2)

Quantizer has no dead zone

DC coefficient is encoded differentially with previousDCT block (take into account the macroblock structure)

VLC codes: 2-D Huffman based on zig-zag scan and (zero-runlength,

amplitude)

Only combinations of high probability are VLC encoded Others get escape code plus FLC to avoid large Huffman

tables

Macroblock types: Default Q-setting / New Q-setting


16/30

Coding of P pictures (1)Coding of P pictures (1)

=

lk

lk

lkWQ ,

,, nint

16 16 16 16 16 16 16 16

16 16 16 16 16 16 16 16

16 16 16 16 16 16 16 16

16 16 16 16 16 16 16 1616 16 16 16 16 16 16 16

16 16 16 16 16 16 16 16

16 16 16 16 16 16 16 16

16 16 16 16 16 16 16 16

Quantization:

plus dead zone

Use fixed inter quantization matrix

Wk,l=


Motion estimation: Not specified

Motion compensation: on macro blocks (16x16pixels) Half pixel accurate or finer

Encoding of motion vectors differentially encoded to last vector (of same time) VLC for base vector FLC for residues


17/30


Badly predicted macroblocks may yield a larger difference signal than

the original picture information

Can decide to use intra coding mode for such macroblock

Within P-pictures macro-blocks have the following coding types:

non-intra / intra

motion vectors / no-motion vectors

quantized prediction difference is zero / not zero default Q-setting / new Q-setting

Coding of B picturesCoding of B pictures

In B-pictures badly predicted macroblocks may occur Can decide to use intra coding mode for such macroblock

Within B-pictures macro-blocks have the following codingtypes:

intra / non-intra

forward predicted / backward predicted / interpolated

quantized prediction difference is zero / not zero

default Q-setting / new Q-setting


18/30

Rate control and bit allocationRate control and bit allocation

Objectives of rate control: Make sure that output buffer at encoder side and input

buffer at decoder side do not overflow or underflow

Try to make the quality of the encoded frames asconstant as possible on frame level

Objectives of bit allocation: Given the budget allocated to a frame by the rate

control, try to make the quality of the picture as good as

possible Uses techniques similar to bit allocation for images Sometimes extended with temporal considerations

Rate control (1)Rate control (1)

Hybrid coders such as MPEG generate a variable number of bitsper frame

Constant bit rate channel - cbr requires that on the average the generated bit rate is fixed encoder uses output buffer for smoothing the produced bit rate decoder uses input buffer to collect enough bits to decode a frame example: digital video broadcasting

Variable bit rate channel - vbr allows for variation of the channel bit rate up to certain maximum encoder still uses buffer for smoothing short time variations decoder uses input buffer to collect enough bits to decode a frame example: digital versatile disk (DVD)


19/30


Hybrid coders such as MPEG generate a variable

number of bits per frame: Need buffer

0

1

2

3

4

5

6

7

8

9

0 1 2 3 4 5

T ime (sec)

Bitrate(Mbit/sec)


0

1

2

3

4

5

6

0 1 2 3 4 5 6 7 8 91011121314151617181920212223

Time (sec)

Bitrate(M

bit/sec)


20/30

Constant bit rate codingConstant bit rate coding

Video encoder needs to know size of input buffer of decoder way the decoder consumer the bits from this input buffer

Model decoder describes the external behavior of any MPEG decoder encoder anticipates this behavior all decoders should internally compensate for deviations from this

model

VBR CBR VBR

Video

encoderVideo

decoder

Output

bufferInput

buffer

Constant bit rateConstant bit rateCBRCBR -- controlcontrol Bit budget is assigned to current picture on basis of

fullness of (decoder input) buffer number of bits used by previous picture of same type (I,P,B) complexity of current picture

Two-pass coding: quantization with fixed Q to get complexity

indication

Measure statistics from current image (variance, motion, ) anticipated video effects

slow motion freeze framescene change ahead

Rate control attempts to make the picture quality as constant aspossible over time


21/30

Variable bit rateVariable bit rate -- VBRVBR -- controlcontrol

Within short time windows behaves similar as CBR Buffering needed to smooth for instance I,P,B effects

Over longer time windows, the CBR rate changes

Objectives: Guarantees constant picture quality, according to quality measure

used

Quality degrades only if maximum rate is surpassed

Bitrate

time (minutes)

0 1 2 3 4 5 6 7 8

maximum

Bit allocationBit allocation

CBR bit allocationGiven the bit budget, determine the Q setting (per

slice/macroblock) such that the picture quality is spatially optimal

VBR quality allocationGiven the (spatial) quality of previous pictures, find the Q setting(per slice/macroblock) such that the picture quality is constant in

time

Most MPEG coders use bit allocation techniquesdiscussed with DCT compression

Two-pass encoding or multi-pass encoding


22/30

MPEGMPEG--1 Constrained Parameter Set1 Constrained Parameter Set

Decoders are advised to be able to handle at least MPEG-1 streams that satisfy the following parameters:

Parameter Maximum value

Horizontal size 768 pixelsVertical size 576 pixels#Macroblocks/picture 396#Macroblocks/sec 396*25 = 330*30 = 9900Picture rate 30

Bit rate 1.856 Mbit/secEncoder buffer size 372 680 bitsMotion vector range +/- 64 pixels

MPEGMPEG--2 levels and profiles (1)2 levels and profiles (1) Given the big amount of possible applications,

MPEG-2 defines levels and profiles to facilitate

design and implementation.

Low

Main

High 1440

High

Level Characteristics

Picture of 352x288

Picture of 720x576x25Hz, 720x480x29,97Hz

HDTV picture up to 1440x1152

HDTV Picture up to 1920x1152

Level: Defines a set of constraints imposed

on parameters in the bit stream of a profile


23/30

MPEGMPEG--2 levels and profiles (2)2 levels and profiles (2) Profiles refer to a subset of tools of MPEG-2

Profiles support different classes of applications

Simple

Profile

Main

SNR

Spatial

High

Characteristics

No B pictures

Very similar to MPEG-1

Main profile with PSNR scalability

Main profile with spatial scalability

Main profile, PSNR, spatial and other options

The most used is Main Level - Main Profile (ML @ MP)

MPEGMPEG--2 levels2 levels -- profiles (3)profiles (3)4:2:0 4:2:0,4:2:2

1920 x 1152 1920 x 1152

HIGH 80 Mb/s 100 Mb/sI, P, B I, P, B

4:2:0 4:2:0 4:2:0,4:2:2HIGH 1440 x 1152 1440 x 1152 1440 x 1152

1440 60 Mb/s 60 Mb/s 80 Mb/s

I, P, B I, P, B I, P, B

4:2:0 4:2:0 4:2:2 4:2:0 4:2:0,4:2:2720 x 576 720 x 576 720 x 608 720 x 576 720 x 576

MAIN 15 Mb/s 15 Mb/s 50 Mb/s 15 Mb/s 20 Mb/sI, P I, P, B I, P, B I, P, B I, P, B

4:2:0 4:2:0352 x 288 352 x 288

LOW 4 Mb/s 4 Mb/sI,P,B I, P, B

LEVEL SIMPLE MAIN 4:2:2 SNR SPATIAL HIGHPROFILE PROFILE


24/30

Freedom within ML@MPFreedom within ML@MP

GOP structure Open/closedGOP size, position I framesFrame types used

Frame coding types I, P or B

Interlacing tools Frame/field picturesFrame/field prediction

Slice structure Fixed/adaptive

Slice quantization parameter Quantizer level for entire sliceMacroblock coding type Coded/Not-coded

Motion/No-motionIntra/Predicted

Macroblock quantization Adapt quantizer level within sliceMotion vectors Estimation algorithm used

Rate control Assignment of bits to a frameFeedforward (pre-analysis)/ FeedbackCBR versus VBR

Bit allocation Distribution of bits over macroblocks

MPEG system aspectsMPEG system aspects


25/30

MPEG 2 multiplexingMPEG 2 multiplexing

Video

codingVideo

Auido

codingAudio

Packetizer

Packetizer

Elementary

Stream

Video

PES

Audio

PES

Program

Stream

Mux

TransportStream

Mux

Data

Program

Stream

Single Program

Transport Stream

PES: Packetized Elementary stream

MPEG 2: Decoding standard

DataDataMPEGMPEG--22

Single Program

Transport StreamTransport Stream Transport Stream Transport Stream


26/30

Elementary video streamElementary video stream -- ESES

Block of

coefficients

Motion

vector

Macro

blockSlice Picture

Group

of

Pictures

Video

sequence

Syncronism I, P, B Time

Global vectors

Open /

closed

Quantization

matrices

Picture

rate

Level

Profile

Picture

sizeAspect

ratio

Progressive /

interlaced

Chroma

type

Elementary stream

Packetized elementaryPacketized elementarystreamsstreams -- PESPES

24 bits

Packet start

code prefixPES packet

length

Stream

identification

Video or audio

identification

Identificacin del comienzo

de paquete

PES packet header

PES packet

Packet

lengthElementary stream

in packets

MPEG defines only the transport stream

The designer can chose to convert ES in Transport Stream

without specifying PES

PES can be used for Programs Streams or Transport Streams

Length depends on

the application


27/30

Program streamsProgram streams -- PSPS

DVD applications (Digital Video / Versatile Disc)

Possibilities

One video program + several audio programs

Only several audio programs

The video Elementary Stream is divided in Access Units (AU). Each AU

contains information of a compressed image (I, P, B)

AU are converted into PST (Program stream packets)

Each PST can be of variable lenght

Variable bit-rate

Transport streamsTransport streams -- TSTS Transmission application for different media

Several different programs of audio and video. Typical: 4 - 6 TV programs,

3 or 4 audio programs, data, teletext, captions, etc.

Eacg program can have a different compression factor

bit-rate can be changed dynamically although the total be constant

(statistical multiplexing). Typical: 45 Mbits/sec

Each PES of video can have a different number of PES of audio and associated

data

Some programs can be protected. The TS must have conditional access

The TS has to provide syncronization for each program individually

This syncronization is called Program Clock Reference - PCR


28/30

Transport stream packetTransport stream packet

Header Payload

188 bytes

Sync

Byte

Adaptationfield

controlScrambling

control

Packet

identificationPID

Transport

priority

Start

indicator

Transport

errorindicator

Continuity

counterAdaptation

field

Discontinuity

indicator

Adaptation

field

length

Random

access

indicator

Elem stream

priority

indicator 5 Flags

Optional

fields

Stuffing

bytes

PCR OPCRSplice

countdown

Transport

private

data

Adaptation

fieldextension

Packet identificationPacket identification -- PIDPID

PID is used by the demultiplexer to distinguish packets which contain

different types of information

All packets which belong to the same ES have the same PID

The data of a program can be selected using the corresponding

PIDs of video, audio and teletext.

To know the PIDs associated to a program the Program Specific

Information PSI is used


29/30

Digital video broadcastingDigital video broadcasting -- DVBDVB

DVB - S: satelite system which could be used by any transponder

DVB - C: cable system for any type of cable network

DVB - T: Terrestrial system

DVB - MC / S: Multipoint microwave distribution system

DVB - SI: System service information allowing the user to browse

in a DVB environment

DVB - CA: Common scrambling system

DVB - CI: Common interface for conditional access

Objetive (1993):

To provide a common format allowing the transmission of TV digital

DVB characteristicsDVB characteristics

MPEG 2 compression system for audio and video

DVB provides modulation techniques and error correction coding techniques

for satellite, cable and terrestrial systems

DVB provides format for data insertion in the tranmission channel


30/30

DVB tranmission systemDVB tranmission system

Video coding

MultiplexerAudio coding

Video coding

Audio coding

Error

correctionModulation

Video270Mbits/seg

Audio6Mbits/seg

MPEG 2 Transport Stream Mbits/seg

DVB - C: QAM

DVB - S: QPSK

DVB - T: COFDM

Energy dispersion

Reed Solomon

InterleavingConvolutional codes: DVB - S, DVB - T

DVB integrated systemDVB integrated system

Standards Video Mpeg English

Documents