JVT Coding ITU-T H.26L; ISO MPEG-4, Part 10

JVT CodingJVT CodingITU-T H.26L; ISO MPEG-4, Part 10ITU-T H.26L; ISO MPEG-4, Part 10

Thomas WiegandThomas WiegandHeinrich Hertz Institute, Berlin, GermanyHeinrich Hertz Institute, Berlin, Germany

Associated Rapporteur ITU-T VCEGAssociated Rapporteur ITU-T VCEGCo-Chair ITU-T/ISO JVTCo-Chair ITU-T/ISO JVT

[email protected]@hhi.de

T. Wiegand: JVT/H.26L Coding 2

OutlineOutline

JVT/H.26L: History, Goals, Applications, Structure

Video Coding Layer• Syntax and Decoder• Coder Control• Comparisons

Network Adaptation Layer


The The JVT JVT ProjectProject

New ITU-T Q.6/SG16 (VCEG - Video Coding Experts Group) standardization activity for video compression

August 1999: 1st test model (TML-1) December 2001: Formation of the Joint Video Team

(JVT) between VCEG and MPEG to finalize H.26L as a joint project: JVT CodingJVT Coding (similar to MPEG-2/H.262)

February 2002: WD-2 (11 th test model: TML-11)

Schedule: • February 2002: Last major feature adoptions• November 2002: Final approval


Simple syntax specification • Targeting simple and clean solutions • Avoiding any excessive quantity of optional features or

profile configurations Improved Coding Efficiency

• Average bit rate reduction of 50% given fixed fidelity compared to any other standard

Improved Network Friendliness• Issues examined in H.263 and MPEG-4 are further

improved• Major targets are mobile networks and Internet

Goals of the JVT/Goals of the JVT/H.26L H.26L ProjectProject


Conversational H.32X Services• H.320 Conversational• 3GPP Conversational H.324/M• 3GPP Conversational IP/RTP/SIP• H.323 Conversational Internet/unmanaged/best effort IP/RTP

Streaming Services• 3GPP Streaming IP/RTP/RTSP• Streaming IP/RTP/RTSP

Other Services • Entertainment Satellite/Cable/DVD, 0.5 – 8 Mbit/s• Digital Cinema Application• 3GPP Multimedia Messaging Services

ApplicationsApplications


JVT/H.26L Layer StructureJVT/H.26L Layer Structure

Video Coding Layer

Data Partitioning


H.320 H.324 H.323/IP H.324M etc.

Con

trol D

ata

Macroblock

Slice/Partition


JVT/H.26L Layer StructureJVT/H.26L Layer Structure

Video Coding Layer

Data Partitioning


H.320 H.324 H.323/IP H.324M etc.

Con

trol D

ata

Macroblock

Slice/Partition


EntropyCoding

H.26L Video Coding LayerH.26L Video Coding Layer

Deq./Inv. Transform

Motion-Compensated

Predictor

ControlData

Quant.Transf. coeffs

MotionData

0

Intra/Inter

CoderControl

Decoder

MotionEstimator

Transform/Quantizer-


Common Elements with other StandardsCommon Elements with other Standards

16x16 macroblocks Conventional sampling of chrominance and association

of luminance and chrominance data Block motion displacement Motion vectors over picture boundaries Variable block-size motion Block transforms (not wavelets or fractals) Run-length coding of transform coefficients Scalar quantization I- and P-picture types


Motion-Compensated

Predictor

MotionData

EntropyCoding

Motion Compensation AccuracyMotion Compensation Accuracy

Deq./Inv. Transform

ControlData


0

Intra/Inter

CoderControl

Decoder

MotionEstimator


1/4 (QCIF) or 1/8 (CIF) pel

8x8

0

4x8

0 10 12 3

4x48x4

1

08x8Modes

0

16x16

0 1

8x16MB

Modes

8x80 12 3

16x8

1

0


INTRA

INTRA

MB:

8x8:

Allows motion segmentation shapes like

INTRA

Tree-Structured MB PartitionTree-Structured MB Partition


EntropyCoding

MotionData

Multiple Reference FramesMultiple Reference Frames

Deq./Inv. Transform

Motion-Compensated

Predictor

ControlData


0

Intra/Inter

CoderControl

Decoder

MotionEstimator


Multiple Reference Frames for Motion Compensation


EntropyCoding

Residual CodingResidual Coding

Deq./Inv. Transform

Motion-Compensated

Predictor

ControlData


MotionData

0

Intra/Inter

CoderControl

Decoder

MotionEstimator


Residual coding is based on 4x4 blocks

Integer Transform


Residual and Intra CodingResidual and Intra Coding Transform

• Based primarily on 4x4 transform (all prior standards: 8x8)

• Expanded to 8x8 for chroma by 2x2 transform of the DC values

Intra Coding Structure• Directional spatial prediction

(6 types luma, 1 chroma)• Expanded to 16x16 for luma intra by 4x4 transform of the DC

values

1 1 1 12 1 1 21 1 1 11 2 2 1

H


Quantization and DeblockingQuantization and Deblocking

Quantization• Two inverse scan patterns• Logarithmic step size control• Smaller step size for chroma (per H.263

Annex T)

Deblocking Filter (in loop)


Entropy CodingEntropy Coding

Deq./Inv. Transform

Motion-Compensated

Predictor

ControlData


MotionData

0

Intra/Inter

CoderControl

Decoder

MotionEstimator


EntropyCoding


Exp-Golomb CodingExp-Golomb Coding One table that is used universally for all

symbols

Simple, but has the following disadvantages• Probability distribution may not be a good fit

• Probability distribution is static• Correlations between symbols are ignored,

i.e. no conditional probabilities are used• Code words must have integer number of bits

(Low coding efficiency for highly peaked pdfs)


Context-based Adaptive Binary Context-based Adaptive Binary Arithmetic Codes (CABAC)Arithmetic Codes (CABAC)

Usage of adaptive probability models Exploiting symbol correlations by using contexts Non-integer number of bits per symbol by using

arithmetic codes Restriction to binary arithmetic coding

• Simple and fast adaptation mechanism• Fast binary arithmetic coders are available• Binarization is done using the UVLC


SPSP-Pictures-Pictures

Bitstream 1

Bitstream 2 S 2

S 1 P

P P

P

P

P P

P

S 12

Time n-2 n-1 n n+1 n+2

SP-pictures are much smaller than I-pictures


Comparison of JVT/H.26L and MPEG-4Comparison of JVT/H.26L and MPEG-4 Both:

• Sequence structure IBBPBBP...• Search range: 32x32 around 16x16 predictor• Encoders use similar D+R optimization techniques

MPEG-4: Advanced Simple Profile (ASP)• Motion Compensation: 1/4 pel• Global Motion Compensation• QPB=1.2 x QPP

H.26L: • Motion Compensation: 1/4 pel (QCIF), 1/8 pel (CIF)• Using CABAC entropy coding• 5 reference frames • QPB=QPP+2


Comparison to MPEG-2, H.263, MPEG-4Comparison to MPEG-2, H.263, MPEG-4

27282930313233343536373839

0 50 100 150 200 250Bit-rate [kbit/s]

Foreman QCIF 10Hz

Left-hand sideLeft-hand side

Right-hand sideRight-hand side

QualityY-PSNR [dB]

MPEG-2H.263

MPEG-4JVT/H.26L


Comparison to MPEG-2, H.263, MPEG-4Comparison to MPEG-2, H.263, MPEG-4Tempete CIF 30Hz

2526272829303132333435363738

0 500 1000 1500 2000 2500 3000 3500Bit-rate [kbit/s]

QualityY-PSNR [dB]

MPEG-2H.263

MPEG-4JVT/H.26L

???-hand side???-hand side

???-hand side???-hand side


Comparison to MPEG-2, H.263, MPEG-4Comparison to MPEG-2, H.263, MPEG-4Tempete CIF 30Hz

2526272829303132333435363738

0 500 1000 1500 2000 2500 3000 3500Bit-rate [kbit/s]

QualityY-PSNR [dB]

MPEG-2H.263

MPEG-4JVT/H.26L

Left-hand sideLeft-hand side

Right-hand sideRight-hand side


JVT/JVT/H.26L Layer StructureH.26L Layer Structure

Video Coding Layer

Data Partitioning


H.320 H.324 H.323/IP H.324M etc.

Con

trol D

ata

Macroblock

Slice/Partition


Network Adaptation LayerNetwork Adaptation Layer Tasks

• Mapping of slice structure on transport layer• Setup, framing, encapsulation, interleaving, logical

channels, closing, timing issues, synchronization, etc.• Transport of control and header information• Further network specific issues (feedback,

prioritization,…)

The specification for each NAL includes • Verbal description• Encapsulation process (processing of slice structure)• Header and parameter set specification


Slice Structure Coding• Slices for a specified number of macroblocks• Slices for a specified number of Bytes

Data Partitioning: header, motion vectors, Intra, and Inter transform coefficients

Signalling of header info and parameter sets via appropriate means

Network Adaptation FeaturesNetwork Adaptation Features


ConclusionsConclusions JVT/H.26L contains a video coding and a network

adaptation layer Video coding layer is based on hybrid video coding and

similar in spirit to other standards but with important differences

Bit-rate savings up to 50 % against any other standard Network adaptation layer and error resilience features

consider transport over MPEG2/H.222.0 IP and 3GPP networks Download of documents and software via anonymous

ftp to standard.pictel.com/video-site

JVT Coding ITU-T H.26L; ISO MPEG-4, Part 10

Documents

JVT Coding ITU-T H.26L; ISO MPEG-4, Part 10