JVT Coding ITU-T H.26L; ISO MPEG-4, Part 10

JVT CodingITU-T H.26L; ISO MPEG-4, Part 10Thomas WiegandHeinrich Hertz Institute, Berlin, GermanyAssociated Rapporteur ITU-T VCEGCo-Chair ITU-T/ISO JVTwiegand@hhi.de

T. Wiegand: JVT/H.26L Coding*

OutlineJVT/H.26L: History, Goals, Applications, StructureVideo Coding LayerSyntax and DecoderCoder ControlComparisonsNetwork Adaptation Layer

T. Wiegand: JVT/H.26L Coding*

The JVT ProjectNew ITU-T Q.6/SG16 (VCEG - Video Coding Experts Group) standardization activity for video compressionAugust 1999: 1st test model (TML-1)December 2001: Formation of the Joint Video Team (JVT) between VCEG and MPEG to finalize H.26L as a joint project: JVT Coding (similar to MPEG-2/H.262)February 2002: WD-2 (11 th test model: TML-11) Schedule: February 2002: Last major feature adoptionsNovember 2002: Final approval

T. Wiegand: JVT/H.26L Coding*

Simple syntax specification Targeting simple and clean solutions Avoiding any excessive quantity of optional features or profile configurations Improved Coding EfficiencyAverage bit rate reduction of 50% given fixed fidelity compared to any other standardImproved Network FriendlinessIssues examined in H.263 and MPEG-4 are further improvedMajor targets are mobile networks and InternetGoals of the JVT/H.26L Project

T. Wiegand: JVT/H.26L Coding*

Conversational H.32X ServicesH.320 Conversational3GPP Conversational H.324/M3GPP Conversational IP/RTP/SIPH.323 Conversational Internet/unmanaged/best effort IP/RTP Streaming Services3GPP Streaming IP/RTP/RTSPStreaming IP/RTP/RTSPOther Services Entertainment Satellite/Cable/DVD, 0.5 8 Mbit/sDigital Cinema Application3GPP Multimedia Messaging ServicesApplications

T. Wiegand: JVT/H.26L Coding*

JVT/H.26L Layer StructureVideo Coding LayerData PartitioningNetwork Adaptation LayerH.320H.324H.323/IPH.324Metc.Control DataMacroblockSlice/Partition

T. Wiegand: JVT/H.26L Coding*

JVT/H.26L Layer StructureVideo Coding LayerData PartitioningNetwork Adaptation LayerH.320H.324H.323/IPH.324Metc.Control DataMacroblockSlice/Partition

T. Wiegand: JVT/H.26L Coding*

H.26L Video Coding LayerDeq./Inv. TransformMotion-CompensatedPredictorControlDataQuant. Transf. coeffsMotionData0Intra/InterCoderControlDecoderMotionEstimatorTransform/ Quantizer-

T. Wiegand: JVT/H.26L Coding*

Common Elements with other Standards16x16 macroblocksConventional sampling of chrominance and association of luminance and chrominance dataBlock motion displacementMotion vectors over picture boundariesVariable block-size motionBlock transforms (not wavelets or fractals)Run-length coding of transform coefficientsScalar quantizationI- and P-picture types

T. Wiegand: JVT/H.26L Coding*

Motion-CompensatedPredictorMotionDataMotion Compensation AccuracyDeq./Inv. TransformControlDataQuant. Transf. coeffs0Intra/InterCoderControlDecoderMotionEstimatorTransform/ Quantizer-1/4 (QCIF) or 1/8 (CIF) pel

T. Wiegand: JVT/H.26L Coding*

INTRAINTRAMB:8x8:Allows motion segmentation shapes likeINTRATree-Structured MB Partition

T. Wiegand: JVT/H.26L Coding*

MotionDataMultiple Reference FramesDeq./Inv. TransformMotion-CompensatedPredictorControlDataQuant. Transf. coeffs0Intra/InterCoderControlDecoderMotionEstimatorTransform/ Quantizer-

T. Wiegand: JVT/H.26L Coding*

Motion CompensationVarious block sizes and shapes for motion compensation (7 segmentations of the macroblock: 16x16, 16x8, 8x16, 8x8, 8x4, 4x8, 4x4)1/4 sample (sort of per MPEG-4) and 1/8 sample accuracy motion6x6 tap filtering to 1/2 sample accuracy, bilinear filtering to 1/4 sample accuracy, special position with heavier filtering8x8 tap filtering applied repeatedly for 1/8 pel motionMultiple reference pictures (per H.263++ Annex U)Temporally-reversed motion and generalized B-framesB-frame prediction weighting

T. Wiegand: JVT/H.26L Coding*

Residual CodingDeq./Inv. TransformMotion-CompensatedPredictorControlDataQuant. Transf. coeffsMotionData0Intra/InterCoderControlDecoderMotionEstimatorTransform/ Quantizer-Residual coding is based on 4x4 blocksInteger Transform

T. Wiegand: JVT/H.26L Coding*

Residual and Intra CodingTransformBased primarily on 4x4 transform (all prior standards: 8x8)

Expanded to 8x8 for chroma by 2x2 transform of the DC valuesIntra Coding StructureDirectional spatial prediction (6 types luma, 1 chroma)Expanded to 16x16 for luma intra by 4x4 transform of the DC values

T. Wiegand: JVT/H.26L Coding*

Quantization and DeblockingQuantizationTwo inverse scan patternsLogarithmic step size controlSmaller step size for chroma (per H.263 Annex T)

Deblocking Filter (in loop)

T. Wiegand: JVT/H.26L Coding*

Entropy Coding

T. Wiegand: JVT/H.26L Coding*

Exp-Golomb CodingOne table that is used universally for all symbolsSimple, but has the following disadvantagesProbability distribution may not be a good fitProbability distribution is staticCorrelations between symbols are ignored, i.e. no conditional probabilities are usedCode words must have integer number of bits (Low coding efficiency for highly peaked pdfs)

T. Wiegand: JVT/H.26L Coding*

Context-based Adaptive Binary Arithmetic Codes (CABAC)Usage of adaptive probability modelsExploiting symbol correlations by using contextsNon-integer number of bits per symbol by using arithmetic codesRestriction to binary arithmetic codingSimple and fast adaptation mechanismFast binary arithmetic coders are availableBinarization is done using the UVLC

T. Wiegand: JVT/H.26L Coding*

SP-PicturesSP-pictures are much smaller than I-pictures

DOCUMENTTYPE

1 (1)

TypeUnitOrDepartmentHere

TypeYourNameHere

TypeDateHere

Time n-2 n-1 n n+1 n+2

Bitstream 1

Bitstream 2

12

S

P

P

P

P

P

P

P

P

1

S

2

S

_935227290.doc

T. Wiegand: JVT/H.26L Coding*

Comparison of JVT/H.26L and MPEG-4Both:Sequence structure IBBPBBP...Search range: 32x32 around 16x16 predictorEncoders use similar D+lR optimization techniquesMPEG-4: Advanced Simple Profile (ASP)Motion Compensation: 1/4 pelGlobal Motion CompensationQPB=1.2 x QPPH.26L: Motion Compensation: 1/4 pel (QCIF), 1/8 pel (CIF)Using CABAC entropy coding5 reference frames QPB=QPP+2

T. Wiegand: JVT/H.26L Coding*

Comparison to MPEG-2, H.263, MPEG-427282930313233343536373839050100150200250Bit-rate [kbit/s]Foreman QCIF 10HzQualityY-PSNR [dB]MPEG-2H.263MPEG-4JVT/H.26L

T. Wiegand: JVT/H.26L Coding*

Comparison to MPEG-2, H.263, MPEG-4Tempete CIF 30Hz25262728293031323334353637380500100015002000250030003500Bit-rate [kbit/s]QualityY-PSNR [dB]MPEG-2H.263MPEG-4JVT/H.26L

T. Wiegand: JVT/H.26L Coding*

Comparison to MPEG-2, H.263, MPEG-4Tempete CIF 30Hz25262728293031323334353637380500100015002000250030003500Bit-rate [kbit/s]QualityY-PSNR [dB]MPEG-2H.263MPEG-4JVT/H.26L

T. Wiegand: JVT/H.26L Coding*

JVT/H.26L Layer StructureVideo Coding LayerData PartitioningNetwork Adaptation LayerH.320H.324H.323/IPH.324Metc.Control DataMacroblockSlice/Partition

T. Wiegand: JVT/H.26L Coding*

Network Adaptation LayerTasksMapping of slice structure on transport layerSetup, framing, encapsulation, interleaving, logical channels, closing, timing issues, synchronization, etc.Transport of control and header informationFurther network specific issues (feedback, prioritization,)

The specification for each NAL includes Verbal descriptionEncapsulation process (processing of slice structure)Header and parameter set specification

T. Wiegand: JVT/H.26L Coding*

Slice Structure CodingSlices for a specified number of macroblocksSlices for a specified number of BytesData Partitioning: header, motion vectors, Intra, and Inter transform coefficientsSignalling of header info and parameter sets via appropriate meansNetwork Adaptation Features

T. Wiegand: JVT/H.26L Coding*

ConclusionsJVT/H.26L contains a video coding and a network adaptation layerVideo coding layer is based on hybrid video coding and similar in spirit to other standards but with important differencesBit-rate savings up to 50 % against any other standardNetwork adaptation layer and error resilience features consider transport over MPEG2/H.222.0IP and 3GPP networksDownload of documents and software via anonymous ftp to standard.pictel.com/video-site