PPT

H.264 to VC 1 Transcoding

Vidhya VijayakumarMultimedia Processing Lab

MSEE, University of Texas @ [email protected]

Guided by Dr. KR Rao

What is…

H.264– The new industry standard – Massive quality, Minimal files – Scalable from 3G to HD and Beyond

VC 1– Informal name of the SMPTE 421M video codec– Standard initially developed by Microsoft – Supported standard for HD DVDs, Blu-ray Discs,

and Windows Media Video

What is…

Transcoding – Converting a previously compressed video signal into

another one with different format Change in bit rate, frame rate, frame size, or even

compression standard – 2 Ways

Decode fully and encode in target standard Change the bit stream format from one standard to another

without its undergoing the complete decoding and encoding process.

– Limitations Compression artifacts are cumulative

Why Trancode H.264 to VC-1?

The two high definition DVD formats HD-DVD and Blue ray have mandated MPEG-2, H.264 and VC-1 as video compression formats

As H.264 based and VC-1 based content and products become available, transcoding in both directions will become widely used capabilities.

From an end user point of view, any VC-1 decoder can now become twice as powerful as it was earlier.

Why VC1?

Requires less computational power and can be decoded at full 1080i/p resolution on today’s off-the-shelf PC

Advanced Profile delivers compression efficiencies far superior to MPEG-2

Delivers HD content at bit rates as low as 6-8 Mbps

Better visual quality against H.264 and MPEG-2 demonstrated in independent tests

More of VC1…

DCT-based video codec design Coding tools for interlaced video sequences

as well as progressive encoding 8-bit, 4:2:0 format Uses block based transform and motion

compensation with quantization and entropy coding.

Decoder – Simple & Main profile

DECODED FRAME BUFFER

Decoding Process Block Diagram for VC-1 Simple and Main Profiles

BITSTREAM PARSING VLC

DECODE COEFFS

INVERSE QUANT

INVERSE TRANS

VLC DECODE

MVMV PRED

MOTION COMP

½ or ¼ pel interpolation

1 MV /4 MV

RANGE RED/ INTENSITY

COMP

+

VLC DECODE COEFFS

INVERSE QUANT

INVERSE TRANS

INTER

INTRA

AC/DC PRED

OVERLAP SMOOTH

LOOP FILTER

DECODED FRAME

DERING/DEBLOCK

DISPLAY PROCESS

COLOR CONV

RESIZE

Etc.

IMPLEMENTATION SPECIFIC

CONFORMING IMPLEMENTATION

Decoder – Advanced Profile

DECODED PICTURE BUFFER

Decoding Process Block Diagram for VC-1 Advanced Profile

BITSTREAM PARSING VLC

DECODE COEFFS

INVERSE QUANT

INVERSE TRANS

VLC DECODE

MVMV PRED

MOTION COMP

½ or ¼ pel interpolation

1 MV /4 MV

INTENSITY COMP

+

VLC DECODE COEFFS

INVERSE QUANT

INVERSE TRANS

INTER

INTRA

AC/DC PRED

OVERLAP SMOOTH

LOOP FILTER

RANGEMAP

DERING/DEBLOCK

DISPLAY PROCESS

COLOR CONV

RESIZE

Etc.

IMPLEMENTATION SPECIFIC

CONFORMING IMPLEMENTATION

DECODED FRAME

Block Transforms (Integer DCT)

8x8 blocks can be encoded using

1_8x8 2_8x4

2_4x8 4_4x4 - Frame / Macroblock/Block signaling- Block level for coarse and fine level specification - Frame level for overhead reduction - Only 8x8 used for I frames

8x8 Integer DCT Matrices

WMV 9 H.264 HP

Key features of the Transforms

The norms of vectors of the ratio 288:289:299 The variation in the norm accounted for in the

encoder itself At the decoder, inverse transform (rows) ->

rounding-> inverse transform (columns) ->rounding (to operate in the 16 bit range)

Quantization

Same rule applied to all block sizes Both types with (bit savings at low bit rates) and without dead

zone available Type used signaled at the frame level to the decoder At the encoder side automatic switch from uniform quantization

to dead zone quantization as Q – parameter increases Other factors like noise and rate control can be used to control

this switch

Loop Filtering

Done to remove blocky artifacts and thus quality of current frame for future prediction

Operates on pixels on the border of blocks

The process involves a discontinuity measurement

Checks are computationally expensive so done only for one set of pixel per boundary

Motion Estimation and Compensation

Max resolution of ¼ pixel (i.e. ¼, ½, ¾) allowed 16x 16 motion vectors used by default but 8x8 allowed Bicubic filter with 4 taps/ Bilinear filters with 2 taps to generate subpixel

precision. 4 combined modes 1.Mixed block size (16x16 and 8x8), ¼ p ,bicubic 2.16x16, ¼ p, bicubic 3.16x16, ½ p, bicubic 4.16x16, ½ p, bilinear Bilinear filters for chroma components

Advanced entropy coding

Simple VLC codes Multiple code tables for encoding each particular alphabet out A possible set of code tables is chosen (based on frame level

quantization parameter) and signaled in the bitstream Additional information e.g. motion vectors resolution coded

using bitplane coding

Interlaced coding Supports field and frame coding

Advanced B frame coding

B frames:- employ bi-directional prediction Fractional position definition with respect to the

reference frames for better scaling of motion vectors Intra coded B frames for scene changes Allow inter field reference

Overlap smoothing

The deblocking filter smoothens out the block as well as true edges and it may be disabled in less complex profiles

A lapped transform (input spans to pixels from other blocks as well) is used at the edges

Used in spatial domain as pre and post processing Used only at low bit rates where blocking artifacts are higher Signaled at macroblock level so can be turned off in smooth

regions

Low rate tools (<100 Kbps)

Code frames at multiple resolutions (both in X and Y direction)

A frame can be downscaled at the encoder and then upscaled at the decoder for LBR transmission

The downscaling factor needs to remain same from the start of 1 I frame to the start of next I frame.

The frame must be upscaled prior to display (upscaling out of scope of the standard).

Fading compensation

Large amount of bits required for scenes having effects like fade-to-black ,fade-from-black

Not possible to predict motion using normally used techniques. Effective fading detection (original reference image- current

video image > threshold = fading) If detected then encoder computes fading parameters which

specify a pixel-wise first order linear transform for the reference image.

Also signaled to the decoder

Profiles

Simple Main Advanced

Baseline intra frame compression Yes Yes Yes

Variable-sized transform Yes Yes Yes

16-bit transform Yes Yes Yes

Overlapped transform Yes Yes Yes

4 motion vector per macroblock Yes Yes Yes

¼ pixel luminance motion compensation Yes Yes Yes

¼ pixel chrominance motion compensation No Yes Yes

Start codes No Yes Yes

Extended motion vectors No Yes Yes

Loop filter No Yes Yes

Dynamic resolution change No Yes Yes

Adaptive macroblock quantization No Yes Yes

B frames No Yes Yes

Intensity compensation No Yes Yes

Range adjustment No Yes Yes

Field and frame coding modes No No Yes

GOP Layer No No Yes

Display metadata No No Yes

Simple Main Advanced

Comparison of H.264 and VC-1

Overview VC 1 H.264

Goals

Designed to offer veryhigh image quality withexcellent compression

efficiency

Designed to meet a variety of industry needs with many profiles andlevels, allowing for varying compression, quality and CPU usage levels,where the lowest level is for portable devices, designed with low CPUusage in mind, while the high levels are designed with very high qualityand compression efficiency in mind

Example industry use

Supports 4:2:0compression / color

space

Supports studio archiving requirements with 4:4:4 color space;separate black and white (BW) video mode

Licensing costsSimilar

Documentation

Not free. Reference decoder, which is notfree by itself, comes withexternal documentation.FFMPeg project gives afree decoder.

Free. Reference encoder and decoder free as well. Plus JVT, M4IF mail-lists where one may receive answers on the AVC related questions.

Comparison of H.264 and VC-1

Features VC-1 H.264

Bitstream formats single bit stream NAL and byte stream

Bitstream format

In advanced profile each Bit stream Data Unit has itsown header. Simple and Main profile do not provideneither sequence nor entry point headers.

SPS (sequence parameter set),PPS (picture parameters set),slice header, macroblock

Deblocking filter In-loop and out-of-loop algorithms, overlap transform In-loop only

CABAC No Only supported in Main and higher profiles

Variable transformSize

Yes Only in High profile and above

VLC Yes

Slice Contiguous (integer number of macro block rows only) Contiguous/non contiguous

B frame used forpredicting other

pictures

No Yes

Sub pixelInterpolationmethods

bicubic, bilinear 6-tap filter for half pixel, averaging for quarter pixels

Comparison of H.264 and VC-1

Feature VC-1 H.264

Partition sizes 16x16, 16x8, 8x16, and 8x816x16, 16x8, 8x16, 8x8, 8x4,

4x8, and 4x4

Integer transform 8x8, 4x8, 8x4, and 4x44x4; 8x8 available in High

Profile only

FrameUsed for interlace content. Consists of

bottom and top fieldUsed for progressive or

interlaced content

Macroblock sizes 16x16 only

Motion vector Two dimensional vector offset from current position to reference frame

Picture A field or frame

Skipped Mb No data is encoded for macroblock

• 8x8, 4x8, 8x4, 4x48x8, 4x8, 8x4, 4x4adaptive block sizeadaptive block size

•Frequency-independent Frequency-independent dequantization scalingdequantization scaling

•VLC-based entropy codingVLC-based entropy coding

•4 tap bicubic filters for MC4 tap bicubic filters for MC

•Relatively-simple loop filterRelatively-simple loop filter

•Overlap intra filteringOverlap intra filtering

•Range reduction/expansionRange reduction/expansion

•Resolution red./exp.Resolution red./exp.

• 8x8 and 4x48x8 and 4x4adaptive block sizeadaptive block size

•Frequency-dependent Frequency-dependent dequantization matrixdequantization matrix

•CABAC or VLCCABAC or VLC

•Long filters for MCLong filters for MC

•Complex loop filterComplex loop filter

•Spatial intra predictionSpatial intra prediction

•Multi-picture arbitrary-order Multi-picture arbitrary-order referencingreferencing

•Intra PCMIntra PCM

VC-1VC-1H.264H.264

Block motionBlock motion

16-bit integer 16-bit integer transformstransforms

Bit-exact specBit-exact spec

Fading Fading predictionprediction

Loop filterLoop filter

Graphically…

Transcoding point of view

Adaptive In High profile

Stepping forward…

Algorithm to deduce the picture type in VC-1 from H.264 picture types

Algorithm to effectively handle transform size mismatch between H.264 and VC-1

Algorithm to choose the best reference picture of H.264 to be used for MC in VC-1

References

An efficient algorithm for VC-1 to H.264 video transcoding in progressive compression - Jae-Beom Lee and Hari Kalva

http://www.avsforum.com/avs-vb/showthread.php?p=9931723&&#post9931723 http://www.microsoft.com/windows/windowsmedia/howto/articles/vc1techovervi

ew.aspx

Windows Media Video 9: overview and applications

Sridhar Srinivasan, Pohsiang (John) Hsu, TomHolcom b, Kunal Mukerjee, Shankar L. Regunathan, Bruce Lin, Jie Liang, Ming-Chieh Lee, Jordi Ribas-Corbera

Windows Digital Media Division, Microsoft Corporation, Redmond, WA 98052, USA, available online at www.sciencedirect.com

Thank You

Vidhya [email protected]