Technion - IIT Dept. of Electrical Engineering Signal and Image Processing lab Transrating and Transcoding of Coded Video Signals David Malah Ran Bar-Sella.

Technion - IITDept. of Electrical Engineering

Signal and Image Processing lab

Transrating and Transcoding of Coded

Video Signals

David Malah

Ran Bar-Sella Michael Lavrentiev

Research Proposal to

STRIMM Meeting June 25, 2003

TranscoderCoded

Bit StreamTranscodedBit Stream

Converting a coded bit stream to another bitstream to match a destination profile in terms of:

– Media Formats {e.g., MPEG-2 H.264}

– Resolutions (Spatial, Temporal)

– Bit Rates (Transrating)

– Color depths / formats {e.g., 4:4:4 4:2:0}

• • •

Transcoding

Cell-phone

Generic Hybrid Video Encoder

..Image sequence

0101...

Bit stream+

DCT Q VLC

Q-1

DCT -1

+

MEMMC

-

+

Intra / Interswitch

++

ME

MC -Motion Compensation

ME -Motion Estimation

MEM -Frame store

DCT -Discrete Cosine Transform

Q -Quantization

VLC -Variable Length Code

MPEG Frame Structure

H.264

Intra Spatial Prediction

• Frame dropping (B, P)

• Color suppression

• Discarding high-frequency DCT coefficients

• Reducing spatial resolution (size reduction)

• DCT coefficients Re-quantization

Transrating Methods (Bit-Rate Reduction - BRR)

Cascading Decoder and Encoder

FullDecoder

FullEncoder

CodedBit Stream

TranscodedBit Stream

Transcoder Architectures

Related Issues• Open Loop • Closed Loop - Pixel-Domain - DCT-Domain

PartialEncoder

Complexity-Reduced Transcoding(Compressed Domain)

PartialDecoder

Processing

Motion vectors / coding modes

CodedBit Stream

TranscodedBit Stream

Re-quantization Error

Q1

Q2 Original pixel value

Quantized values with Q1Quantized values with Q2

q2/q1

q2/q1

q2/q1

q2/q1

Re-quantization Step-size Selection(Wang and Woods, 1998)

1q

2q

Preferred Step-size Ratios(Wang and Woods, 1998)

MSE Re-quantization

Finer quantizer

p(x)

x

d1 d2

Coarser quantizerR1 R2

D

c

c

Centroid

Q1

Q2

MAP Re-quantization

p(x)

xp1 p2

d1 d2

p2 < p1 p2 > p1

Finer quantizer

R1 R2

DCoarser quantizer

Q1

Q2

Constrained Minimization problem:

Lagrangian cost function becomes sum of independent MB level calculated parts:

( ) min{ ( ) ( )}k

k k k k kq

J d q r q

Lagrangian parameter, λ≥0, is iteratively updated to achieve desired bit-rate RT

Re-quantization by Lagrangian Optimization

(Assuncao and Ghanbari, 1997)

• Recent result (Laventer and Porat – ICIP-03): Optimal quantizer steps are obtained at values close to specific multiples of the input quant. step.

Variable Length Coding

Macro-Block

Trellis Re-quantization

q*kmax

q*k2

block_1

q*k1

block_2 block_N

HVS-based Bit Allocation

• Different regions have different perceptual importance and may be encoded at different rates for similar subjective quality.

• Video frames can be partitioned (segmented) into regions having different characteristics:

- Textured regions- Smooth regions- Edges- Moving objects

• For efficiency, segmentation should be based on compressed- domain data: Block DCT Coeff., MB type, MB q-step, MB rate, and Motion Vectors (MV).

• Possible approach: Block classification, Region-growing, and Segment tracking.

• Bit allocation to each region should reflect perceptual importance.

MPEG-2 H.264 Transcoding

• MEC

- 7 modes (block sizes 16x16 – 4x4) - ¼ Pel resolution- Improved MV coding

• Transform- Integer transform (4x4)

• Intra Prediction

- spatial prediction (9 modes)- 16x16 or 4x4 blocks

• Inter Prediction

- Multiple reference frames (1 – 5)- SP frame for stream transition

H.264 – Main Differences vis-à-vis MPEG-2

• Quantization and Coeff. Scanning- 32 logarithmic steps- no ‘dead-zone’- 2 scan modes

• De-blocking Filter (4x4)

• Entropy Coding - Universal VLC (UVLC) - Arithmetic Coding (CABAC)

• Network Friendly Representation

- Video Coding Layer - Network Adaptation Layer

(packetization, priority control)

Proposed Activity

Re-quantization• Restricted Lagrangian optimization with MSE / MAP• Trellis re-quantization (MSE optional)

• Bit-rate Control

• Integrating other rate-reduction techniques (temporal and spatial resolution reduction – BGU)

I. Transrating (MPEG2 MPEG2)

HVS-based bit-rate reduction• Frame segmentation and tracking

• HVS-based bit allocation to segments

• Restricted Lagrangian optimization in each segment• Trellis re-quantization in each segment

Proposed Activity (Cont’d)

II. Transcoding (MPEG-2 H.264)

Base-line Transcoder

• Maximum matching to MPEG-2

- MEC – MB based; ½ pixel res.- B&P frames, Single ref. frame- No Intra spatial prediction- Same MB types decisions

• Adapted H.264 features:

- Syntax

- UVLC

- Transform

- Quantization

• Reduced rate- Adapting developed re-quantization techniques from MPEG-2- Integrating other rate- reduction techniques (temporal and spatial resolution reduction – BGU)

Full Transcoder

• Similar rate- Intra prediction- All motion modes- Multiple ref. Frames- Coding control decisions

END