Variable Bit Rate Video Coding

Variable Bit Rate Video Coding

April 18, 2002

(Compressed Video over Networks: Chapter 9)

Topics Introduction Variable rate compression of video Delay constraints Impact of transmission modes Encoder rate constraints Rate control algorithms Conclusions

Introduction VBR/CBR

• Coding modes (no. of bits per frame)

• Transport modes (no. of bits that can be transmitted during certain periods of time)

Video quality (end-to-end), distortion• Depends on decisions at encoder (if lossy and online)

• Depends on transport (loss, bandwidth, delay)

• Tolerance depends on application

Variable rate compression Variable rate nature of compressed video:

• Input video characteristics (scene changes, redundancy varies)

• Coding parameters (quantizers, motion compensation)

Constant quality requires variable bits per frame Offline encoding (two-pass approach for DVD) Distortion measurement (MSE)

Variable rate compression Ways of creating variable rate video

• Quantizer selection (with each block/macroblock; complicates for video since RD of future frames depend on motion vectors)

• Frame type selection (I, P, B; I frames may be placed at fixed interval, eg. every 0.5 sec)

• Frame skipping (use more bits/frame but less frame/s; use interpolation at receiver)

Delay constraints Prevent decoder buffer underflow

i

i

kk

Ni

kk RC

d

,1

1

1

Ri = no. of bits assigned to frame iCi = no. of bits recd. by decoder buffer during ith frame intervalTd = I corresponds to ith frame intervals having passed

Delay constraints We assumed no frame skipping, and const. frame rate Encoding and decoding delay constant. Delays not considered:

• Transmitter buffer delay tb

• Transmission channel delay ch

tb+ ch should not violate decoder buffer underflow constraint

Constraints vary on• Pre-encoded video

• Live video

• Interactive video

Impact of transmission modes CBR vs VBR transmission

• Better video quality, shorter delay, increased call capacity QoS, best-effort model of networks Constrained vs. unconstrained transmission

• Eg. Transmission rate subject to conditions (peak rate/sustainable rate in QoS) or congestion control in TCP/IP within best-effort transmission

Feedback vs No Feedback• Encoders can modulate data(quality/resolution/rate) based on feedback

Modes of operation: U-VBR / U-SVBR/ C-VBR / F-VBR

Encoder Rate constraints Derive rate constraints that encoder/transmitter must meet

• Avoid violating delay constraints

• Use channel information

• Consider type of application (PEV/LIV/LNIV)

• Memory restrictions (PDA)

To prevent decoder buffer underflow, encoder must avoid the encoder buffer size exceeding• CBR transmission mode

• VBR with known channel rate

• VBR with unpredictable channel rates

Encoder Rate constraints

CNB deff

Ni

ijjeff CiB

1

)(

Ni

ijje CEiB

1

)(

threshold)(1

Ni

ijje CiBP

Encoder Rate constraints Video caching

• Proxy caching of web objects• Efficient sharing of resources• Reduce initial latency• Less likelihood of packet losses

• Replicate entire sequence

• store prefix (for td seconds)

• Store intermediate samples so that decoder does not starve• Coarse layers cached

Rate control algorithms Constraints placed so as to guarantee that decoder always has

data to decode Video encoder should produce bitstream that meets these

constraints To accomplish this, we need rate control algorithms

H.263, MPEG-2 do not define operation of encoder, so free to use any rate control techniques

Rate control algorithms Problem formulation

• Rate constraints for set of frames, not of individual frame

• Decision on how to allocate bits among the set, so that the average distortion is minimized – RD optimization

CBR algorithms• Each coding parameter is associated with a rate and distortion pair.

Find coding parameters within a finite set (for each frame) such that distortion is lowest

• First calculate RD at all points and then optimize

• Gets complicated when dependency exists among frames

• Allow change of frame rate (find optimum, no jerky motion, redundancy decreases among frames at low frame rate)

Rate control algorithms VBR algorithms

• Many parameters (rate, delay, etc.)

• Algorithms monitor the long-term average transmission rate and keep it close to a value

Real-time adaptation to channel conditions• Encoder should react to channel changes (channel memory is long,

channel feedback is fast) Layered/Scalable video

• Incorporated in MPEG-2 standard, high complexity.

Conclusions VBR coding is the natural representation of video Constraints need to be placed at encoder for best

quality Transmission medium and service plays a critical role Optimization algorithms used to find ideal rate at

encoder Rate control gets even more challenging with

wireless mediums