Rate-Control in Video Codec

Rate-Control in Video Codec

Heejune AHNEmbedded Communications Laboratory

Seoul National Univ. of TechnologyFall 2008

Last updated 2008. 11. 16

Heejune AHN: Image and Video Compression p. 2

Agenda

Basic Concepts Rate-Distortion Theory Practical Rate-Control Algorithms Other issues


1. Rate Control Concepts

Key players coded video quality: (usu. Lower limits) bit-rate: some constraints (usu. upper limits) input video sequence: (noisy, degree of details, motion complexity) coding parameters (standards, coded mode, quantization step)

Rate-optimization select the parameters for best video quality under constraints Poor algorithm : low quality, large fluctuation, dropped frames

encoder

Input Video Coded Video

Bit-rate contraints

network/

storage

Rate Control

Coding para. selection


Video stream Bit-rate characteristics

Factors Encoding algorithms

(intra/inter, forward/bi-direction, DCT/Wavelet , standards, etc)

Input video Encoding parameters

• Q step, picture &mb mode, mv search area, GOP structures

4000 4200 4400 4600 4800 50001.0

1.5

2.0

2.5

4000 4200 4400 4600 4800 50000

1

2

3

0

1

2

3

화면 번호

화

면당

트래

픽 (

x 10

5 bi

ts/f

ram

e)

(c) MPEG-1 기법으로 부호화

(b) H.261 기법으로 부호화

(a) JPEG 기법으로 부호화

4000 4200 4400 4600 4800 5000


Bit-rate Constraints

Typical constraints parameters Average (mean) bit-rate (with duration) Peak (maximum) bit-rate Maximum variation in bit rate Prevention of underflow and overflows at encoder &decoder buffer Required latency

Examples DVD video

• Total 4.7GB (play time 3 hours), peak rate => 3.5Mbps with some peak limits

Video conferencing in ISDN or Circuit switching network• Constant transmission bit-rate and delay => Based on buffer size, peak, average rate is calcuated

Vide streaming over internet and packet switched network• Network condition is variable. Should be adaptive rate control


2. Rate Control Theory

Revisit to Quantization For Uniform distributed Input & Uniform quantizer

Rate Distortion Theory General trends are same as above

2 , 2 )2(3

12

12

1

1222

2

222R

xR

rN

A

N

AD

distortion ~ input signal variance and exp. desc.(bitrate) !!


Rate-Distortion Curve Rate decreases as distortion increases

• depends on input statistics (complex image => large variance)

• PSNR measure is 10log10(2552/D)

3.0

2.0

1.0

0.1 0.2 0.4 1

R DI( ) ( )

D / 2

0.1 0.5 1.0

)()( RD I

R

2

R D DD

D

I( ) ( )log ,

,

1

20

0

22

2


Rate-Distortion Optimization

Motivation Which subject I have to study if I gets 90pts in Math, 50pts in Eng. ? Image coding

• Condition: we can use 10KBytes for one picture and each parts have different complexity (i.e, simpler and complex, variance)

• Which parts I have to assign more bits?

simpler blocks

More complex blocks


Lagrange multiplier

Constrained Optimization Problem Minimize distortion with bits no larger than max bits

Cannot use partial differential for minima and maxima Lagrange Multiplier technique

Insert one more imaginary variable (called Lagrange) We have multivariable minimization problem

maxR R subject to )}(min{ RD

})(min{ RRDJ


Side note: why not partial differential

Simple example min subject to

Solution• By substitution• minimum ½ at x = y = 1/sqrt(2)

Wrong Solution With complex contraints, we cannot use substitution of variable

method We cannot change partial different with ordinary difference

• If we check,

• x = y = 0 ?

22 yx 1

122),( yxyxf 1),( yxyx

02)( 22

xx

yx 02)( 22

yy

yx


An Example: Lagrange Multiplier

Optimization Problem multiple independent Gaussian variables

Lagrange multiplier optimization

BbM

ii

)(1

i hλb

J

B b b DJ

ibi

i

M

ii

ii

22

1

2)2ln( 2 0

)()(

1 ,22)( ),()( 2222

1

hhhbDbDBD ii b

ib

iiii

M

iii

iChibi for ])2 ln 2[( 2 122


Solution M 개를 모두 곱하고 M square

를 하면

Bit-allocation

Lagrange value

CbM

bMbM

ii

i

ii

222

2

12

1

2 222

M

bb i

M

ii

,

1

22

2

2

2 log2

1

0 i

i bb

bM

ii

b hh 2

1

222 22ln222ln2


RDO in Image Coding

Bit allocation problem Modified Lagrange Method First proposed by H. Everett, then signal processing domain by

Shoham and Gersho Theorem

For any the solution B*(to the ‘unconstrained’ problem

is also a solution to the ‘constrained’ problem

subject to

Here B = {r1,r2,. . ., rN}, ie. Bit allocaton

each ri such that

])},,,([]),,,([{min 2121 NNSB rrrRrrrD

)}({min RDSB

CRBR )(

*)( ii

rDr


RDO algorithm

How to find such that results to Algorithm

Set Get Ri for each i independently ie. tagent to slope of RD curve Get R (e.g. sum of ri’s) Repeat until R = Rc, changing (note: large small b)

Diverse Constraints Sum Ri < R Embedded sum Ri < Rx Buffer controls Dependency condition

*)( ii

rDr

CRBR *)(


RDO algorithm

Intuitive understanding Reverse Water filling theory

• larger bits for larger variance block

• The returns will be same at the equivalent point

Large variance

RD curve

small variance

RD curve

RS* RL*


Practical Rate Control

Problem in Lagrange optimization too much computational complexity for real-time use

Rate control Model Buffer occupancy level B at the end of i-th frame

• BiPRE = Bi-1 + Rv – Rc

• Bi = BiPRE if 0< Bi

PRE <Bmax

= Bmax if overflow (BiPRE >Bmax)

= 0 if underflow (BiPRE < 0)

Rate Control (a negative feedback loop) : see Fig. 10.9• High level => large Q step => smaller Rv => level down• Low level => smaller Q step => larger Rv => level up

Rate Control

Encoder

Quant step

Rv Rc

Input Video


Practical Rate Control

H263 Test Model Rate control Frame Level

• Bit allocation for preventing buffer underflow and overflow (some times even frame skip)

Macroblock level • Allocate bits for each MB

• A : # of pixles in a MB, : standard deviation, Qi : quant. step

• Step 1: calculate the MB complexity

• Step 2: calculate Qi from K, A, C, a,

• Step 3: encode MB• Step 4: update the model parameters K, C

)(2

2

CQ

KABi

ii


MPEG-2 Test Model 5 Bit allocation

• Assign a target bits to current GOP

• Assign to each current picture– Based on Complexity of previous I, P, B pictures (Q-Step and

resultant bit)

Rate control • Compare coded bit with expected values, and adjust Q-Step

Modulation • block activity, increase Q-step

bbbpppiii QSXQSXQSX ,,

PR

C

KX

XKNN

BT

PR

C

KX

XKNN

BT

PR

C

KX

XN

KX

XNB

T

pb

pbpb

rb

bp

bpbp

rp

bi

bb

pi

pp

ri 8

,max,8

,max,8

,1

max

cntMB

jTBdd

cntMB

jTBdd

cntMB

jTBdd b

jbjbp

jpjpi

jiji _

)1(,

_

)1(,

_

)1(101010

r

dQ j

j

31

actavgact

actavgactactN

j

jj _2

_2_


4. Other Rate Control Issue

Variable Complexity Algorithm Constraints on computational power

• Motion search algorithm & search range • Rate Control Mode selection • Frame rate (skipping)

Trade off• Computation complexity v.s. video quality • Also depends on input image (time-varying)

Usage • Computation limited Mobile device (usu. Software based codec)• Iain E.G. Richardson, the athor of textbook,’s interest

Performance Measure PSNR is not enough (for subject quality measure) Some objective measures are under development SSIM(Structural SIMiliarity)

Appendix Rate distortion theory


Rate-Distortion Theory

Distortion of X’ from X D = H(X) – H(X|X’)

• X 전체의 정보양에서 X’ 을 통하여 확보한 X 의 정보양을 뺀것 (make sense? Yeh ~~~)

• 참고 : mutual information : I (X:X’) = H(X:X’) = H(X’:X)

Information theoretic definition

I(X:X’)

H(X’)H(X)

D = H(X) – I(X:X’)

Note: 붉음색은 X’ = Quant(X) 인경우


Rate Distortion

Rate-distortion function Inverse relation between Information/Data Rate and Distortion Example with Gaussian R.V. (known as most difficult distribution)

• Refer to element of Information theory

3.0

2.0

1.0

0.1 0.2 0.4 1

R DI( ) ( )

D / 20.1 0.5 1.0

)()( RD I

R

2

R D DD

D

I( ) ( )log ,

,

1

20

0

22

2


Derivation

Rate-Control in Video Codec

Documents

video compressionp

video stream

peak rate

average rate

rate control theoryrevisit

ratedistortion curverate

best video quality

rate maximum variation