Computational Complexity Management of a Real-Time H.264/AVC Encoder C S Kannangara, I E Richardson, and A J Miller CSVT 2008 1.

Computational Complexity

Management of a Real-Time H.264/AVC

Encoder

C S Kannangara, I E Richardson, and A J Miller

CSVT 2008

1

Outline Coding Complexity of H.264/AVC Frame Rate vs. Coding Complexity Objective Architecture Frame Level Control Algorithm Per-frame Complexity Control Algorithm Experimental Results

2

Coding Complexity of H.264/AVC

Variable block size Increase 2.5% complexity for each additional mode Reduce 4~20% bit rate

Hadamard transform Increase 20% access frequency

CABAC Increase 25~30% access frequency Reduce 16% bit-rate

Multiple reference frame Increase 20% access frequency for each added frame Reduce 2~14% bit rate

J. Ostermann, J. Bormans, P. List, D. Marpe, M. Narroschke, F. Pereira, T. Stockhammer, and T. Wedi, "Video Coding with H.264/AVC: Tools, Performance and Complexity," IEEE Circuits and Systems Magazine, vol. 4, pp. 7-28, 2004.

3

Frame Rate vs. Coding Complexity Video encoding over limited computational

resources Reduce frame rate or encoding complexity?

Subjective quality of high activity videos

Subjective quality of low activity videos

J. A. F. Ronnie, T. Apteker, Valentin S. Kisimov, and Hanoch Neishlos, "Video Acceptability and Frame Rate," IEEE Multimedia, vol. 2, pp. 32-40, 1995.

4

Objective Managed complexity (MC) coder

Maintain a smooth video frame rate Frame level control algorithm

Calculate total delay Calculate target encoding time for the next frame Drop the frame if coding delay is too high

Per-frame complexity control algorithm Lagrangian cost Complexity control

5

Architecture

6

Frame Level Control Algorithm (1/2) Timing scenario of the buffer

Encoding delay Total_Delayn = Tn

st – Tlast + Tf(Buffer_level-1) Tf + Tf(B-1) (drop frames in this situation)

F1

T1 T2 T3 T4 T5

TfTf Tf Tf

T1st T2

st T3st T4

st

T1enc T2

enc T3enc

F1 F2 F2

F3

F3 F3

F4F4

F5

F2 F3 F4

F5

T3st – T3 T4

st – T5 + Tf

The last input frame timeThe maximum buffer size

7

Frame Level Control Algorithm (2/2) Target encoding time for the next frame

Total_Delayn+k=Total_Delayn+k-1–Tf+Tn+k-1target

Total_Delayn+k=Total_Delayn–kTf+Tntarget +Tn+1

target +…+Tn+k-1

target

Assume Total_Delayn+k = 0 (Tn

target+Tn+1target+…+Tn+k-1

target)/k = Tf –Total_Delayn/k

Assume TntargetTn+1

target…Tn+k-1target

Tntarget = Tf –Total_Delayn/k

Max Total_Delayn Min Tntarget = 0

0 = Tf - TfB/k k = B

Tntarget = Tf – [Tn

st – Tlast + Tf(Buffer_level-1) ]/B8

Delay due to Fn+k-1

Tn+k-1st

Tlast

Tf

Tn+k-1target

Per-frame Complexity Control Algorithm (1/3) Encode Fn within time Tn

target Lagrangian cost (code or skip a MB)

Code a MB Jcode = Dcode + rRcode + cCcode

Skip a MB Jskip = Dskip (MV=0 without residual)

Jskip < Jcode ? skip : coding

r

r Aexp(BQP) A=7.410-8F+5.210-5 B=-3.710-5F+0.3 F = (Avg. MSE per MB) (Avg. bits per MB)

Set to 1

Detail 9

Per-frame Complexity Control Algorithm (2/3) r (RD control) D(xi,Mi)=FiR-1(Xi,Mi)

F = (1/N)Fi

r Aexp(BQP)

A=7.410-8F+5.210-5

B=-3.710-5F+0.3 (by linear regression)

Carephone(100-150)

Foreman(100-150)

Foreman(275-325)

RFD

RdDdr

C.S. Kannangara, I.E.G. Richardson, M. Bystrom, J.R. Solera, Y. Zhao, A. MacLennan and R. Cooney, “ Low Complexity Skip Prediction for H.264 Through Lagrangian Cost Estimation” , IEEE Trans. Circuits and Systems for Video Tech., vol. 16, no. 2, pp. 202-208, February 2006.

10

Per-frame Complexity Control Algorithm (3/3) c (Complexity control)

cn = c

n-1 + cn

cn =

c2 = 0.2(1-)2(T1

enc – T2enc)

c1 = 0

Limitation: -2 c

n 1 (handling Non-stationary behaviour) Similar number of drop MBs between successive frames

cn Limitc (limiting the number of drop MBs) Drop frames rather than drop MBs

Actual encoding time

Target percentage of saved coding time

11C. S. Kannangara and I. E. G. Richardson, "Computational Control of an H.264 Encoder through Lagrangian Cost Function Estimation", VLBV 2005, Sardinia, Italy, September 2005.

Experimental Results (1/6) Environment

JM 7.4 Baseline Profile One reference frame, IPPP… RDO disabled Start at 15fps frame rate

Multiple processing threads Proposed algorithm Encoding process

12

Experimental Results (2/6) Initial environment

13

Experimental Results (3/6)

14

Experimental Results (4/6)

15

Experimental Results (5/6)

16

Experimental Results (6/6)

17