Top Banner
COMPLEXITY REDUCTION OF MOTION ESTIMATION IN HEVC Jayesh Dubhashi Department of Electrical Engineering University of Texas at Arlington Advisor: Dr. K R. Rao
52

COMPLEXITY REDUCTION OF MOTION ESTIMATION IN HEVC Jayesh Dubhashi Department of Electrical Engineering University of Texas at Arlington Advisor: Dr. K.

Apr 01, 2015

Download

Documents

Norman Manner
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: COMPLEXITY REDUCTION OF MOTION ESTIMATION IN HEVC Jayesh Dubhashi Department of Electrical Engineering University of Texas at Arlington Advisor: Dr. K.

COMPLEXITY REDUCTION OF MOTION ESTIMATION IN HEVC

Jayesh Dubhashi

Department of Electrical Engineering

University of Texas at Arlington

Advisor: Dr. K R. Rao

Page 2: COMPLEXITY REDUCTION OF MOTION ESTIMATION IN HEVC Jayesh Dubhashi Department of Electrical Engineering University of Texas at Arlington Advisor: Dr. K.

Outline

• Introduction of HEVC• Features of HEVC• Inter prediction in HEVC• Proposed algorithm• Experimental conditions and results• Conclusions• Future work

Page 3: COMPLEXITY REDUCTION OF MOTION ESTIMATION IN HEVC Jayesh Dubhashi Department of Electrical Engineering University of Texas at Arlington Advisor: Dr. K.

Introduction to HEVC• HEVC is the latest standard by Joint Collaborative Team

of ITU-T VCEG and ISO/IEC MPEG

• 50% bit-rate reduction over the H.264 standard with same perceptual quality

• Uses variable block sizes as compared to fixed block sizes in the H.264 standard

• Supports parallel processing architectures

Page 4: COMPLEXITY REDUCTION OF MOTION ESTIMATION IN HEVC Jayesh Dubhashi Department of Electrical Engineering University of Texas at Arlington Advisor: Dr. K.

HEVC Encoder

Page 5: COMPLEXITY REDUCTION OF MOTION ESTIMATION IN HEVC Jayesh Dubhashi Department of Electrical Engineering University of Texas at Arlington Advisor: Dr. K.

Coding structure in HEVC

Page 6: COMPLEXITY REDUCTION OF MOTION ESTIMATION IN HEVC Jayesh Dubhashi Department of Electrical Engineering University of Texas at Arlington Advisor: Dr. K.

Coding structure in HEVC

Page 7: COMPLEXITY REDUCTION OF MOTION ESTIMATION IN HEVC Jayesh Dubhashi Department of Electrical Engineering University of Texas at Arlington Advisor: Dr. K.

Inter prediction• Exploits temporal redundancies• Motion prediction using one or more frames called

reference frames

Page 8: COMPLEXITY REDUCTION OF MOTION ESTIMATION IN HEVC Jayesh Dubhashi Department of Electrical Engineering University of Texas at Arlington Advisor: Dr. K.

Motion estimation and compensation• Motion estimation and compensation process is used to

find the best matching block• Motion search is performed• Motion vectors are obtained which are the horizontal and

vertical displacements which direct to the Prediction Unit (PU) in the reference picture

Page 9: COMPLEXITY REDUCTION OF MOTION ESTIMATION IN HEVC Jayesh Dubhashi Department of Electrical Engineering University of Texas at Arlington Advisor: Dr. K.

Motion estimation• motion vector prediction

• Neighboring blocks are likely to correspond to the same moving object with similar motion and the motion of the object is not likely to change abruptly over time

Page 10: COMPLEXITY REDUCTION OF MOTION ESTIMATION IN HEVC Jayesh Dubhashi Department of Electrical Engineering University of Texas at Arlington Advisor: Dr. K.

Motion estimation• Motion vectors of the current block are usually correlated with the

motion vectors of neighboring blocks in the current picture or in the earlier coded pictures

• Motion vectors are coded in terms of horizontal (x) and vertical (y) components as a difference to a motion vector predictor (MVP)

Page 11: COMPLEXITY REDUCTION OF MOTION ESTIMATION IN HEVC Jayesh Dubhashi Department of Electrical Engineering University of Texas at Arlington Advisor: Dr. K.

Motion estimation in HEVC

Page 12: COMPLEXITY REDUCTION OF MOTION ESTIMATION IN HEVC Jayesh Dubhashi Department of Electrical Engineering University of Texas at Arlington Advisor: Dr. K.

Diamond search pattern

Small diamond pattern Large diamond pattern

Page 13: COMPLEXITY REDUCTION OF MOTION ESTIMATION IN HEVC Jayesh Dubhashi Department of Electrical Engineering University of Texas at Arlington Advisor: Dr. K.

Proposed algorithm• Step 1: Start the motion estimation process

• Step 2: Get the best motion vector predictor and compute the threshold using the formula threshold=(β x SAD of predictor) where β= (block size/(SAD of predictor)2 ) - α

• Step 3: Perform a small diamond search pattern on the initial search point

• Step 4: Calculate the SAD for the first point and check if it is less than the threshold

Page 14: COMPLEXITY REDUCTION OF MOTION ESTIMATION IN HEVC Jayesh Dubhashi Department of Electrical Engineering University of Texas at Arlington Advisor: Dr. K.

Proposed algorithm• Step 5: If yes, go to step 7 else step 6

• Step 6: Continue with search process on the next search point

• Step 7: Start the next step with a large diamond search pattern

Page 15: COMPLEXITY REDUCTION OF MOTION ESTIMATION IN HEVC Jayesh Dubhashi Department of Electrical Engineering University of Texas at Arlington Advisor: Dr. K.

Test conditions • Source code: HEVC Reference software HM 13.0 [38]

• Platform: Windows 7 64-bit OS on 8 GB RAM at 2.2 GHz on Intel I-7 processor

• Profile: ‘random access profile’

• CTB size: 64x64 with minimum CU size of 8x8.

• Quantization Parameters: 22, 27, 32, 37

Page 16: COMPLEXITY REDUCTION OF MOTION ESTIMATION IN HEVC Jayesh Dubhashi Department of Electrical Engineering University of Texas at Arlington Advisor: Dr. K.

Test sequences

No Sequence Resolution Type No of frames

1 Race Horses 416x240 WQVGA 30

2 BQ Mall 832x480 WVGA 30

3 BasketBallDrillText 832x480 WVGA 30

4 Kristen and Sara 1280x1080 SD 30

5 Parkscene 1920x1080 HD 30

Page 17: COMPLEXITY REDUCTION OF MOTION ESTIMATION IN HEVC Jayesh Dubhashi Department of Electrical Engineering University of Texas at Arlington Advisor: Dr. K.

Test sequences

Page 18: COMPLEXITY REDUCTION OF MOTION ESTIMATION IN HEVC Jayesh Dubhashi Department of Electrical Engineering University of Texas at Arlington Advisor: Dr. K.

Encoding time gain

22 27 32 370

200

400

600

800

1000

1200

1400

1600 1394.02

1110.753903.756

753.122

1323.65

1022.239

812.419

656.715

original

proposed

Race Horses-WQVGA-30 frames

Qp

Enc

odin

g ti

me

(sec

)

22 27 32 370

500

1000

1500

2000

2500

3000

3500

40003344.474

2759.522383.185

2133.866

3049.212

2467.316 2075.006

1839.711

original

proposed

BQ Mall-WVGA-30 frames

Qp

Enc

odin

g ti

me

(sec

)

22 27 32 370

500

1000

1500

2000

2500

3000

3500

40003795.112

3114.593 2611.741

2282.861

3513.106

2795.1512268.961

1961.797

original

proposed

BasketBallDrillText-WVGA-30 frames

Qp

Enc

odin

g ti

me

(sec

)

22 27 32 370

1000

2000

3000

4000

5000

6000 5230.557

4560.632 4031.441 4308.201

4544.291

3899.955 3562.4713374.153

original

proposed

Kristen and Sara-SD-30 frames

Qp

Enc

odin

g ti

me

(sec

)

Page 19: COMPLEXITY REDUCTION OF MOTION ESTIMATION IN HEVC Jayesh Dubhashi Department of Electrical Engineering University of Texas at Arlington Advisor: Dr. K.

Encoding time gain

22 27 32 370

2000

4000

6000

8000

10000

12000

14000

16000

1800016588.689

13091.34611353.34

10315.036

15210.024

11733.109

9648.017 8583.977

original

proposed

Park Scene - HD - 30 frames

QP

Enc

odin

g ti

me(

sec)

Page 20: COMPLEXITY REDUCTION OF MOTION ESTIMATION IN HEVC Jayesh Dubhashi Department of Electrical Engineering University of Texas at Arlington Advisor: Dr. K.

Encoding time gain22 27 32 37

-14

-12

-10

-8

-6

-4

-2

0

-5

-7.9

-10.1

-12.8

Race Horses-WQVGA-30 frames

QP

% d

ecre

ase

in e

ncod

ing

time

22 27 32 37

-16

-14

-12

-10

-8

-6

-4

-2

0

-8.8

-10.5

-12.9-13.7

BQ Mall-WVGA-30 frames

QP

% d

ecre

ase

in e

ncod

ing

time

22 27 32 37

-16

-14

-12

-10

-8

-6

-4

-2

0

-7.4

-10.2

-13.1-14

BasketBallDrillText-WVGA-30 frames

QP

% d

ecre

ase

in e

ncod

ing

time 22 27 32 37

-20-18-16-14-12-10-8-6-4-20

-13.1-14.5

-17.3-16.3

Kristen and Sara-SD-30 frames

QP

% d

ecre

ase

in e

ncod

ing

time

Page 21: COMPLEXITY REDUCTION OF MOTION ESTIMATION IN HEVC Jayesh Dubhashi Department of Electrical Engineering University of Texas at Arlington Advisor: Dr. K.

Encoding time gain

22 27 32 37

-18

-16

-14

-12

-10

-8

-6

-4

-2

0

-8.31

-10.37

-15.2

-16.8

Parkscene -HD-30 frames

QP

% d

ecre

ase

in e

ncod

ing

time

Page 22: COMPLEXITY REDUCTION OF MOTION ESTIMATION IN HEVC Jayesh Dubhashi Department of Electrical Engineering University of Texas at Arlington Advisor: Dr. K.

Bit-rate increase

22 27 32 370

200

400

600

800

1000

1200

1400

1600

1800 1443.872

724.936

362.28185.016

1643.664

786.544

379.408

188.528

original

proposed

Bit-

rate

incr

ease

(Kbp

s)

Race Horses -WQVGA-30 frames

QP22 27 32 37

0

500

1000

1500

2000

25001935.968

942.528

474.12256.664

2135.416

1004.936

490.208

261.2

original

proposed

Bit-

rate

incr

ease

(Kbp

s)

BQ Mall -WVGA-30 frames

QP

22 27 32 370

500

1000

1500

2000

2500

30002222.256

1070.696

553.344296.952

2400.064

1163.656

584.696

309.496

Original

proposed

Bit-

rate

incr

ease

(Kbp

s)

BasketBallDrillText -WVGA-30 frames

QP22 27 32 37

0

200

400

600

800

1000

1200

1400 1171.816

507.784

266.296148.184

1218.336

522.408

269.648

150.36

Original

proposed

Bit-

rate

incr

ease

(Kbp

s)

Kristen and Sara -SD-30 frames

QP

Page 23: COMPLEXITY REDUCTION OF MOTION ESTIMATION IN HEVC Jayesh Dubhashi Department of Electrical Engineering University of Texas at Arlington Advisor: Dr. K.

Bit-rate increase

22 27 32 370

2000

4000

6000

8000

10000

120009610.968

4071.8

1834.792834.32

10480.408

4306.96

1885.136

843.936

Originalpro-posed

Bit-

rate

incr

ease

(Kbp

s)

Park Scene -HD-30 frames

QP

Page 24: COMPLEXITY REDUCTION OF MOTION ESTIMATION IN HEVC Jayesh Dubhashi Department of Electrical Engineering University of Texas at Arlington Advisor: Dr. K.

Bit-rate increase

22 27 32 370

2

4

6

8

10

12

14

1613.83

8.4

4.7

1.8

original vs proposed

Bit-

rate

incr

ease

(%)

Race Horses -WQVGA-30 frames

QP22 27 32 37

0

2

4

6

8

10

1210.3

6.6

3.3

1.7

original vs proposed

Bit-

rate

incr

ease

(%)

BQ Mall -WVGA-30 frames

QP

22 27 32 370

1

2

3

4

5

6

7

8

9

10

88.6

5.6

4.2

original vs proposed

Bit-

rate

incr

ease

(%)

BasketBallDrillText -WVGA-30 frames

QP22 27 32 37

0

0.5

1

1.5

2

2.5

3

3.5

4

4.53.9

2.8

1.21.4

original vs proposed

Bit-

rate

incr

ease

(%)

Kristen and sara -SD-30 frames

QP

Page 25: COMPLEXITY REDUCTION OF MOTION ESTIMATION IN HEVC Jayesh Dubhashi Department of Electrical Engineering University of Texas at Arlington Advisor: Dr. K.

Bit-rate increase

22 27 32 370

1

2

3

4

5

6

7

8

9

109

5.7

2.7

1.1

original vs proposed

Bit-

rate

incr

ease

(%)

Park Scene -HD-30 frames

QP

Page 26: COMPLEXITY REDUCTION OF MOTION ESTIMATION IN HEVC Jayesh Dubhashi Department of Electrical Engineering University of Texas at Arlington Advisor: Dr. K.

PSNR vs Bit-rate

100 300 500 700 900 1100 1300 1500 1700 19002527293133353739414345

Original Proposed

BitRate(Kbps)

Race Horses-WQVGA-30 frames

PSN

R (

dB)

100 300 500 700 900 1100 1300 1500 1700 1900 2100 23002527293133353739414345

Original Proposed

BitRate(Kbps)

BasketBallDrillText-WVGA-30 frames

PSN

R (

dB)

100 600 1100 1600 2100 2600252729313335373941

Original Proposed

BitRate(Kbps)

BQ Mall-WVGA-30 frames

PSN

R (

dB)

100 200 300 400 500 600 700 800 900 100011001200130032

34

36

38

40

42

44

Original Proposed

BitRate(Kbps)

Kristen and Sara-SD-30 framesP

SNR

(dB

)

Page 27: COMPLEXITY REDUCTION OF MOTION ESTIMATION IN HEVC Jayesh Dubhashi Department of Electrical Engineering University of Texas at Arlington Advisor: Dr. K.

PSNR vs Bit-rate

100 1600 3100 4600 6100 7600 9100 10600 1210025

27

29

31

33

35

37

39

41

Original Proposed

BitRate(Kbps)

Park Scene-HD-30 frames

PSN

R (

dB)

Page 28: COMPLEXITY REDUCTION OF MOTION ESTIMATION IN HEVC Jayesh Dubhashi Department of Electrical Engineering University of Texas at Arlington Advisor: Dr. K.

BD-PSNR loss

22 27 32 37

-3

-2.5

-2

-1.5

-1

-0.5

0

-2.4183

-1.7854

-1.1622

-0.63

original vs proposed

QP

BD

-PSN

R (

dB)

Race Horses WQVGA 30 frames

22 27 32 37

-2-1.8-1.6-1.4-1.2

-1-0.8-0.6-0.4-0.2

0

-1.763

-1.2547

-0.7477

-0.4354

original vs proposed

QP

BD

-PSN

R (

dB)

BasketBallDrillText WVGA 30 frames

22 27 32 37

-1.6

-1.4

-1.2

-1

-0.8

-0.6

-0.4

-0.2

0

-1.2644-1.4409

-1.0617 -1.0617

original vs proposed

QP

BD

-PSN

R (

dB)

BQ Mall WVGA 30 frames

22 27 32 37

-0.8

-0.7

-0.6

-0.5

-0.4

-0.3

-0.2

-0.1

0

-0.7473

-0.6338

-0.3395 -0.3574

original vs proposed

QP

BD

-PSN

R (

dB)

Kristen and Sara SD 30 frames

Page 29: COMPLEXITY REDUCTION OF MOTION ESTIMATION IN HEVC Jayesh Dubhashi Department of Electrical Engineering University of Texas at Arlington Advisor: Dr. K.

BD-PSNR loss

22 27 32 37

-1.4

-1.2

-1

-0.8

-0.6

-0.4

-0.2

0

-1.3003

-0.9629

-0.5494

-0.2679

original vs proposed

QP

BD

-PSN

R (

dB)

Park Scene HD 30 frames

Page 30: COMPLEXITY REDUCTION OF MOTION ESTIMATION IN HEVC Jayesh Dubhashi Department of Electrical Engineering University of Texas at Arlington Advisor: Dr. K.

BD-Bitrate increase

22 27 32 3705

101520253035404550

39.467943.9024

33.1171

21.4365

original vs proposed

QP

BD

-Bit

Rat

e (K

bps)

Race Horses WQVGA 30 frames

Series10

5

10

15

20

25 22.5899

19.1387

14.124

8.9455

original vs proposed

QP

BD

-Bit

Rat

e (K

bps)

BasketBallDrillText WVGA 30 frames

22 27 32 370

2468

1012141618

13.1191

16.5348 16.4486 15.8582

original vs proposed

QP

BD

-Bit

Rat

e (K

bps)

BQ Mall WVGA 30 frames

Series10

1

2

3

4

5

6

7

5.74836.1477

4.618

3.008

original vs proposed

QP

BD

-Bit

Rat

e (K

bps)

Kristen and Sara SD 30 frames

Page 31: COMPLEXITY REDUCTION OF MOTION ESTIMATION IN HEVC Jayesh Dubhashi Department of Electrical Engineering University of Texas at Arlington Advisor: Dr. K.

BD-Bitrate increase

22 27 32 370

5

10

15

20

25 23.160321.5803

15.0043

7.8196

original vs proposed

QP

BD

-Bit

Rat

e (K

bps)

Park Scene HD 30 frames

Page 32: COMPLEXITY REDUCTION OF MOTION ESTIMATION IN HEVC Jayesh Dubhashi Department of Electrical Engineering University of Texas at Arlington Advisor: Dr. K.

Conclusions• The proposed algorithm of early termination of motion

search has resulted in reduction of encoding time by 5- 17% over the existing algorithm

• PSNR loss of less than 1dB shows negligible loss in the quality

• Increase in the bit-rate as a trade-off

Page 33: COMPLEXITY REDUCTION OF MOTION ESTIMATION IN HEVC Jayesh Dubhashi Department of Electrical Engineering University of Texas at Arlington Advisor: Dr. K.

Future work• The proposed algorithm can be used with different search

patterns like hexagon or octagon search patterns or adaptive patterns

• Parallel processing techniques can be used in the search algorithms or for prediction units in CUs.

• Complexity reduction can also be done using hardware implementations of the algorithms which require higher computations.

Page 34: COMPLEXITY REDUCTION OF MOTION ESTIMATION IN HEVC Jayesh Dubhashi Department of Electrical Engineering University of Texas at Arlington Advisor: Dr. K.

References• 1. B. Bross, W. J. Han, J. R Ohm and T Wiegand, “High efficiency video coding (HEVC) text specification draft 8”, ITU-T/ISO/IEC Joint

Collaborative Team on Video Coding (JCTVC) document JCTVC-J1003, July 2012

 • 2. G. J. Sullivan, J.-R. Ohm, W.-J. Han, and T. Wiegand, "Overview of the high efficiency video coding (HEVC) Standard," IEEE Transactions on

Circuits and Systems for Video Technology, vol 22 , pp.1649-1668, December 2012.

 • 3. F. Bossen, B. Bross, K. Sühring, and D. Flynn, "HEVC complexity and implementation analysis," IEEE Transactions on Circuits and Systems

for Video Technology, vol 22 , pp.1685-1696, December 2012.

 • 4. H. Samet, “The quadtree and related hierarchical data structures,”Comput. Surv, vol. 16 , pp. 187-260, 1984

 • 5. N.Purnachand , L. N. Alves and A.Navarro, “Fast motion estimation algorithm for HEVC ,” IEEE Second International Conference on

Consumer Electronics - Berlin (ICCE-Berlin), 2012.

 • 6. X Cao, C. Lai and Y. He, “ Short distance intra coding scheme for HEVC”, Picture Coding Symposium, 2012.

• 7. M. A. F. Rodriguez, “CUDA: Speeding up parallel computing”, International Journal of Computer Science and Security, November 2010.

 • 8. NVIDIA, NVIDIA CUDA Programming Guide, Version 3.2, NVIDIA, September 2010.

http://docs.nvidia.com/cuda/cuda-c-programming-guide/

 • 9. “http://drdobbs.com/high-performance-computing/206900471” Jonathan Erickson, GPU Computing Isn’t Just About Graphics Anymore,

Online Article, February 2008.

 • 10. J. Nickolls and W. J. Dally,” The GPU computing era” , IEEE Computer Society Micro-IEEE, vol. 30, Issue 2, pp . 56 - 69, April 2010.

 

 •  

Page 35: COMPLEXITY REDUCTION OF MOTION ESTIMATION IN HEVC Jayesh Dubhashi Department of Electrical Engineering University of Texas at Arlington Advisor: Dr. K.

References• 11. M. Abdellah, “High performance Fourier volume rendering on graphics processing units”, M.S. Thesis, Systems and Bio-Medical

Engineering Department, Cairo University, 2012. •  • 12. J. Sanders and E. Kandrot, “CUDA by example: an introduction to general-purpose GPU programming” Addison-Wesley, 2010. •  • 13. NVIDIA, NVIDIA’s Next Generation CUDA Compute Architecture:Fermi, White Paper, Version 1.1, NVIDIA 2009.• http://www.nvidia.com/content/PDF/fermi_white_papers/NVIDIA_Fermi_Compute_Architecture_Whitepaper.pdf•  • 14. W.-N.Chen, et al, “H.264/AVC motion estimation implementation on compute unified device architecture (CUDA)” , IEEE

International Conference on Multimedia and Expo, pp. 697 – 700, 2008.

• 15. CUDA reference manual: http://developer.nvidia.com/cuda-downloads

• 16. C. Fogg, “Suggested figures for the HEVC specification”, ITU-T/ISO/IEC Joint Collaborative Team on Video Coding (JCT-VC) document JCTVC- J0292r1, July 2012.

•  • 17. F.Dufaux and F.Moscheni, “Motion estimation techniques for digital TV – a review and a new contribution”, Proc. IEEE , vol.83, pp

858 – 876, 1995.•  • 18. J.R.Jain and A.K.Jain , “ Displacement measurement and its application in interframe image-coding” IEEE Trans. Commun.,

Vol.com -29, pp 1799-1808, Dec 1981.•  • 19. I.E.G. Richardson, Video codec design : Developing image and video compression systems , John Wiley and Sons, Chichester,

2002.•  • 20. J.B. Lee and H. Kalva , The VC-1 and H.264 video compression standards for broadband video Services , Springer Science +

Business Media , New York, 2008.

Page 36: COMPLEXITY REDUCTION OF MOTION ESTIMATION IN HEVC Jayesh Dubhashi Department of Electrical Engineering University of Texas at Arlington Advisor: Dr. K.

References• 21. M. Jakubowski and G. Pastuszak , “Block –based motion estimation algorithms - a survey”, Opto-Electronics Review vol. 21, no. 1, pp

86-102, 2013. • 22. B. Li, G. J. Sullivan, and J. Xu, “Comparison of compression performance of HEVC working draft 4 with AVC high profile,” JCTVC-G399,

Nov. 2011. • 23. P.Hanhart et al, “ Subjective quality evaluation of the upcoming HEVC video compression standard”, SPIE Applications of digital image

processing XXXV , vol. 8499, paper 8499-30, Aug. 2012. • 24. M. Horowitz et al, “Informal subjective quality comparison of video compression performance of the HEVC and H.264/MPEG-4 AVC

standards for low delay applications” , SPIE Applications of digital image processing XXXV , vol. 8499, paper 8499-31, Aug. 2012. • 25. Y. Su and M.-T. Sun, “Fast multiple reference frame motion estimation for H.264/AVC”, IEEE Transactions on circuits and systems for

video technology, vol. 16, pp. 447-452, March 2006.  • 26. Information about quad tree structure of HEVC http://codesequoia.wordpress.com/2012/10/28/hevc-ctu-cu-ctb-cb-pb-and-tb/ • 27. Information on developments in HEVC NGVC –Next generation video coding.

http://www.h265.net

• 28. JVT KTA reference software http://iphome.hhi.de/suehring/tml/download/KTA

 • 29. F.Bossen,D.Flynn and K.Suhring (July 2011), “HEVC reference software manual”

http://phenix.int-evry.fr/jct/doc_end_user/documents/6_Torino/wg11/JCTVC-F634-v2.zip 

Page 37: COMPLEXITY REDUCTION OF MOTION ESTIMATION IN HEVC Jayesh Dubhashi Department of Electrical Engineering University of Texas at Arlington Advisor: Dr. K.

References• 30. JCT-VC documents are publicly available at

http://ftp3.itu.ch/av-arch/jctvc-sitehttp://phenix.it-sudparis.eu/jct/

• 31. B.Bross et al, “High efficiency video coding (HEVC) text specification draft 8”, JCTVC-J1003,July 2012.• http://phenix.int-evry.fr/jct/doc_end_user/current_document.Php?id=5889•  • 32. Special issue on emerging research and standards in next generation video coding , IEEE Trans. CSVT, vol. 22, pp. 321 ,Dec 2012.•  • 33. M.E.Sinangil, A.P.Chandrakasan, V.Sze and M.Zhou , “Memory cost vs coding efficiency trade-offs for HEVC motion estimation

engine ", IEEE International conference on image processing, pp. 1533-1536, 2012.•  • 34. K.R.Rao , D.N.Kim and J.J.Hwang, “ Video coding standards: AVS China, H.264/MPEG-4 PART 10,HEVC, VP6, DIRAC and VC-1 “,

Springer , 2014.•  • 35. Y.He , J.Ostermann , M.Domanski , O.C.Au and N.Ling ,"Introduction to the issue on video coding : HEVC and beyond ", IEEE

journal of selected topics in signal processing, Vol. 7, no. 6 ,Dec 2013.•  • 36. Information about quad tree structure of HEVC http://codesequoia.wordpress.com/2012/10/28/hevc-ctu-cu-ctb-cb-pb-and-tb/ •  • 37. X. Cao, C. Lai and Y.He, “Short distance intra coding scheme for HEVC”, Picture Coding Symposium, pp. 501-504, 2012.•  • 38. HEVC reference software HM 13.0 [online]. • http://hevc.kw.bbc.co.uk/svn/jctvc-a124/branches/•  • 39. X. Li et al, “Rate-Complexity-Distortion evaluation for hybrid video coding”, IEEE Transactions on Circuits and Systems for Video

Technology, vol. 21, pp. 957 - 970, July 2011.

Page 38: COMPLEXITY REDUCTION OF MOTION ESTIMATION IN HEVC Jayesh Dubhashi Department of Electrical Engineering University of Texas at Arlington Advisor: Dr. K.

References• 40. HEVC test sequences: ftp://ftp.tnt.uni-hannover.de/testsequences

• 41. C. Zhaepong, W. Dujuan, J. Guang, W. Chengke, “Octagonal Search Algorithm with Early Termination for Fast Motion Estimation on H.264”, IAS Fifth International Conference on Information Assurance and Security, vol. 1, pp. 123-126, 2009

•  • 42. N. Ling, “High efficiency video coding and its 3D extension: A research perspective,” 2012 ,7th IEEE conference on

Industrial Electronics and Applications (ICIEA), pp. 2150-2155, July 2012•  • 43. S. Lui et al, “Video Prediction Block Structure and the Emerging High Efficiency Video • Coding Standard”, IEEE proceedings on Signal & Information Processing Association • Annual Summit and Conference (APSIPA ASC), 2012 Asia-Pacific, pp. 1-4, 2012. • 44. Basics of video : http://lea.hamradio.si/~s51kq/V-BAS.HTM•  • 45 MPL website: http://www.uta.edu/faculty/krrao/dip/•  • 46. Website for downloading test sequences:• https://media.xiph.org/video/derf/•  • 47. G. J. Sullivan et al, “Standardized extensions of high efficiency video coding (HEVC)”, IEEE Journal on Selected Topics in

Signal Processing, v 7, n 6, p 1001-1016, December 2013.•  

Page 39: COMPLEXITY REDUCTION OF MOTION ESTIMATION IN HEVC Jayesh Dubhashi Department of Electrical Engineering University of Texas at Arlington Advisor: Dr. K.

References• 48 Special issue on emerging research and standards in next generation video coding, IEEE trans. CSVT, vol. 22, pp. 1646-

1909, Dec. 2012.•  • 49. M. T. Pourazad et al,” HEVC: The new gold standard for video compression”, IEEE CE magazine, vol. 1, issue 3, pp. 36-

46, July 2012.•  • 50. HEVC encoded bitstreams:• ftp://ftp.kw.bbc.co.uk/hevc/hm-11.0-anchors/bitstreams/•  • 51. V. Sze, M. Budagavi, G. J. Sullivan, “High Efficiency Video Coding-Algorithm and architectures”, Springer, 2014.

Page 40: COMPLEXITY REDUCTION OF MOTION ESTIMATION IN HEVC Jayesh Dubhashi Department of Electrical Engineering University of Texas at Arlington Advisor: Dr. K.

Acronyms• AVC - Advanced Video Coding• AMVP – Advanced Motion Vector Prediction• BD - Bjontegaard Delta• CABAC – Context Adaptive Binary Arithmetic Coding• CB – Coding Block• CBF – Coding Block Flag• CFM – CBF Fast Mode• CTU – Coding Tree Unit• CTB – Coding Tree Block• CU – Coding Unit• DCT – Discrete Cosine Transform• DST – Discrete Sine Transform• HDTV - High Definition Tele Vision• HDR - High Dynamic Range• HDRI - High Dynamic Range Imaging• HEVC – High Efficiency Video Coding• HM – HEVC Test Model• HVS – Human Visual System• ISO – International Standards Organization• ITU – International Telecommunication Union• JCT-VC - Joint Collaborative Team on Video Coding

Page 41: COMPLEXITY REDUCTION OF MOTION ESTIMATION IN HEVC Jayesh Dubhashi Department of Electrical Engineering University of Texas at Arlington Advisor: Dr. K.

Acronyms• MB – Macro block• MC – Motion Compensation• ME – Motion Estimation• MPEG – Moving Picture Experts Group• NAL – Network Abstraction Layer• PB – Prediction Block• PSNR – Peak Signal to Noise Ratio• PU – Prediction Unit• QP – Quantization Parameter• RDOQ – Rate Distortion Optimization Quantization• RGB – Red Green Blue • RMD – Rough Mode Decision• SATD – Sum of Absolute Transform Differences• SD – Standard Definition• SSIM – Structural Similarity• TB – Transform Block• TU – Transform Unit• URQ – Uniform Reconstruction Quantization• VCEG – Video Coding Experts Group• VPS – Video Parameter Set• WQVGA – Wide Quarter Video Graphics Array• WVGA – Wide Video Graphics Array

Page 42: COMPLEXITY REDUCTION OF MOTION ESTIMATION IN HEVC Jayesh Dubhashi Department of Electrical Engineering University of Texas at Arlington Advisor: Dr. K.

Code for proposed algorithm• __inline Void TEncSearch::xTZ8PointDiamondSearch( TComPattern* pcPatternKey, IntTZSearchStruct& rcStruct, TComMv*

pcMvSrchRngLT, TComMv* pcMvSrchRngRB, const Int iStartX, const Int iStartY, const Int iDist )• {• Int iSrchRngHorLeft = pcMvSrchRngLT->getHor();• Int iSrchRngHorRight = pcMvSrchRngRB->getHor();• Int iSrchRngVerTop = pcMvSrchRngLT->getVer();• Int iSrchRngVerBottom = pcMvSrchRngRB->getVer();• UInt cost=rcStruct.min_cost;• // 8 point search, // 1 2 3• // search around the start point // 4 0 5• // with the required distance // 6 7 8• assert ( iDist != 0 );• const Int iTop = iStartY - iDist;• const Int iBottom = iStartY + iDist;• const Int iLeft = iStartX - iDist;• const Int iRight = iStartX + iDist;• rcStruct.uiBestRound += 1;• UInt c=rcStruct.uiBestSad;• if ( iDist == 1 ) // iDist == 1• {• //if (c > cost)• //{• if ( iTop >= iSrchRngVerTop && c > cost) // check top• {• xTZSearchHelp( pcPatternKey, rcStruct, iStartX, iTop, 2, iDist );• }• //if(c < cost)• //goto label;

Page 43: COMPLEXITY REDUCTION OF MOTION ESTIMATION IN HEVC Jayesh Dubhashi Department of Electrical Engineering University of Texas at Arlington Advisor: Dr. K.

Code for proposed algorithm• if ( iLeft >= iSrchRngHorLeft && c > cost ) // check middle left• {• xTZSearchHelp( pcPatternKey, rcStruct, iLeft, iStartY, 4, iDist );• }• //if(rcStruct.uiBestSad < rcStruct.min_cost)• // goto label;• if ( iRight <= iSrchRngHorRight && c > cost ) // check middle right• {• xTZSearchHelp( pcPatternKey, rcStruct, iRight, iStartY, 5, iDist );• }• //if(rcStruct.uiBestSad < rcStruct.min_cost)• // goto label;• if ( iBottom <= iSrchRngVerBottom && c > cost) // check bottom• {• xTZSearchHelp( pcPatternKey, rcStruct, iStartX, iBottom, 7, iDist );• }• //}• }• //label:• else • //if (iDist != 1)• {• if ( iDist <= 8 )• {•

• const Int iTop_2 = iStartY - (iDist>>1);• const Int iBottom_2 = iStartY + (iDist>>1);• const Int iLeft_2 = iStartX - (iDist>>1);• const Int iRight_2 = iStartX + (iDist>>1);

Page 44: COMPLEXITY REDUCTION OF MOTION ESTIMATION IN HEVC Jayesh Dubhashi Department of Electrical Engineering University of Texas at Arlington Advisor: Dr. K.

Code for proposed algorithm• • if ( iTop >= iSrchRngVerTop && iLeft >= iSrchRngHorLeft &&• iRight <= iSrchRngHorRight && iBottom <= iSrchRngVerBottom && c > cost) // check border• {• xTZSearchHelp( pcPatternKey, rcStruct, iStartX, iTop, 2, iDist );• xTZSearchHelp( pcPatternKey, rcStruct, iLeft_2, iTop_2, 1, iDist>>1 );• xTZSearchHelp( pcPatternKey, rcStruct, iRight_2, iTop_2, 3, iDist>>1 );• xTZSearchHelp( pcPatternKey, rcStruct, iLeft, iStartY, 4, iDist );• xTZSearchHelp( pcPatternKey, rcStruct, iRight, iStartY, 5, iDist );• xTZSearchHelp( pcPatternKey, rcStruct, iLeft_2, iBottom_2, 6, iDist>>1 );• xTZSearchHelp( pcPatternKey, rcStruct, iRight_2, iBottom_2, 8, iDist>>1 );• xTZSearchHelp( pcPatternKey, rcStruct, iStartX, iBottom, 7, iDist );• }• else // check border• {• • • //if(rcStruct.uiBestSad < rcStruct.min_cost)• //goto label1;

Page 45: COMPLEXITY REDUCTION OF MOTION ESTIMATION IN HEVC Jayesh Dubhashi Department of Electrical Engineering University of Texas at Arlington Advisor: Dr. K.

Code for proposed algorithm• if ( iTop >= iSrchRngVerTop && c > cost) // check top• {• xTZSearchHelp( pcPatternKey, rcStruct, iStartX, iTop, 2, iDist );• }• //if(c < cost)• //goto label1;• if ( iTop_2 >= iSrchRngVerTop && c > cost) // check half top• {• if ( iLeft_2 >= iSrchRngHorLeft ) // check half left• {• xTZSearchHelp( pcPatternKey, rcStruct, iLeft_2, iTop_2, 1, (iDist>>1) );• }• // if(rcStruct.uiBestSad < rcStruct.min_cost)• //goto label1;• if ( iRight_2 <= iSrchRngHorRight && c > cost) // check half right• {• xTZSearchHelp( pcPatternKey, rcStruct, iRight_2, iTop_2, 3, (iDist>>1) );• }• } // check half top• //if(rcStruct.uiBestSad < rcStruct.min_cost)• //goto label1;• if ( iLeft >= iSrchRngHorLeft && c > cost) // check left• {• xTZSearchHelp( pcPatternKey, rcStruct, iLeft, iStartY, 4, iDist );

Page 46: COMPLEXITY REDUCTION OF MOTION ESTIMATION IN HEVC Jayesh Dubhashi Department of Electrical Engineering University of Texas at Arlington Advisor: Dr. K.

Code for proposed algorithm• }• //if(rcStruct.uiBestSad < rcStruct.min_cost)• //goto label1;• if ( iRight <= iSrchRngHorRight && c > cost) // check right• {• xTZSearchHelp( pcPatternKey, rcStruct, iRight, iStartY, 5, iDist );• }• //if(rcStruct.uiBestSad < rcStruct.min_cost)• //goto label1;• if ( iBottom_2 <= iSrchRngVerBottom && c > cost) // check half bottom• {• if ( iLeft_2 >= iSrchRngHorLeft ) // check half left• {• xTZSearchHelp( pcPatternKey, rcStruct, iLeft_2, iBottom_2, 6, (iDist>>1) );• }• // if(rcStruct.uiBestSad < rcStruct.min_cost)• //goto label1;• if ( iRight_2 <= iSrchRngHorRight && c > cost)// check half right• {• xTZSearchHelp( pcPatternKey, rcStruct, iRight_2, iBottom_2, 8, (iDist>>1) );• }• } // check half bottom• //if(rcStruct.uiBestSad < rcStruct

Page 47: COMPLEXITY REDUCTION OF MOTION ESTIMATION IN HEVC Jayesh Dubhashi Department of Electrical Engineering University of Texas at Arlington Advisor: Dr. K.

Code for proposed algorithm• //goto label1;• if ( iBottom <= iSrchRngVerBottom && c > cost) // check bottom• {• xTZSearchHelp( pcPatternKey, rcStruct, iStartX, iBottom, 7, iDist );• }• • } // check border• }• //label1:• else // iDist > 8• //if(iDist>8)• {• if ( iTop >= iSrchRngVerTop && iLeft >= iSrchRngHorLeft &&• iRight <= iSrchRngHorRight && iBottom <= iSrchRngVerBottom && c > cost) // check border• {• xTZSearchHelp( pcPatternKey, rcStruct, iStartX, iTop, 0, iDist );• xTZSearchHelp( pcPatternKey, rcStruct, iLeft, iStartY, 0, iDist );• xTZSearchHelp( pcPatternKey, rcStruct, iRight, iStartY, 0, iDist );• xTZSearchHelp( pcPatternKey, rcStruct, iStartX, iBottom, 0, iDist );• for ( Int index = 1; index < 4; index++ )• {• Int iPosYT = iTop + ((iDist>>2) * index);• Int iPosYB = iBottom - ((iDist>>2) * index);• Int iPosXL = iStartX - ((iDist>>2) * index);• Int iPosXR = iStartX + ((iDist>>2) * index);• xTZSearchHelp( pcPatternKey, rcStruct, iPosXL, iPosYT, 0, iDist );

Page 48: COMPLEXITY REDUCTION OF MOTION ESTIMATION IN HEVC Jayesh Dubhashi Department of Electrical Engineering University of Texas at Arlington Advisor: Dr. K.

Code for proposed algorithm• xTZSearchHelp( pcPatternKey, rcStruct, iPosXR, iPosYT, 0, iDist );• xTZSearchHelp( pcPatternKey, rcStruct, iPosXL, iPosYB, 0, iDist );• xTZSearchHelp( pcPatternKey, rcStruct, iPosXR, iPosYB, 0, iDist );• }• }• else // check border• {• if ( iTop >= iSrchRngVerTop && c > cost) // check top• {• xTZSearchHelp( pcPatternKey, rcStruct, iStartX, iTop, 0, iDist );• }• if ( iLeft >= iSrchRngHorLeft && c > cost) // check left• {• xTZSearchHelp( pcPatternKey, rcStruct, iLeft, iStartY, 0, iDist );• }• if ( iRight <= iSrchRngHorRight && c > cost) // check right• {• xTZSearchHelp( pcPatternKey, rcStruct, iRight, iStartY, 0, iDist );• }• if ( iBottom <= iSrchRngVerBottom && c > cost) // check bottom• {• xTZSearchHelp( pcPatternKey, rcStruct, iStartX, iBottom, 0, iDist );• }

Page 49: COMPLEXITY REDUCTION OF MOTION ESTIMATION IN HEVC Jayesh Dubhashi Department of Electrical Engineering University of Texas at Arlington Advisor: Dr. K.

Code for proposed algorithm• for ( Int index = 1; index < 4; index++ )• {• Int iPosYT = iTop + ((iDist>>2) * index);• Int iPosYB = iBottom - ((iDist>>2) * index);• Int iPosXL = iStartX - ((iDist>>2) * index);• Int iPosXR = iStartX + ((iDist>>2) * index);• • if ( iPosYT >= iSrchRngVerTop && c > cost) // check top• {• if ( iPosXL >= iSrchRngHorLeft ) // check left• {• xTZSearchHelp( pcPatternKey, rcStruct, iPosXL, iPosYT, 0, iDist );• }• if ( iPosXR <= iSrchRngHorRight ) // check right• {• xTZSearchHelp( pcPatternKey, rcStruct, iPosXR, iPosYT, 0, iDist );• }• } // check top• if ( iPosYB <= iSrchRngVerBottom && c > cost) // check bottom• {• if ( iPosXL >= iSrchRngHorLeft ) // check left• {• xTZSearchHelp( pcPatternKey, rcStruct, iPosXL, iPosYB, 0, iDist );• }• if ( iPosXR <= iSrchRngHorRight && c > cost) // check right• {• xTZSearchHelp( pcPatternKey, rcStruct, iPosXR, iPosYB, 0, iDist );• }

Page 50: COMPLEXITY REDUCTION OF MOTION ESTIMATION IN HEVC Jayesh Dubhashi Department of Electrical Engineering University of Texas at Arlington Advisor: Dr. K.

Code for proposed algorithm• } // check bottom• } // for ...• } // check border• } // iDist <= 8• } // iDist == 1• }• Void TEncSearch::xCheckBestMVP ( TComDataCU* pcCU, RefPicList eRefPicList, TComMv cMv, TComMv& rcMvPred, Int& riMVPIdx,

UInt& ruiBits, UInt& ruiCost )• {•

•  • AMVPInfo* pcAMVPInfo = pcCU->getCUMvField(eRefPicList)->getAMVPInfo();• • assert(pcAMVPInfo->m_acMvCand[riMVPIdx] == rcMvPred);• • if (pcAMVPInfo->iN < 2) return;• • m_pcRdCost->getMotionCost( 1, 0 );• m_pcRdCost->setCostScale ( 0 );• • Int iBestMVPIdx = riMVPIdx;• • m_pcRdCost->setPredictor( rcMvPred );• Int iOrgMvBits = m_pcRdCost->getBits(cMv.getHor(), cMv.getVer());• iOrgMvBits += m_auiMVPIdxCost[riMVPIdx][AMVP_MAX_NUM_CANDS];• //x+=m_auiMVPIdxCost[riMVPIdx][AMVP_MAX_NUM_CANDS];• Int iBestMvBits = iOrgMvBits;

Page 51: COMPLEXITY REDUCTION OF MOTION ESTIMATION IN HEVC Jayesh Dubhashi Department of Electrical Engineering University of Texas at Arlington Advisor: Dr. K.

Code for proposed algorithm• for (Int iMVPIdx = 0; iMVPIdx < pcAMVPInfo->iN; iMVPIdx++)• {• if (iMVPIdx == riMVPIdx) continue;• • m_pcRdCost->setPredictor( pcAMVPInfo->m_acMvCand[iMVPIdx] );• //x=AMVPInfo->m_acMvCand[iMVPIdx];• Int iMvBits = m_pcRdCost->getBits(cMv.getHor(), cMv.getVer());• //x=m_pcRdCost->getCost(1);• //x=x<<4;• //min_cost=m_pcRdCost->getCost(1);• //min_cost=(min_cost/2);• //Int iMvBits = m_pcRdCost->getBits(cMv.getHor(), cMv.getVer())/AMVP_MAX_NUM_CANDS;• //x=m_pcRdCost->getCost(cMv.getHor(), cMv.getVer());• //iMvBits += m_auiMVPIdxCost[iMVPIdx][AMVP_MAX_NUM_CANDS]/AMVP_MAX_NUM_CANDS;• iMvBits += m_auiMVPIdxCost[iMVPIdx][AMVP_MAX_NUM_CANDS];• //x=m_auiMVPIdxCost[iMVPIdx][AMVP_MAX_NUM_CANDS];• float b_stop;• b_stop= ((width*height))/x*x;• b_stop= b_stop-alpha;• min_cost=b_stop*x;• if (iMvBits < iBestMvBits)• {• iBestMvBits = iMvBits;• iBestMVPIdx = iMVPIdx;• }• }

Page 52: COMPLEXITY REDUCTION OF MOTION ESTIMATION IN HEVC Jayesh Dubhashi Department of Electrical Engineering University of Texas at Arlington Advisor: Dr. K.

Thank you