Srikanth Vasireddy 1001101538 [email protected] Multimedia Processing Lab,UTA1.

Multimedia Processing Lab,UTA 1

Fast Block Based Motion Estimation Algorithms in HEVC

Srikanth Vasireddy1001101538

[email protected]

mailto:[email protected]


Growing demand for Video HEVC Encoder and Decoder Features of Moving pictures Block based Motion Estimation Different Motion Estimation Algorithms Test Sequences Simulation Results Work done and In Progress Acronyms References

Contents


Growing Demand for Video• Increase in Applications• Higher efficiency

• Broadcast Quality• Higher throughput

• Increase in Mobile data traffic• Low power


HEVC Encoder [5]

ME has 84% coding complexity and time to encode [1] [5]

Fig .1: HEVC Encoder[5]


HEVC Decoder [1]

Fig .2: HEVC Decoder[1]


Moving images contain significant temporal redundancy• successive frames are very similar

Features of Moving Pictures


Video coding algorithms usually contain two coding schemes :• Intraframe coding : Intraframe coding does not exploit the correlation

among adjacent frames; Intraframe coding therefore is similar to the still image coding.

• Interframe coding :The interframe coding should include motion estimation/compensation process to remove temporal redundancy.

Intra frame and Inter frame coding

“The amount of data can be reduced significantly if the previous frame is subtracted from the current frame.”[4]

Fig.3: Motion Estimation and Motion Compensation [4]


Temporal Redundancy Removal [1]

M.J.Jakubowski and G.Pastuszak, “Block-based motion estimation algorithms – a survey ,” Opto-Electronic Review , Volume 21, pp 86-102,,March2013.


Block based Motion Estimation [6][11] The MPEG and H.26X standards[4] use block based technique

for motion estimation /compensation. In this technique, each current frame is divided into equal-size

blocks, called source blocks. Each source block is associated with a search region in the

reference frame. The objective of block-matching is to find a candidate block in

the search region best matched to the source block. The relative distances between a source block and its

candidate blocks are called motion vectors.


Video Sequence

X: Source block for block-matching

Bx: Search area associated with X

MV: Motion vector

current frame Reference frame

Fig.4: Block matching Scenario [6]

time


Motion Vector and Search Area

Search Area

Source block

Candidate block

Search Area: pnpn 22

Motion vector: (u, v)


Motion Estimation Algorithms [11]

• Full Search Algorithm• Three Step Search Algorithm• Four Step Search Algorithm• Diamond Search Algorithm• Hexagonal Search Algorithm

If p=7, then there are(2p+1)(2p+1)=225 candidate blocks.

u

vSearch Area

CandidateBlock

Full Search Algorithm

Fig.5 : Full Search Scenario [6][11]


• In order to get the best match block in the reference frame, it is necessary to compare the current block with all the candidate blocks of the reference frames.

• Full search motion estimation calculates the sum of absolute difference (SAD) value at each possible location in the search window.

• Full search computes the all candidate blocks intensively for the large search window.

• Computational complexity is of order n^2 for a block size of nxn


• The first step involves block-matching based on 4-pel resolution at the nine location.(step size m).Now they check for minimum cost distance and shift center to the new point of minimum.

• The second step involves block-matching based on 2-pel resolution around the location determined by the first step.(step size is m/2)

• The third step repeats the process in the second step (but with resolution 1-pel).

3SS Algorithm

Fig.6: 3 Step Search Scenario [6] [11]

The position with minimum cost will give us the motion vector and also position of the best match.


• 4SS algorithm utilizes a center-biased search pattern with nine checking points on a 5 x 5 window in the instead of a 9 x 9 window in the 3SS.

• This algorithm helps in reducing the number of search points compared to the 3SS and hence is more robust.

• Block distortion method (BDM) point is used

4SS Algorithm

Fig.7: 4Step Search Scenario [6] [11]


Diamond Search Algorithm

The DS algorithm employs two search patterns. Large diamond search pattern(LDSP) comprises 9 checking points from which eight points surround the center one to compose a diamond shape. Small diamond search pattern (SDSP) consisting of 5 checking points forms a small diamond shape.

LDSP is repeatedly used until the minimum block distortion (MBD) occurs at the center point.

Fig.8 : Diamond Search Scenario for ME [7] [11]


Hexagonal Search Algorithm

Fig.9:Hexagonal Search Scenario for ME [7][11]

In block motion estimation, a search pattern with a different shape or size has a very important impact on search speed and distortion performance.

• HEXBS algorithm can find a same motion vector with fewer search points than the DS algorithm. (Calculate the minimum cost at 6 corner points of Hexagon)

• Generally speaking, the larger the motion vector, the more search points the HEXBS algorithm can save.


RaceHorses_416x240_30.yuv sequence BQMall_832x480_60.yuv sequence

Test Sequences[23]

KirstenAndSara_1280x720_60.yuv sequence ParkScene_1920x1080_24.yuv sequence


RaceHorses_416x240_30.yuv, Number of frames encoded = 20

Random Access profile (FAST SEARCH) Random Access profile (FULL SEARCH)

QP PSNR in dB Bit rate in kbps

Encoding time in seconds

PSNR in dB

Bit rate in kbps


22 39.5858 1504.992 120.810 39.5969 1494.096 1399.795

27 35.9841 769.740 101.093 35.9926 762.0240 1508.792

32 32.7600 391.116 88.600 32.7769 389.296 1306.499

37 30.1072 202.572 71.437 30.1423 202.3080 1231.534

Table 1: Results for RaceHorses_416x240_30.yuv sequence in Random Access Configuration

BQMall_832x480_60.yuv , Number of frames encoded = 20

Random Access profile (FAST SEARCH) Random Access profile (FULL SEARCH)

QP PSNR in dB Bit rate kbps


PSNR in dB Bit rate in kbps


22 40.6653 4538.160 308.840 40.7583 4528.440 4875.343

27 38.2347 2257.968 259.764 38.2446 2252.592 5214.150

32 35.4849 1200.864 244.682 35.4965 1196.352 4837.578

37 32.7343 656.088 224.326 32.7370 655.2240 5248.942

Table 2: Results for BQMall_832x480_60.yuv sequence in Random Access Configuration

Simulation Results


KristenAndSara_1280x720_60.yuv Number of fames encoded =20

ParkScene_1920x1080_24.yuv Number of fames encoded =20

Random Access profile (FAST SEARCH) Random Access profile (FAST SEARCH)

QP PSNR in dB Bit rate in kbps


PSNR in dB Bit rate in kbps


22 44.3166 2676.456 533.038 40.7117 8421.072 1808.863

27 42.5646 1232.856 570.403 38.3703 3673.3632 1356.223

32 40.4650 667.344 481.128 36.0006 1697.606 1211.177

37 38.0715 381.720 444.487 33.7621 791.712 1107.963

Table 3: Results for KirstenAndSara_1280x720_60.yuv & ParkScene_1920x1080_24.yuv sequences in Random Access Configuration

Processor Intel Core(TM) i5 CPU 4210U 2.40 GHz

Number of cores 2

Memory 8GB

Operating System 64 bit (x64-based processor),Windows 8.1

Testing Platform:


Figure 10: Snapshot of encoder_randomaccess_main.cfg[15]


20 24 28 32 360

1000

2000

3000

4000

5000

6000

7000

8000

9000

Bit rate vs QP

RaceHorses_416x240

BQMall_832x480_60

KirstenAndSara_1280x720_60

ParkScene_1920x1080_24

QP

Bit

rate

(K

bps)

Fig 11: Bit rate vs QP for all test sequences

20 22 24 26 28 30 32 34 36 380

5

10

15

20

25

30

35

40

45

50

PSNR vs QP

RaceHorses_416x240

BQMall_832x480_60

KirstenAndSara_1280x720_60

ParkScene_1920x1080_24

QP

PSN

R (

dB

)

Fig 12: PSNR vs QP for all test sequences


0 200 400 600 800 1000 1200 1400 16000

5

10

15

20

25

30

35

40

45

R-D plot for RaceHorses_416x240_30.yuv

Fast Search PSNR in dB Full Search PSNR in dB

Bit rate (kbps)

PSN

R (

dB

)

Fig 13: R-D plot for RaceHorses_416x240_30.yuv sequence

0 500 1000150020002500300035004000450050000

5

10

15

20

25

30

35

40

45

R-D plot for BQMall_832x480_60.yuv

Fast Search PSNR in dB Full Search PSNR in dB

Bit rate (Kbps)

PSN

R (

dB

)Fig 14: R-D plot for BQMall_832x480_60.yuv sequence


0 500 1000 1500 2000 2500 300034

36

38

40

42

44

46

R-D plot for KirstenAndSara_1280x720_60.yuv

Bit rate (Kbps)

PSN

R (

dB

)

0 1000 2000 3000 4000 5000 6000 7000 8000 90000

5

10

15

20

25

30

35

40

45

R-D plot for ParkScene_1920x1080_24.yuv

Bit rate (Kbps)

PSN

R (

dB

)

Fig 15: R-D plot for KirstenAndSara_1280x720_60.yuv sequence Fig 16: R-D plot for ParkScene_1920x1080_24.yuv sequence


22 27 32 370

1000

2000

3000

4000

5000

6000

Fast search vs Full search for RaceHorses_416x240_30.yuv

Fast Search Full Search

QP

Enco

din

g t

ime(s

ec)

Fig 17: Encoding time comparison for RaceHorses_416x240_30.yuv sequence


22 27 32 370

1000

2000

3000

4000

5000

6000

Fast search vs Full search for BQMall_832x480_60.yuv

Fast Search Full Search

QP

Enco

din

g t

ime(s

ec)

Fig 18: Encoding time comparison for RaceHorses_416x240_30.yuv sequence


22 27 32 370

200

400

600

800

1000

1200

1400

1600

1800

2000

Fast Search Encoding time

RaceHorses_416x240_30.yuv BQMall_832x480_60.yuv

KirstenAndSara_1280x720_60.yuv ParkScene_1920x1080_24.yuv

QP

Enco

din

g t

ime(s

ec)

Fig 19: Fast Search Encoding time comparison for all test sequences


22 27 32 370

1000

2000

3000

4000

5000

6000

Full Search Encoding time

RaceHorses_416x240 BQMall_832x480_60

QP

Enco

din

g t

ime (

sec)

Fig 20: Full Search Encoding time comparison for RaceHorses & BQMall


Analyzed Full Search,3SS and Diamond Search.

Developed functions in MATLAB like To find PSNR w.r.t reference image To compute motion compensated image To find minimum distance among macro blocks To compute Mean Absolute difference

Developing functions for Full Search ,3SS and Diamond Search

Work done and In Progress


AcronymsBBME : Block Based Motion EstimationBD-BR: Bjontegaard Delta Bitrate.BD-PSNR: Bjontegaard Delta Peak Signal to Noise Ratio. CABAC: Context Adaptive Binary Arithmetic Coding.CTB: Coding Tree Block.CTU: Coding Tree Unit. CU: Coding Unit.DBF: De-blocking Filter.DCT: Discrete Cosine Transform. fps: frames per second. HEVC: High Efficiency Video Coding. HM: HEVC Test Model.ISO: International Organization for Standardization.ITU-T: International Telecommunication Union- Telecommunication Standardization Sector. JCT-VC: Joint Collaborative Team on Video Coding. MAD: Mean Absolute DifferenceMC: Motion Compensation.ME: Motion Estimation.MPEG: Moving Picture Experts Group. MSE: Mean Square Error.PB: Prediction Block.PSNR: Peak Signal to Noise Ratio.QP: Quantization Parameter SAO: Sample Adaptive Offset.TB: Transform Block.TU: Transform Unit.VCEG: Visual Coding Experts Group.


[1] V. Sze and M. Budagavi, “Design and Implementation of Next Generation Video Coding Systems (H.265/HEVC Tutorial)”, IEEE International Symposium on Circuits and Systems (ISCAS), Melbourne, Australia, June 2014, available on http://www.rle.mit.edu/eems/publications/tutorials/

[2] HEVC tutorials http://www.vcodex.com/h265.html[3] G.J. Sullivan; J. Ohm; W.J, Han and T. Wiegand, “Overview of the High Efficiency Video Coding (HEVC) Standard”, IEEE Trans. on

Circuits and Systems for Video Technology, Volume: 22, Issue: 12, pp. 1649-1668, Dec. 2012.[4] I.E. Richardson “Video Codec Design : Developing Image and Video compression systems”,Wiley,2002.[5] G. J. Sullivan et al “Standardized Extensions of High Efficiency Video Coding (HEVC).”IEEE Journal of selected topics in Signal

Processing” vol. 7, pp.1001-1016, Dec. 2013[6] L.C.Manikandan et al “A new survey on Block Matching Algorithms in Video Coding” in International Journal of Engineering

Research ,Volume 3,pp.121-125,Feb 2014.[7] ] L.N.A. Alves, and A. Navarro, " Fast Motion Estimation Algorithm for HEVC ", Proc IEEE International Conf. on Consumer

Electronics -ICCE Berlin , Germany , vol.11 , pp. 11 - 14 , Sep. , 2012[8] F. Bossen, et al, “HEVC complexity and implementation analysis”, IEEE Trans. on Circuits and Systems for Video Technology,

Volume: 22, Issue: 12, pp. 1685 - 1696, Dec. 2012.[9] J. Ohm, et al, “Comparison of the Coding Efficiency of Video Coding Standards –Including High Efficiency Video Coding (HEVC)”,

IEEE Trans. on Circuits and Systems for Video Technology, volume: 22, Issue: 12, pp. 1669 - 1684, Dec. 2012.[10] K. Kim, et al, “Block partitioning structure in the HEVC standard,” IEEE Trans. on circuits and systems for video technology, vol.

22, pp.1697-1706, Dec. 2012.[11] M. Jakubowski and G. Pastuszak, “Block-based motion estimation algorithms-a survey,” Journal of Opto-Electronics Review, vol.

21, pp 86-102, Mar. 2013. http://link.springer.com/article/10.2478%2Fs11772-013-0071-0#page-1[12] A. Abdelazim, W. Masri and B. Noaman "Motion estimation optimization tools for the emerging high efficiency video coding

(HEVC)", SPIE vol. 9029, Visual Information Processing and Communication V, 902905, Feb. 17, 2014, doi:10.1117/12.2041166

REFERENCES

http://www.rle.mit.edu/eems/publications/tutorials/

http://www.vcodex.com/h265.html

http://www.vcodex.com/h265.html

http://ieeexplore.ieee.org/xpl/articleDetails.jsp?tp=&arnumber=6316136&contentType=Journals+&+Magazines&searchField=Search_All&queryText=overview+of+HEVC




http://link.springer.com/article/10.2478/s11772-013-0071-0#page-1


[13] Software repository for HEVC - https://hevc.hhi.fraunhofer.de/svn/svn_HEVCSoftware/tags/HM-16.0/[14] HEVC white paper –Ittiam systems - http://www.Ittiam.com/Downloads/en/documentation.aspx[15] HM Software Manual - https://hevc.hhi.fraunhofer.de/svn/svn_HEVCSoftware/[16] G. Bjontegaard, "Calculation of average PSNR difference between RD curves", VCEG-M33,ITU-T SG 16/Q 6,Austin, TX,

April 2001.[17] Multimedia Processing Lab at UTA: http://www.uta.edu/faculty/krrao/dip/ Analysis of Motion Estimation (ME) Algorithms. By Tuan Phan Minh Ho (Spring 2014) Comparative study of Motion Estimation (ME) Algorithms by Khyati Mistry (Spring 2008)[18] http://www.h265.net has info on developments in HEVC NGVC – Next generation video coding.[19] Detailed Overview of HEVC/H.265 by Shevach Riabtsev : https://app.box.com/s/rxxxzr1a1lnh7709yvih[20] W. Hong, “Coherent Block-Based Motion Estimation for Motion-Compensated Frame Rate Up-Conversion", IEEE

International Conference on Consumer Electronics, pp. 165-166, Jan.2010. [21] N.Purnachand and L.N. Alves, A. Navarro “Improvements to TZ search motion estimation algorithm for multiview video

coding” 19th International Conference on Systems, Signals and Image Processing IWSSIP, pp. 388 -391, 2012.[22] Video test sequences - http://forum.doom9.org/archive/index.php/t-135034.html or

http://media.xiph.org/video/derf/ or ftp://ftp.kw.bbc.co.uk/hevc/hm-11.0-anchors/bitstreams/ or http://forum.doom9.org/archive/index.php/t-135034.html

[23] M. Wien, “High efficiency video coding: Tools and specification”, Springer, 2015.[24] I.E. Richardson, “Coding video: A practical guide to HEVC and beyond”, Wiley, 11 May 2015[25] V.Sze ,M.Budagavi and G.J.Sullivan “ High Efficiency Video Coding(HEVC) –Algorithms and Architectures”, Springer, 2014.[26] X. Li et al, “Rate-complexity-distortion evaluation for hybrid video coding”, IEEE International Conference on Multimedia and Expo (ICME), pp. 685-690, July. 2010.[27] G. Correa et al, “Performance and computational complexity assessment of high efficiency video encoders”, IEEE Trans. on Circuits and Systems for Video Technology, Vol.22, pp.1899-1909, Dec.2012.

https://hevc.hhi.fraunhofer.de/svn/svn_HEVCSoftware/tags/HM-16.0/

http://www.ittiam.com/Downloads/en/documentation.aspx

http://www.ittiam.com/Downloads/en/documentation.aspx

https://hevc.hhi.fraunhofer.de/svn/svn_HEVCSoftware/

http://www.uta.edu/faculty/krrao/dip/

http://www.uta.edu/faculty/krrao/dip/

http://www.h265.net/

https://app.box.com/s/rxxxzr1a1lnh7709yvih

https://app.box.com/s/rxxxzr1a1lnh7709yvih

http://forum.doom9.org/archive/index.php/t-135034.html

http://media.xiph.org/video/derf/

ftp://ftp.kw.bbc.co.uk/hevc/hm-11.0-anchors/bitstreams/

ftp://ftp.kw.bbc.co.uk/hevc/hm-11.0-anchors/bitstreams/

http://forum.doom9.org/archive/index.php/t-135034.html

Srikanth Vasireddy 1001101538 [email protected] Multimedia Processing Lab,UTA1.

Documents

search motion estimation

current block

search scenario

search region

objective of block

block size of nxn slide

time slide

block matching scenario