PERFORMANCE COMPARISON OF HEVC AND H.264 DECODER FINAL PRESENTATION SPRING 2014 ADVISOR: Dr. K.R.Rao VASAVEE VIJAYARAGHAVAN 1001037366 [email protected].

PERFORMANCE COMPARISON OF HEVC AND H.264 DECODER

FINAL PRESENTATIONSPRING 2014

ADVISOR: Dr. K.R.Rao VASAVEE VIJAYARAGHAVAN 1001037366

[email protected]

TABLE OF CONTENTS Acronyms And Abbreviations Goal of the Project Overview Of HEVC Overview of H.264 Profiles Used For Comparison Video Resolutions Used For Comparison Test Sequences Configuration of HM 14.0 and JM 18.6 Results References

ACRONYMS AND ABBREVIATIONS

AMVP: Advanced motion vector prediction AVC: Advanced Video CodingBD-PSNR: Bjontegaard metric calculationCB: Coding Block CIF: Common Intermediate FormatCU: Coding UnitCTB: Coding Tree BlockCTU: Coding Tree Unit DCT: Discrete Cosine Transforms

ACRONYMS AND ABBREVIATIONS(contd..)

DST: Discrete Sine Transform HEVC: High Efficiency Video Coding JCT-VC: Joint Collaborative Team on Video Coding MC: Motion Compensation ME: Motion Estimation MPEG: Moving Picture Experts Group MV: Motion Vector QP: Quantization Parameter QCIF: Quarter Common Intermediate Format PSNR: Peak Signal To Noise Ratio PU: Prediction Unit

ACRONYMS AND ABBREVIATIONS(contd..)

RD: Rate DistortionSAO: Sample Adaptive OffsetSAD: Sum of Absolute Differences SATD: Sum of Absolute Transformed Differences (SATD) SHVC: Scalable HEVCSSIM: Structural SimilaritySVC: Scalable Video CodingTU: Transform Unit URQ: Uniform Reconstruction QuantizationVCEG: Video Coding Experts Group

OBJECTIVE

The goal of this project is to compare the performance of the HEVC and H.264 decoders in various profiles, using different sequences and in different resolutions.

ABOUT HIGH EFFICIENCY VIDEO CODING (HEVC) [43]

HEVC is the latest video coding standard jointly presented by ITU-T Video Coding Experts Group and the ISO/IEC Moving Picture Experts Group. The main advantage of HEVC is that it reduces the bit rate by 50% compared to the existing video coding standards at the same time giving equal perceptual quality.

HEVC ENCODER BLOCK DIAGRAM (with decoder elements shaded in grey) [43]

MACRO BLOCKS IN HEVC [57]

ABOUT H.264

H.264 ENCODER AND DECODER BLOCK DIAGRAM [23,24]

How does an H.264 codec work ?

An H.264 video encoder carries out prediction, transform and encoding processes to produce a compressed H.264 bitstream. An H.264 video decoder carries out the complementary processes of decoding, inverse transform and reconstruction to produce a decoded video sequence.

INTRA PREDICTION IN H.264 [23]

INTER PREDICTION IN H.264 [23]

PROFILES USED FOR COMPARISON

The profiles that will be used for comparison in HEVC will be main and main 10.

The profiles that will be used for H.264 will be high and high 10.

VIDEO RESOLUTIONS USED FOR COMPARISON

FORMAT RESOLUTION

CIF 352*288

QCIF 176*144

TEST SEQUENCESBRIDGE-CLOSE_CIF(352*288)

TEST SEQUENCES(contd.)BRIDGE-CLOSE_QCIF(176*144)

HIGHWAY_CIF(352*288)

TEST SEQUENCES(contd.)

TEST SEQUENCES(contd..)HIGHWAY_QCIF(176*144)

TEST SEQUENCES(contd.)CONTAINER-QCIF.YUV

TEST SEQUENCES(contd.)COAST-GUARD-QCIF.YUV

CONFIGURATION OF HM 14.0 [61]FILE TO BE MODIFIED- encoder_intra_main.cfgPARAMETERS MODIFIED:F-number of frames to be encoded(10)Fr-frame rate(10)Wdt-width of the video sequence(352 or 176)Hgt-height of the video sequence(288 or 144)Profile-main/main10SAMPLE COMMAND LINE PARAMETERS FOR HM 14.0ENCODING:C:\Users\Vasavee\Desktop\HM_14.0\bin\vc10\Win32\Release>TAppEncoder.exe –c encoder_intra_main.cfg –i

highway_qcif .yuv –f 10 –fr 10 –wdt 176 –hgt 144 –o C:\Users\Vasavee\Desktop\HM_14.0\bin\vc10\Win32\Release\Output_Encoded\highway_qcif_main34.bin>>C:\Users\Vasavee\Desktop\HM_14.0\bin\vc10\Win32\Release\LogFiles_Encoded\highway_qcif_main.txt

DECODINGC:\Users\Vasavee\Desktop\HM_14.0\bin\vc10\Win32\Release>TAppDecoder.exe -b C:\Users\Vasavee\Desktop\

HM_14.0\bin\vc10\Win32\Release\Encoded_bin_files\highway_qcif_main34.bin -o C:\Users\Vasavee\Desktop\HM_14.0\bin\vc10\Win32\Release\Output_D

ecoded\highway_qcif_main34.yuv>>C:\Users\Vasavee\Desktop\HM_14.0\bin\vc10\Win32\Release\LogFiles_Decoded\highway_qcif_main.txt

CONFIGURATION OF JM 18.6 [62]

FILE TO BE MODIFIED- encoder.cfgPARAMETERS MODIFIED:FramesToBeEncoded-number of frames to be encoded(10)FrameRate-frame rate(10)SourceWidth-width of the video sequence(352 or 176)SourceHeight-height of the video sequence(288 or 144)ProfileIDC-high(100)/high10(110)SAMPLE COMMAND LINE PARAMETERS FOR JM 18.6ENCODING:C:\Users\Vasavee\Downloads\jm18.6\JM\bin>lencod.exe -f encoder.cfg -p

InputFile="highway_cif.yuv" -p SourceHeight=288 -p SourceWidth=352 -p OutputFile=”encoded_highway.bin” –p ProfileIDC=100>>Log.txt

DECODINGC:\Users\Vasavee\Downloads\jm18.6\JM\bin>ldecod.exe -i encoded_highway.bin -o

output_hughway.yuv>>log.txt

RESULTS

COMPLEXITY PLOTS MEASURED IN UNITS OF TIME

ENCODING TIME COMPARISON BETWEEN HEVC AND H.264 FOR THE SEQUENCE BRIDGE-

CLOSE_CIF.YUV

ENCODING TIME COMPARISON BETWEEN HEVC AND H.264 FOR THE SEQUENCE COAST-

GUARD_QCIF.YUV

ENCODING TIME COMPARISON BETWEEN HEVC AND H.264 FOR THE SEQUENCE

CONTAINER_QCIF.YUV

ENCODING TIME COMPARISON BETWEEN HEVC AND H.264 FOR THE SEQUENCE

HIGHWAY_CIF.YUV

PSNR PLOTS COMPARISON BETWEEN HEVC AND H.264

BRIDGE-CLOSE_CIF

COAST-GUARD_QCIF

CONTAINER_QCIF

HIGHWAY_CIF

BIT RATE PLOTS COMPARISON BETWEEN HEVC AND H.264

BRIDGE-CLOSE_CIF

COAST-GUARD_QCIF

CONTAINER_QCIF

HIGHWAY_CIF

SOME ADDITIONAL INFORMATION

• HEVC decoders, that consume low power, have been implemented on hand held devices by few companies. [58], [65], [66]

• When the same HEVC decoder implemented in HM, is made of low power decoder by modifying few components, then it can be used as low power decoder for handheld and mobile devices. [59]

REFERENCES[1] Joint Collaborative Team On Video Coding Information website-

http://www.itu.int/en/ITU-T/studygroups/2013-2016/16/Pages/video/jctvc.aspx[2] H.261: Video Codec for Audiovisual Services at px64 kbit/s,”

http://www.itu.int/rec/T-REC-H.261-199303-I/en” [3] H.263: Video Coding for Low Bit Rate Communication, “http://www.itu.int/rec/T-

REC-H.263/en”[4] Coding of Moving Pictures and Associated Audio for Digital Storage Media at up to

About 1.5 Mbit/s- http://www.digitalpreservation.gov/formats/fdd/fdd000035.shtml

[5] Coding of Audio-Visual Objects—Part 2: http://www.digitalpreservation.gov/formats/fdd/fdd000155.shtml

[6] Generic Coding of Moving Pictures and Associated Audio Information— Part 2: Video- http://www.itu.int/rec/T-REC-H.222.0, ITU-T Rec. H.262 and ISO/IEC 13818-2 (MPEG 2 Video), ITU-T and ISO/IEC JTC 1, Nov. 1994.

REFERENCES

[7] J. Ostermann et al, ” Video coding with H.264/AVC: tools, performance, and complexity”, IEEE Transactions on CSVT, Vol. 14, no.1, pp. 7-28, Aug. 2004

[8] H. Samet, “The quadtree and related hierarchical data structures,” Comput. Survey, vol. 16, no. 2, pp. 187–260, Jun. 1984.

[9] G. J. Sullivan et al, “Overview of the High Efficiency Video Coding (HEVC) Standard”, IEEE Trans. Circuits and Systems for Video Technology, Vol. 22, No. 12, pp. 1649-1668, Dec. 2012.

[10] K. Shah, “Time Optimization of HEVC Encoder over X86 Processors using SIMD”, Thesis proposal, http://www-ee.uta.edu/Dip/Courses/EE5359/Projects/Spring2013/Kushal_Shah/Kushal_Shah_Proposal.pdf

[11] F. Pescador et al, “On an Implementation of HEVC Video Decoders with DSP Technology”, IEEE ICCE, pp. 121-122, Las Vegas, NV, Jan 2013.

[12] CODE: Sequoia Website on HEVC http://codesequoia.wordpress.com/2012/10/28/hevc-ctu-cu-ctb-cb-pb-and-tb/

REFERENCES

[13] O. L. Meur, “ Video Compression Beyond H.264”, http://people.irisa.fr/Olivier.Le_Meur/teaching/HEVC_CAV_ESIR3_2011_2012.pdf , November 7 2011.

[14] Multi-Frame Motion Compensated Prediction from Fraunhofer website- http://www.hhi.fraunhofer.de/fields-of-competence/image-processing/research-groups/image-video-coding/h264mpeg4-avc/multi-frame-motion-compensated-prediction.html

[15] C. Fogg, “Suggested figures for the HEVC specification”, ITU-T / ISO-IEC Document: JCTVC J0292r1, July 2012.

[16] H.Zhang et al, “Fast Intra-Prediction Mode Selection for H.264”: http://mathcs.slu.edu/~fritts/pres/spie05_IVC_h264_fast_mode_sel_pres.pdf

REFERENCES

[17] G. Bjontegaard, “Calculation of Average PSNR Differences between RD Curves”, document VCEG-M33, ITU-T SG 16/Q 6, Austin, TX, Apr. 2001.

[18] F. Bossen, “Common test conditions and software reference configurations”, Technical report, JCT-VC, 2012.

[19] Multimedia Processing course website. http://www-ee.uta.edu/Dip/Courses/EE5359/

[20] Texas Instruments Product file. http://downloadt.advantech.com/ProductFile/PIS/DSPC-8681/Product%20-%20Datasheet/DSPC-8681_DS20121018134716.pdf

[21] Video sequences- http://basakoztas.net[22] Multimedia Communications with SVC, HEVC, and SHVC-

http://r2d2n3po.tistory.com/50

REFERENCES [23] H.264 White Paper by Iain Richardson:

http://www.vcodex.com/images/uploaded/469323879727520.pdf[24] T. Wiegand et al, “Overview of the H.264/AVC Video Coding Standard”, IEEE

Transactions on Circuits and Systems for Video Technology, Vol. 13, No. 7, pp. 560-576, Jul. 2003.

[25] H.264 tutorial by I.E.G. Richardson: https://www.vcodex.com/h264.html [26] J. R. Ohm et al, “Comparison of the Coding Efficiency of Video Coding Standards—

Including High Efficiency Video Coding (HEVC)”, IEEE Transactions on Circuits and Systems for Video Technology, Vol. 22, No. 12, pp. 1669-1684, Dec. 2012.

[27] A. Puri et al, “Video coding using the H.264/MPEG-4 AVC compression standard”, Signal Processing: Image Communication, vol. 19, pp. 793-849, Oct. 2004.

[28] Eurasip Journal- http://jes.eurasipjournals.com/content/2013/1/15[29] Vcodex- http://www.vcodex.com/h265.html[30] HEVC Fraunhofer site containing all the information on HEVC- http://hevc.info/

REFERENCES[31] Access the website http://www-ee.uta.edu/Dip/Courses/EE5359/ and refer to the project by S.

Kulkarni on “Transcoding from H.264/AVC to High Efficiency Video Coding (HEVC)”, University of Texas, Arlington, Spring 2013.[32] Access to HM 14.0 Reference Software: http://hevc.hhi.fraunhofer.de/[33] HEVC download link-

https://hevc.hhi.fraunhofer.de/svn/svn_HEVCSoftware/branches/HM-14.0-dev/doc/software-manual.pdf

[34] Visual studio download for students for free- www.dreamspark.com[35] Tortoise SVN download- http://tortoisesvn.net/downloads.html [36]MPL Website- http://www-ee.uta.edu/Dip/Courses/EE5359/index.html[37] Video Sequence Download Link- http://media.xiph.org/video/derf/[38] K .R .Rao and J .J .Hwang, “Techniques and standards for Image Video and Audio Coding”

Prentice Hall, 1996.[39] R.C. Gonzalez and R.E.Woods, “Digital Image Processing” ,Pearson, Edition 3,2009[40] Test Sequences: ftp://ftp.kw.bbc.co.uk/hevc/hm-11.0-anchors/bitstreams/ [41] T.Wiegand et al “Overview of the H.264/AVC Video Coding Standard,” IEEE Trans. CSVT, Vol. 13,

pp. 560-576, July 2003.

REFERENCES

[42] M.A. Isnardi, ” Historical Overview of video compression in consumer electronic devices”, IEEE ICCE, pp. 1-2, Las Vegas, NV, January2007.

[43] HEVC tutorial by I.E.G. Richardson: http://www.vcodex.com/h265.html [44] K. Iguchi et al, “HEVC Encoder for Super Hi-Vision”, 2014 IEEE

International conference on Consumer Electronics (ICCE), pp. 61-62, 2014

[45] F. Pescador et al, “A DSP HEVC decoder implememtation based on Open HEVC”, IEEE ICCE, pp. 65-66, 2014.

[46] G J. Sullivan et al,” Standardized Extensions of HEVC”, IEEE Journal of Selected topics in Signal Processing, Vol.7, No.6, pp.1001-1016, December 2013.

[47] K.R. Rao, D.N. Kim and J.J. Hwang, “Video Coding Standards: AVS China, H.264/MPEG-4 Part 10, HEVC, VP6, DIRAC and VC-1”, Springer, 2014.

REFERENCES

[48] F. Pescador et al, “ Complexity analysis of an HEVC Decoder based on a Digital Signal Processor”, IEEE Transactions on Consumer Electronics, Vol.59, No.2, pp. 391-399, May 2013.

[49] JCT-VC Video Subgroup, “HM9: High Efficiency Video Coding (HEVC) Test Model 9 Encoder Description”, Shanghai, China, October 2012.

[50] F. Bossen et al, “HEVC Complexity and Implementation Analysis”, IEEE Transactions on CSVT, Vol.22, no. 12, pp. 1685 - 1696 , Dec. 2012.

[51] J.-R. Ohm et al , “Comparison of the Coding Efficiency of Video Coding Standards—Including High Efficiency Video Coding (HEVC)”, IEEE Transactions on Circuits and Systems for Video Technology, vol. 22, no. 12, pp. 1669-1684, December 2012.

REFERENCES

[52] N. Ahmed, T. Natarajan, and R.K.Rao, “Discrete Cosine Transform”, IEEE Transactions on Computers, Vol.C-23, pp.90-93, January 1974.

[53] S. Kwon, A. Tamhankar and K.R. Rao, ”Overview of H.264 / MPEG-4 Part 10”, J. Visual Communication and Image Representation, vol. 17, pp.186-216, April 2006.

[54] I. E. Richardson, “The H.264 Advanced Video Compression Standard”, 2nd Edition, Wiley 2010.

[55] D. Marpe, T. Wiegand and G. J. Sullivan, “The H.264/MPEG-4 AVC standard and its applications”, IEEE Communications Magazine, vol. 44, pp. 134-143, Aug. 2006.

REFERENCES

[56] H.264/MPEG-4 AVC Reference Software Manual - http://vc.cs.nthu.edu.tw/home/courses/CS553300/97/project/JM%20Reference%20Software%20Manual%20(JVT-X072).pdf

[57] HM Software Manual- https://hevc.hhi.fraunhofer.de/svn/svn_HEVCSoftware/branches/HM-14.0-dev/doc/software-manual.pdf

[58] HEVC decoder for handheld devices implemented by Ace Thought- http://www.acethought.com/index.php/products/hevc-decoder/

[59] Z. Ma and A. Segall, “Low Resolution Decoding For High-Efficiency Video Coding”, IASTED SIP 2011, Dallas, TX, Dec. 2011.

[60] Link to access JCTVC Documents - http://www.itu.int/en/ITU-T/studygroups/2013-2016/16/Pages/video/jctvc.aspx

REFERENCES[60]I.E.G. Richardson, “Video Codec Design: Developing Image and Video Compression Systems”, Wiley,

2002.[61]HM 14.0 (HEVC Software) Download Link-

https://hevc.hhi.fraunhofer.de/svn/svn_HEVCSoftware/branches/HM-14.0-dev/[62] JM 18.6 (H.264 Software) Download Link- http://iphome.hhi.de/suehring/tml/download/[63] Special Issue on emerging research and standards in next generation video coding, IEEE

Transactions On Circuits and Systems for Video Technology, vol. 22, no. 12, December 2012.[64] Special Issue on emerging research and standards in next generation video coding , IEEE

Transactions On Circuits and Systems for Video Technology, vol. 23, no. 12, December 2013.[65] Imagination company website- http://www.imgtec.com/news/release/index.asp?NewsID=780[66] NGCODEC website-

http://ngcodec.com/news/2014/2/16/vixs-partners-with-ngcodec-to-license-worlds-first-silicon-proven-10-bit-hevc-decoder-for-ultra-hd-4k-60p

[67] Video Coding: HEVC and Beyond- IEEE Journal of Selected Topics In Signal Processing,vol.7, pp. 93-1151, Dec. 2013

[68] H. Zhang and Z.Ma, “Fast Intra Mode Decision for HEVC”, IEEE Trans. On CSVT, vol.24, pp.660-668, April 2014.

REFERENCES

• [67] Video Coding: HEVC and Beyond- IEEE Journal of Selected Topics In Signal Processing,vol.7, pp. 93-1151, Dec. 2013