Comparative study of Intra Frame Coding efficiency in HEVC and VP9 By: Shwetha Chandrakant Kodpadi 1001051972 [email protected] Spring.

Comparative study of Intra Frame Coding efficiency in HEVC and VP9

By: Shwetha Chandrakant Kodpadi 1001051972

[email protected] 2014

EE5359 Multimedia Processing Interim Report

Under the guidance of Dr.K.R.Rao

University of Texas at ArlingtonDept. of Electrical Engineering

Contents• Objective• General Compression Dataflow• HEVC Coding Standard• Google VP9• Intra Prediction Modes in HEVC and VP9• Performance Metrics• Test sequences and Implementation• Implementation Results• Conclusions• Acronyms and abbreviations• References

Objective

This project aims at studying and comparing intra frame coding efficiency between video coding technologies HEVC and VP9.Overview of different coding tools in HEVC and VP9 such as Prediction, Transforms, Entropy Coding and Post-processingDetailed analysis and implementation of HEVC and VP9 Intra Frame coding [1][3] and compare the performances by using various performance metrics.PSNR, MSE, BD-PSNR and BD-BR [14] are used as comparison metrics.

General compression dataflow

Figure 1: Hybrid block-based codec dataflow [9]

Both HEVC and VP9 video compression standards are hybrid block-based codecs relying on spatial transformations [9]. General compression data flow of hybrid block-based encoders is illustrated in Figure 1.

General compression dataflow (Continued..)• The input video frame is initially partitioned into blocks of the same

size called macroblocks• A macroblock is subpartitioned into smaller blocks to perform

prediction (intra and inter predictions)• The aim of the prediction process is to reduce data redundancy and

therefore, not store excessive information in coded bitstream• Intra-prediction works within a current video frame and is based upon

the compressed and decoded data available for the block being predicted

• Predicted frame is subtracted from the original data to get residuals which are subjected to transforms such as forward discrete Fourier transform and discrete cosine transform [22]

• The transform coefficients are further quantized and subjected to Entropy Coding which makes it possible to get compressed bit-stream.

High Efficiency Video Coding

• High Efficiency Video Coding (HEVC) is the latest Video Coding format [4].

• It challenges the state-of-the-art H.264/AVC [21] Video Coding standard which is in current use in the industry by being able to reduce the bit rate by 50%, retaining the same video quality [4].

• Also HEVC supports increased use of parallel processing architectures [1].

Figure 2: Encoder block diagram for HEVC [4]

Figure 3: Decoder block diagram for HEVC [17]

VP9

• VP9 is an open and royalty free video compression standard being developed by Google [2][3].

• One of the goals for VP9 is to reduce the bit rate by 50% compared to VP8 while having the same video quality [7]

• VP9 expands techniques used in AVC and VP8 and is very likely to replace H.264/AVC at least in the YouTube video service [9].

Figure 4: Encoder block diagram for VP9 [19]

Figure 5: Decoder block diagram for VP9 [19]

HEVC and VP9 Coding Tools

• Macroblock concept and Prediction block sizes• Prediction Modes (Intra and Inter)• Transform and Quantization • Entropy Coding• Post Processing

Intra Prediction modes in HEVC•HEVC has 35 luma intra prediction modes (Figure 6)•Intra prediction can be done at different block sizes, ranging from 4 X 4 to 64 X 64 (whatever size the PU has) (Figure 7)•HEVC also includes a planar and DC intra prediction modes

Figure 6: Modes and directional orientations for intra picture prediction for HEVC [1]

Figure 7: Luma intra prediction modes for different PU sizes in HEVC [8]

VP9 techniques

• Recursive variable block size support (64x64 to 4x4, square and rectangular)• Finer motion vector precision (16th-pel) and adaptive motion filters (sharp, lowpass, regular) Figure 8: Example partitioning of a 64x64

Super-block

• Multi-level past and future reference (alt-ref) frames• larger dct transforms (32x32,16x16, 8x8, 4x4) and adst

transforms (16x16, 8x8, 4x4)

Intra Prediction modes in VP9

•There are total of ten intra-prediction modes in VP9 [9]: DC, True Motion (TM), and eight angular modes (H, V, D207, D153, D135, D117, D63 and D45 on Figure 8)

Figure 9: VP9 angular intra-prediction modes [9]

Performance Metrics

Criteria to evaluate the compression qualityTwo types of quality measures Objective quality measure- PSNR, MSE Structural quality measure- SSIM [10]MSE and PSNR for a NxM pixel image are defined as

where O is the original image and R is the reconstructed image. M and N are the width and height of an image and ‘L’ is the maximum pixel value in the NxM pixel image.

M

m

N

n

nmRnmONM

MSE1 1

2,,*

1

dBMSE

LPSNR ,log10

2

10

Performance Metrics(Continued..)

Bjøntegaard-Delta Bit-Rate and peak signal-to-noise ratio Measurements

•The Bjøntegaard delta peak signal-to-noise ratio (dB) [14][20] and the Bjøntegaard delta bit-rate (%) will be used compare performance of HEVC and VP9 Codecs.•As rate-distortion (R-D) performance assessment [14], Bjøntegaard-Delta bit-rate measurement method is used for calculating average bit-rate differences between R-D curves for the same objective quality (e.g., for the same PSNR-YUV values), where negative BD-BR values Indicate actual bit-rate savings.•Encoding time is used to compare the implementation complexity

Implementation

• For comparison purposes open-source implementations of the reviewed codecs will be used. HEVC compression efficiency will be measured with the HM Test Model [12].

• Evaluation of VP9 compression performance will be carried out with the VPX encoder from The WebM Project [13].

• Since HEVC has more Intra Prediction modes and few other features better than VP9, both the codecs are configured to establish a fair comparison.

Implementation(Continued..)• The test video sequence set that will be used for the comparison

are; BlowingBubbles, BasketballDrill, FourPeople, Kimono1 [9].

Sequence Resolution, Pixels Frame-rate, Hz Number of Frames

RaceHorses 416x240 30 500

BasketballDrill 832x480 50 500

Kimono1 1920x1080 24 240

PeopleOnStreet 2560x1600 30 150

Implementation(Continued..)• Figures 10, 11, 12 and 13 are frames of the test sequences RaceHorses,

BasketballDrill, Kimono1 and PeopleOnStreet, respectively.

Figure 11: BasketballDrill (832x480)

Figure 12: Kimono (1920x1080) Figure 13: PeopleOnStreet (2560x1600)

Figure 10: RaceHorses (416x240)

Implementation resultsRaceHorses_416x240_30.yuv (AI mode)

HEVC VP9

QP PSNR(dB) Bitrate(Kbit/s) Encoding Time(s)

PSNR(dB) Bitrate(Kbit/s) Encoding Time(s)

22 42.5720

5057.16

63.726 41.143

6507.95 52.426

27 38.6146 3152.1000 57.084 40.191

4683.97 50185

32 34.9992 1814.8920 48.975 36.230 2380.11 42.470

37 31.9718 979.5840 41.869 35.913

2262.14

40.828

BasketballDrill_832x480_50.yuv (AI mode)

HEVC VP9

QP PSNR(dB) Bitrate(Kbit/s) Encoding Time(s)

PSNR(dB) Bitrate(Kbit/s) Encoding Time(s)

22 42.3716 20407.88 278.849 42.3070

22101.34 197.060

27 39.0915 11014.04 220.022 39.553

13454.87

229.792

32 36.2986 5847.02 196.536 38.224

10621.66 178.128

37 33.9144 3200.72 162.733 35.831 7410.842 163.153

Implementation results(continued..)Kimono1_1920x1080_24.yuv (AI mode)

HEVC VP9

QP PSNR(dB) Bitrate(Kbit/s) Encoding Time(s)

PSNR(dB) Bitrate(Kbit/s) Encoding Time(s)

22 43.3716 18738.4512 1054.518 43.164 20764.76 940.820

27 42.0109 10746.1920 886.952 42.053

13404.4746

749.586

32 39.7045 6408.2112 830.751 40.955 9977.10 634.479

37 38.0951 3776.95 785.519 39.868 7508.94 659.983

PeopleOnStreet_2560x1600_30_crop.yuv (AI mode)

HEVC VP9

QP PSNR(dB) Bitrate(Kbit/s) Encoding Time(s)

PSNR(dB) Bitrate(Kbit/s) Encoding Time(s)

22 43.8084 104202.5640 2109.036 43.219 101898.87 1714.655

27 40.6998 60435.6720 1773.904 40.387 62989.20 1577.737

32 37.8947 34338.7800 1661.502 38.794 47220.45 1481.304

37 35.4149 19983.8040 1571.038 36.611 35706.77 1347.487

Implementation results(continued..)

Implementation results(continued..)

Implementation results(continued..)

Implementation results(continued..)

Implementation results(continued..)

Implementation results(continued..)

Conclusions

• HEVC provides better compression rates than VP9, but VP9 is patent-free and can be used without licensing expenses.

• For Intra frame coding, HEVC gives 13% more bitrate savings than VP9. And the encoding time taken by VP9 is marginally less than HEVC.

List of Acronyms

• ADST - Asymmetric Discrete Sine Transform

• AVC – Advanced Video Coding• BD-BR- Bjøntegaard-Delta Bit-Rate

Measurements• BD-PSNR - Bjøntegaard-Delta Peak signal to

noise ratio• CU- Coding unit• CTU- Coding tree unit• DBF- Deblocking Filter• DFT – Discrete Fourier Transform• DCT – Discrete Cosine Transform• DST – Discrete Sine Transform• DPB - Decoded Picture Buffer• DC – Direct Current• HD- High definition • HEVC-High Efficiency Video Coding

• ITU-T - International Telecommunication Union (Telecommunication Standardization Sector)

• JPEG - Joint photographic experts group• JCT-VC- Joint collaborative team on video

coding• MSE-Mean square error• MPEG-Moving picture experts group• NGOV- Next Geneneration Open Video• PU- Prediction unit • PSNR-Peak signal to noise ratio• PU – Prediction Unit• RD – Rate Distortion• SAO - Sample Adaptive Offset• SSIM- Structural similarity index• TM- True Motion • TU-Transform units • VCEG – Video Coding Experts Group

References

• [1] G.J. Sullivan et al, “Overview of the high efficiency video coding (HEVC) standard”, IEEE Trans. circuits and systems for video technology, vol. 22, no.12, pp. 1649 – 1668, Dec 2012.

• [2] D. Grois et al, “Performance Comparison of H.265/MPEG-HEVC, VP9, and H.264/MPEG-AVC Encoders” 30th Picture Coding Symposium 2013 (PCS 2013), San José, CA, USA, Dec 8-11, 2013

• [3] D. Mukherjee et al, “The latest open-source video codec VP9–An overview and preliminary results”, Google Inc., United States

• [4] G.J. Sullivan et al, "Standardized Extensions of High Efficiency Video Coding (HEVC)", IEEE Journal of Selected Topics in Signal Processing, vol.7, no.6, pp.1001-1016, Dec. 2013

• [5]Article on HEVC - http://en.wikipedia.org/wiki/High_Efficiency_Video_Coding• [6] Q. Cai et al, “Lossy and lossless intra coding performance evaluation: HEVC, H.264/AVC,

JPEG 2000 and JPEG LS”, Signal & Information Processing Association Annual Summit and Conference (APSIPA ASC), 2012 Asia-Pacific, vol.9, no.12, pp.1-9, Dec 2012.

• [7] "VP-Next Overview and Progress Update" (PDF). WebM Project (Google). Retrieved 2012-12-29. Available on : http://downloads.webmproject.org/ngov2012/pdf/04-ngov-project-update.pdf

• [8]M.T. Pourazad et al, “HEVC:The new gold standard for video compression”, IEEE consumer electronics magazine ,vol.1, no.7, pp.36-46, July 2012.

References(Continued..)• [9] M.P. Sharabayko et al, "Intra Compression Efficiency in VP9 and HEVC" Applied

Mathematical Sciences, Vol. 7, no. 137, pp.6803 – 6824, Hikari Ltd, 2013• [10] Z. Wang et al, “Image quality assessment: From error visibility to structural similarity,”

IEEE Trans. on Image Processing, vol. 13, no. 4, pp. 600-612, Apr. 2004. • [11] H. Jain, “Comparative performance analysis of HEVC and H.264 Intra frame coding

and JPEG2000”, EE5359, UTA, spring 2013.• http://www-ee.uta.edu/Dip/Courses/EE5359/index.html.• [12] HM Reference Software- https://hevc.hhi.fraunhofer.de/HM-doc/• [13] Chromium® open-source browser project, VP9 source code,

Online:http://git.chromium.org/gitweb/?p=webm/libvpx.git;a=tree;f=vp9;hb=aaf61dfbcab414bfacc3171501be17d191ff8506

• [14] G. Bjøntegaard, “Calculation of average PSNR differences between RD-curves”, ITU-T Q.6/SG16 VCEG 13th Meeting, Document VCEG-M33, Austin, USA, Apr. 2001.

• [15] S. Jeong et al., High efficiency video coding for entertainment quality. ETRI J. vol.33, pp.145–154, 2011.

• [16] JVT Draft ITU-T recommendation and final draft international standard of joint video specification (ITU-T Rec. H.264-ISO/IEC 14496-10 AVC), March 2003, JVT-G050-http://ip.hhi.de/imagecom_G1/assets/pdfs/JVT-G050.pdf

References(Continued..)

• [17] White paper on PSNR-NI - http://www.ni.com/white-paper/13306/en/• [18] HEVC tutorial by I.E.G. Richardson: http://www.vcodex.com/h265.html• [19] J. Padia, “Complexity reduction for VP6 to H.264 transcoder using motion vector

reuse,” M.S. Thesis, EE Dept., UTA, Arlington, TX, 2010. Available on : http://www-ee.uta.edu/Dip/Courses/EE5359/index.html

• [20] G. Bjøntegaard , “Improvements of the BD-PSNR model” ITU-T SG16 Q.6, Doc. VCEG-AI11, Berlin, Germany, July 16-18, 2008

• [21] T. Wiegand et al, “Overview of the H.264/AVC Video Coding Standard”, IEEE Transactions on Circuits and Systems for Video Technology, Vol. 13, No. 7, pp. 560-576, Jul. 2003.

• [22] N. Ahmed , T. Natarajan, K.R. Rao, “Discrete Cosine Transform”, IEEE Transactions on Computers, Vol. C-23, pp. 90-93, Jan. 1974.

• [23] I.E.G. Richardson, “The H.264 advanced video compression standard”, 2nd Edition, Hoboken, NJ, Wiley, 2010.

• [24] I.E.G. Richardson, “Video Codec Design: Developing Image and Video Compression Systems”, Wiley, 2002.

• [25] K.R. Rao, D.N. Kim and J.J. Hwang, “Video Coding Standards: AVS China, H.264/MPEG-4 Part 10, HEVC, VP6, DIRAC and VC-1”, Springer, 2014.

• [26] B. Bross et al, “High Efficiency Video Coding (HEVC) Text Specification Draft 10”, Document JCTVC-L1003, ITU-T/ISO/IEC Joint Collaborative Team on Video Coding (JCT-VC), Mar. 2013 available on http://phenix.itsudparis.eu/jct/doc_end_user/current_document.php?id=7243

Thank You