IMPLEMENTATION AND PERFOMANCE ANALYSIS OF H.264 INTRA FRAME CODING, JPEG, JPEG-LS, JPEG- 2000 AND JPEG-XR 1 EE 5359 Multimedia Project Amee Solanki (1000740226) [email protected]
Dec 24, 2015
IMPLEMENTATION AND PERFOMANCE ANALYSIS OF H.264 INTRA FRAME CODING, JPEG, JPEG-LS, JPEG-2000 AND JPEG-XR
1
EE 5359 Multimedia ProjectAmee Solanki (1000740226)[email protected]
Image Compression
2
•Compression is the process of compacting data, reducing the number of bits. •Reduce redundancy of the image or video data in order to be able to store or transmit data in an efficient form.
Fig.1 Comparison of original coronary angiogram (left) with two compression results. Middle: JPEG data compression by factor of CR=12, Right: factor of CR=24[14].
Two Types of Compression
3
Lossless compression: There is no information loss, and the image can be
reconstructed exactly the same as the original Applications: medical imagery, archiving
Lossy compression: Information loss is tolerable. Applications: commercial distribution (DVD) and rate
constrained environment where lossless methods cannot provide enough compression ratio
Evolution of Image Compression Standards
4
Fig.2 Evolution of compression technology[15]
Compression standards
5
Standard Software Main Application
Year
JPEG JPEG-Baseline Ref.
Image 1992-1999
JPEG-LS JPEG-LS DLL*DLL-Dynamic linked library
Image 1999-2000
JPEG-2000 JasPer Image 2000
JPEG-XR JPEG-XR Ref. Image 2009
H.264/AVC Intra Coding
JM Video 2003
Table 1: Comparison of image compression standards[13]
JPEG Standards
6
Baseline JPEG Encoder and Decoder
7
Fig.2 JPEG encoder block diagram [1]
Fig.3 JPEG decoder block diagram [1]
JPEG 2000 Encoder and Decoder
8
Fig. 4 (a) Encoder block diagram (b) Decoder block diagram of JPEG 2000 [2]
JPEG and JPEG-2000
9
Standard
Compression ratio
Main CompressionTechnologies
Main TargetApplications
JPEG Compression ratio 2-30
-DCT-Perceptual quantization-Zig zag reordering-Huffman coding-Arithmetic coding
-Internet imaging-Digital photography-Image and video editing
JPEG-2000
Compression ratio 2-50
-Wavelets EBCOT
-Internet imaging-Digital photography-Image and video Editing-Printing-Medical imaging-Mobile applications-Color fax-Satellite imaging
Table 2: Comparison of JPEG and JPEG 2000 [13]
JPEG-LS and JPEG-XR
10
Standard Compression ratio
Main CompressionTechnologies
Main TargetApplications
JPEG-LS Compression Ratio 2:1
-Context Modeling-Prediction-Golomb Codes -Arithmetic coding
- Lossless and near lossless coding of continuous tone still images
JPEG-XR Higher compression ratio than JPEG
Based on HD Photo of Microsoft (Windows Media Photo)
-Storage and interchange of continuous tone photographic content (lossless and lossy )
Table 3: Comparison of JPEG-LS and JPEG-XR [13]
H.264/AVC(Advanced Video Coding) Standard
11
H.264 Basics
12
H.264/AVC compression video coding is based on the traditional hybrid concept of block-based motion-compensated prediction (MCP) and transform coding
In order to improve the compression efficiency of intra-only compression, the following two coding tools provide major contributions to the significant bit rate savings:
• Entropy encoding improvement, CAVLC and CABAC• Spatial intra prediction conducted by using spatially
neighboring samples of a target block which have been previously coded.
13
Fig.4 Examples of spatial intra prediction modes for (8X8) blocks [15]
Spatial Intra prediction[15]
•H.264/AVC uses both spatial and temporal predictions to increase its coding gain. •The intra-only compression uses spatial prediction and the prediction only occurs within a slice
14
Fig. 5 A 4X4 block and its neighboring pixels[16]
Fig. 6 Direction of 9 4X4 intra-prediction [16]
•Fig. 5 shows a 4x4 block containing 16 pixels labeled from a through p. A prediction block p is calculated based on the pixels labeled A-M obtained from the neighboring blocks.
•A prediction mode is a way to generate these 16 predictive pixel values using some or all of the neighboring pixels in nine different directions as shown in Fig. 6.
•In some cases, not all of the samples A-M are available within the current slice.
•In order to preserve independent decoding of slices, only samples within the current slice are used for prediction.
15
Fig.7 Examples of spatial intra prediction modes for (4X4) blocks[16]
1. Mode 0 is the vertical prediction mode in which pixels a, e, i, and m are predicted by A and so on.
2. Mode 1 is the horizontal prediction mode in which pixels a,b, c, and d are predicted by I and so on.
3. Mode 2 is called DC prediction in which all pixels i.e. (a to p) as shown in fig. 5 are predicted by (A+B+C+D+I+J+K+L)/8.
4. For modes 3-8, the predicted samples are formed from a weighted average of the prediction samples A-M.
H.264 Basic Encoder and Decoder
16Fig.8 (a) H.264 encoder block diagrams [3]
17
Fig.8 (b) H.264 decoder block diagrams [3]
Compressed Image Quality Measures
18
Criteria to evaluate a compressed image are as follows :
1. Compression ratio2. Bit-rate (bandwidth)3. Objective quality measure- PSNR, MSE (quality
of compressed image)4. Structural quality measure- SSIM
PSNR and MSE
19
Peak signal-to-noise ratio, often abbreviated PSNR, is the ratio between the maximum possible power of a signal and the power of corrupting noise that affects the fidelity of its representation
MSE and PSNR for a NxM pixel image are defined as
(1)
(2)
where x is the original image and y is the reconstructed image. M and N are the width and height of an image and ‘L’ is the maximum pixel value in the NxM pixel image
Structural Similarity Index
20
The structural similarity (SSIM) [17] index is a method for measuring the similarity between two images
SSIM is designed to improve on traditional methods like peak signal-to-noise ratio (PSNR) and mean squared error (MSE), which have proved to be inconsistent with human eye perception
SSIM considers image degradation as perceived change in structural information. Structural information is the idea that the pixels have strong inter-dependencies especially when they are spatially close
SSIM Metric [17]
21
wherex and y correspond to two different signals that need to be compared, i.e. two different blocks in two separate images ;
Example with SSIM index
22
Fig. 9 SSIM Index example [4]
Conclusion
23
24
25
TABLE OF ACRONYMS
26
AVC advanced video codingBMPbit map formatCABAC context adaptive binary arithmetic codingDCTdiscrete cosine transformEBCOT embedded block coding with optimized truncationFRExt fidelity range extensionsGIF graphics interchange formatHD-photo high-definition photoHVShuman visual systemI-frame intra frameJMjoint modelJPEG joint photographic experts groupJPEG-LS joint photographic experts group lossless and lossless
codingJPEG-XR joint photographic experts group extended rangeLBT lapped bi-orthogonal transformLOCO-I low complexity lossless compression for imagesMSEmean square errorPSNR peak signal to noise ratioSSIM structural similarity index VLC variable length coding
References
27
[1] JPEG encoder and decoder block diagram :http://www.cmlab.csie.ntu.edu.tw/cml/dsp/training/coding/jpeg/jpeg/decoder.htm
[2] JPEG2000 encoder and decoder block diagram :http://eeweb.poly.edu/~yao/EE3414/JPEG.pdf
[3] H.264 encoder and decoder block diagram :S. Kwon, A. Tamhankar and K.R. Rao, “Overview of H.264 / MPEG-4 Part 10”, J. Visual Communication and Image Representation, vol. 17, pp.186-216, April 2006.
[4] SSIM index example diagram:https://ece.uwaterloo.ca/~z70wang/research/ssim/
[5] H.264/AVC reference software (JM 17.2) website: http://iphome.hhi.de/suehring/tml/download/
[6] JPEG2000 latest reference software (Jasper Version 1.900.0) website: http://www.ece.uvic.ca/~mdadams/jasper/
[7] JPEG reference software website: ftp://ftp.simtel.net/pub/simtelnet/msdos/graphics/jpegsr6.zip
[8] JPEG-LS reference software website: http://www.hpl.hp.com/loco/
28
[9] T. Wiegand, G. J. Sullivan, G. Bjontegaard and A. Luthra,” Overview of the H.264 / AVC video coding standard ” IEEE Trans. on Circuits and Systems for Video Technology,vol. 13, pp. 560-576, July 2003.
[10] A.Skodras, C. Christopoulos and T. Ebrahimi, “The JPEG 2000 still image compression standard”, IEEE Trans. on Signal Processing, vol.18, pp. 36 - 58, Aug 2002.
[11] M. J. Weinberger, G. Seroussi and G. Sapiro, “The LOCO-I lossless image compression algorithm: principles and standardization into JPEG-LS”, IEEE Trans. on Image Processing, vol.9, pp.1309-1324, Aug. 2000
[12] C. Christopoulos, A. Skodras and T.Ebrahimi, “The JPEG2000 still image coding system: anoverview”, IEEE Trans. on Consumer Electronics, vol.46, pp.1103-
1127, Nov. 2000.
[13] T. Ebrahimi and M. Kunt, “ Visual data compression for multimedia applications”, Proc IEEE, vol.86, pp. 1109-1125, June 1998.
29
[14] Image compression test image:
http://www.uni-kiel.de/Kardiologie/dicom/1999/compression1.html.
[15] Evolution of image compression standards :ftp://ftp.panasonic.com/pub/panasonic/drivers/PBTS/papers/WP_AVC-Intra.pdf
[16] Intra-prediction modes image:http://www.atc-labs.com/technology/h264_publication_1.pdf [17] Z. Wang, A. C. Bovik, H. R. Sheikh and E. P. Simoncelli, “Image quality assessment: From error visibility to structural similarity,” IEEE Trans. on Image Processing, vol. 13, no. 4, pp. 600-612, Apr. 2004.
[18] I. E. Richardson, “The H.264 advanced video compression standard”, II Edition, Wiley, 2010.
[19] D. S. Taubman and M. W. Marcellin, "JPEG2000 – Image compression fundamentals, standards, and practice," Kluwer, 2001.
Some Important terms
30
Quality factor –N denotes the scale quantization tables to adjust image quality. Quality factor varies from 0 (worst) to 100 (best); default is 75.
Rate specify target rate as a positive real number. ‘rate’=1 corresponds to no compression. Rate and bits per pixel are related by the expression: compression ratio=24/bpp= 1/rate for a color image and rate=bpp/8 for a gray scale image.
Error value is varied from 1 to 60. Error value of zero corresponds to no compression.T1, T2, T3 are thresholds. While giving the settings the following condition need to be met. (Error value+1)<T1<T2<T3.
Results
31
32
33
34
35
36
37
38
39
40