MULTIMEDIA PROCESSING

MULTIMEDIA PROCESSING

ADAPTIVE INTERPOLATION FILTER FOR H.264/AVC

Bhavana PrabhakarStudent Id: 1000790889Department of Electrical Engineering

H.264/ADVANCED VIDEO CODING:

Encoder block diagram H.264 [18]

H.264/ADVANCED VIDEO CODING:

Decoder block diagram H.264 [18]

What is Adaptive Interpolation Filter [AIF] ?

Interpolation process of (a) the filter in H.264/AVC, (b) the non-separable AIF, and (c) the separable AIF [16]

↑𝑚 = m times the sampling rate due to interpolation.

Integer samples (shaded blocks with upper-case letters) and fractional sample positions (non-shaded blocks with lower-case letters). Example for filter size 6 x6. [16]

AIF PROPOSED IN [3,4]1. Adaptive interpolation filter, which is independently estimated for every image. The filter coefficients, which are used for the calculation of the half-pel positions, are estimated iteratively using a numerical approach. The quarter-pel positions are calculated using a bilinear filter. [3]2. 3-D filter- Combining two techniques[4] 2-D spatial filter Motion compensated interpolation filter (MCIF)The main disadvantage of MCIF is the sensitivity concerning displacement vector estimation errors. Thus, the maximal bit-rate savings could not be achieved in case of quarter-pel motion accuracy.

DISADVANTAGES OF THE AIF IN [3.4] The method is nondeterministic in terms of time and

requires a significantly higher encoder complexity, i.e., the highest gains cannot be guaranteed given a particular increase of encoder complexity due to its numerical approach.

Besides aliasing, there are further distorting factors, which impair the efficiency of motion compensated prediction.

Motion blur typically occurs in video sequences when the relative motion between the camera and the objects in the scene being captured is faster than the camera exposure time allows.

Further distorting factors, caused by limited amplitude resolution of displacement vectors or by large quantization errors in the reference images, were analyzed in [3].

STEPS TO OVERCOME THE DISADVANTAGES [16]

In order to guarantee a limited increase of encoder complexity compared to the standard H.264/AVC on the one hand and to reach the theoretical bound for the coding gain obtained by means of a 2-D filter on the other hand, a non-separable filter scheme is proposed. An individual filter shall be used for the interpolation of each fractional-pel position.

For all fractional-pel positions, the filter coefficients are estimated minimizing the prediction error energy, i.e., the squared difference between the original and the predicted image signals.

H.264/AVC CODER SETTINGSPARAMETER SETTINGSVersion of the reference software JM11.0Version of the kta software KTA1.3Profile: High(100)Number of reference images: 4Intra period 0 (first image only)Context-adaptive binary arithmetic coding (CABAC) on8x8 Transform: onDeblocking filter: onWeighted Prediction: On for H.264

Off for the proposed methodRate Distortion optimisation onSearch Range: 32 Pel for QCIF and CIF

64 Pel for 720p and 1080pBlock sizes for MCP All 16x16 to 4x4Quantization parameter (I/P/B): 22/23/24, 27/28/29,

32/33/34, 37/38/39Adaptive rounding [32] onR-D picture decision off

EXPERIMENTAL RESULTS

Evaluation of the HDTV sequence Raven in terms of PSNR prediction quality (top left), the displacement vectors per frame in quarter-pel resolution (bottom left) and two cut-outs (top right and bottom right) for the standard Wiener filter, symmetric adaptive interpolation filter and non-symmetric adaptive interpolation filter. [16]

Rate-distortion curves for CIF sequences Foreman, Mobile, Concrete, and Waterfall.

Rate-distortion curves for 720p sequences City, Raven, ShuttleStart, and for 1080p sequence Sunflower.

Bit-rate savings in %, achieved by means of adaptive interpolation filter compared to h.264/avc for several Qcif, cif, and hdtv sequences.

CONCLUSION

The non-separable adaptive interpolation filter: Analytically minimizes the energy of the prediction

error (PE). Where is the 2-D filter coefficient for each fractional pel position.

PE = Reduces the distorting effects, caused by aliasing,

motion blur, motion estimation inaccuracies etc. Further improvements were achieved, when

applying locally adaptive filter, which is adapted to local properties of the image.

The following steps taken effectively reduced motion blur: first considering only displacement without blurring effects. Where = average displacement vector and the prediction signal ( ) expressed in frequency domain is given in eq.(1) (jΩ) = (jΩ) eq.(1)With the intention of compensating the blurring effects, the adaptive interpolation filter H(jΩ) for perfect motion compensated prediction has to satisfy the condition given in eq.(2) (jΩ) = (jΩ) . H(jΩ) eq.(2)

LIST OF ACRONYMSAIF: Adaptive interpolation filterAVC: Advanced video codingBD – ROM: Blue ray disc – read only memoryCABAC: Context-adaptive binary arithmetic codingCIF: Common intermediate formatHD – DVD: High definition - digital video discITU: International telecommunication unionKTA: Key technical areaMCIF: Motion compensated interpolation filterMPEG: Moving picture experts groupQCIF: Quarter common intermediate format.VCEG: Video coding experts group

REFERENCES[1] JVT of ISO/IEC & ITU-T, Draft ITU-T Recommendation H.264 and Draft ISO/IEC 14496-10 AVC, Doc JVT-Go50. Pattaya, Thailand, 2003.[2] O. Werner, “Drift analysis and drift reduction for multi resolution hybrid video coding”, Signal processing: image commun., vol. 8, no. 5, pp. 387–409, Jul. 1996.[3] T. Wedi and H. G. Musmann, “Motion and aliasing compensated prediction for hybrid video coding”, IEEE Trans. circuits syst. video technol., vol. 13, no. 7, pp. 577–586, Jul. 2003.[4] T. Wedi, “Adaptive interpolation filter for motion and aliasing compensated prediction”, in Proc VCIP, San Jose, CA, USA, pp. 415–422, Jan. 2002.[5] M. Budagavi, “Video compression using blur compensation”, in Proc.IEEE ICIP, Genova, Italy, pp. 882–885, Sep. 2005.[6] R. E. Crochiere and L. R. Rabiner, “Multi-rate signal processing”, Englewood Cliffs, NJ: Prentice Hall, pp. 88–91, 1983.[7] R. W. Schaefer and A. V. Oppenheim, “Discrete-time signal processing”, Englewood Cliffs, NJ: Prentice-Hall, 1989.[8] T. Wiegand et al, “Overview of the H.264/AVC video coding standard”, IEEE Trans. circuits syst. video technol., vol. 13, no. 7, pp. 560-576, Jul. 2003.[9] Y. Vatis and J. Ostermann, “Locally adaptive non separable interpolation filter for H.264/AVC”, in Proc. IEEE ICIP, Atlanta, GA, pp. 33–36, Oct. 2006.[10] T.Wedi, “Adaptive interpolation filter for motion compensated prediction”, Proc. IEEE ICIP, Rochester, NY, pp. 509–512, Sep. 2002.

[11] H.264/AVC reference software version JM11.0http://iphome.hhi.de/suehring/tml/download/old_jm/jm11.0.zip, Jan. 2007 [Online].[12] KTA software, version JM11.0 KTA1.3. http://www.tnt.uni-hannover.de/~vatis/kta/jm11.0kta1.3.zip, Mar. 2007 [Online].[13] Y. Vatis and J. Ostermann, “Prediction of P- and B-frames using a 2-D non-separable adaptive Wiener interpolation filter”,in ITU-T SG16/Q [15] (VCEG) Doc VCEG-AD08, Hangzhou, China, Oct. 2006.[14] Y. Vatis and J. Ostermann, ITU-T SG16/Q [15] (VCEG) VCEG-AE16, Marrakech, Morocco, Jan. 2007.[[15]] S. Wittman and T. Wedi, “Separable adaptive interpolation filter”,in ITU-T SG16/Q6, Doc. C-0219, Geneva, Switzerland, Jul. 2007.[16] Y. Vatis and J. Ostermann “Adaptive interpolation filter for H.264/AVC”, IEEE Trans. circuits syst. video technol., vol. 19, pp.179-192, Feb. 2009.[17] D. Rusanovskyy, K. Ugur, and J. Lainema, “Adaptive interpolation with directional filters”, in ITU-T SG16/Q.6 Doc. VCEG-AG21, Shenzhen,China, Oct. 2007.[18] D. Marpe, T. Wiegand and G. J. Sullivan, “The H.264/MPEG-4 AVC standard and its applications”, IEEE Communications Magazine, vol. 44, pp. 134-143, Aug. 2006.[19] T. Wiegand and G. J. Sullivan, “The picturephone is here:Really”, IEEE Spectrum, vol.48, pp. 50-54, Sep. 2011.[20] I. E. Richardson, “The H.264 Advanced Video Compression Standard”, 2nd Edition, Wiley 2010.