Digital Video Digital Video Compression Fundamentals Compression Fundamentals and Standards and Standards Speaker: Wei-Yi Wei Speaker: Wei-Yi Wei Advisor: Prof. Jian-Jung Ding Advisor: Prof. Jian-Jung Ding Digital Image and Signal Processing Lab Digital Image and Signal Processing Lab GICE, National Taiwan University GICE, National Taiwan University
53
Embed
Digital Video Compression Fundamentals and Standards.ppt
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Digital Video Compression Digital Video Compression Fundamentals and StandardsFundamentals and Standards
Speaker: Wei-Yi WeiSpeaker: Wei-Yi WeiAdvisor: Prof. Jian-Jung DingAdvisor: Prof. Jian-Jung Ding
Digital Image and Signal Processing LabDigital Image and Signal Processing LabGICE, National Taiwan UniversityGICE, National Taiwan University
2008/12/262008/12/26 Digital Video Compression Fundamentals and StDigital Video Compression Fundamentals and Standardsandards
22
OutlineOutline
IntroductionIntroduction
Video Compression StandardsVideo Compression Standards
Future Work and ConclusionsFuture Work and Conclusions
2008/12/262008/12/26 Digital Video Compression Fundamentals and StDigital Video Compression Fundamentals and Standardsandards
1818
The Development of Video The Development of Video Compression StandardsCompression Standards
…………
5-point 1D DCT
6-point 1D DCT
…………
5-point 1D DCT
6-point 1D DCT…………
5-point 1D DCT
6-point 1D DCT
2008/12/262008/12/26 Digital Video Compression Fundamentals and StDigital Video Compression Fundamentals and Standardsandards
1919
The MPEG-1 StandardThe MPEG-1 Standard
Group of Pictures
Motion EstimationMotion Estimation
Motion CompensationMotion Compensation
Differential CodingDifferential Coding
DCTDCT
QuantizationQuantization
Entropy CodingEntropy Coding
2008/12/262008/12/26 Digital Video Compression Fundamentals and StDigital Video Compression Fundamentals and Standardsandards
2020
Group of Pictures (1/2)Group of Pictures (1/2)
I-frame (Intracoded Frame)I-frame (Intracoded Frame)Coded in one frame such as DCT. Coded in one frame such as DCT. This type of frame do not need previous frame This type of frame do not need previous frame
P-frame (Predictive Frame)P-frame (Predictive Frame)One directional motion prediction from a previous frameOne directional motion prediction from a previous frame
The reference can be either I-frame or P-frameThe reference can be either I-frame or P-frameGenerally referred to as inter-frameGenerally referred to as inter-frame
B-frame (Bi-directional predictive frame)B-frame (Bi-directional predictive frame)Bi-directional motion prediction from a previous or future frameBi-directional motion prediction from a previous or future frame
The reference can be either I-frame or P-frameThe reference can be either I-frame or P-frameGenerally referred to as inter-frameGenerally referred to as inter-frame
2008/12/262008/12/26 Digital Video Compression Fundamentals and StDigital Video Compression Fundamentals and Standardsandards
2121
Group of Pictures (2/2)Group of Pictures (2/2)
The distance between two nearest P-frame or P-frame and I-frameThe distance between two nearest P-frame or P-frame and I-frame
denoted by denoted by MM
The distance between two nearest I-framesThe distance between two nearest I-frames
denoted by denoted by NN
I B B P B B P B B I
GOPGOP
Bidirectional Motion Compensation
Forward Motion Compensation
N=9
M=3
2008/12/262008/12/26 Digital Video Compression Fundamentals and StDigital Video Compression Fundamentals and Standardsandards
2222
The MPEG-1 Encoder (1/4)The MPEG-1 Encoder (1/4)
DCT QEntropyCoding
Q-1
IDCT
MotionCompensation
FrameMemory
MotionEstimation
DCT QEntropyCoding
DCT Q
Q-1
IDCT
MotionCompensation
FrameMemory
MotionEstimation
Intra-frame
EntropyCoding
Residue
Motion Vector
Inter-frame
2008/12/262008/12/26 Digital Video Compression Fundamentals and StDigital Video Compression Fundamentals and Standardsandards
2323
The MPEG-1 Encoder (2/4)The MPEG-1 Encoder (2/4)
Differential CodingDifferential Coding
is the input image is the input image
is the predictive imageis the predictive image
DCT DCT
^
( ) ( ) ( )D t t t ( )t
^
( )t
1 1
0 0
2 (2 1) (2 1)( , ) ( ) ( ) ( , ) cos cos
2 2
for 0,..., 1 and 0,..., 1
1/ 2 for 0where 8 and ( )
1 otherwise
N N
x y
x u y vF u v C u C v f x y
N N N
u N v N
kN C k
2008/12/262008/12/26 Digital Video Compression Fundamentals and StDigital Video Compression Fundamentals and Standardsandards
Inter quantization matrixInter quantization matrix
intra
8 16 19 22 26 27 29 34
16 16 22 24 27 29 34 37
19 22 26 27 29 34 34 38
22 22 26 27 29 34 37 40
22 26 27 29 32 35 40 48
26 27 29 32 35 40 48 58
26 27 29 34 38 46 56 69
27 29 35 38 46 56 69 83
Q
intra
16 17 18 19 20 21 22 23
17 18 19 20 21 22 23 24
18 19 20 21 22 23 24 25
19 20 21 22 23 24 26 27
20 21 22 23 25 26 27 28
21 22 23 24 26 27 28 30
22 23 24 26 27 28 30 31
23 24 35 27 28 30 31 33
Q
2008/12/262008/12/26 Digital Video Compression Fundamentals and StDigital Video Compression Fundamentals and Standardsandards
2525
The MPEG-1 Encoder (4/4)The MPEG-1 Encoder (4/4)Motion CompensationMotion Compensation
Exploit motion vector and the previous reconstructed frame to generate Exploit motion vector and the previous reconstructed frame to generate the predictive framethe predictive frame
is the compensated image is the compensated image
is the previous imageis the previous image
is the motion vectoris the motion vector
( , ) ( ( , ), ( , )), ( , ) ( , )n n x yx y x v p q y v p q x y MB p q ( , )n x y
The neighboring frames are not the most similar in some casesThe neighboring frames are not the most similar in some cases
The B-frame can be reference frameThe B-frame can be reference frameB-frame is close to the target frame in many situations B-frame is close to the target frame in many situations
2008/12/262008/12/26 Digital Video Compression Fundamentals and StDigital Video Compression Fundamentals and Standardsandards
3535
Integer Transform for Reducing The Integer Transform for Reducing The Spatial Redundancy (1/2)Spatial Redundancy (1/2)
The transform coefficients are The transform coefficients are CXCCXCTT
1 1 1 1
2 1 1 2
1 1 1 1
1 2 2 1
C
2008/12/262008/12/26 Digital Video Compression Fundamentals and StDigital Video Compression Fundamentals and Standardsandards
3636
Integer Transform for Reducing The Integer Transform for Reducing The Spatial Redundancy (2/2)Spatial Redundancy (2/2)
00 01 02 03
10 11 12 13
20 21 22 23
30 31 32 33
0.5 0.5 0.5 0.5 0.5 0.6325 0.5 0.3162
0.6325 0.3162 0.3162 0.6325 0.5 0.3162 0.5 0.6325
0.5 0.5 0.5 0.5 0.5 0
0.3162 0.6325 0.6325 0.3162
T
x x x x
x x x xY AXA
x x x x
x x x x
.3162 0.5 0.6325
0.5 0.6325 0.5 0.3162
00 01 02 03
10 11 12 13
20 21 22 23
30 31 32 33
1 1 1 1 0.5 0.6325 0.5 0.3162
2 1 1 2 0.5 0.3162 0.5 0.6325=
1 1 1 1 0.5 0.3162 0.5 0.6325
1 2 2 1 0.5 0.6325 0.5 0.3162
x x x x
x x x x
x x x x
x x x x
00 01 02 03
10 11 12 13
20 21 22 23
30 31 32 33
0.5 0.5 0.5 0.5
0.3162 0.3162 0.3162 0.3162
0.5 0.5 0.5 0.5
0.3162 0.3162 0.3162 0.3162
1 1 1 1 1 2 1 1
2 1 1 2 1 1 1 2=
1 1 1 1 1 1 1 2
1 2 2 1 1 2 1
x x x x
x x x x
x x x x
x x x x
0.25 0.1581 0.25 0.1581
0.1581 0.1 0.1581 0.1
0.25 0.1581 0.25 0.1581
1 0.1581 0.1 0.1581 0.1
CC XX CCTT EE
2008/12/262008/12/26 Digital Video Compression Fundamentals and StDigital Video Compression Fundamentals and Standardsandards
3737
Intra PredictionIntra PredictionPredict the similarity between the neighboring pixels in one Predict the similarity between the neighboring pixels in one frame in advance, and exploit differential coding transform frame in advance, and exploit differential coding transform coding to remove the redundancy. coding to remove the redundancy.
VerticalHorizontal + + + +
++++
Mean
DC Diagonal down-left
Horizontal up
Diagonal right
Vertical right Vertical leftHorizontal down
Intra Prediction
Transform/Quantization
EntropyCoding
2008/12/262008/12/26 Digital Video Compression Fundamentals and StDigital Video Compression Fundamentals and Standardsandards
Implementation Based on TI DSPImplementation Based on TI DSP
2008/12/262008/12/26 Digital Video Compression Fundamentals and StDigital Video Compression Fundamentals and Standardsandards
4646
Fast Mode Decision AlgorithmFast Mode Decision AlgorithmThe computational cost of H.264 is large
Variable block-size ME
Multiple reference frames
Fast mode decision is needed for reducing the computation time
2008/12/262008/12/26 Digital Video Compression Fundamentals and StDigital Video Compression Fundamentals and Standardsandards
4747
Interpolation Filter DesignInterpolation Filter DesignIn order to estimate and compensate the fractional-pel displacements
Adaptive Interpolation filter for replacing the fixed coefficient filter
2008/12/262008/12/26 Digital Video Compression Fundamentals and StDigital Video Compression Fundamentals and Standardsandards
4848
Deblocking Filter DesignDeblocking Filter DesignBlock based ME and Transform result in the annoying blockinBlock based ME and Transform result in the annoying blocking artifactg artifact
Reduce the blocking artifact can increase the quality of comprReduce the blocking artifact can increase the quality of compressed videoessed video
2008/12/262008/12/26 Digital Video Compression Fundamentals and StDigital Video Compression Fundamentals and Standardsandards
4949
DCT Based Motion EstimationDCT Based Motion Estimation
Robust even in noisy environmentRobust even in noisy environment
Complexity comparisonComplexity comparisonDCT Based MEDCT Based ME
O(MO(M22))
Block Based MEBlock Based MEO(NO(N22××MM22))
ConceptConceptPseudo PhasePseudo Phase
Similar toSimilar to { ( )} { ( )}jF x t e F x t
2008/12/262008/12/26 Digital Video Compression Fundamentals and StDigital Video Compression Fundamentals and Standardsandards
5050
Implementation Based on TI DSPImplementation Based on TI DSP
TMS320C6416TMS320C6416
TI DM642 TI DM642
2008/12/262008/12/26 Digital Video Compression Fundamentals and StDigital Video Compression Fundamentals and Standardsandards
5151
ConclusionsConclusions
Thank YouThank YouQ & AQ & A
2008/12/262008/12/26 Digital Video Compression Fundamentals and StDigital Video Compression Fundamentals and Standardsandards
5252
References (1/2)References (1/2)[1] Yun Q.Shi and Huifang Sun, “Image and Video Compression for Multimedia Engineering: Fundamentals, Algorithms, and Standards”, CRC press, 2000.[2] Yao Wand, Jorn Ostermann and Ya-Qin Zhang, “Video Processing and Communications”, Prentice Hall, 2007.[3] Richardson, Lain E. G., “Video Codec Design: Developing Image and Video Compression Systems”, John Wiley & Sons Inc, 2002.[4] Barry G, Haskell, Atul Puri and Arun N. Netravali, “Digital Video : An Introduction to MPEG-2”, Boston : Kluwer Academic, 1999.[5] T. Wiegand, G. J. Sullivan, G. Bjontegaard, and A. Luthra, “Overview of the H.264/AVC video coding standard”, IEEE Trans. on Circuits and systems for video Technology, vol. 13, no. 7, pp. 560-576, July 2003.[6] G. Sullivan and T. Wiegand, “Video Compression - From Concepts to the H.264/AVC Standard”, Proceedings of the IEEE, Special Issue on Advances in Video Coding and Delivery, December 2004.[7] 酒井善則、吉田俊之 共著,白執善 編譯,“影像壓縮技術”,全華,2004.
2008/12/262008/12/26 Digital Video Compression Fundamentals and StDigital Video Compression Fundamentals and Standardsandards
5353
References (2/2)References (2/2)[8] Thomas Wedi, “Adaptive Interpolation Filters and High-Resolution Displacements For Video Coding”, IEEE Trans. on Circuits and Systems For Video Technology, vol.[9] Dong-Hwan Kim, Hwa-Yong Oh, O˘guzhan Urhan, Sarp Ertürk and Tae-Gyu Chang, “Optimal Post-Process/In-Loop Filtering for Improved Video Compression Performance”, IEEE Trans. on Consumer Electronics, vol. 53, no. 4, Nov. 2007.[10] Shu-Fa Lin, Meng-Ting Lu, and Homer Chen, ” Fast Multi-Frame Motion Estimation for H.264 and Its Applications to Complexity-Aware Streaming”, IEEE International Symposium on Circuits and Systems Circuits and Systems, 2005., 2005.[11] Kai-Ting Cheng and Hsueh-Ming Hang, “Acceleration and Implementation of H.264 Encoder and Scalable Extension of H.264 Decoder on TI DSP Platform”, master thesis, June 2007