1 Video Coding Concept Kai-Chao Yang
Jan 04, 2016
1
Video Coding ConceptKai-Chao Yang
2
Video Sequence and Picture
Video sequence Large amount of temporal redundancy
Intra Picture/VOP/Slice (I-Picture) (p.295)
Encoded without referencing others All MBs are intra coded
Inter Picture/VOP/Slice (P-Picture, B-Picture) (p.295)
Encoded by referencing other pictures Some MBs are intra coded, and some are inter
coded
Intra 0 Inter 1 Inter 2 Inter 3 Inter 4 Inter 5
3
Group of Pictures
Group of Pictures (GOP) (p.295, p.455-457)
GOP in H.264/AVC Multiple reference frames B-pictures can be referenced Variable number of B-pictures
GOP in Scalable Video Coding (SVC) of H.264/AVC Hierarchical prediction structure
I B B P B B P … B B I B B P …
Encoding order:
Frame order: 0 1 2 3 4 5
0 2 3 1 5 6
6
4
Video stream
GOP GOP GOP…
Picture, Slice, MacroBlock, and Block
H.264/AVC Variable MB size 4x4 transform block
4
Picture
SliceMBblock
8
8
8
8
5
Flow Chart of Video Coding (p.293-295)
Transform Quantization
Motion Estimation
Entropy Coding
Quantization-1
Inverse Transform
Motion Compensation
Decoder
MB (spatial frequency) (lossy compression) (lossless compression)
(frame reconstruction)
(estimating similar blocks)
(removing temporal redundancy)
-
+
High frequency
Low frequency
Reference frameCurrent frame
6
Coding of I-Slice
Example
Original block Transformed block Quantization matrix
15 0 -2 -1 -1 -1 0 …Bit-stream
Zig-zag scanEntropy coding
DCT
7
Coding of P-Slice
Example
=
=
+Motion Vectors
Motion Estimation
Residual
Motion Compensation
Original current frame
Reconstructed reference frame
Frame buffer
-
8
Structure of H.264/AVC Encoder
Representation of video content
Formats of VCL representation
9
H.264/AVC Flow Chart
EntropyCoding
Scaling & Inv. Transform
Motion-Compensation
ControlData
Quant.Transf. coeffs
MotionData
Intra/Inter
CoderControl
Decoder
MotionEstimation
Transform/Scal./Quant.-
InputVideoSignal
Split intoMacroblocks16x16 pixels
Intra-frame Prediction
De-blockingFilter
OutputVideoSignal
10
New Features of H.264/AVC
Multiple reference frames Variable block size 1/4 and 1/8 motion vector precision Weighted prediction 4x4 transform block Intra prediction De-blocking filter CABAC & CAVLC Various block sizes and shapes
8x8
0
4x8
0 10 12 3
4x48x4
108x8
Types
0
16x16
0 1
8x16MB
Types
8x80 12 3
16x8
1
0
11
FMO and ASO
Examples of Flexible MB Order (FMO)
Arbitrary Slice Order (ASO) sending and receiving the slices of the picture in
any order relative to each other
Slice Group #2
Slice Group #1
Slice Group #0Slice Group #0
Slice Group #1
Slice #0
Slice #1
Slice #2
12
H.264/AVC Profiles
13
H.264/AVC FRExt Amendment
Completed in July 2004 For coding of high-fidelity
video material Professional film production,
video post production, or high-definition TV/DVD
Higher quality, higher rates Professional extensions
Fidelity Range extensions (FRExt)
http://plusd.itmedia.co.jp/lifestyle/articles/0407/09/news074.html
14
Profiles of H.264/AVC FRExt
High profiles High profile (HP)
Supporting 8-bit with 4:2:0 sampling High 10 profile (Hi10P)
Supporting 10-bit with 4:2:0 sampling High 4:2:2 profile (H422P)
Supporting 10-bit with 4:2:2 sampling High 4:4:4 profile (H444P)
Supporting 12-bit with 4:4:4 sampling, and efficient lossless coding and an integer residual color transform for coding RGB video
References
T. Wiegand, G.J. Sullivan, G. Bjntegaard, and A. Luthra, “Overview of the H. 264/AVC video coding standard,” in IEEE Transaction on Circuits and Systems for Video Technology, 2003.
Iain E. G. Richardson, H.264 and MPEG-4 Video Compression: Video Coding for Next-generation Multimedia, Wiley, 2003.
G. J. Sullivan, P. Topiwala, and A. Luthra, “The H.264/AVC Advanced Video Coding Standard: Overview and Introduction to the Fidelity Range Extensions,” in SPIE Conference on Applications of Digital Image Processing, 2004.
15