Basics of Video Coding Basics of Video Coding and H.263 Video Coding and H.263 Video Coding 김김김 Sep. 6, 2000
Basics of Video CodingBasics of Video Codingand H.263 Video Codingand H.263 Video Coding
김성재
Sep. 6, 2000
Sep. 6, 2000Basics of Video Coding and H.263 Video Coding
2
ContentContent
• Basics of Video Coding– Introduction– Color Coordinate and Picture Format– Transform Coding - DCT– Quantization– Entropy Coding
• H.263 Video Coding– Introduction– Encoder Block Diagram– Overall Process– Motion Estimation– Quantization– VLC encoding of quantized transform coefficient
Sep. 6, 2000Basics of Video Coding and H.263 Video Coding
3
Introduction (1/4)Introduction (1/4)
• Image Coding (Video Coding)– To represent an image(video)
with as few bits as possible
while preserving the level of distortion (quality)
Bit R ate
D istortion(Q uality)
Basics of Video Coding
Sep. 6, 2000Basics of Video Coding and H.263 Video Coding
4
Introduction (2/4)Introduction (2/4)
• Typical Image Coding System
• Three Components in Image Coder
Im ageCoder
Im ageSource
ChannelIm age
DecoderReconstructed
Im age
Transform ation orFeature Extraction
Im ageSource
Q uantizerCodeword
Assignm entS tringof B its
Basics of Video Coding
Sep. 6, 2000Basics of Video Coding and H.263 Video Coding
5
Introduction (3/4)Introduction (3/4)
• Classification of Image Coding Techniques
Transform ation orFeature Extraction
Q uantizationC odeword
Assignm ent
W aveformC oding
Im age M odelC oding
Predictive coding Transform coding Subband coding (W avelet)
Segm entation based M odel based Fracta l based
Scalar quantization Vector quantization
Entropy coding Huffm an coding A rithm etic coding Run-length coding
Basics of Video Coding
Sep. 6, 2000Basics of Video Coding and H.263 Video Coding
6
Introduction (4/4)Introduction (4/4)
• Features of Video Compression Algorithm– Random Access– Fast Forward/Reverse Searches– Reverse Playback– Audio-Visual Synchronization– Robustness to Errors– Coding/Decoding Delay– Format Flexibility
Basics of Video Coding
Sep. 6, 2000Basics of Video Coding and H.263 Video Coding
7
Color Coordinate and Picture Format (1/3)Color Coordinate and Picture Format (1/3)
• Human Visual System– Cone : near to fovea, for day vision, 7 million– Rod : away from fovea, for night vision, 120 million
• Color Coordinate– RGB– YUV
– YCrCb
)(877.0
)(493.0
114.0587.0299.0
YRV
YBU
BGRY
)(577.0
)(730.0
114.0587.0299.0
YBC
YRC
BGRY
b
r
Basics of Video Coding
Sep. 6, 2000Basics of Video Coding and H.263 Video Coding
8
Color Coordinate and Picture Format (2/3)Color Coordinate and Picture Format (2/3)
• Picture Format
Y
U V
xY
yY
xC
yC
Picture Format xY yY xC yC
Sub-QCIF 128 96 64 48
QCIF 176 144 88 72
CIF 352 288 176 144
4CIF 704 576 352 288
16CIF 1408 1152 704 576
Basics of Video Coding
Sep. 6, 2000Basics of Video Coding and H.263 Video Coding
9
Color Coordinate and Picture Format (3/3)Color Coordinate and Picture Format (3/3)Basics of Video Coding
Sep. 6, 2000Basics of Video Coding and H.263 Video Coding
10
DCT(Discrete Cosine Transform) (1/2)DCT(Discrete Cosine Transform) (1/2)
• N 개의 입력 신호 에 대 해서 1-D forward DCT 는 다음과 같이 정의
}10),({ Nnnu
)(kv
0,1
0,2
1)(
10,2
)12(cos)()(
2)(
1
0
C
NkN
knnukC
Nkv
N
n
W av eN u m b e r, k
0
1
2
3
4
5
6
7
0 4 8n
Basics of Video Coding
Sep. 6, 2000Basics of Video Coding and H.263 Video Coding
11
DCT(Discrete Cosine Transform) (2/2)DCT(Discrete Cosine Transform) (2/2)
• 4x4 2-D DCTBasics of Video Coding
Sep. 6, 2000Basics of Video Coding and H.263 Video Coding
12
Scalar QuantizationScalar QuantizationBasics of Video Coding
0n bits/pixel Q r bits/pixel
Original image Reconstructed image
Sep. 6, 2000Basics of Video Coding and H.263 Video Coding
13
Vector QuantizationVector QuantizationBasics of Video Coding
Original image Reconstructed imageCodebook
Sep. 6, 2000Basics of Video Coding and H.263 Video Coding
14
Huffman CodingHuffman Coding
Color Frequency
Red 19
Black 17
Green 16
Blue 5
Cyan 4
Magenta 2
Yellow 1
Color Bits assigned
Red 00
Black 01
Green 10
Blue 111
Cyan 1100
Magenta 11010
Yellow 11011
Basics of Video Coding
Sep. 6, 2000Basics of Video Coding and H.263 Video Coding
15
Arithmetic CodingArithmetic Coding
• Green-Green-Red
Pixel Probability Range Assigned (0x<1)
Red 0.2 [0.0, 0.2)
Green 0.6 [0.2, 0.8)
Blue 0.2 [0.8, 1.0)
0.0 0.2 0.8 1.0
Red G reen B lue
Red G reen B lue
R ed G reen B lue
Basics of Video Coding
Sep. 6, 2000Basics of Video Coding and H.263 Video Coding
16
Introduction (1/2)Introduction (1/2)
• Two basic compression algorithm– Block-based motion estimation/compensation
: Reduction of the temporal redundancy– Transform domain(DCT)-based compression
: Reduction of the spatial redundancy
• 3 types of pictures– I(Intra picture), P(Predicted picture), B(Interpolated picture)
I B B P B B P
B idirectional In terporlation
Prediction
H.263 Video Coding
Sep. 6, 2000Basics of Video Coding and H.263 Video Coding
17
Introduction (2/2)Introduction (2/2)
• Structure of Video Data
– Macroblock¤ Four Luminance(Y) Block
¤ Two Chrominence(Cr, Cb) Block
– Block : 8x8 pixel
Syntax layers Function
Sequence
Group of pictures(GOP) Random access
Picture Primary coding unit
Macroblock Motion compensation
Block DCT unit
H.263 Video Coding
Sep. 6, 2000Basics of Video Coding and H.263 Video Coding
18
Encoder Block Diagram (1/2)Encoder Block Diagram (1/2)
D C T Q
P R E D
M 2M E 1
ID C T
V LC (C )
M B TYP
V LC (M )
M U X B U F
M 1
C C
M E 2
IQ
V ideoIN
+-
+
+
IN TR A /IN TE R
0
D A TAO U T+
+
H.263 Video Coding
Sep. 6, 2000Basics of Video Coding and H.263 Video Coding
19
Encoder Block Diagram (2/2)Encoder Block Diagram (2/2)
ME1 Integer pixel motion estimation and INTRA/INTER
decision ME2 Half pixel motion estimation M1 Input frame store M2 Decoded frame store PRED Make prediction block MBTYPE Decide block type and block pattern VLC(C) VLC coding of transform coefficients VLC(M) VLC coding of motion vectors CC Coding control DCT Discrete Cosine Transform Q Quantizer
H.263 Video Coding
Sep. 6, 2000Basics of Video Coding and H.263 Video Coding
20
Overall Process (1/2)Overall Process (1/2)
Y
U V
Y
U V
Y
U V
Y
U V
Y
U V
FRAME 1
DC T Quantize to decoder
Dequantize
InverseDC T
Save for FRAME 2
Save for FRAME 2
ReconstructedFRAME 1
Sep. 6, 2000Basics of Video Coding and H.263 Video Coding
21
Overall Process (2/2)Overall Process (2/2)
Y
U V
Y
U V
Y
U V
Y
U V
FRAME 2
ME
DC T
to decoder
Y
U V
M otionVector
FRAME 1
ReconstructedFRAME 1
MC
to d
ecoder
Quantize
Difference Y
U V
Y
U V
Sep. 6, 2000Basics of Video Coding and H.263 Video Coding
22
Motion Estimation (1/4)Motion Estimation (1/4)
• Integer pixel motion estimation– 움직임 탐색은 Y 에 대해 정수 화소 단위로 이루어진다 .– 현재 입력 Block 과 이전 Frame 의 영상과 Macroblock
단위로 비교하여 SAD(Sum of Absolute Difference) 가 가장 작은 위치를 찾는다 .
– Full search 가 사용되며 탐색범위는 현재 Macroblock 의 위치에서 수평 , 수직방향으로 화소가 된다15
100)0,0()0,0(
816,"15",|,|),(
1616
,
1,1
SADSAD
orNtoupyxpreviousoriginalyxSADNN
jiN
H.263 Video Coding
Sep. 6, 2000Basics of Video Coding and H.263 Video Coding
23
Motion Estimation (2/4)Motion Estimation (2/4)
• INTRA/INTER Mode Decision– Integer pixel motion estimation 을 한 후에 Coder 는
INTRA 로 부호화할 것인지 INTER 로 부호화 할 것인지를 결정한다 .
– A < (SAD - 500) 이면 INTRA 모드로 선택된다 .– INTER 모드가 선택되면 계속해서 Half pixel search 가
이루어진다 .
16,16
1,1
16,16
1,1
||
256
1
jimean
jimean
MBoriginalA
originalMB
H.263 Video Coding
Sep. 6, 2000Basics of Video Coding and H.263 Video Coding
24
Motion Estimation (3/4)Motion Estimation (3/4)
• Half pixel search– 이전에 재생된 영상을 이용– 탐색 범위는 정수단위의 움직임 벡터 위치에서
Half pixel 이 된다 .– SAD(0,0) 는 100 을 뺀 값과 비교
C
B
Dc
Ab
d
a
Half pixel position
Integer pixel position
a = A b = (A+B)//2c = (A+C)//2 d = (A+B+C+D)//4
// : Integer division with rounding to nearest integer
1
H.263 Video Coding
Sep. 6, 2000Basics of Video Coding and H.263 Video Coding
25
Motion Estimation (4/4)Motion Estimation (4/4)
• Differential coding of motion vectors– Motion vector 의 전송은 이전 Macroblock 의 Motion
vector 와의 차이를 전송
– Motion vector difference :
MV2 MV3
MVMV1
MV2 MV3
MV(0, 0)
MV1 MV1
MVMV1
MV2 (0, 0)
MVMV1
: Picture or GOB border
MV : Current motion vectorMV1 : Previous motion vectorMV2 : Above motion vectorMV3 : Above right motion vector
}{}3,2,1{
MVimedianMVMVDi
H.263 Video Coding
Sep. 6, 2000Basics of Video Coding and H.263 Video Coding
26
Quantization (1/2)Quantization (1/2)
· Notation- QP(Quantization Parameter) : 1~32 사이의 정수값- Quantization step size : 2xQP- COF : 양자화 될 DCT 영역의 계수값- LEVEL : COF 를 양자화한 절대값- COF’ : 재생된 DCT 영역의 계수값
· Quantization – For INTRA : – For INTER :
)*2/(|| QPCOFLEVEL )*2(/)2/|(| QPQPCOFLEVEL
H.263 Video Coding
Sep. 6, 2000Basics of Video Coding and H.263 Video Coding
27
Quantization (2/2)Quantization (2/2)
• Dequantization
• Quantization for INTRA DC coefficient– 8 bit 가 사용– Quantization :– Dequantization :
,12|'|
,2|'|
,0|'|
QPLEVELQPCOF
QPLEVELQPCOF
COF
evenisQPLEVELif
oddisQPLEVELif
LEVELif
,0
,0
0
|'|)(' COFCOFSignCOF
8/COFLEVEL
8' LEVELCOF
H.263 Video Coding
Sep. 6, 2000Basics of Video Coding and H.263 Video Coding
28
VLC encoding of quantized transform coefficientVLC encoding of quantized transform coefficient
• Event 는 3 개의 Parameter 의 조합으로 이루어진다 . (LAST, RUN, LEVEL)– LAST
¤ 0 : Not last coefficient¤ 1 : Last nonzero coefficient
– RUN : Number of zero coefficients preceding the current
nonzero coefficient
– LEVEL : Magnitude of the coefficient
1 2 6 7 15 16 28 293 5 8 14 17 27 30 434 9 13 18 26 31 42 44
10 12 19 25 32 41 45 5411 20 24 33 40 46 53 5521 23 34 39 47 52 56 6122 35 38 48 51 57 60 6236 37 49 50 58 59 63 64
H.263 Video Coding