A hybrid error concealment A hybrid error concealment scheme for MPEG-2 video scheme for MPEG-2 video transmission based on best transmission based on best neighborhood matching algorithmneighborhood matching algorithm
Li-Wei Kang and Jin-Jang Leou
Journal of Visual Communication & Image Representation 2005
OutlineOutlineIntroductionProposed hybrid error
concealment scheme for MPEG-2 intra-coded I frames
Proposed hybrid error concealment scheme for MPEG-2 inter-coded (P or B) frames
Simulation results
Introduction (1/2)Introduction (1/2)Transmission errors and error
propagation
Network
Encoded stream
Encoded stream
Original pictures
Decoded pictures
Introduction (2/2)Introduction (2/2)Error control
◦Channel coding
◦Error resilient coding
◦Detection and correction
◦Detection and concealment
Encoded stream
Encoded stream
Encoded stream + = Encoded
stream
currentprevious future current
Encoder control
Decoder control
Proposed hybrid error Proposed hybrid error concealment scheme for concealment scheme for MPEG-2MPEG-2I frame
◦Spatial search for similar blocks◦Selection between H.26L test model
and FBNM P frame
◦Temporal search for similar blocks◦Determination of block behavior
Small motions, middle motions, and large motions
◦Search for motion vectors
Proposed hybrid error Proposed hybrid error concealment scheme for concealment scheme for MPEG-2 intra-coded I frames MPEG-2 intra-coded I frames (1/7)(1/7)H.26L test model [25]
Best neighborhood matching (BNM) [15]
a1
a2
a3
a4
4321 2222),( a
Nja
Nia
BjNa
NiNjif
N+mN+m
H
L
Best match
Proposed hybrid error Proposed hybrid error concealment scheme for concealment scheme for MPEG-2 intra-coded I frames MPEG-2 intra-coded I frames (2/7)(2/7)Proposed fast BNM
◦Step 1: Initial candidate block by MAD
◦Step 2: Hexagon-based fast search by MAD
Proposed hybrid error Proposed hybrid error concealment scheme for concealment scheme for MPEG-2 intra-coded I frames MPEG-2 intra-coded I frames (3/7)(3/7)Proposed hybrid error concealment
scheme◦Fitness function
H.26L or fast BNM? Check the smoothness of the boundaries
between the concealed block and its neighbors Check the similarity between the concealed
block and its neighbors Components
AIDB AMD AVD
Proposed hybrid error Proposed hybrid error concealment scheme for concealment scheme for MPEG-2 intra-coded I frames MPEG-2 intra-coded I frames (4/7)(4/7)Average intersample difference
across the block boundaries between a concealed block and its four neighbors (AIDB)◦Smoothness of the boundaries
1
0
1
0
1
0
1
0
ofneighbor right theis if ,)0,()1,(1
ofneighbor left theis if ,)1,()0,(1
ofneighbor bottom theis if ,),0(),1(1
ofneighbor top theis if ,),1(),0(1
),(
N
i
N
i
N
j
N
j
BXiXNiBN
BXNiXiBN
BXjXjNBN
BXjNXjBN
XBABD
Bi BNB
iB
BBABDBN
BAIDB ),(1)(
Proposed hybrid error Proposed hybrid error concealment scheme for concealment scheme for MPEG-2 intra-coded I frames MPEG-2 intra-coded I frames (5/7)(5/7)Average mean difference
between a concealed block and its eight neighbors (AMD)
Bi BNB
iB
BmeanBmeanBN
BAMD )()(||
1)(
mean
meanmean
mean
meanmeanmean
mean
mean
Proposed hybrid error Proposed hybrid error concealment scheme for concealment scheme for MPEG-2 intra-coded I frames MPEG-2 intra-coded I frames (6/7)(6/7)Average variance difference
between a concealed block and its eight neighbors (AVD)
Bi BNB
iB
BBBN
BAVD )var()var(||
1)(
Proposed hybrid error Proposed hybrid error concealment scheme for concealment scheme for MPEG-2 intra-coded I frames MPEG-2 intra-coded I frames (7/7)(7/7)Fitness function
Choose of concealed block
◦Purpose of TC: BH.26L is usually smaller than BFBNM, but it’s visual quality is not always better than BFBNM
)()()()( BAVDBAMDBAIDBBCF
o.w.
)()( if 26.26. CFBNMLH
FBNM
LHC
TBCFBCFBB
B
Proposed hybrid error Proposed hybrid error concealment scheme for concealment scheme for MPEG-2 inter-coded (P or B) MPEG-2 inter-coded (P or B) frames (1/2)frames (1/2)MV magnitude
Determination of types of the corrupted blocks
Block matching
bN
ibiaveNMVMV
1
/
Small motion blocksCandidates:MV0
MVave
Middle motion blocksCandidates:MVi for i = 1 to 8MVave
MVmedium
Large motion blocksCandidates:Fast search algorithm
Tl Th
Proposed hybrid error Proposed hybrid error concealment scheme for concealment scheme for MPEG-2 inter-coded (P or B) MPEG-2 inter-coded (P or B) frames (2/2)frames (2/2)Fast search algorithm for large
motion blocks
Search range for the corrupted block
2D+1
2D+1
Simulation results (1/8)Simulation results (1/8)MPEG-2
◦GOP size = 12◦IBBP structure◦1.5 Mbps◦30 fps◦One packet contains one slice
CIF◦Coastguard◦Football◦Salesman◦Table tennis
Simulation results (2/8)Simulation results (2/8)parameters
◦I frame N = 16; m = 4; L = H = 120
◦P or B frame Tl = 8; Th = 12; TC = 200; D = 2
N+mN+m
H
L
o.w.
)()( if 26.26. CFBNMLH
FBNM
LHC
TBCFBCFBB
B
Tl Th
Simulation results (3/8)Simulation results (3/8)Discussions
◦H and L for I frame Smaller: good candidates may be excluded Larger: bad candidates may be included
◦Tl and Th for P frame Smaller: better concealment results but high
computation cost Larger: worst concealment results but low
computation cost◦D for P frame
Smaller: worst concealment results but low computation cost
Larger: better concealment results but high computation cost
Simulation results (4/8)Simulation results (4/8)1. replace all corrupted blocks by
zeros2. I frame: H.26L; P/B frame:
overlapped region matching [21]3. I frame: split matching; P/B frame:
forward-backward block matching [22]
4. I frame: H.26L; P/B frame: neighbor matching [23]
5. Proposed
MV interpolation
Simulation results (5/8)Simulation results (5/8)Football
Salesman
Processing time (10% PLR)
12 34
Simulation results (6/8)Simulation results (6/8)Visual quality (15% PLR)
Simulation results (7/8)Simulation results (7/8)Visual quality (15% PLR)
Simulation results (8/8)Simulation results (8/8)FootballPLR = 15%
ReferencesReferences[15] Z. Wang, Y. Yu, D. Zhang, Best neighborhood matching: an
information loss restoration technique for block-based image coding systems, IEEE Trans. Image Process. 7 (7) (1998) 1056–1061.
[21] M.C. Hong, L. Kondi, H. Scwab, A.K. Katsaggelos, Video error concealment techniques, Signal Process.: Image Commun. 14 (1999) 437–492.
[22] S. Tsekeridou, I. Pitas, MPEG-2 error concealment based on block-matching principles, IEEE Trans. Circuits Syst. Video Technol. 10 (4) (2000) 646–658.
[23] S. Valente, C. Dufour, F. Groliere, D. Snook, An efficient error concealment implementation for MPEG-4 video streams, IEEE Trans. Consumer Electron. 47 (3) (2001) 568–578.
[25] Y.K. Wang, M.M. Hannuksela, V. Varsa, A. Hourunranta, M. Gabbouj, The error concealment feature in the H.26L test model, in: Proceedings IEEE International Conference on Image Processing, Rochester, NY, USA, 2002, pp. 729–732.