Distributed Video Coding
Post on 31-Dec-2015
40 Views
Preview:
DESCRIPTION
Transcript
Distributed Video CodingDistributed Video CodingDistributed Video CodingDistributed Video Coding
VLBVVLBV, Sardinia, September 16, 2005, Sardinia, September 16, 2005
Bernd GirodBernd Girod
Information Systems LaboratoryInformation Systems LaboratoryStanford UniversityStanford University
B. Girod: Distributed Video Coding 2
Outline
Foundations of distributed coding– Slepian-Wolf Theorem and practical Slepian-Wolf coding– Wyner-Ziv results and practical Wyner-Ziv coding
Low-complexity video encoding– Pixel-domain and transform-domain coding– Hash-based receiver motion estimation– Wyner-Ziv residual coding
Error-resilient video transmission– Systematic lossy joint source-channel coding– Improving the error-resiliency of MPEG (or anything else)
by Wyner-Ziv coding
B. Girod: Distributed Video Coding 3
Outline
Foundations of distributed coding– Slepian-Wolf Theorem and practical Slepian-Wolf coding– Wyner-Ziv results and practical Wyner-Ziv coding
Low-complexity video encoding– Pixel-domain and transform-domain coding– Hash-based receiver motion estimation
Error-resilient video transmission– Systematic lossy joint source-channel coding– Improving the error-resiliency of MPEG by Wyner-Ziv coding
B. Girod: Distributed Video Coding 4
Compression of Dependent Sources
Source X
Source X
Source Y
Source Y
JointDecoder
JointDecoder
X
Y
,R H X YJoint
Encoder
JointEncoder
,XY
X Y
p x y
p x p y
B. Girod: Distributed Video Coding 5
Distributed Compression of Dependent Sources
Source X
Source X
Source Y
Source Y
Encoder X
Encoder X
Encoder Y
Encoder Y
JointDecoder
JointDecoder
X
Y
??XR
??YR
,XY
X Y
p x y
p x p y
B. Girod: Distributed Video Coding 6
Slepian Wolf Theorem
[bits]XR
[bits]YR
H X
H Y
Independent decoding
Achievable rate region for i.i.d sequences
B. Girod: Distributed Video Coding 7
Slepian Wolf Theorem
[bits]XR
[bits]YR
H X
H Y
|H Y X
|H X Y
,X YR R H X Y
Joint decoding:Vanishing error probabilityfor long sequences
Independent decoding:No errors
[Slepian, Wolf, 1973]
Achievable rate region for i.i.d sequences
B. Girod: Distributed Video Coding 8
Lossless Compression with Receiver Side Information
[bits]XR
[bits]YR
H X
H Y
|H Y X
|H X Y
,X YR R H X Y
Source Encoder Decoder
Y
X
Y
|R H X Y
X
B. Girod: Distributed Video Coding 9
Distributed Compression and Channel Coding
Idea
Interpret Y as a “noisy” version of X
with “channel errors” Encoder generates “parity bits” P to
protect against errors Decoder concatenates Y and P and
performs error-correcting decoding
Idea
Interpret Y as a “noisy” version of X
with “channel errors” Encoder generates “parity bits” P to
protect against errors Decoder concatenates Y and P and
performs error-correcting decoding
SourceX|Y
Encoder Decoder
Y
X
YX Y
01001100010101
01001101010101
00000001000000
X
Y
P
B. Girod: Distributed Video Coding 10
Towards Practical Slepian-Wolf Coding
Convolution coding for data compression [Blizard, 1969, unpublished] Convolutional source coding [Hellman, 1975] Coset codes [Pradhan and Ramchandran, 1999] Trellis codes [Wang and Orchard, 2001] Turbo codes
[Garcia-Frias and Zhao, 2001]
[Bajcsy and Mitran, 2001]
[Aaron and Girod, 2002] LDPC codes [Liveris, Xiong, and Georghiades, 2002] . . . . . .
B. Girod: Distributed Video Coding 11
Slepian-Wolf Coding Using Turbo Codes
Systematic Convolution
al Code
X
Systematic Convolution
al Code
Interleaver
SISO Decode
r
SISO Decode
rDecision X
Parity bits
Parity bits
Systematicbits X . . . . . . . Y
X Y“Correlation channel”
[Aaron and Girod, 2002]
B. Girod: Distributed Video Coding 12
X
Lossy Compression with Side Information
'XSource Encoder Decoder
Y Y
X 'XSource Encoder Decoder
Y Y Y
[Wyner, Ziv, 1976] For mse distortion and Gaussian statistics, rate-distortion functions of the two systems are the same.
[Wyner, Ziv, 1976] For mse distortion and Gaussian statistics, rate-distortion functions of the two systems are the same.
B. Girod: Distributed Video Coding 13
Practical Wyner-Ziv Encoder and Decoder
Wyner-Ziv Decoder
QuantizerSlepian-
Wolf Encoder
Wyner-Ziv Encoder
Slepian-
WolfDecode
r
Minimum Distortion
Reconstruction
Y Y
X 'XQ Q
B. Girod: Distributed Video Coding 14
Non-Connected Quantization Regions
Example: Non-connected intervals for scalar quantization
Decoder: Minimum mean-squared error reconstruction with side information
x
1q 2q 3q
x
| |X Yf x y 2
ˆ
conditional centroid
ˆ ˆ | ,arg min x
x E X x y q
B. Girod: Distributed Video Coding 15
Outline
Foundations of distributed coding– Slepian-Wolf Theorem and practical Slepian-Wolf coding– Wyner-Ziv results and practical Wyner-Ziv coding
Low-complexity video encoding– Pixel-domain and transform-domain coding– Hash-based receiver motion estimation
Error-resilient video transmission– Systematic lossy joint source-channel coding– Improving the error-resiliency of MPEG by Wyner-Ziv coding
B. Girod: Distributed Video Coding 16
Interframe Video Coding
PredictiveInterframe Decoder
PredictiveInterframe Encoder
X’
Side Information
YX Y
B. Girod: Distributed Video Coding 17
Video Coding with Low Complexity
Wyner-ZivInterframe Decoder
Wyner-ZivIntraframe Encoder
X’
Side Information
YX
[Witsenhausen, Wyner, 1978][Puri, Ramchandran, Allerton 2002][Aaron, Zhang, Girod, Asilomar 2002]
B. Girod: Distributed Video Coding 19
Low Complexity Encoding and Decoding
B. Girod: Distributed Video Coding 20
Pixel Domain Wyner-Ziv Coder
Interframe DecoderIntraframe Encoder
Reconstruction X’
Y
Video frame
XScalar
QuantizerTurbo
EncoderBuffer Turbo
Decoder
Request bits
Slepian-Wolf Codec
InterpolationKey frames
previous
next[Aaron, Zhang, Girod, Asilomar 2002][Aaron, Rane, Zhang, Girod, DCC 2003]
B. Girod: Distributed Video Coding 21
Decoder side informationgenerated by motion-
compensated interpolationPSNR 30.3 dB
After Wyner-Ziv Decoding16-level quantization – 1.375 bpp
11 pixels in errorPSNR 36.7 dB
Pixel Domain Wyner-Ziv Coder
B. Girod: Distributed Video Coding 22
Pixel Domain Wyner-Ziv Coder
Decoder side informationgenerated by motion-
compensated interpolationPSNR 24.8 dB
After Wyner-Ziv Decoding16-level quantization – 2.0 bpp
0 pixels in errorPSNR 36.5 dB
B. Girod: Distributed Video Coding 23
Stanford Camera Array
Courtesy Marc Levoy, Stanford Computer Graphics Lab
B. Girod: Distributed Video Coding 24
…WZ-ENC
WZ-DEC
WZ-ENC
WZ-DEC
…
GeometryReconstruction
Rendering
Wyner-Ziv Cameras Conventional Cameras
Distributed Compression
Distributed Encoding
Joint Decoding
[Zhu, Aaron, Girod, 2003]
B. Girod: Distributed Video Coding 25
Light Field Compression
Rate: 0.11 bppPSNR 39.9 dB
Rate: 0.11 bppPSNR 37.4 dB
Wyner-Ziv, Pixel-Domain JPEG-2000
B. Girod: Distributed Video Coding 26
DCT-Domain Wyner-Ziv Video Encoder
For each low frequency coefficient band k
level Quantizer
DCTkM2 Turbo
EncoderExtract bit-
planes
bit-plane 1
bit-plane 2
bit-plane Mk
…Inputvideo frame
QuantizerEntropy Coder
Comparison
Previous-frame quantized high freq coefficients
Wyner-Ziv parity bits
High frequency
bits
Low freq coeffs
High freq coeffs
B. Girod: Distributed Video Coding 27
IDCT
Wyner-Ziv Video Decoder with Motion Compensation
Reconstruction
DCT
Entropy Decoder and
Inverse Quantizer
Side information
Wyner-Ziv parity bits
High frequency
bits
Turbo Decoder
Motion-compensated Extrapolation
Previous frame
DCT
Refinedside information
ExtrapolationRefinement
For each low frequency band Decoded frame
Reconstructed high frequency coefficients
B. Girod: Distributed Video Coding 28
Rate-Distortion Performance - Salesman
Every 8th frame is a key frame
Salesman QCIF sequence at 10fps 100 frames
6 dB
3 dB
B. Girod: Distributed Video Coding 29
Rate-Distortion Performance – Hall Monitor
8 dB
3 dB
Every 8th frame is a key frame
Hall Monitor QCIF sequence at 10fps 100 frames
B. Girod: Distributed Video Coding 30
Rate-Distortion Performance – Foreman
2 dB
1.5 dB
Every 8th frame is a key frame
Foreman QCIF sequence at 10fps 100 frames
DCT-based Intracoding 149 kbps
PSNRY=30.0 dB
Wyner-Ziv DCT codec 152 kbps
PSNRY=35.6 dB GOP=8
Salesman at 10 fps
DCT-based Intracoding 156 kbps
PSNRY=30.2 dB
Wyner-Ziv DCT codec 155 kbps
PSNRY=37.1 dB GOP=8
Hall Monitor at 10 fps
DCT-based Intracoding 290 kbps
PSNRY=34.4 dB
Wyner-Ziv DCT codec 293 kbps
PSNRY=35.5 dB GOP=8
Foreman at 10 fps
B. Girod: Distributed Video Coding 34
Wyner-Ziv Residual Coding
Wyner-ZivDecoder
Wyner-Ziv Encoder
Side Information
Y X’n
X’n-1
Side Information
-Xn
X’n-1
Frame differenceXn
B. Girod: Distributed Video Coding 35
Rate-Distortion Performance – Foreman
Every 8th frame is a key frame
Foreman QCIF sequence at 30fps 100 frames
B. Girod: Distributed Video Coding 36
Rate-Distortion Performance – Foreman
Every 8th frame is a key frame
Foreman QCIF sequence at 30fps 100 frames
B. Girod: Distributed Video Coding 37
Outline
Foundations of distributed coding– Slepian-Wolf Theorem and practical Slepian-Wolf coding– Wyner-Ziv results and practical Wyner-Ziv coding
Low-complexity video encoding– Pixel-domain and transform-domain coding– Hash-based receiver motion estimation
Error-resilient video transmission– Systematic lossy joint source-channel coding– Improving the error-resiliency of MPEG by Wyner-Ziv coding
B. Girod: Distributed Video Coding 38
Systematic Lossy Error Protection (SLEP)of Compressed Video
Any OldVideo
Encoder
Video Decoder with Error
Concealment
Err
or-P
rone
cha
nnel
S S’
Wyner-Ziv Decoder A S*
Wyner-Ziv Encoder A
Wyner-Ziv Decoder B S**Wyner-Ziv
Encoder B
Graceful degradation without a layered signal representation
Analog channel (uncoded)
[Aaron, Rane, Girod, ICIP 2003]
B. Girod: Distributed Video Coding 39
MPEG with Systematic Lossy Error Protection
Cha
nnel
Slepian-WolfEncoder
Wyner-Ziv Encoder
ED T-1Q-1 +
MC
S*MPEGEncoder
main
S
Side information
MPEGEncoder
coarse
T-1q-1ED +
MC
S’
R-SDecoder
ReconstructedFrame atEncoder
MPEGEncoder
coarse
R-SEncoder
[Rane, Aaron, Girod, VCIP 2004]
Parityonly
B. Girod: Distributed Video Coding 40
Reed-Solomon Coding Across Slices
1 byte in slice
filler byte
parity byte
RS code across slices
Transmit parity slices only
B. Girod: Distributed Video Coding 41
Results: CIF Foreman
Main Stream @ 1.092 MbpsFEC (n,k) = (40,36) FEC bitrate = 120 KbpsTotal = 1.2 Mbps
WZ Stream @ 270 KbpsSLEP (n,k) = (52,36)WZ bitrate = 120 KbpsTotal = 1.2 Mbps
SLEP
FEC
FECSLEP
B. Girod: Distributed Video Coding 42
MPEG with Systematic Lossy Error Protection
Cha
nnel
Slepian-WolfEncoder
Wyner-Ziv Encoder
ED T-1Q-1 +
MC
S*MPEGEncoder
main
S
Side information
MPEGEncoder
coarse
T-1q-1ED +
MC
S’
R-SDecoder
ReconstructedFrame atEncoder
MPEGEncoder
coarse
R-SEncoder
[Rane, Aaron, Girod, VCIP 2004]
Parityonly
B. Girod: Distributed Video Coding 43
Quantized transformed Prediction error
Coarse Quantizer
Entropy CodingQ1
Q-1
Quantization parameter (Q)
MPEG2Encoder
Conventionally encoded streamInputVideo
Err
or-
pro
ne
Ch
an
ne
l
Entropy Decoding
MPEG2Decoder
T-1 +
MC
LEGACY BROADCASTING SYSTEM
WYNER-ZIV ENCODER WYNER-ZIV DECODER
RS Decoder
Fallback to coarser version
Decoded motion vecs
Entropy Decoding
-11Q
Parityonly
RS Encoder
Side Information (motion vectors,mode decisions)
SLEP MPEG Codec with Simple Decoder
Q1
Entropy Coding
Sid
e I
nfo
(mo
tion
ve
cs,
mo
de
de
cisi
on
s)
B. Girod: Distributed Video Coding 44
Performance at Symbol Error Rate 10-4
MPEG-2 video: 2 Mbps+ FEC 222 Kbps(PSNR 29.78 dB)
MPEG-2 video: 2 Mbps+ Wyner-Ziv 222 Kbps
(PSNR 35.45 dB)
B. Girod: Distributed Video Coding 45
Distributed Coding of Video:Why Should We Care?
Chance to reinvent compression from scratch– Entropy coding– Quantization– Signal transforms– Adaptive coding– Rate control– . . .
Enables new compression applications– Very low complexity encoders– Compression for networks of cameras– Error-resilient transmission of signal waveforms– Digitally enhanced analog transmission– Unequal error protection without layered coding– . . .
The EndThe EndFurther interest:Further interest:
B. Girod, A. Aaron, S. Rane, D. Rebollo-Monedero, "Distributed Video Coding," B. Girod, A. Aaron, S. Rane, D. Rebollo-Monedero, "Distributed Video Coding," Proceedings of the IEEE,Proceedings of the IEEE, Special Issue on Video Coding and Delivery. Special Issue on Video Coding and Delivery. January 2005.January 2005.
http://www.stanford.edu/~bgirod/pdfs/DistributedVideoCoding-IEEEProc.pdf http://www.stanford.edu/~bgirod/pdfs/DistributedVideoCoding-IEEEProc.pdf
top related