Distributed Video Coding

Distributed Video CodingDistributed Video CodingDistributed Video CodingDistributed Video Coding

VLBVVLBV, Sardinia, September 16, 2005, Sardinia, September 16, 2005

Bernd GirodBernd Girod

Information Systems LaboratoryInformation Systems LaboratoryStanford UniversityStanford University

B. Girod: Distributed Video Coding 2

Outline

Foundations of distributed coding– Slepian-Wolf Theorem and practical Slepian-Wolf coding– Wyner-Ziv results and practical Wyner-Ziv coding

Low-complexity video encoding– Pixel-domain and transform-domain coding– Hash-based receiver motion estimation– Wyner-Ziv residual coding

Error-resilient video transmission– Systematic lossy joint source-channel coding– Improving the error-resiliency of MPEG (or anything else)

by Wyner-Ziv coding


Outline


Low-complexity video encoding– Pixel-domain and transform-domain coding– Hash-based receiver motion estimation

Error-resilient video transmission– Systematic lossy joint source-channel coding– Improving the error-resiliency of MPEG by Wyner-Ziv coding


Compression of Dependent Sources

Source X

Source X

Source Y

Source Y

JointDecoder

JointDecoder

X

Y

,R H X YJoint

Encoder

JointEncoder

,XY

X Y

p x y

p x p y


Distributed Compression of Dependent Sources

Source X

Source X

Source Y

Source Y

Encoder X

Encoder X

Encoder Y

Encoder Y

JointDecoder

JointDecoder

X

Y

??XR

??YR

,XY

X Y

p x y

p x p y


Slepian Wolf Theorem

[bits]XR

[bits]YR

H X

H Y

Independent decoding

Achievable rate region for i.i.d sequences


Slepian Wolf Theorem

[bits]XR

[bits]YR

H X

H Y

|H Y X

|H X Y

,X YR R H X Y

Joint decoding:Vanishing error probabilityfor long sequences

Independent decoding:No errors

[Slepian, Wolf, 1973]

Achievable rate region for i.i.d sequences


Lossless Compression with Receiver Side Information

[bits]XR

[bits]YR

H X

H Y

|H Y X

|H X Y

,X YR R H X Y

Source Encoder Decoder

Y

X

Y

|R H X Y

X


Distributed Compression and Channel Coding

Idea

Interpret Y as a “noisy” version of X

with “channel errors” Encoder generates “parity bits” P to

protect against errors Decoder concatenates Y and P and

performs error-correcting decoding

Idea

Interpret Y as a “noisy” version of X

with “channel errors” Encoder generates “parity bits” P to

protect against errors Decoder concatenates Y and P and

performs error-correcting decoding

SourceX|Y

Encoder Decoder

Y

X

YX Y

01001100010101

01001101010101

00000001000000

X

Y

P


Towards Practical Slepian-Wolf Coding

Convolution coding for data compression [Blizard, 1969, unpublished] Convolutional source coding [Hellman, 1975] Coset codes [Pradhan and Ramchandran, 1999] Trellis codes [Wang and Orchard, 2001] Turbo codes

[Garcia-Frias and Zhao, 2001]

[Bajcsy and Mitran, 2001]

[Aaron and Girod, 2002] LDPC codes [Liveris, Xiong, and Georghiades, 2002] . . . . . .


Slepian-Wolf Coding Using Turbo Codes

Systematic Convolution

al Code

X

Systematic Convolution

al Code

Interleaver

SISO Decode

r

SISO Decode

rDecision X

Parity bits

Parity bits

Systematicbits X . . . . . . . Y

X Y“Correlation channel”

[Aaron and Girod, 2002]


X

Lossy Compression with Side Information

'XSource Encoder Decoder

Y Y

X 'XSource Encoder Decoder

Y Y Y

[Wyner, Ziv, 1976] For mse distortion and Gaussian statistics, rate-distortion functions of the two systems are the same.

[Wyner, Ziv, 1976] For mse distortion and Gaussian statistics, rate-distortion functions of the two systems are the same.


Practical Wyner-Ziv Encoder and Decoder

Wyner-Ziv Decoder

QuantizerSlepian-

Wolf Encoder

Wyner-Ziv Encoder

Slepian-

WolfDecode

r

Minimum Distortion

Reconstruction

Y Y

X 'XQ Q


Non-Connected Quantization Regions

Example: Non-connected intervals for scalar quantization

Decoder: Minimum mean-squared error reconstruction with side information

x

1q 2q 3q

x

| |X Yf x y 2

ˆ

conditional centroid

ˆ ˆ | ,arg min x

x E X x y q


Outline





Interframe Video Coding

PredictiveInterframe Decoder

PredictiveInterframe Encoder

X’

Side Information

YX Y


Video Coding with Low Complexity

Wyner-ZivInterframe Decoder

Wyner-ZivIntraframe Encoder

X’

Side Information

YX

[Witsenhausen, Wyner, 1978][Puri, Ramchandran, Allerton 2002][Aaron, Zhang, Girod, Asilomar 2002]


Low Complexity Encoding and Decoding


Pixel Domain Wyner-Ziv Coder

Interframe DecoderIntraframe Encoder

Reconstruction X’

Y

Video frame

XScalar

QuantizerTurbo

EncoderBuffer Turbo

Decoder

Request bits

Slepian-Wolf Codec

InterpolationKey frames

previous

next[Aaron, Zhang, Girod, Asilomar 2002][Aaron, Rane, Zhang, Girod, DCC 2003]


Decoder side informationgenerated by motion-

compensated interpolationPSNR 30.3 dB

After Wyner-Ziv Decoding16-level quantization – 1.375 bpp

11 pixels in errorPSNR 36.7 dB




Decoder side informationgenerated by motion-

compensated interpolationPSNR 24.8 dB

After Wyner-Ziv Decoding16-level quantization – 2.0 bpp

0 pixels in errorPSNR 36.5 dB


Stanford Camera Array

Courtesy Marc Levoy, Stanford Computer Graphics Lab


…WZ-ENC

WZ-DEC

WZ-ENC

WZ-DEC

…

GeometryReconstruction

Rendering

Wyner-Ziv Cameras Conventional Cameras

Distributed Compression

Distributed Encoding

Joint Decoding

[Zhu, Aaron, Girod, 2003]


Light Field Compression

Rate: 0.11 bppPSNR 39.9 dB

Rate: 0.11 bppPSNR 37.4 dB

Wyner-Ziv, Pixel-Domain JPEG-2000


DCT-Domain Wyner-Ziv Video Encoder

For each low frequency coefficient band k

level Quantizer

DCTkM2 Turbo

EncoderExtract bit-

planes

bit-plane 1

bit-plane 2

bit-plane Mk

…Inputvideo frame

QuantizerEntropy Coder

Comparison

Previous-frame quantized high freq coefficients

Wyner-Ziv parity bits

High frequency

bits

Low freq coeffs

High freq coeffs


IDCT

Wyner-Ziv Video Decoder with Motion Compensation

Reconstruction

DCT

Entropy Decoder and

Inverse Quantizer

Side information

Wyner-Ziv parity bits

High frequency

bits

Turbo Decoder

Motion-compensated Extrapolation

Previous frame

DCT

Refinedside information

ExtrapolationRefinement

For each low frequency band Decoded frame

Reconstructed high frequency coefficients


Rate-Distortion Performance - Salesman

Every 8th frame is a key frame

Salesman QCIF sequence at 10fps 100 frames

6 dB

3 dB


Rate-Distortion Performance – Hall Monitor

8 dB

3 dB


Hall Monitor QCIF sequence at 10fps 100 frames


Rate-Distortion Performance – Foreman

2 dB

1.5 dB


Foreman QCIF sequence at 10fps 100 frames

DCT-based Intracoding 149 kbps

PSNRY=30.0 dB

Wyner-Ziv DCT codec 152 kbps

PSNRY=35.6 dB GOP=8

Salesman at 10 fps


PSNRY=30.2 dB


PSNRY=37.1 dB GOP=8

Hall Monitor at 10 fps


PSNRY=34.4 dB


PSNRY=35.5 dB GOP=8

Foreman at 10 fps


Wyner-Ziv Residual Coding

Wyner-ZivDecoder

Wyner-Ziv Encoder

Side Information

Y X’n

X’n-1

Side Information

-Xn

X’n-1

Frame differenceXn










Outline





Systematic Lossy Error Protection (SLEP)of Compressed Video

Any OldVideo

Encoder

Video Decoder with Error

Concealment

Err

or-P

rone

cha

nnel

S S’

Wyner-Ziv Decoder A S*

Wyner-Ziv Encoder A

Wyner-Ziv Decoder B S**Wyner-Ziv

Encoder B

Graceful degradation without a layered signal representation

Analog channel (uncoded)

[Aaron, Rane, Girod, ICIP 2003]


MPEG with Systematic Lossy Error Protection

Cha

nnel

Slepian-WolfEncoder

Wyner-Ziv Encoder

ED T-1Q-1 +

MC

S*MPEGEncoder

main

S

Side information

MPEGEncoder

coarse

T-1q-1ED +

MC

S’

R-SDecoder

ReconstructedFrame atEncoder

MPEGEncoder

coarse

R-SEncoder

[Rane, Aaron, Girod, VCIP 2004]

Parityonly


Reed-Solomon Coding Across Slices

1 byte in slice

filler byte

parity byte

RS code across slices

Transmit parity slices only


Results: CIF Foreman

Main Stream @ 1.092 MbpsFEC (n,k) = (40,36) FEC bitrate = 120 KbpsTotal = 1.2 Mbps

WZ Stream @ 270 KbpsSLEP (n,k) = (52,36)WZ bitrate = 120 KbpsTotal = 1.2 Mbps

SLEP

FEC

FECSLEP


MPEG with Systematic Lossy Error Protection

Cha

nnel

Slepian-WolfEncoder

Wyner-Ziv Encoder

ED T-1Q-1 +

MC

S*MPEGEncoder

main

S

Side information

MPEGEncoder

coarse

T-1q-1ED +

MC

S’

R-SDecoder

ReconstructedFrame atEncoder

MPEGEncoder

coarse

R-SEncoder

[Rane, Aaron, Girod, VCIP 2004]

Parityonly


Quantized transformed Prediction error

Coarse Quantizer

Entropy CodingQ1

Q-1

Quantization parameter (Q)

MPEG2Encoder

Conventionally encoded streamInputVideo

Err

or-

pro

ne

Ch

an

ne

l

Entropy Decoding

MPEG2Decoder

T-1 +

MC

LEGACY BROADCASTING SYSTEM

WYNER-ZIV ENCODER WYNER-ZIV DECODER

RS Decoder

Fallback to coarser version

Decoded motion vecs

Entropy Decoding

-11Q

Parityonly

RS Encoder

Side Information (motion vectors,mode decisions)

SLEP MPEG Codec with Simple Decoder

Q1

Entropy Coding

Sid

e I

nfo

(mo

tion

ve

cs,

mo

de

de

cisi

on

s)


Performance at Symbol Error Rate 10-4

MPEG-2 video: 2 Mbps+ FEC 222 Kbps(PSNR 29.78 dB)

MPEG-2 video: 2 Mbps+ Wyner-Ziv 222 Kbps

(PSNR 35.45 dB)


Distributed Coding of Video:Why Should We Care?

Chance to reinvent compression from scratch– Entropy coding– Quantization– Signal transforms– Adaptive coding– Rate control– . . .

Enables new compression applications– Very low complexity encoders– Compression for networks of cameras– Error-resilient transmission of signal waveforms– Digitally enhanced analog transmission– Unequal error protection without layered coding– . . .

The EndThe EndFurther interest:Further interest:

B. Girod, A. Aaron, S. Rane, D. Rebollo-Monedero, "Distributed Video Coding," B. Girod, A. Aaron, S. Rane, D. Rebollo-Monedero, "Distributed Video Coding," Proceedings of the IEEE,Proceedings of the IEEE, Special Issue on Video Coding and Delivery. Special Issue on Video Coding and Delivery. January 2005.January 2005.

http://www.stanford.edu/~bgirod/pdfs/DistributedVideoCoding-IEEEProc.pdf http://www.stanford.edu/~bgirod/pdfs/DistributedVideoCoding-IEEEProc.pdf

Distributed Video Coding

Documents

distributed video codingvlbv

wynerziv codingb

practical wynerziv encoder

lossy compression

lossless compression

data compression blizard

channel errors dencoder

channel codingideainterpret