Interframe Video Compression

1

Department of Electrical Engineering,Stanford University

EE 392J Final Project PresentationShantanu Rane

Hash-Aided Motion Estimation & Rate ControlFor

Distributed Video Coding

S. Rane – Hash-Aided Motion Estimation & Rate Control – EE 392J Project, March 10, 2004 2

Interframe Video Compression

Interframe predictive coding for compression

Encoder is 5-10 times more complex than decoder

X’i-1

Interframe Encoder

Interframe Decoder

Xi Xi’

Standard codec

S. Rane – Hash-Aided Motion Estimation & Rate Control – EE 392J Project, March 10, 2004 3

Low-complexity Video Coding

Wyner-Ziv Coding Lossy source coding with decoder side information

Interframe Decoder

Intraframe EncoderXi

Xi-1’

Xi’

Side Information

Low-complexity encoding, possibly more complex decoding

Applications: video sensors for surveillance, wireless PC cameras, mobile camera-phones

Imagine dependence channel between source and side-info. WZ coding = sending parity symbols to correct errors in the dependence channel.

S. Rane – Hash-Aided Motion Estimation & Rate Control – EE 392J Project, March 10, 2004 4

Outline

How to improve side-information at decoder w/o access to current frame? Hash-Aided Motion Estimation

How to decide encoding bit-rate needed to correct errors in dependence channel? Hash-Aided Rate Control

Project motivated by Wyner-Ziv video codec architecture described in [Aaron et al., VCIP ’04, ICIP ’03].

S. Rane – Hash-Aided Motion Estimation & Rate Control – EE 392J Project, March 10, 2004 5

Generating the Hash

Visual hash originally used in digital watermarking [Fridrich, 1997]

E.g., Generate a 16 bit hash for an image-block of size NxN. Generate 16 NxN matrices with elements i.i.d. ~ U[0,1]. Low-pass filter each matrix. (e.g., repeated 2x2 averaging.)

Write out image block as vector Write out each low-pass pattern as vector Find inner products

21 NI

16,...,1, iLi

16,...,1, T iLIP ii0 else ,1,)abs( If iii HHTP

……

S. Rane – Hash-Aided Motion Estimation & Rate Control – EE 392J Project, March 10, 2004 6

Hash-aided Motion Estimation

Projecting onto low frequency patterns is equivalent to finding the low-frequency DCTcoefficients. Similar looking blocks have almost equal low-frequency DCT coefficientsand produce almost similar hashes.

Frame n-1 Frame n

Side info

UnavailableAvailable

B=00110110 received from encoderas helper informationA=00111110

Motion CompensatedSide info

S. Rane – Hash-Aided Motion Estimation & Rate Control – EE 392J Project, March 10, 2004 7

Motion-Estimation Results

2.5 dB

3.3 dB

S. Rane – Hash-Aided Motion Estimation & Rate Control – EE 392J Project, March 10, 2004 8

Motion-Estimation Results

Foreman QCIF Frames 301 to 350Error between current frame and side information

With previous frameAs side info

With hash-aided motionestimation with excess

bit-rate = 0.14 bpp

S. Rane – Hash-Aided Motion Estimation & Rate Control – EE 392J Project, March 10, 2004 9

Hash-Aided Rate Control

Encoder can perform motion-estimation at low-complexity, with small hash store.

Experimentally found that there is a high correlation between MSE of MC prediction and parity bitrate, which the encoder needs to transmit.

For given code parameters, can characterize the required bitrate as a function of MSE of MC prediction.

S. Rane – Hash-Aided Motion Estimation & Rate Control – EE 392J Project, March 10, 2004 10

Rate Control Results

Avg Hash bitrate= 0.175 bpp

Avg WZ bitrate= 1.27 bpp

S. Rane – Hash-Aided Motion Estimation & Rate Control – EE 392J Project, March 10, 2004 11

Conclusions

Hash-aided motion estimation can be used at the decoder to improve the quality of the side-information at very low bit-rate overhead. This reduces the overall Wyner-Ziv bit-rate.

With a small complexity addition, motion estimation can be performed at the encoder. This enables the encoder to control the Wyner-Ziv bit-rate depending upon the quality of the MC prediction.

S. Rane – Hash-Aided Motion Estimation & Rate Control – EE 392J Project, March 10, 2004 12

Wyner-ZivEncoder

Wyner-ZivDecoder

Input currentvideo frame

Side Info: previous decoded frame(s)+ Helper information from encoder

Hash-aided motion

estimation

Motion compensated prediction (extrapolation)

Frames encoded independently Frames decoded conditionally

DecodedFrame

Hash-aidedrate control