Yu Liu and King Ngi Ngan Department of Electronic Engineering, The Chinese University of Hong Kong

11

Anisotropic Double Cross Search Algorithm Anisotropic Double Cross Search Algorithm using Multiresolution-Spatio-Temporal Context using Multiresolution-Spatio-Temporal Context

for Fast Lossy In-Band Motion Estimationfor Fast Lossy In-Band Motion Estimation

Yu Liu and King Ngi NganYu Liu and King Ngi Ngan

Department of Electronic Engineering,Department of Electronic Engineering,The Chinese University of Hong KongThe Chinese University of Hong Kong

PCS2006, April 24-26, Beijing, ChinaPCS2006, April 24-26, Beijing, China

22

OutlineOutline

IntroductionIntroduction BackgroundBackground Proposed AlgorithmProposed Algorithm Experimental ResultsExperimental Results ConclusionConclusion

33

IntroductionIntroduction

• Motion Estimation in Critically-Sampled Wavelet Domain• Pro: basically free form the blocking effects• Con: inefficient in high bands

• Motion Estimation in Shift-Invariant Wavelet Domain• Pro: perform ME more precisely and efficiently• Con: computational complexity

e.g. low-band-shift (LBS) and complete-to-overcomplete DWT (CODWT)

44

BackgroundBackground

Motion Estimation in Shift-Invariant Wavelet Domain (1)Motion Estimation in Shift-Invariant Wavelet Domain (1)

• Two Level Shift-Invariant Wavelet Decomposition by using Low-Band-Shift (LBS) or Complete-to-Overcomplete DWT (CODWT)

55

BackgroundBackground

Motion Estimation in Shift-Invariant Wavelet Domain (2)Motion Estimation in Shift-Invariant Wavelet Domain (2)

• Generation of Wavelet BlocksGeneration of Wavelet Blocks

jifjiS pkltpklt ,),( ,,,,,,

• The coefficient of the pth wavelet block of current frame t can be represented by

• The v-pixel-shifted or {dx,dy}-pixel-shifted coefficient of the pth wavelet block of reference frame t’ can be represented by

llllpkltvpklt dyjdxidydxfjiS 2/,2/,2%,2%),( ,,,',,,'

66

BackgroundBackground

Motion Estimation in Shift-Invariant Wavelet Domain (3)Motion Estimation in Shift-Invariant Wavelet Domain (3)

• The sum of absolute difference (SAD) of the pth wavelet block for the motion vector v is computed as follows:

• The optimum motion vector v ∗ of the pth wavelet block, which has minimum displacement error, is given by:

77

BackgroundBackground

Anisotropic Motion Model in Wavelet DomainAnisotropic Motion Model in Wavelet Domain

• Traditional 2D ME in spatial domain

• suffers from the aperture problem

• 2D ME in wavelet domain

• the aperture problem can in fact be exploited as an advantage.

Aperture 1 Aperture 2

Aperture 3

Normal flow

(a)

Normal flow

LL LH

HL HH

(b)

(a) Aperture problem in spatial domain, (b) Anisotropic motion model in wavelet domain

88

Proposed AlgorithmProposed Algorithm

Multiresolution-Spatio-Temporal Context (1)Multiresolution-Spatio-Temporal Context (1)

(a) multiresolution context, (b) spatial context, (c) temporal context

• Traditional MRME algorithmsTraditional MRME algorithms• Multiresolution contextMultiresolution context

• Not enough for reducing the risk of getting trapped into a local minimum.Not enough for reducing the risk of getting trapped into a local minimum.

• The proposed algorithmThe proposed algorithm• Multiresolution-spatio-temporal ContextMultiresolution-spatio-temporal Context

• Consists of one multiresolution context, four spatial contexts, and five temConsists of one multiresolution context, four spatial contexts, and five temporal contexts.poral contexts.

c1

c2

X

c3 c4 c7

c5c6 c8

c9

c0

(a) (b) (c)

99

Proposed AlgorithmProposed Algorithm

Multiresolution-Spatio-Temporal Context (2)Multiresolution-Spatio-Temporal Context (2)

• For LL subbandFor LL subband• Initialization: Initialization: spatio-temporal context, plus the spatio-temporal context, plus the

candidate points in shifted LL subband, candidate points in shifted LL subband, where the median predictor is located where the median predictor is located

• Refinement:Refinement: diamond search algorithmdiamond search algorithm

• For other levelsFor other levels• Initialization:Initialization: multiresolution-spatio-temporal context multiresolution-spatio-temporal context

• Refinement: Refinement: anisotropic double cross search algorithmanisotropic double cross search algorithm

1010

Proposed AlgorithmProposed Algorithm

Anisotropic Double Cross Search Algorithm (1)Anisotropic Double Cross Search Algorithm (1)

• Anisotropic motion model suggests that the 2D ME problem in wavelet domain can be approximated by 1D ME along the normal flow direction for the vertical/horizontal subbands.

• During the 1D window searching, only the coefficients in the corresponding subbands and LL subband are computed.

1111

Proposed AlgorithmProposed Algorithm

Anisotropic Double Cross Search Algorithm (2)Anisotropic Double Cross Search Algorithm (2)

(a) (b)

LHyd

LHyd

HLxd

HLxd

LHyd LHyd

HLxd

HLxd

(c)

HLxd

HLxd

LHyd LHyd

Starting point or the center of the searchChecking points for HL subband in the first cross search routeChecking points for LH subband in the first cross search routeChecking points for LH subband in the second cross search routeChecking points for HL subband in the second cross search route

Best matching point obtained from the second cross search routeBest matching point obtained from the first cross search route

1212

Experimental Results (1)Experimental Results (1)

• PSNR

• MAD

• operation number

• speed-up ratio

Simulation results are reported in the following ways:

For performance comparison

• Full Search Algorithm (FSA)

• FMRME [6]

• FIBME [7]

• proposed MR-STC-ADCS

1313

Experimental Results (2)Experimental Results (2)

• Comparison of the Tested Algorithms for QCIF Video Sequences

1414

Experimental Results (3)Experimental Results (3)

• Comparison of the Tested Algorithms for CIF Video Sequences

1515

Experimental Results (4)Experimental Results (4)

• Comparison of the Tested Algorithms for 4CIF Video Sequences

On average, for all sequences examined in the experimental tests:

MR-STC-ADCS is roughly 11.5 and 2.6 times faster whereas its PSNR is approximately 1.46 dB and 0.6 dB higher than FMRME and FIBME; and its MAD is approximately 0.426 and 0.165 lower than FMRME and FIBME.

MR-STC-ADCS is about 271 times faster than FSA for QCIF, 667 times for CIF, and 1313 times for 4CIF, while having an average PSNR loss of only 0.04 dB or an average MAD increase of only 0.018 compared to the FSA.

1616

ConclusionConclusion

Fast Lossy In-Band Motion Estimation Algorithm• Anisotropic property of the motion field in shif

t-invariant wavelet domain• Multiresolution-spatio-temporal Context• Anisotropic Double Cross Search