研究生：何銘哲 指導教授：蔣依吾博士 中山大學資訊工程學系

H.264/AVC基於影像複雜度與提早結束之快速

階層運動估計方法

Content-Based Hierarchical Fast Motion Estimation with

Early Termination in H.264/AVC

研究生：何銘哲指導教授：蔣依吾博士中山大學資訊工程學系

Terms

• FME = Fast Motion Estimation

• MB = Macroblock

• MVCOST = Motion Vector Cost

• PSNR = Peak Signal-to-Noise Ratio

• RDO = Rate Distortion Optimization

Video Compression Intro.

Video Compression Intro. (cont.)• ITU-T

– H.26x

• ISO

– Mpeg-x

Video Compression Indicator

• PSNR (dB)–

• Bit Rate (kbit/s)–

• Time (msec, sec)–

Quality

Compression ratio

Encoding time

Level Two

Level One

H.264 vs previous standards

• Variable block size

16x16 16x8 8x16 8x8

8x8 8x4 4x8 4x4

H.264 vs previous standards

• Quarter-pel motion compensation

H.264 vs previous standards

• Multiple reference picture motion compensation

H.264 vs previous standards

• Directional spatial prediction for intra coding

1. INTRA_4x4

2. INTRA_16x16

H.264 vs previous standards

• In-the-loop deblocking filtering

Visual Comparison(MPEG-4 Simple Profile VS JVT/H.264)

H.264 Encoder Dataflow

Motion Estimation

Block-based Matching Algorithms• Full Search (Exhaustive Search)

– Exhaustively searching each pixel in the search range.

43 56 76 78 89 31 34 54

44 35 66 75 34 22 35 90

54 33 45 66 48 37 44 57

73 76 50 53 50 18 36 43

49 61 55 65 35 53 32 29

83 124 100 110 52 64 65 46

98 101 99 105 55 34 45 13

75 89 83 72 68 56 44 23

3454104100

6452110102

43346555

20515350

Current Block

Search range Minimum MSE

Block-based FME in H.264

• Why FME?

• Spatial and Temporal analysis

– Spatial Homogeneity and Temporal Stationarity

[Pan, 2005]

– Motion Vector Merging [Chen, 2005]

Block-based FME in H.264 (cont.)

• Early Termination

– Predicted Vector Block [Yang, 2005]

Block-based FME in H.264 (cont.)

• Adaptive Search Pattern

– Adaptive Search Window [Bailo, 2004]

– UMHS(Uneven Multi-Hexagon Search) & CBDS(Center-Biased Diamond Search)

[Chen, 2002][Tham, 1998]

Proposed Algorithm

• Fast Hierarchical Motion Search (FHMS)

• Early Termination

Encode one MB

Mode Decision

Motion Estimation

RDO

FHMS

Hierarchical Motion Search (HMS)• Sub-sampling

Pre-processing

• We apply our sub-sampling algorithm to only four larger block types to prevent imprecision.

Pre-processing (cont.)Start FHMS

Satisfy

?Yes No

Fast Sub-sampling Search Fast Integer-pel Search

16x16 16x8 8x16 8x8

Proposed Algorithm (cont.)

• Step 1:

Utilize spatial median prediction schemes for initial search point prediction

Current MB

b c

a

• Step 2-1:

After the initial search point is found, two local search patterns, namely, diamond and cross search, are initiated.

• Step 2-2:

After local search is finished, we come across a complex search condition which indicates that if current MVCOST is below some empirical threshold T, the search process is terminated immediately.

complex_offset[blocktype] = {0, 0, 1, 1, 2, 3, 3, 1}

• Step 3:

If the complex search condition is satisfied, continue other search patterns as in the data flow diagram below.

No

Yes

YesNo

STOP

Satisfy Complex Search Condition

Need Refinement?

Start : Check Predictors

Small Local Search

Cross Search

Multi big Hexagon Search

Hexagon search

Small Local Search

Refinement Search

• Step 4:

After all search patterns are performed, if the current block type belongs to fast sub-sampling search, a further refinement stage proceeds. Otherwise, the motion search is terminated.

Proposed Search Patterns

Test Parameters (JM10.2)

Experimental Results

• PSNR

Experimental Results (cont.)

• Motion Estimation Time

Experimental Results (cont.)

• Bit Rate

Y-PSNR

35.3

35.5

35.7

35.9

36.1

36.3

1 10 19 28 37 46 55 64 73 82 91 100

#frame

PSN

R

full search

SHEX FME

hierarchical FME

Diagram of Stefan

ME Time

0

500

1000

1500

2000

2500

3000

3500

4000

4500

1 9 17 25 33 41 49 57 65 73 81 89 97

#frame

time(ms)

full searchSHEX FMEhierarchical FME

Bits per frame

11000

16000

21000

26000

31000

36000

1 10 19 28 37 46 55 64 73 82 91 100

#frame

bit(s)

full searchSHEX FMEhierarchical FME

Conclusion

• Our proposed algorithm makes H.264/AVC feasible for real-time applications.

• Adaptively choosing the proper block type to further improve the bit rate and ME time will be the focus of the following research efforts.