Efficient Stereo Matching Based on a New Confidence Metric

Efficient Stereo Matching Based on a New Confidence Metric

Won-Hee Lee, Yumi Kim, and Jong Beom Ra

Department of Electrical Engineering, KAIST, Daejeon, Korea

20th European Signal Processing Conference (EUSIPCO 2012)

2

Outline• Introduction

• Related Work

• Proposed Algorithm

• Experimental Results

• Conclusion

3

Introduction

4

Introduction• For the TV application, stereo matching should be

performed in real-time.

• Aggregation kernel size is to be small• Aggregation process takes large computation loads • May cause problems in a textureless area

• Texture area information incorrect textureless area

• Propose a new confidence metric for stereo matching• For efficient refinement (with small kernel size)

Objective:

5

Introduction

35X35

5X35

[4] K. Zhang, J. Lu, and G. Lafruit, “Cross-based local stereo matching using orthogonal integral images,” IEEE TCSVT, 2009.

6

Related Work

7

Related Work• Cross-based stereo matching algorithm[4]

• Raw matching cost:

• Aggregated cost:

Ud(x) : local support region : the number of pixels in Ud(x)

8

Related Work• Cross-based stereo matching algorithm[4]

• Winner-take-all:

d0(x) : the initial disparitydmax(x) : the maximum disparity

9

Related Work• Confidence metrics[5]:

• Several metrics were proposed to measure the confidence level of match

• Utilizing:

• Aggregated cost

• Curvature of the cost curve

• Left-right consistency

[5] X. Hu and P. Mordohai, “Evaluation of stereo confidence indoors and outdoors,” in CVPR, 2010

10

Confidence metrics• 1) Matching score metric (MSM)

C : aggregated costdi : the disparity that reveals the ith minimum cost

White(High confidence)

Black(Low confidence)

11

Confidence metrics• 2) Curvature of cost curve metric (CUR)

12

Confidence metrics• 3) Naive peak ratio metric (PKRN)

13

Confidence metrics• 4) Naive winner margin metric (WMNN)

• computes a margin between two minimum costs • normalize it with the sum of total costs

14

Confidence metrics• 5) Left right difference metric (LRD)

min{cR(x - d1, dR)}: the minimum value of a cost curve at the corresponding pixel in the right image.

15

ProposedAlgorithm

16

Framework

17

Proposed Confidence Metric

‧:Correct estimated pixels X : Incorrect estimated pixels

18

Proposed Confidence Metric• The new metric is proposed as

• Characteristic:• extracts the curvature information across a range larger than that

including three cost values in the CUR metric

• : improve the metric performance for a cost graph with a small curvature.

LoG : a Laplacian of Gaussian filter of n-taps

19

Refinement• Weighted median filter

• Weight :

: the initial disparity of neighboring pixels (same color segment)

: duplication operator

offset a slope of function

20

Refinement• Histogram-based color segmentation algorithm[6]:

[6] J. Delon, A. Desolneux, J. L. Lisani, and A. B. Petro, “A nonparametric approach for histogram segmentation,” IEEE TIP, 2007.

21

Refinement• The filtering is applied only to the limited number if pixels

• Due to small size of filtering kernel

• To enlarge the filtering range

• Vertically propagate the filtered result of a current pixel

A

B

C

(current)

If Weightpropagate > WeightB

DisparityB = Disparitypropagate

Else Datapropagate = DataB

Datapropagate = DataAFiltered Disparity

Weight

Color segment

index

Propagation Data After weighted median filtering…

22

ExperimentalResults

23

Experimental Results• Parameters:

Aggregation kernel

Filtering kernel

T n σ τ

5 x 35 5 x 63 60 7 10 2

n : Laplacian of Gaussian filter of n-taps

offset a slope of function

24

Experimental ResultsBad pixel Confidence mapInitial disparity map

25

Experimental Results AUC: Area Under the Curve

Venus Tsukuba

Teddy Cones

26

Experimental Results

Error rate (Threshold = 1)

Error rate (Threshold = 1)

Cross-based

Adaptivesupport-weight

27

Experimental Results[4]

[10]

Proposed

After Aggregation

After Aggregation

After Aggregation

31

Conclusion

32

Conclusion• Presented an efficient stereo matching algorithm

• Applying a weighted median filter that is based on the proposed confidence metric.

• Successfully refine initial disparities.

• Competitive to the existing algorithms with a large size of aggregation kernel.