Joint Histogram Based Cost Aggregation for Stereo Matching - TPAMI 2013 M.S. Student, Hee-Jong Hong Sep 24, 2013 Dongbo Min, Member, IEEE, Jiangbo Lu,

Joint Histogram Based Cost Aggregationfor Stereo Matching - TPAMI 2013

M.S. Student, Hee-Jong HongSep 24, 2013

Dongbo Min, Member, IEEE, Jiangbo Lu, Member, IEEE,

Minh N. Do, Senior Member, IEEE

2

• Introduction• Related Works• Proposed Method

: Improve Cost Aggregation• Experimental Results• Conclusion

Outline

Joint Histogram Based Cost Aggregation for Stereo Matching - TPAMI 2013

3

Introduction


• Goal： Perform efficient cost aggregation.• Solution : Joint histogram + reduce redundancy • Advantage : Low complexity but keep high-quality.

Cost Initial-ization≈70~75%

≈20~25%

≈5%

4

Related Works


• Complexity of aggregation： O(NBL)

• Reduce complexity approach• Scale image : Multi Scale Approach

D. Min and K. Sohn, “Cost aggregation and occlusion handling with WLS in stereo matching,” IEEE Trans. on Image Processing, 2008.

• Bilateral filter : Bilateral Approximation C. Richardt, D. Orr, I. P. Davies, A. Criminisi, and N. A. Dodgson, “Real-time spatiotemporal stereo matching using the dual-cross- bilateral grid,” in European Conf. on Computer Vision, 2010S. Paris and F. Durand, “A fast approximation of the bilateral filter using a signal processing approach,” International Journal of Computer Vision, 2009.

• Guided filter : Run in constant time => O(NL)C.Rhemann,A.Hosni,M.Bleyer,C.Rother,andM.Gelautz,“Fast cost-volume filtering for visual correspondence and beyond,” in IEEE Conf. on Computer Vision and Pattern Recognition, 2011

N : all pixels (W*H)B : window sizeL : disparity level

5


Proposed Method

6

Local Method Algorithm


• Cost initialization : Truncated Absolute Difference

=>• Cost aggregation : Weighted filter

• Disparity computation : Winner take all

[4,8]

7

Improve Cost Aggregation


• New formulation for aggregation• Remove normalization• Joint histogram representation

• Compact representation for search range• Reduce disparity levels

• Spatial sampling of matching window• Regularly sampled neighboring pixels• Pixel-independent sampling

8

New formulation for aggregation


• Remove normalization

=>

• Joint histogram representation

9

Compact Search Range


• Cost aggregation

=>

• MC(q)： a subset of disparity levels whose size is Dc.

O( NBD )

O( NBDc )

N : all pixels (W*H)B : window sizeD : disparity level

10

Compact Search Range


• Non-occluded region of ‘Teddy’ image Dc = 60

Final Accuracy = 93.7%

Dc = 6Final Accuracy =

94.1%

Dc = 5 (Best)Final Accuracy =

94.2%

11

Spatial Sampling of Matching Window


• Reason• A large matching window and a well-defined weighting function leads to high

complexity.• Pixels should aggregate in the same object, NOT in the window.

• Solution• Color segmentation => Time consuming (Heavy)• Spatial Sampling => Easy but powerful• Regular Sampling => Independent from reference pixel => Reduce Complexity

12

Spatial Sampling of Matching Window


• Cost aggregation

=>

• S : sampling ratio

O( NBDc )

O( NBDc / S2)

13


Parameter defini-tionN : size of image B : size of matching win-dow N(p)=W×WMD : disparity levels size=DMC : The subset of dispar-ity size=DC<<DS : Sampling ratio

Pre-procseeing

14


Experimental Result

15

Experimental Results


• Pre-processing• 5*5 Box filter

• Post-processing• Cross-checking technique• Weighted median filter (WMF)

• Device： Intel Xeon 2.8-GHz CPU (using a single core only) and a 6-GB RAM• Parameter setting

( ) = (1.5, 1.7, 31*31, 0.11, 13.5, 2.0)

16



(a) (b)

(c) (d)

17



• Using too large box windows (7×7, 9×9) deteriorates the quality, and incurs more computational overhead.

• Pre-filtering can be seen as the first cost aggregation step and serves the removal of noise.

18



Fig. 5. Performance evaluation: average per-cent (%) of bad matching pixels for ‘nonocc’, ‘all’ and ‘disc’ regions according to Dc and S.

2 better than 1

The smaller S, the better

19



The smaller S, the longer

The bigger Dc, the longer

20



• APBP : Average Percentage of Bad Pixels

21



Ground truthError mapsResultsOriginal images

22



23


Conclusion

24

Conclusion


• Contribution• Re-formulate the problem with the relaxed joint histogram.• Reduce the complexity of the joint histogram-based aggregation.• Achieved both accuracy and efficiency.

• Future work• More elaborate algorithms for selecting the subset of label hypotheses.• Estimate the optimal number Dc adaptively.• Extend the method to an optical flow estimation.

25

Thank you!

Joint Histogram Based Cost Aggregation for Stereo Matching - TPAMI 2013 M.S. Student, Hee-Jong Hong Sep 24, 2013 Dongbo Min, Member, IEEE, Jiangbo Lu,

Documents