Depth Enhancement via Low-rank Matrix Completion Si Lu 1 , Xiaofeng Ren 2 , and Feng Liu 1 Department of Computer Science, Portland State University 1 Department of Computer Science and Engineering, University of Washington 2 Depth maps captured by consumer RGB-D cameras are often noisy and miss values at some pixels. This paper presents a depth enhancement algorithm via low rank matrix completion that performs depth map completion and de-noising simultaneously. SUMMARY EXPERIMENTS This work was supported in part by NSF grants IIS-1321119, CNS- 1205746, and CNS-1218589. FRAMEWORK OBSERVATIONS: Similar RGB-D patches approximately lie in a low- dimensional subspace. The subspace constraint essentially captures the potentially scene-dependent image structures in the RGB- D patches in both the color and depth domain. This low-rank subspace constraint can be enforced through incomplete matrix factorization. Patch samples. (a): clean patch. (b): noisy patch. (c): the top ten eigen- vectors of the patch matrix formed by similar patches to each noisy patch. Similar patch searching Rank prediction for patch matrix Enhancement via low-rank matrix completion Output color Output depth Input color Input depth Output color Output depth Input color Input depth Output color Output depth Input color Input depth Output color Output depth Features capturing patch structure properties Predicted rank Training data Noisy color Color edge Depth edge Noisy depth (a) Ground truth (b) Noisy RGB-D image BM3D Our method (c) Color denoising (d) Joint bilateral filter (e) BM3D + joint bilateral filter (f) Our depth enhancement result Rank distribution of 30,000 RGB-D patch matrices (a) (b) (c) (a) (b) (c) (a) (b) (c) ^ = Regress ion model Comparisons among depth enhancement methods. patch matrix M matrix rank r