Top Banner
Recovering Transparent Shape from Time-of-Flight Distortion (CVPR2016) 1 K. Tanaka Y. Mukaigawa H. Kubo Y. Matsushita Y. Yagi
14

MIRU2016 invited talk - Recovering Transparent Shape from Time-of-Flight Distortion (CVPR 2016)

Apr 16, 2017

Download

Science

Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: MIRU2016 invited talk - Recovering Transparent Shape from Time-of-Flight Distortion (CVPR 2016)

Recovering Transparent Shapefrom Time-of-Flight Distortion

(CVPR2016)

1

K. Tanaka Y. Mukaigawa H. Kubo Y. Matsushita Y. Yagi

Page 2: MIRU2016 invited talk - Recovering Transparent Shape from Time-of-Flight Distortion (CVPR 2016)

Transparent Objects

• Invisible, but distorted background can be seen.

• 3D reconstruction of transparent material is challenging.

3

Sensor

Estimated pointby triangulation

BackgroundReference

Distorted

Page 3: MIRU2016 invited talk - Recovering Transparent Shape from Time-of-Flight Distortion (CVPR 2016)

Time-of-Flight (ToF) Camera

• Depth sensor based on time delay of light

• Kinect v2, Project Tango, etc.

4

time

Light signal

Observation

𝑡Δ𝑑 =

𝑐𝑡Δ2

(speed of light x time delay)

Page 4: MIRU2016 invited talk - Recovering Transparent Shape from Time-of-Flight Distortion (CVPR 2016)

Time of Flight Distortion

• Speed of light slows down depending on refractive index.

• Depth becomes longer ( = ToF Distortion).

• We use this distortion for transparent shape recovery.

5

cc

c

Page 5: MIRU2016 invited talk - Recovering Transparent Shape from Time-of-Flight Distortion (CVPR 2016)

Contributions

6

1. ToF distortion can be used for transparent shape recovery.

2. Easy multi-path mitigation using retroreflective sheet.

cc

c

Page 6: MIRU2016 invited talk - Recovering Transparent Shape from Time-of-Flight Distortion (CVPR 2016)

Problem Setting

Input

• Known refractive index

• 1 distorted ToF depth

• 2 references (3D points)

7

r

f

br

t

v vr

s

Output

• 3D points of both surfaces

• Surface normals

Page 7: MIRU2016 invited talk - Recovering Transparent Shape from Time-of-Flight Distortion (CVPR 2016)

Parameters and Candidate Shapes

• Candidate shapes• Front surface is on camera ray at distance 𝑡

• Back surface is on reference ray at distance 𝑠

• Many candidates. (2 degree of freedom)

8

ToF camera

𝑡

Glass object

Display or known pattern

𝑠

Page 8: MIRU2016 invited talk - Recovering Transparent Shape from Time-of-Flight Distortion (CVPR 2016)

Candidate Shape using ToF Distortion

• Candidate shapes• Front surface is on camera ray at distance 𝑡

• Back surface is on reference ray at distance 𝑠• such that 𝑠 + 𝑡 + 𝜂 = 𝑙𝑇𝑜𝐹 : ToF distortion

• One degree of freedom

9

ToF camera

𝑡

Glass object

Display or known pattern

𝑠

Page 9: MIRU2016 invited talk - Recovering Transparent Shape from Time-of-Flight Distortion (CVPR 2016)

Surface Normal Consistency

• Surface normal is uniqueRefractive normal

Geometric normal

• They should coincide.

10

ToF camera

𝑡

Glass object

Display or known pattern

𝑠

𝑛 =sin 𝜃1sin 𝜃2

Refractive normal

camera ray

Geometric normal

Page 10: MIRU2016 invited talk - Recovering Transparent Shape from Time-of-Flight Distortion (CVPR 2016)

Real world experiment setup

• modified Kinect v2 and LCD panel

11

Kinect v2

(IR Lens changed)LCD panel

Linear stage

Target object

Page 11: MIRU2016 invited talk - Recovering Transparent Shape from Time-of-Flight Distortion (CVPR 2016)

Results and evaluations

• Target materials and estimated results

• Evaluation• Fit estimated points to ground-truth CAD model by ICP

12

Cube Wedge prism Schmidt prism

Object Mean error Std. dev.

Cube 0.188 mm 0.458 mm

Wedge 0.226 mm 1.137 mm

Schmidt 0.381 mm 1.398 mm

Page 12: MIRU2016 invited talk - Recovering Transparent Shape from Time-of-Flight Distortion (CVPR 2016)

Summary

Input

• 1 distorted ToF depth

• 2 references (3D points)

• Known refractive index

13

r

f

br

t

v vr

s

Output

• 3D points of both surfaces

• Surface normals

Page 13: MIRU2016 invited talk - Recovering Transparent Shape from Time-of-Flight Distortion (CVPR 2016)

Time-of-Flight as alternative imager

• Light-in-flight [Gkioulekas+2015]

• Parameter tunable ToF camera (Texas instruments)

14

Page 14: MIRU2016 invited talk - Recovering Transparent Shape from Time-of-Flight Distortion (CVPR 2016)

Imaging, Analyzing using ToF Camera

• Recently Emerging Topic

[Heide+2013], [Kadambi+2013], [Naik+2013], [Godbaz+2013], [Freedman+2014], [Lin+2014], [O’Toole+2014], [Gupta+2015], [Heide+2015], [Xiao+2015], [Kadambi+2015], [Peters+2015], [Tadano+2015], and more!

• CVPR 2016• 1 oral, 2 posters (including ours)

[Kadambi et al.], [Su et al.]

• SIGGRAPH 2016• 2 technical papers.

[Shrestha et al.], [Kadambi et al.]

15

We will continue working on ToF camera