Underwater Image Dehazing with a Light Field Cameralightfield-analysis.net/LF4CV/slides/...dehazing_katherine_skinner.pdf · Underwater Image Dehazing with a Light Field Camera Katherine

Underwater Image Dehazing with a Light Field Camera

Katherine A. Skinner1 and Matthew Johnson-Roberson2

1Robotics Program, University of Michigan, USA2Department of Naval Architecture and Marine Engineering, University of Michigan, USA

Motivation

2

Photo: Volodymyr Goinyk

Motivation

3

Lizard Island, Australia Port Royal, Jamaica

Motivation

4

Photo: Jaffe Lab

Motivation

5

Photo: Jordt (2014)

Light Field Cameras

6

• Real-time RGB-D

• Passive optical sensors

• Compact form factor

• Increased depth-of-field Photo: Raytrix

Photo: Lytro

Real-time Underwater 3D Reconstruction

7

Katherine A. Skinner and Matthew Johnson-Roberson, "Towards real-time underwater 3D reconstruction with plenoptic cameras.“ IROS, 2016.

Real-time Underwater 3D Reconstruction

8

Katherine A. Skinner and Matthew Johnson-Roberson, "Towards real-time underwater 3D reconstruction with plenoptic cameras.“ IROS, 2016.

Pure vs. Turbid Water

9

Disparity map in turbid waterDisparity map in pure water

Underwater Image Restoration

10

Contribution: We present a pipeline for dehazing of underwater light field images incorporating a physical model of underwater light propagation.

Underwater Light Field Image Dataset

11

Contribution: Provide a comprehensive light field dataset for underwater image dehazing.

https://github.com/kskin/data

Technical Approach

12

Dehazing Model

13

𝐼(𝒙) = 𝐽(𝒙)𝑒−𝜂𝑑(𝒙) + 𝐴(1 − 𝑒−𝜂𝑑(𝒙))

Dehazing Model

14

𝐼(𝒙) = 𝐽 𝒙 𝑡(𝒙) + 𝐴(1 − 𝑡 𝒙 )

Dehazing Model

15

Reference: Y. Li et al. “Nighttime haze removal with glow and multiple light colors”. ICCV 2015.

𝐼(𝒙) = 𝐽 𝒙 𝑡(𝒙) + 𝐴(𝒙) 1 − 𝑡 𝒙 +𝐿𝛼 (𝒙) ∗ 𝑨𝑷𝑺𝑭

• Non-uniform illumination• Glow patterns

Synthesizing Views

16

Recovering Epipolar Images

17

Results

18

In air Raw underwater Our result

Results

19

Disparity map using raw underwater image Disparity map using corrected image

Conclusions

• Presented an underwater light field image dehazing pipeline that incorporates a physical model of light propagation

• Provided an underwater light field image dataset

20

21

[1] Dansereau, et al. Light field image denoising using a linear 4D frequency-hyperfan all-in-focus filter. Computational Imaging, 34(2):15–1, 2013.

• Gather light field images of more complex scenes• Greater depth disparity

• Varying turbidity

• Further leverage light field structure• All-in-focus filter [1]

• Consistent motion between sub-aperture images

• Incorporate more components of physical model for underwater light propagation

• Recovering range in underwater imaging

Future Work

Final Thoughts

22

Underwater light field camera – contact [email protected]

Deep learning approach to underwater image restoration

Honolulu, Hawaii