Numerical Geometry in Image Processing

www.cs.technion.ac.il/~ron

Numerical Geometry in Image Processing

Ron Kimmel

Geometric Image Processing Lab

Computer Science Department Technion-Israel Institute of Technology

Heat Equation in Image Analysis

Linear scale space (T. Iijima 59, Witkin 83, Koenderink 84)

)(tIIt )0(*)()( ItGtI

Geometric Heat Equation in Image Analysis

Geometric scale space, Euclidean (Gage-Hamilton 86, Grayson 89, Osher-Sethian 88, Evans Spruck 91, Alvarez-Guichard-Lions-Morel 93)

Geometric Heat Equation in Image Analysis

Gabor 65 anisotropic reaction-diffusion Geometric, Special Affine. (Alvarez-Guichard-Lions-Morel

93, Sapiro-Tannenbaum 93)

Geometric Heat Equation in Image Analysis

Multi Channel, Euclidean.(Chambolle 94, Whitaker-Gerig 94, Proesmans-Pauwels-van Gool 94,Sapiro-Ringach 96, Shah 96, Blomgren-Chan 96, Sochen-Kimmel-Malladi 96, Weickert, Romeny, Lopez, and van Enk 97,…)

Geometric, Bending.(Curves: Grayson 89, Kimmel-Sapiro 95 (via Osher-Sethian),Images: Kimmel 97)

Bending Invariant Scale Space

Invariant to surface bending. Embedding: The gray level sets embedding is preserved. Existence: The level sets exist for all evolution time,

disappear at points or converge into geodesics. Topology: Image topology is simplified. Shortening flow:The scale space is a shortening flow of the

image level sets. Implementation: Simple, consistent, and stable numerical

implementation.

Curves on Surfaces: The Geodesic Curvature

From Curve to Image Evolution

Geodesic curvature flow

The Beltrami Framework

Brief history of color line element theories. A simplified color image formation model. The importance of channel alignment. Images as surfaces. Surface area minimization via Beltrami flow. Applications: Enhancement and scale space. Beyond the metric, the Gabor connection

Images as Surfaces Gray level analysis is sometimes misleading…

Is there a `right way’ to link color channels? process texture? enhance volumetric data?

We view images as embedded maps that flow towards minimal surfaces: Gray scale images are surfaces in (x,y, I), and color images are surfaces embedded in (x,y,R,G,B).

Joint with Sochen & Malladi, IEEE T-IP 98, IJCV 2000.

Helmholtz 1896: Schrodinger 1920:

Stiles 1946: Vos and Walraven 1972: inductive line elements (above), empirical line

elements (MacAdam 1942, CIELAB 1976). Define: the simplest hybrid spatial- color space:

Spatial-Spectral Arclength

Color Image Formation

F. Guichard 93Mondrian world:Lambertian surface patches

Image formationLambetian

model

V

lN

)cos(,),( lNyxI

)cos(,),(

)cos(,),(

)cos(,),(

BB

GG

RR

lNyxB

lNyxG

lNyxR

Color Image Formation

The gradient directions should agree since

Example: Demosaicing

Color image reconstruction Solution: Edges support the colors and the colors support the edges

Color Image Formation

Lambertian shading model: R(x,y) = <N,L> G(x,y) = <N,L> B(x,y) = <N,L>Thus Within an object R/G= / =constant We preserve color ratio weighted by an edge

indication function.

R

G

B

R G

Demosaicing ResultsOriginal Bilinear interpolation Weighted interpolation

Demosaicing ResultsBilinear interpolation Weighted interpolation

Demosaicing ResultsOriginal Bilinear interpolation Weighted interpolation

Demosaicing ResultsBilinear interpolation Weighted interpolation

Demosaicing ResultsOriginal Bilinear interpolation Weighted interpolation

Demosaicing ResultsBilinear interpolation Weighted interpolation

From Arclength to Area

Gray level arclength:

Color arclength

Area

Multi Channel Model

The Beltrami Flow

Gray level:

The Beltrami Flow

Color :

where

Matlab Program

Signal processing viewpoint

Beltrami Smoothing

Gaussian Smoothing

Sochen, Kimmel, Bruckstein, JMIV, 2001.

The Beltrami Flow

Texture:

Inverse Diffusion Across the Edge

Inverse Diffusion Across the Edge

Summary: Geometric Framework

From color image formation to the importance of channel alignment.

From color line element theories to the definition of area in color images.

Area minimization as a unified framework for enhancement and scale space.

Inverse heat operator across the edges. Related applications: Color movies segmentation

and demosaicing

www.cs.technion.ac.il/~ron

Open Questions

Is there a maximum principle to the Beltrami flow?

Are there simple geometric measures to minimize in color image processing subject to more complicated image formation models?

Can we really invert the geometric heat operator?

Is there a real-time numerical implementation for the Beltrami flow in color?

www.cs.technion.ac.il/~ron