Static Image Mosaicing Amin Charaniya ([email protected]) EE 264: Image Processing and Reconstruction.

Static Image Mosaicing

Amin Charaniya

([email protected])

EE 264: Image Processing and Reconstruction

Presentation Overview

Problem definition Background

Literature Survey Image transformations

Image Registration Coarse Image registration Transformation Optimization

Image Blending Implementation and Results Conclusions (limitations and enhancements)

The Problem

Q: “Static” ?Ans.: No moving objects in the scene.

+Image 1 Image 2 Mosaiced image

The Solution

Original images

Image Registration /Alignment / Warping Image Blending

Constraints

Scene Static / Dynamic Planar / Non planar (perspective distortion)

Camera Motion Translation (sideways motion) Panning and Tilting (rotation about the Y and X axes) Scaling (zooming, forward / backward motion) General motion

Other Constraints Automated / User input

Background and Literature survey

Barnea & Silverman, 1972 (L1 Norm) Kuglin & Hines, 1975 (Phase Correlation) Mann & Picard, 1994 (Cylindrical projection) Irani & Anandan, 1995 (Static and Dynamic mosaics) Szeliski, 1996 (Transformation optimization) Badra, 1998 (Rotation and Zooming) Peleg and Rousso, 2000 (Adaptive Manifolds, Mosaicing

using strips)

Image transformations

TransformationInputimage

Output

image

w

y

x

w

y

x

876

543

210

'

'

'

mmm

mmm

mmm

100

543

210

mmm

mmm

affineM

Affine transformation

876

543

210

mmm

mmm

mmm

projectiveM

Projective transformation

100

cossin

sincos

y

x

rigid t

t

M

Rigid transformationOriginal

shape

Presentation Overview

Problem definition Background

Literature Survey Image transformations

Image Registration Coarse Image registration Transformation Optimization

Image Blending Implementation and Results Conclusions (limitations and enhancements)

Image Registration

Coarse ImageRegistration

Initial transformation TransformationOptimization

ErrorImproved ?

{Phase Correlation

L1 Norm

User input

Phase Correlation

Kuglin & Hines, 1975 Translation property of Fourier Transform

)(2..00

00),(),( yx yxjyxTFeFyyxxf

1|| 1..1j

TFeFf

2|| 2..2

jTF

eFf

)( 21 jeInverse

transformd(x,y)

maximum

(x0, y0)

Spatial Correlation, L1 Norm

Barnea and Silverman

E(x0,y0) = |f1(x,y) – f2(x- x0, y- y0)|

f1

f2 f2

Spatial correlation techniques User input

Transformation Optimization

Richard Szeliski, “Video Mosaics for Virtual Environments”, 1996. Optimization of initial transformation matrix M, to minimize error. Levenberg-Marquardt non-linear minimization algorithm.

yx

yxfyxfeerror,

221

)),('),(()(minimize

Compute partial derivatives

}7..0{,

km

e

k

bIAm1

)( mMM )()1( tt

Transformation Optimization

Advantages Faster convergence Statistically optimal solution

Limitations Local minimization (need a good initial guess)

Presentation Overview

Problem definition Background

Literature Survey Image transformations

Image Registration Coarse Image registration Transformation Optimization

Image Blending Implementation and Results Conclusions (limitations and enhancements)

Image Blending

Simple averaging Weighted averaging

2/)),('),((),( 21 yxfyxfyxf

),('),(),(),(),( 2211 yxfyxwyxfyxwyxf

Smooth transition (edges, illumination artifacts)

Sample weight function – “hat filter”

0 xmax

2

|2

|1)(

max

max

x

xx

xw

More weight at the center of the image, less at the edges

Image blending

Simple averaging Weighted averaging

Presentation Overview

Problem definition Background

Literature Survey Image transformations

Image Registration Coarse Image registration Transformation Optimization

Image Blending Implementation and Results Conclusions (limitations and enhancements)

Implementation

Implemented using Matlab Source Images

BE 230 lab images (fixed tripod) College 8 images (free hand motion, perpective distortion) East Field House images (free hand motion)

Equipment: Sony DCR-TRV 900 3CCD digital camcorder

Sample results

Sample results

Conclusions/Enhancements

Better automatic coarse registration techniques needed.

Need to handle more general camera motion.

Thanks for listening !!

Questions ?