Lecture 17: Mosaicing and Stabilizationrtc12/CSE486/lecture17.pdf · CSE486, Penn State Robert Collins Applications: Stabilization Given a sequence of video frames, warp them into

CSE486, Penn StateRobert Collins

Lecture 17:

Mosaicing and Stabilization


Recall: Planar Projection

x

y

Point on planeRotation + Translation

Perspectiveprojection

Pixel coordsu

v

Internalparams

Homography


Destination image

Recall: Projective (un)Warping

from Hartley & Zisserman

Source Image

H


Recall : Planar Projection

H2H1

H


Applications: Stabilization

Given a sequence of video frames, warp theminto a common image coordinate system.

This “stabilizes” the video to appear as if thecamera is not moving.

frame k k+1 k+2k-1k-2

destinationframe


Stabilization Example

VIVID project


Stabilization by Chaining

What if the reference image does not overlap with all the source images? As long as there are pairwise overlaps, wecan chain (compose) pairwise homographies.

frame k k+1 k+2

destination

k+3 k+4

Not recommended for long sequences, as alignment errors accumulate over time.

H43H3

2H21H1

0

H40 = H1

0 * H21 * H3

2 * H43


Applications: Mosaicing

from Hartley & Zisserman

Destination imageSource 1 Source 2

Mosaic

H1 H2


Note on Planar Mosaicing

Assumes scene is roughly planar.

What if scene isn’t planar? Alignment will not be good if significant 3D relief

“Ghosting”


Ghosting Example

Source image Reference imageH


Ghosting Example (cont)

Mosaic


Mosaics from Rotating Cameras

However, there is a mitigating factorin regards to ghosting…

Images taken from a rotating camera are related by a 2D homography…

regardless of scene structure!


Rotating Camera (top-down view)






Rotating Camera (top-down view)Rays in camera coord system are invariant!






Special Case : Rotating Camera

Relative Rotation of camera Translation is 0

This is important!

Relative R,TProjectionInternalparams


Special Case : Rotating Camera

Relative R,TProjectionInternalparams

CSE486, Penn StateRobert Collins Relations among Images

Taken by Rotating Camera

Image 1

Image 2

Same ray!

-1


Mosaicing ExampleOriginal Images (from a pan/tilt camera)

Panoramic (Mosaic) View


One more detail: Blending!


Approaches to Blending

1) Straight averaging

2) Feathering

3) Equalize intensity statistics (gain, offset)

P = (P1 + P2) / 2

P = (w1*P1 + w2*P2 ) / (w1+w2)

How to combine colors in area of overlap?

With wi being distance from image border


Before and After Blending


360 Degree Panoramas?

Problem: Can’t just choose a reference imageto map all other images to.

Solution: Use cylindrical or spherical mosaicsurface rather than a plane.


Panorama Input images

Sarnoff


Spherical Panorama Result

Sarnoff


Panorama Unwarped

Sarnoff


Quicktime VR

This example from www.panoguide.com/gallery/

Also big list at http://www.multimedialibrary.com/diana/qtvr_sites.asp


Quicktime VR


Quicktime VR

This example from www.ems.psu.edu/~fraser/qtvr/

Lecture 17: Mosaicing and Stabilizationrtc12/CSE486/lecture17.pdf · CSE486, Penn State Robert Collins Applications: Stabilization Given a sequence of video frames, warp them into

Documents