Computer Vision A Hand-Held “Scanner” for Large-Format Images COMP 256 Adrian Ilie Steps Towards.

ComputerVision

A Hand-Held “Scanner” forLarge-Format Images

COMP 256

Adrian Ilie

Steps Towards

ComputerVision Previous Work: Panoramas

• Feature extracting: use SIFT, since they are scale-invariant and partially invariant to affine illumination changes - done

• Feature matching: approximate nearest neighbor - done

• Image matching: probabilistic model using RANSAC inliers/outliers - N/A

• Bundle adjustment: add images one by one and iterate using Levenberg-Marquardt - N/A

• Blending: multi-band - not done

ComputerVision Extracting the Features

• Use SIFT features– Location: peaks in DoG pyramids– Descriptors: gradient orientation

histograms

ComputerVision Matching Features

• Look for closest 2 descriptors in a k-d tree (logarithmic speed)

• If distance(descriptor, 1st closest) < 0.36*distance(descriptor, 2nd closest), descriptor is a good match

ComputerVision Computing the Homography

• MLESAC

ComputerVision Warping the Images

• Use bilinear interpolation

ComputerVision Algorithm

• Take an “overview” image• Extract its features and build a k-d tree• Take N “detail” images• For each image i

Extract the features Match the features against the ones in

the k-d tree Use MLESAC to compute the homography Warp the image Blend the image into the current estimate Update the k-d tree

ComputerVision Blending the Warped Images

• Detail image has higher resolution!• Resample the current estimate so that

the area corresponding to the warped image is equal to the area of the unwarped image

• Can blend using some weights, or just use the detail image pixel (since it is of higher quality)

ComputerVision Results

ComputerVision Scanning My Favorite Poster ;)

ComputerVision Scanning My Favorite Poster ;)

ComputerVision Scanning My Favorite Poster ;)

ComputerVision Scanning My Favorite Poster ;)

ComputerVision Issues and Future Work

• Issues– Radial distortion in the “overview” image– Numerical instability of the homography

computation– Illumination changes across images

• Future work– Super-resolution would be nice to have– It would be nice to have a nice viewer

that would take images and homographies as input, then blend and render them at the appropriate level of detail, depending on the zoom level