Top Banner
SIFT on GPU Changchang Wu 5/8/2007
24

SIFT on GPU

Jan 19, 2016

Download

Documents

SIFT on GPU. Changchang Wu 5/8/2007. Outline. Background and related implementation SIFT on GPU (SiftGPU) Goal: fast, general, flexible Conclusion and Future Work. SIFT (Lowe, IJCV04). Scale Invariant Feature Transform - PowerPoint PPT Presentation
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: SIFT on GPU

SIFT on GPU

Changchang Wu

5/8/2007

Page 2: SIFT on GPU

Outline

• Background and related implementation

• SIFT on GPU (SiftGPU)•Goal: fast, general, flexible

• Conclusion and Future Work

Page 3: SIFT on GPU

SIFT (Lowe, IJCV04)

• Scale Invariant Feature Transform•Detect and describe features that are invariant to similarity transformation

• Popular technique in computer vision•Panorama Generation•Microsoft Photosynth•Content Based Image Retrieval

Page 4: SIFT on GPU

SIFT (Lowe, IJCV04)

• Scale-space extrema detection•Difference-of-Gaussian function

•A close approximate of scale normalized Laplacian of Gaussian , more stable than gradient, Hessian, or Harris corner function.

•Maximum and minimum of DOG are Invariant to scale change

),()),,(),,((),,( yxIyxGkyxGyxD

Page 5: SIFT on GPU

Scale-space Construction

For each octave: s Intervals, k=21/s,s+3 Gaussian images

σdoubles for the next octave, just resample

Page 6: SIFT on GPU

Finding Local Extrema

Comparing a pixel (marked

with X) to its 26 neighbors in 3x3 regions at the current and adjacent scales (markedwith circles).

DOG space

Also assign orientations to keypoints using the maxima in local gradient orientation histogram (In a window of size 3*sigma)

))),1(),1(/())1,()1,(((2atan),( yxLyxLyxLyxLyx

Page 7: SIFT on GPU

Refinement

• Sub-pixel localization•Fitting 3D quadratic function in the 3x3x3 cube to find sub-pixel location

• Edge elimination

Page 8: SIFT on GPU

Feature descriptor• Select Gaussian image at expected scale• Compute weighted histogram of gradient orientation

(relative to keypoint orientations)

128D vector =16 squares x 8(directions)

Page 9: SIFT on GPU

Existing implementations

• CPU version• Lowe’s binary (http://www.cs.ubc.ca/~lowe/keypoints/)

• Andrea Vedaldi’s SIFT++• (http://vision.ucla.edu/~vedaldi/code/siftpp/siftpp.html)

• C#(autopanosift), Matlab…

• GPU version• Sudipta Sinha’s GPUSIFT• Sebastian Heymann’s

Page 10: SIFT on GPU

Current Progress• Intensity conversion and sampling (cg + GLSL)• Image pyramid (cg + GLSL)• keypoint detection (cg + GLSL)• Sub-pixel localization (none)• Edge elimination (cg only)• Feature List generation (cg + GLSL+CPU)• Orientation (cg fp40 only)• Display List generation (cg + GLSL)• Descriptor generation (cg)• Visualization (cg + GLSL, Glut+win32)

• + means multiple versions of implementations

Page 11: SIFT on GPU

Scale Space Construction

• Run horizontal and vertical Gaussian filtering separately

• When # of DOG level in an octave is 3, the largest Gaussian kernel can be 19x19

• Compute difference of Gaussian in the same pass since it is already read out

• Didn’t use Ping-pong, sometimes write and read same texture, because not all channels need to be changed.

Page 12: SIFT on GPU

Color channel mapping

• Use Texture from Destination instead of PingPong

Page 13: SIFT on GPU

Keypoint Detection• Compare with 26 neighbors?

• Do in 4 steps• Intra-level comparing with 8 neighbors, (compute

gradient in this pass, and edge elimination)• Store the maximum and minimum of the 9 pixels in

an auxiliary texture• Early z culling based on the in-level suppression• Comparing with the maximum and minimum of the

pixel at upper level and lower level

Page 14: SIFT on GPU

Feature List Generation on GPU

• Use Gernot Ziegler’s histogram pyramid method. Use all RGBA chanels

1. Do reduction, and read back the highest level.

2. Allocate texture to hold the feature list

3. Traverse the pyramid to get location

Page 15: SIFT on GPU

Feature Orientation• Use a circular window (use 3*sigma as radius)• Compute weighted histogram of orientations

(36 bins as 9 float4)Binary search to locate desired bin

bin+=float4(fmod(idx,4)==float4(0,1,2,3))

• Smoothing the histogramsmoothing kernel can easily be largeone (1 3 6 7 6 3 1 )/27 as three (1 1 1)/3

• next

Page 16: SIFT on GPU

Feature Orientation

• Find the bins that are • larger than 0.8 times the maximum•Local maximum•Do interpolation to get sub-bin orientation

• Save the largest N<=4M to RGBA of M output textures

• Save N to the original texture, and set N to 0 when N is larger than a threshold

Page 17: SIFT on GPU

Reshape Feature List

• Rebuild the feature list according to orientations (variable # of orientations)

• Use the histogram pyramid method

Page 18: SIFT on GPU

Feature Descriptor

• Use 4 textures for MRT, and 8 RGBA pixels in each. (8*4*4 = 128)

• Trilinear interpolation is implemented

A Geometry Shader Version is also available. Geometry shader can write as many

pixels as possible by emitting point primitives

Page 19: SIFT on GPU

Use 2*sigma ( instead of 6*sigma ) as box size to display here

Page 20: SIFT on GPU

Display VBO generation• Display SIFT features as rotated/scaled

square to illustrate scale and orientation. • Say feature texture is WxH (normally H is 1,

because no more than 2048..)• Make a texture that is Wx(4H)

• For point (x, y), the index is Idx=y*W+x• Then original index is idxo=Idx/4• And sub-index is fmod(idx,4), and use sub-index to

offset and rotate this point• Copy render result to VBO (vertex buffer

object)

Page 21: SIFT on GPU

Parameterization

• This SIFT on GPU also tries to give flexibility by providing parameters

• # of octaves, # of levels, sigma0• Starting octave, starting level• Filter window size• Orientation window size• Descriptor window size• …

• Shaders are dynamically generated

Page 22: SIFT on GPU

Result

• Speed on nVidia 8800

• 13 Hz on a 640*480 image

• 4 Hz on a 2048*1536 image

• Part can run on laptop•Raedon X300 (Maximum instruction is 96)•No orientation/Edge elimination/Descriptor

Page 23: SIFT on GPU

Conclusion

• Very close to sift++

• Finished a basic and also flexible framework of SIFT

• Reduced CPU/GPU data transfer by feature list generation on GPU

Page 24: SIFT on GPU

Future work

• Sub-pixel localization

• Try CUDA for descriptor generation

• Try the packed texture format of Sebastian Heymann’s implementation

• Compatibility with more Graphic Cards