Top Banner
CSE486, Penn State Robert Collins Lecture 3: Linear Operators
35
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
  • 1. Robert CollinsCSE486, Penn StateLecture 3: Linear Operators

2. Robert CollinsCSE486, Penn State AdministriviaI have put some Matlab image tutorials on Angel.Please take a look if you are unfamiliar with Matlabor the image toolbox.I have posted Homework 1 on Angel. It is duenext Friday at beginning of class. 3. Robert CollinsCSE486, Penn StateRecall: 2D Gradient Gradient = vector of partial derivatives of image I(x,y)= [dI(x,y)/dx , dI(x,y)/dy] Gradient vector field indicates the direction and slope of steepest ascent (when considering image pixel values as a surface / height map). 4. Robert CollinsCSE486, Penn State Numerical DerivativesSee also T&V, Appendix A.2 Finite forward difference Finite backward difference Finite central difference More accurate 5. Robert Collins Example: Spatial Image GradientsCSE486, Penn StateIx=dI(x,y)/dxI(x+1,y) - I(x-1,y)I(x,y)2 Partial derivative wrt xI(x,y+1) - I(x,y-1)2 Iy=dI(x,y)/dy Partial derivative wrt y 6. Robert Collins Example: Spatial Image GradientsCSE486, Penn StateIx=dI(x,y)/dxI(x+1,y) - I(x-1,y)I(x,y)2 Partial derivative wrt xI(x,y+1) - I(x,y-1)2 Iy=dI(x,y)/dy Partial derivative wrt y Note: From now on we will drop the constant factor 1/2. We can divide by it later. 7. Robert CollinsCSE486, Penn StateMore SpecificallyImage II2,3 I2,1 dI/dxI1,1 I1,2 I1,3 I1,4 I1,5 -1+1I2,1 I2,2 I2,3 I2,4 I2,5I3,1 I3,2 I3,3 I3,4 I3,5I4,1 I4,2 I4,3 I4,4 I4,5I5,1 I5,2 I5,3 I5,4 I5,5 8. Robert CollinsCSE486, Penn State More Specifically Image I I2,4 I2,2 dI/dxI1,1 I1,2 I1,3 I1,4 I1,5 -1+1I2,1 I2,2 I2,3 I2,4 I2,5I3,1 I3,2 I3,3 I3,4 I3,5I4,1 I4,2 I4,3 I4,4 I4,5I5,1 I5,2 I5,3 I5,4 I5,5 9. Robert CollinsCSE486, Penn StateMore Specifically Image II2,5 I2,3 dI/dxI1,1 I1,2 I1,3 I1,4 I1,5 -1+1I2,1 I2,2 I2,3 I2,4 I2,5I3,1 I3,2 I3,3 I3,4 I3,5I4,1 I4,2 I4,3 I4,4 I4,5I5,1 I5,2 I5,3 I5,4 I5,5 10. Robert CollinsCSE486, Penn StateMore SpecificallyImage II3,3 I3,1 dI/dxI1,1 I1,2 I1,3 I1,4 I1,5I2,1 I2,2 I2,3 I2,4 I2,5 -1+1I3,1 I3,2 I3,3 I3,4 I3,5I4,1 I4,2 I4,3 I4,4 I4,5I5,1 I5,2 I5,3 I5,4 I5,5And so on 11. Robert CollinsCSE486, Penn State Linear Filters General process: Example: smoothing by Form new image whose averaging pixels are a weighted sum form the average of pixels in a of original pixel values, neighbourhood using the same set of Example: smoothing with a weights at each point. Gaussian Properties form a weighted average of Output is a linear function pixels in a neighbourhood of the input Example: finding a derivative Output is a shift-invariant form a weighted average of function of the input (i.e. pixels in a neighbourhood shift the input image two pixels to the left, the output is shifted two pixels to the left)Note: The Linear in Linear Filters means linearcombination of neighboring pixel values. 12. Robert CollinsCSE486, Penn State Image FilteringFreeman, MIT 13. Robert CollinsCSE486, Penn StateLinear Filtering think of this as a weighted sum (kernel specifies the weights):10*0+5*0+3*0+4*0+5*.5+1*0+1*0+1*1+7*.5 = 7Freeman, MIT 14. Robert CollinsCSE486, Penn StateLinear Filtering We dont want to only do this at a single pixel, of course, but want instead to run the kernel over the whole image.Freeman, MIT 15. Robert CollinsCSE486, Penn State Convolution (2D) Given a kernel (template) f and image h, theconvolution f*h is defined as 1) Note strange indexing into neighborhood of h(x,y). As a result, f behaves as if rotated by 180 degrees before combining with h. 2) That doesnt matter if f has 180 deg symmetry 3) If it *does* matter, use cross correlation instead. 16. Robert CollinsCSE486, Penn State Convolution 12 (p,q)5 f(u,v)h(u,v) 34f(u,v) h(x-u,y-v) (-p,-q) (x-p,y-q) Integral of red area is theh(-u,-v)h(x-u,y-v) convolution for this x and yAdapted from Rosenfeld and Kak, Digital Picture Processing 17. Robert CollinsCSE486, Penn State Convolution Examplef h f*h 1-1 -1 ? ? ? ?2 2 2 3 12-1 ? ? ? ?2 1 3 3 111? ? ? ?2 2 1 2Rotate1 3 2 2 ? ? ? ? 111-121Apply-1 -11adapted from C. Rasmussen, U. of Delaware 18. Robert CollinsCSE486, Penn StateConvolution Examplef 1 11 f*h 1 -1 -1-1215?? ? ? 2 2 2 3 12-1 -1 -11 ? ? ? ? 2 1 3 3 111 ? ? ? ? 2 2 1 2Rotate 1 3 2 2 ? ? ? ? 111-121 2*2+1*2+(-1)*2+1*1 = 5 Apply-1 -11 19. Robert CollinsCSE486, Penn StateConvolution Examplef11 1 f*h 1 -1 -1-12 154 ? ?2 2 2 3 12-1 -1 -1 1 ? ? ? ?2 1 3 3 111? ? ? ?2 2 1 2Rotate1 3 2 2 ? ? ? ? 111-1212*2+1*2+(-1)*2+1*1 = 5Apply-1*2+2*2+1*2-1*2-1*1+1*3= 4-1 -11 20. Robert CollinsCSE486, Penn StateConvolution Examplef 11 1 f*h 1 -1 -1-1 2 1 5 4 4 ?22 2 3 12-1-1 -1 1 ? ? ? ?2 1 3 3 111 ? ? ? ?22 1 2Rotate13 2 2 ? ? ? ? 111-121 2*2+1*2+(-1)*2+1*1 = 5Apply -1*2+2*2+1*2-1*2-1*1+1*3= 4-1 -11 -1*2+2*2+1*3-1*1-1*3+1*3= 4 21. Robert CollinsCSE486, Penn StateConvolution Examplef 11 1 f*h 1 -1 -1-1 2 1 5 4 4 -22 22 3 12-1-1 -1 1 ? ? ? ?2 1 3 3 111 ? ? ? ?2 21 2Rotate1 32 2 ? ? ? ? 111-1212*2+1*2+(-1)*2+1*1 = 5Apply-1*2+2*2+1*2-1*2-1*1+1*3= 4-1 -11-1*2+2*2+1*3-1*1-1*3+1*3= 4-1*2+2*3-1*3-1*3 = -2 22. Robert CollinsCSE486, Penn StateConvolution Exampleff*h 1 -1 -11 11 2 2 2 35 4 4-2 12-1 -121 2 1 3 39 ?? ? 111-1 -11 2 2 1 2? ?? ?Rotate 1 3 2 2? ?? ? 111-121 2*2+1*2+(-1)*2+1*1 = 5 Apply -1*2+2*2+1*2-1*2-1*1+1*3= 4-1 -11 -1*2+2*2+1*3-1*1-1*3+1*3= 4 -1*2+2*3-1*3-1*3 = -21*2+1*2+2*2+1*1-1*2+1*2= 9 23. Robert CollinsCSE486, Penn StateConvolution Examplef f*h 1 -1 -1 11 12 2 2 3 5 44-2 12-1 -12 12 1 3 3 9 6? ? 111-1 -1 12 2 1 2??? ?Rotate1 3 2 2??? ? 111-1212*2+1*2+(-1)*2+1*1 = 5Apply-1*2+2*2+1*2-1*2-1*1+1*3= 4-1 -11-1*2+2*2+1*3-1*1-1*3+1*3= 4-1*2+2*3-1*3-1*3 = -21*2+1*2+2*2+1*1-1*2+1*2= 91*2+1*2+1*2-1*2+2*1+1*3-1*2-1*2+1*1= 6 24. Robert CollinsCSE486, Penn StateConvolution Exampleand so onFrom C. Rasmussen, U. of Delaware 25. Robert Collins Practical Issue: Border HandlingCSE486, Penn State Problem: what do we do for border pixels where the kernel does not completely overlap the image? ??? for interior pixels where but what values do we there is full overlap, we use for pixels that are know what to do.off the image ? 26. Robert Collins Practical Issue: Border HandlingCSE486, Penn State Different border handling methods specify different ways of defining values for pixels that are off the image. One of the simplest methods is zero-padding, which we used by default in the earlier example.0 0 0 0 ...0000 27. Robert Collins Practical Issue: Border HandlingCSE486, Penn State Other methods... Replication replace each off-image pixel with the value from the nearest pixel that IS in the image. Example: 1 2 3 1s 1 2 33s1 2 31 2 31 1 1 1 2 3 3 3 34 5 64 4 4 4 5 6 6 6 67 8 97 7 7 7 8 9 9 9 97 8 9 7s 7 8 99s7 8 9 28. Robert Collins Practical Issue: Border HandlingCSE486, Penn State Other methods... Reflection reflect pixel values at the border (as if there was a little mirror there) Example: 9 8 7 7 8 9 9 8 76 5 4 4 5 6 6 5 43 2 1 1 2 3 3 2 1 1 2 33 2 1 1 2 3 3 2 1 4 5 66 5 4 4 5 6 6 5 4 7 8 99 8 7 7 8 9 9 8 79 8 7 7 8 9 9 8 76 5 4 4 5 6 6 5 43 2 1 1 2 3 3 2 1 29. Robert Collins Practical Issue: Border HandlingCSE486, Penn State Other methods... Wraparound when going off the right border of the image, you wrap around to the left border. Similarly, when leaving the bottom of the image you reenter at the top. Basically, the image is a big donut (or torus).1 2 31 2 31 2 3 Example: 4 5 64 5 64 5 67 8 97 8 97 8 9 1 2 31 2 31 2 3 1 2 3 4 5 64 5 64 5 6 4 5 6 7 8 97 8 97 8 9 7 8 91 2 31 2 3 1 2 34 5 64 5 6 4 5 67 8 97 8 9 7 8 9 30. Robert CollinsCSE486, Penn State Convolution in MatlabCould use conv and conv2, but newer versions use:Imfilter(image,template{,option1,option2,})Boundary options: constant, symmetric, replicate, circularOutput size options: same as image, or full size (includespartial values computed when mask is off the image).Corr or conv option: convolution rotates the template (aswe have discussed). Correlation does not).Type help imfilter on command line for more details 31. Robert CollinsCSE486, Penn State Properties of Convolution Commutative: f * g = g * f Associative: (f * g) * g = f * (g * h) Distributive: (f + g) * h = f * h + g * h Linear: (a f + b g) * h = a f * h + b g * h Shift Invariant: f(x+t) * h = (f * h)(x+t) Differentiation rule: You will see some of these again! 32. Robert CollinsCSE486, Penn State Linear Filtering IFilter F O I NxM input image O NxM output imageF (2m+1)x(2m+1) mask a b c e.g. F= d e f for m=1 g h i hmk mO(i, j ) I (i h, j k ) F (h, k ) h m k mO.Camps, PSU 33. Robert CollinsCSE486, Penn StateBack to the Gradient(-1) * I2,1 + (0) * I2,2 + (+1) * I2,3Image II2,3 I2,1dI/dxI1,1 I1,2 I1,3 I1,4 I1,5 -1 0+1I2,1 I2,2 I2,3 I2,4 I2,5I3,1 I3,2 I3,3 I3,4 I3,5I4,1 I4,2 I4,3 I4,4 I4,5I5,1 I5,2 I5,3 I5,4 I5,5 34. Robert CollinsCSE486, Penn State Finite Difference Filters Finite Differences computed using convolution kernelsVertical Edges: Convolve with:-1 0 1 -1Question for class: Horizontal Edges: Is this correct?0Convolve with:1O.Camps, PSU 35. Robert Collins Example: Spatial Image GradientsCSE486, Penn State Ix=dI(x,y)/dxI(x,y)Partial derivative wrt x Iy=dI(x,y)/dyPartial derivative wrt yNote that thereis a difference betweenconvolving with a 1xn rowfilter and an nx1 col filter.