Lecture02

CSE486, Penn StateRobert Collins

Lecture 2:Intensity Surfaces

and Gradients

CSE486, Penn StateRobert Collins

Intensity pattern

Visualizing ImagesRecall two ways of visualizing an image

2d array of numbers

We “see it” at this level Computer works at this level

CSE486, Penn StateRobert Collins

Bridging the Gap

Motivation: we want to visualize images at a level highenough to retain human insight, but low enough to allowus to readily translate our insights into mathematicalnotation and, ultimately, computer algorithms that operate on arrays of numbers.

CSE486, Penn StateRobert Collins

Surface heightproportional topixel grey value(dark=low, light=high)

Images as Surfaces

CSE486, Penn StateRobert Collins

Examples

Mean = 164 Std = 1.8

Note: see demoImSurf.m in matlab examples directory oncourse web site if you wantto generate plots like these.

CSE486, Penn StateRobert Collins

Examples

CSE486, Penn StateRobert Collins

Examples

CSE486, Penn StateRobert Collins

Examples

CSE486, Penn StateRobert Collins

Examples

Mean = 111 Std = 15.4

CSE486, Penn StateRobert Collins

Examples

CSE486, Penn StateRobert Collins

How does this visualization help us?

CSE486, Penn StateRobert Collins

Terrain Concepts

CSE486, Penn StateRobert Collins

Terrain Concepts

www.pianoladynancy.com/ wallpapers/1003/wp86.jpg

CSE486, Penn StateRobert Collins

Terrain ConceptsBasic notions:

Uphill / downhillContour lines (curves of constant elevation)Steepness of slopePeaks/Valleys (local extrema)

Gradient vectors (vectors of partial derivatives) will help us define/compute all of these.

More mathematical notions:Tangent PlaneNormal vectorsCurvature

CSE486, Penn StateRobert Collins

Math Example : 1D GradientConsider function f(x) = 100 - 0.5 * x^2

CSE486, Penn StateRobert Collins

Math Example : 1D GradientConsider function f(x) = 100 - 0.5 * x^2

Gradient is df(x)/dx = - 2 * 0.5 * x = - x

Geometric interpretation: gradient at x0 is slope of tangent line to curve at point x0

x0

f(x)

tangent line

Δy

Δx

slope = Δy / Δx

= df(x)/dx

x0

CSE486, Penn StateRobert Collins

Math Example : 1D Gradientf(x) = 100 - 0.5 * x^2 df(x)/dx = - x

x0 = -2

grad = slope = 2

CSE486, Penn StateRobert Collins

Math Example : 1D Gradientf(x) = 100 - 0.5 * x^2 df(x)/dx = - x

gradients 3

2

10

-3

-2

-1

CSE486, Penn StateRobert Collins

Math Example : 1D Gradientf(x) = 100 - 0.5 * x^2 df(x)/dx = - x�

gradients 3

2

10

-3

-2

-1

Gradientson this sideof peak arepositive

Gradientson this sideof peak arenegative

Note: Sign of gradient at point tells you what direction to go to travel “uphill”

CSE486, Penn StateRobert Collins

Math Example : 2D Gradientf(x,y) = 100 - 0.5 * x^2 - 0.5 * y^2df(x,y)/dx = - x df(x,y)/dy = - y

Gradient = [df(x,y)/dx , df(x,y)/dy] = [- x , - y]

Gradient is vector of partial derivs wrt x and y axes

CSE486, Penn StateRobert Collins

Math Example : 2D Gradientf(x,y) = 100 - 0.5 * x^2 - 0.5 * y^2

Gradient = [df(x,y)/dx , df(x,y)/dy] = [- x , - y]

Plotted as a vector field,the gradient vector at each pixel points “uphill”

The gradient indicates thedirection of steepest ascent.

The gradient is 0 at the peak(also at any flat spots, and local minima,…butthere are none of those for this function)

CSE486, Penn StateRobert Collins

Math Example : 2D Gradientf(x,y) = 100 - 0.5 * x^2 - 0.5 * y^2

Gradient = [df(x,y)/dx , df(x,y)/dy] = [- x , - y]

Let g=[gx,gy] be the gradient vector at point/pixel (x0,y0)

Vector g points uphill (direction of steepest ascent)

Vector - g points downhill (direction of steepest descent)

Vector [gy, -gx] is perpendicular,and denotes direction of constantelevation. i.e. normal to contourline passing through point (x0,y0)

CSE486, Penn StateRobert Collins

Math Example : 2D Gradientf(x,y) = 100 - 0.5 * x^2 - 0.5 * y^2

Gradient = [df(x,y)/dx , df(x,y)/dy] = [- x , - y]

And so on for all points

CSE486, Penn StateRobert Collins

Image Gradient

The same is true of 2D image gradients.

The underlying function is numerical(tabulated) rather than algebraic. So need numerical derivatives.

CSE486, Penn StateRobert Collins

Numerical DerivativesSee also T&V, Appendix A.2

Taylor Series expansion

Finite forward difference

Manipulate:

CSE486, Penn StateRobert Collins

Numerical DerivativesSee also T&V, Appendix A.2

Taylor Series expansion

Finite backward difference

Manipulate:

CSE486, Penn StateRobert Collins

Numerical Derivatives

Finite central difference

See also T&V, Appendix A.2

Taylor Series expansion

subtract

CSE486, Penn StateRobert Collins

Numerical DerivativesSee also T&V, Appendix A.2

Finite backward difference

Finite forward difference

Finite central differenceMoreaccurate

CSE486, Penn StateRobert Collins

Example: Temporal Gradient

derivative

df(x)/dx

A video is a sequence of image frames I(x,y,t).

Each frame has two spatial indices x, y andone temporal (time) index t.

CSE486, Penn StateRobert Collins

Example: Temporal Gradient

Consider the sequence ofintensity values observedat a single pixel over time.

derivative

df(x)/dx

1D function f(x) over time 1D gradient (deriv) of f(x) over time

CSE486, Penn StateRobert Collins

Temporal Gradient (cont)What does the temporal intensity gradient at each pixel look like over time?

CSE486, Penn StateRobert Collins

Example: Spatial Image Gradients

I(x+1,y) - I(x-1,y)

2I(x,y)

I(x,y+1) - I(x,y-1)

2

Ix=dI(x,y)/dx

Iy=dI(x,y)/dy

Partial derivative wrt x

Partial derivative wrt y

CSE486, Penn StateRobert Collins

Functions of GradientsI(x,y) Ix Iy

Magnitude of gradient sqrt(Ix.^2 + Iy.^2)

Measures steepness ofslope at each pixel

CSE486, Penn StateRobert Collins

Functions of GradientsI(x,y) Ix Iy

Angle of gradient atan2(Iy, Ix)

Denotes similarityof orientation of slope

CSE486, Penn StateRobert Collins

Functions of Gradients

What else dowe observe inthis image?

atan2(Iy, Ix)

Enhanced detailin low contrastareas (e.g. foldsin coat; imagingartifacts in sky)

I(x,y)

CSE486, Penn StateRobert Collins

Next Time: Linear Operators

Gradients are an example of linear operators,i.e. value at a pixel is computed as a linearcombination of values of neighboring pixels.