Convolutional Neural Networkshomes.soic.indiana.edu/classes/spring2018/info/i529-yye/... · 2017-04-17 · Convolutional Neural Networks Yuzhen Ye School of Informatics and Computing,

ConvolutionalNeuralNetworks

YuzhenYeSchoolofInformaticsandComputing,IndianaUniversity

Contents

• CNNbasics• CNNforvisualrecognition(toexplaintheconceptofconvolution)• CNNforbioinformatics

ABeginner'sGuideToUnderstandingCNN• NeuralNetworkswithConvolutionlayers

Ref1:http://cs231n.stanford.eduRef2:https://adeshpande3.github.io/adeshpande3.github.io/A-Beginner%27s-Guide-To-Understanding-Convolutional-Neural-Networks/

MovingAverageIn2D

0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 0 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 0 0 0 0 0 0 0

0 0 90 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

0 10 20 30 30 30 20 10

0 20 40 60 60 60 40 20

0 30 60 90 90 90 60 30

0 30 50 80 80 90 60 30

0 30 50 80 80 90 60 30

0 20 30 50 50 60 40 20

10 20 30 30 30 30 20 10

10 10 10 0 0 0 0 0

5Slidecredit:SteveSeitz

],[ yxf ],[ yxg

CorrelationfilteringSaytheaveragingwindowsizeis2k+1x2k+1:

Loopoverallpixelsinneighborhoodaroundimagepixelf[i,j]

Attributeuniformweighttoeachpixel

Nowgeneralizetoallowdifferentweightsdependingonneighboringpixel’srelativeposition:

Non-uniformweights

6SlideadaptedfromKristenGrauman

Correlationfiltering

Filteringanimage:replaceeachpixelwithalinearcombinationofitsneighbors.

Thefilter“kernel”or“mask”h[u,v]istheprescriptionfortheweightsinthelinearcombination.

Thisiscalledcross-correlation,denoted

7Slidecredit:MichaelS.Ryoo

],[ yxf ],[ yxg

Averagingfilter• Whatvaluesbelonginthekernelh forthemovingaverageexample?

0 10 20 30 30

0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 0 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 0 0 0 0 0 0 0

0 0 90 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

111111111

“boxfilter”

?

8Slidecredit:MichaelS.Ryoo

],[ vuh

• Whatifwewantnearestneighboringpixelstohavethemostinfluenceontheoutput?

• Removeshigh-frequencycomponentsfromtheimage(“low-passfilter”).

Gaussianfilter

0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 0 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 0 0 0 0 0 0 0

0 0 90 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

1 2 1

2 4 2

1 2 1

This kernel is an approximation of a 2d Gaussian function:

9Slidecredit:SteveSeitz

],[ yxf

],[ vuh

f

h

Convolution• Convolutionisasimplemathematicaloperationwhichisfundamentaltomanycommonimageprocessingoperators.

• Convolutionisperformedbyslidingthekernelovertheimage,generallystartingatthetopleftcorner,soastomovethekernelthroughallthepositionswherethekernelfitsentirelywithintheboundariesoftheimage.

• Convolution:• Flipthefilterinbothdimensions(bottomtotop,righttoleft)• Thenapplycross-correlation

Notationforconvolutionoperator

10Slidecredit:MichaelS.Ryoo

Convolutionvs.correlationConvolution

Cross-correlation

11SlideadaptedfromKristenGrauman

Derivativesandedges

imageintensity function

(along horizontal scanline) first derivative

edges correspond toextrema of derivative

An edge is a place of rapid change in the image intensity function.

12Slidecredit:SvetlanaLazebnik

DerivativeswithconvolutionFor2Dfunction,f(x,y),thepartialderivativeis:

Fordiscretedata,wecanapproximateusingfinitedifferences:

Toimplementaboveasconvolution,whatwouldbetheassociatedfilter?

ee

e

),(),(lim),(0

yxfyxfxyxf -+=

¶¶

®

1),(),1(),( yxfyxf

xyxf -+»

¶¶

13Slidecredit:KristenGrauman

Partialderivativesofanimage

Which shows changes with respect to x?

-1 1

1 -1

or?

-1 1

xyxf

¶¶ ),(

yyxf

¶¶ ),(

(showing filters for correlation) 14Slidecredit:KristenGrauman

Filtersasfeature(edge)detectors

sobel filter

15Slidecredit:KristenGrauman http://homepages.inf.ed.ac.uk/rbf/HIPR2/index.htm

ImagegradientThegradientofanimage:

Thegradientpointsinthedirectionofmostrapidchangeinintensity

The gradient direction (orientation of edge normal) is given by:

The edge strength is given by the gradient magnitude

16Slidecredit:SteveSeitz

EffectsofnoiseConsiderasingleroworcolumnoftheimage

• Plottingintensityasafunctionofpositiongivesasignal

Where is the edge?17

Slidecredit:SteveSeitz

Effectsofnoise

•Differencefiltersrespondstronglytonoise• Imagenoiseresultsinpixelsthatlookverydifferentfromtheirneighbors• Generally,thelargerthenoisethestrongertheresponse

•Whatcanwedoaboutit?

Slidecredit:MichaelS.Ryoo

Where is the edge?

Solution:smoothfirst

Look for peaks in 19Slidecredit:KristenGrauman

Derivativetheoremofconvolution

Differentiationpropertyofconvolution.

20Slidecredit:SteveSeitz

LaplacianofGaussianConsider

Laplacian of Gaussianoperator

Where is the edge? Zero-crossings of bottom graph 21Slidecredit:SteveSeitz

2Dedgedetectionfilters

• is the Laplacian operator:

Laplacian of Gaussian

Gaussian derivative of Gaussian

22Slidecredit:SteveSeitz

Two commonly used discrete approximations to the Laplacian filter

http://onlinelibrary.wiley.com/doi/10.15252/msb.20156651/full#msb156651-fig-0002

CNNforbiologicalimageanalysis

MolecularSystemsBiologyVolume12,Issue7,29JUL2016DOI:10.15252/msb.20156651http://onlinelibrary.wiley.com/doi/10.15252/msb.20156651/full#msb156651-fig-0002

CNNforpredictingmoleculartraits

Inputdata:one-dimensionalgenomicsequenceswithonechannelpernucleotide

[Visualrecognition:2D-imagewiththreecolorchannels]