Unit8-SDG

Digital Image Processing Morphology Basic Morphology Concepts MathematicalMorphologyisbasedonthealgebraofnon-linearoperators operatingonobjectshapeandinmanyrespectssupersedesthelinear algebraic system of convolution. Itperformsinmanytaskspre-processing,segmentationusingobject shape,andobjectquantificationbetterandmorequicklythanthe standard approach. Mathematical morphology tool is different from the usual standard algebra and calculus. Morphology tools are implemented in most advanced image analysis. Mathematical morphology is very often used in applications where shape of objectsandspeedisanissueexample:analysisofmicroscopicimages, industrial inspection, optical character recognition, and document analysis. Thenon-morphologicalapproachtoimageprocessingisclosetocalculus, beingbasedonthepointspreadfunctionconceptandlinear transformations such as convolution. Mathematicalmorphologyusestoolsofnon-linearalgebraandoperates with point sets, their connectivity and shape. Morphologyoperationssimplifyimages,andquantifyandpreservethe main shape characteristics of objects. Morphological operations are used for the following purpose: Image pre-processing (noise filtering, shape simplification). Enhancingobjectstructure(skeletonzing,thinning,thickening,convex hull, object marking). Segmenting objects from the background. Quantitativedescriptionofobjects(area,perimeter,projections,Euler-Poincare characteristics). Mathematicalmorphologyexploitspointsetproperties,resultsofintegral geometry, and topology. Therealimagecanbemodeledusingpointsetsofanydimension;the Euclidean2Dspaceanditssystemofsubsetsisanaturaldomainfor planar shape description. Set difference is defined by \ =

ComputervisionusesthedigitalcounterpartofEuclideanspacesetsof integerpairs( )forbinaryimagemorphologyorsetsofinteger triples( ) for gray-scale morphology or binary 3D morphology. Discrete grid can be defined if the neighborhood relation between points is welldefined.Thisrepresentationissuitableforbothrectangularand hexagonal grids. Amorphologicaltransformation isgivenbytherelationoftheimage withanothersmallpointsetBcalledstructuringelement.Bisexpressed with respect to a local origin . Structuringelementisasmallimage-usedasamovingwindow--whose support delineates pixel neighborhoods in the image plane. It can be of any shape, size, or connectivity (more than 1 piece, have holes). Toapplythemorphologicaltransformation()totheimagemeans thatthestructuringelementBismovedsystematicallyacrosstheentire image.AssumethatBispositionedatsomepointintheimage;thepixelinthe imagecorrespondingtotherepresentativepointOofthestructuring element is called the current pixel. The result of the relation between the image X and the structuring element B in the current position is stored in the output image in the current image pixel position.

Fig 1: Typical structuring elements. Thedualityofmorphologicaloperationsisdeducedfromtheexistenceof thesetcomplement;foreachmorphologicaltransformation()there exists a dual transformation () () = ()

Thetranslationofthepointsetbythevectorisdenotedby ;itis defined by = , = + . Fig 2: Translation by a vector Morphological Principles:1.Compatibilitywithtranslation:Letthetransformationdependonthe positionoftheoriginoftheco-ordinatesystem,anddenotesucha transformationby .Ifallpointsaretranslatedbythevector ,itis expressed as . The compatibility with translation principle is given by() = () . Ifdoesnotdependonthepositionoftheorigin,thenthecompatibility with translation: principle reduces to invariance under translation () = () . 2.Compatibilitywithchangeofscale:Let representthehomothetic scalingofapointset(i.e.,theco-ordinatesofeachpointofthesetare multipliedbysomepositiveconstant ).Thisisequivalenttochangeof scalewithrespecttosomeorigin.Let denoteatransformationthat depends on the positive parameter(change of scale). Compatibility with change of scale is given by () = 1Ifdoesnotdependonthescale ,thencompatibilitywithchangeof scale reduces to invariance to change of scale () = ()3.Localknowledge:Thelocalknowledgeprincipleconsidersthesituationin which only a part of a larger structure can be examinedthis is always the caseinreality,duetotherestrictedsizeofthedigitalgrid.The morphologicaltransformation satisfiesthelocalknowledgeprincipleif foranyboundedpointset inthetransformation ()thereexistsa boundedset ,knowledgeofwhichissufficienttoprovide .Thelocal knowledge principle may be written symbolically as ) = ) .4.Uppersemi-continuity:Theuppersemi-continuityprinciplesaysthatthe morphological transformation does not exhibit any abrupt changes. Binary Dilation and Erosion Thesetsofblackandwhitepixelsconstituteadescriptionofabinary image.Assumeonlyblackpixelsisconsidered,andtheothersaretreated as a background. Theprimarymorphologicaloperationsaredilationanderosion,andfrom thesetwo,morecomplexmorphologicaloperationssuchasopening, closing, and shape decomposition can be constituted. Dilation Themorphologicaltransformationdilation combinestwosetsusing vector addition (e.g., (a, b) +(c, d) = (a+c, b+d)). The dilation is the point set of all possible vector additions of pairs of elements, one from each of the sets and = : = +, } Example: ={(1,0),(1,1),(1,2),(2,2),(0,3),(0,4)}, = {(0, 0), (1, 0)}, ={(1,0),(1,1),(1,2),(2,2),(0,3),(0,4),(2,0),(2,1),(2,2),(3,2),(1,3),(1,4)} Fig 3: Dilation Fig4shows256x256originalimageontheleft.Astructuringelementsize 3x3 is used. TheresultofdilationisshownontherightsideofFig4.Inthiscasethe dilation is an isotropic expansion (Fill or Grow).

Fig 4: Dilation as isotropic expansion Dilationwithanisotropic3x3structuringelementmightbedescribedasa transformation which changes all background pixels neighboring the object to object pixels. Dilation properties: Dilation operation is commutative = Dilation operation is associative ( ) = ( ) Dilation may also be expressed as a union of shifted point sets = Invariant to translation = ( ) Dilation is an increasing transformation Dilationisusedtofillsmallholesandnarrowgulfsinobjects.Itincreases the object size if the original size needs to be preserved, and then dilation is combined with erosion. Fig 5: Dilation where the representative point is not a member of the structuring element. Fig5illustratestheresultofdilationiftherepresentativepointisnota memberofthestructuringelementB,ifthisstructuringelementisused; the dilation result is substantially different from the input set. Erosion Erosion combines two sets using vector subtraction of set elements and is dual operator of dilation. = : = + }. This formula says that every point from the image is tested; the result of the erosion is given by those points for which all possible + are in X. Example: ={(1,0),(1,1),(1,2),(0,3),(1,3),(2,3),(3,3),(1,4)}, = {(0, 0), (1, 0)}, = {(0, 3), (1, 3), (2, 3)}. Fig 6: Erosion The result of the erosion is shown in the right side of the Fig 7. Erosion with an isotropic structuring element is called as shrink or reduce. Fig 7: Erosion as isotropic shrink Basicmorphologicaltransformationscanbeusedtofindthecontoursof objectsinanimageveryquickly.Thiscanbeachieved,forinstance,by subtraction from the original picture of its eroded version as in Fig 8. Fig 8: Contours obtained by subtraction of an eroded image from the original (left). Erosionisusedtosimplifythestructureofanobject.Itdecomposes complicated object into several simple ones. The equivalent definition for erosion = : }. Theerosionmightbeinterpretedbystructuringelementslidingacross theimage ;then,iftranslatedbythevectoriscontainedinthe image , the point corresponding to the representative point belongs to the erosion . Animplementationoferosionmightbesimplifiedbynotingthatan image erodedbythestructuringelementcanbeexpressedasan intersection of all translations of the image by the vector = . Iftherepresentativepointisamemberofthestructuringelement,then erosionisananti-extensivetransformation;thatis,if(0,0) ,then .Erosion properties: Erosion is translation invariant = ( ), = ( ), Erosion as increasing transformation: Erosion and dilation are dual transformations ( ) = . Erosion is not commutative . Erosion and intersection combined together ( ) = ( ) ( ), ( ) ( ) ( ). Image intersection and dilation cannot be interchanged; the dilation of the intersection of two images is contained in the intersection of their dilations ( ) = ( ) ( ) ( ). The order of erosion may be interchanged with set union. This fact enables thestructuringelementtobedecomposedintoaunionofsimpler structuring elements ( ) = ( ) = ( ) ( ), ( ) ( ) ( ), ( ) = ( ) ( ). Successivedilationoftheimagefirstbythestructuringelementand then by the structuring element is equivalent to the dilation of the image X by ( ) = ( ), ( ) = ( ). Hit-or-miss transformation Hit-or-misstransformationisthemorphologicaloperatorforfindinglocal patterns of pixels, where local means the size of the structuring element. Itisavariantoftemplatematchingthatfindscollectionsofpixelswith certain shape properties. Structuringelement ,TestedpointsX,operationdenotedbyapairof disjoint sets = (, ), called a composite structuring element. The Hit-or-miss transformation is defined as = : }. Findinglocalpatternsinimagetestsobjects,background (complement), Useful for finding corners, for instance. Thehit-or-misstransformationoperatesasabinarymatchingbetweenan imageandthestructuringelement(, ).Itmaybeexpressedusing erosions and dilations as well = ( ) ( ) = ( )( ). Opening and closing Erosion and dilation are not inverse transformationif an image is eroded and then dilated, the original image is not re-obtained. Erosionfollowedbydilationiscalledopening.Theopeningofanimage by the structuring element is denoted by and is defined as = (X B) B. Dilation followed by erosion is called Closing. The closing of an image by the structuring element is denoted by and is defined as = ( ) . If an image is unchanged by opening with the structuring element , it is calledopenwithrespectto .Similarly,ifanimageisunchangedby closing with , it is called as closed with respect to . Openingandclosingwithanisotropicstructuringelementisusedto eliminatespecificimagedetailssmallerthanthestructuringelementthe global shape of the objects is not distorted. Closingconnectsobjectsthatareclosetoeachother,fillsupsmallholes, and smoothes the object outline by filling up narrow gulfs. Fig 9: Opening (original on left) Fig 10: Closing (original on left) Opening and closing are invariant to translation of the structuring element. Opening is anti-extensive ( ) and closing is extensive ( ). Opening and closing are dual transformations ( ) = . Iterativelyusedopeningandclosingsareidempotent,meaningthat reapplication of these transformations does not change the previous result. = ( ) , = ( ) . Gray scale dilation and erosion Binarymorphologicaloperationsareextendibletogray-scaleimagesusing the min and max operations. Erosionassignstoeachpixelminimumvalueinaneighborhoodof corresponding pixel in input imagestructuring element is richer than in binary casestructuringelementisafunctionoftwovariables,specifies desired local gray-level propertyValueofstructuringelementissubtractedwhenminimumis calculated in the neighborhood. Dilationassignsmaximumvalueinneighborhoodofcorrespondingpixel in input imagevalueofstructuringelementisaddedwhenmaximumis calculated in the neighborhoodSuch extension permits topographic view of gray scale images Gray-levelisinterpretedasheightofaparticularlocationofa hypothetical landscape Lightanddarkspotsintheimagecorrespondtohillsand valleys Suchmorphologicalapproachpermitsthelocationofglobal propertiesoftheimagesasvalleys,mountainridges(crests), watersheds. Top surface, Umbra, and gray-scale dilation and erosion Consider a point set in -dimensional Euclidean space, Assumefirst( 1)co-ordinatesofthesetconstituteaspatialdomain and the co-ordinate corresponds to the value of a function or functions at a point. The top surface of a set is a function defined on the ( 1) dimensional support. Foreach( 1)tuple,thetopsurfaceisthehighestvalueofthelastco-ordinate of (Fig 11). Fig 11: Top surface of the set corresponds to maximal values of the function (, ) Let andthesupport = , (, )}.The Top surface of , denoted by [], is a mapping defined as []() = max, (, ) }. Umbraofafunctiondefinedonsomesubset(support)of( 1) dimensional space.Umbra is a region of complete shadow resulting from obstructing the light by a non-transparent object. Inmathematicalmorphology,theumbraof isasetthatconsistsofthe top surface of and everything below it (Fig 12). Fig 12: Umbra of the top surface of a set is the whole subspace below it. Let and : .The umbra of , denoted by[], [] ,is defined by [] = (, ) , ()}. Umbra of an umbra of is an umbra. Top surface and umbra in the case of a simple 1D gray scale image (Fig 13)

Fig 13: Example of a 1D function (left) and its umbra (right). The gray-scale dilation of two functions as the top surface of the dilation of their umbras can be defined. Let, : : .Thedilationof , : is defined by = [] []}. on the left-hand side is dilation in the gray-scale image, on the left-hand side is dilation of binary image. No new symbol is introduced here, same applies to erosion also. Forbinarydilation,onefunction,sayrepresentsanimageandthe second,smallstructuringelement.Fig14showsthefunctionthatwill play the role of structuring element. Fig 15 shows the dilation of the umbra of by the umbra of . Fig 14: A structuring element: 1D function (left) and its umbra (right). Fig15:1Dexampleofgray-scaledilation.Theumbrasofthe1Dfunctionandstructuring elementaredilatedfirst,[] [].Thetopsurfaceofthisdilatedsetgivestheresult, =[] []}. Acomputationallyplausiblewaytocalculatedilationcanbeobtainedby taking the maximum of set of sums: ( )() = max( ) +(), , }. Thecomputationalcomplexityisthesameasforconvolutioninlinear filtering, where a summation of products is performed. The definition of gray-scale erosion is analogous to gray-scale dilation. The gray-scale erosion of two functions(point sets) Takes their umbras. Erodes them using binary erosion. Gives the result as the top surface. Let , : : . The erosion of , : is defined by = [] []}. Todecreasecomputationalcomplexity,theactualcomputationsare performed in another way as the minimum of a set of differences: ( )() = min( + ) ()}. Fig16:1Dexampleofgray-scaleerosion:Theumbrasof1Dfunctionandthestructuring elementareerodedfirst,[] [].Thetopsurfaceofthiserodedsetgivestheresult, = [] []}. Example (Fig 17) This figure illustrates morphological pre-processing on a microscopic image of cells corrupted by noise.The aim is to reduce noise and locate individual cells. A 3x3 structuring element was used in erosion and dilation. Theindividualcellscanbelocatedbythereconstructionoperation.The originalimageisusedasamaskandthedilatedimageisaninputfor reconstruction. Fig17:Morphologicalpre-processing:(a)cellsinamicroscopicimagecorruptedby notes;(b)Eroded image;(c)Dilation of original image;(d)Reconstructed cells. Umbrahomeomorphismtheorem,propertiesoferosionanddilation,opening and closing. The Umbra homeomorphism theorem states that the umbra operation is a homeomorphism from gray-scale morphology to binary morphology. Let , : : . Then (a) [ ] = [] [], (b) [ ] = [] []. Theumbrahomeomorphismisusedforderivingpropertiesofgray-scale operations. Gray-scale openingis defined as = ( ) . Gray-scale closing is defined as = ( ) . The duality between opening and closing is expressed as( )() = () (). The opening of f by structuring element k can be interpreted as sliding k on the landscape f. The position of all highest points reached by some part of k during the slide gives the opening, similar interpretation exists for erosionGray-scaleopeningandclosingoftenusedtoextractpartsofagray-scale image with given shape and gray-scale structure. Top hat transformation The top hat transformation is used as a simple tool for segmenting objects ingray-scaleimagesthatdifferinbrightnessfrombackground,evenwhen the background is of uneven gray-scale. Thetophattransformissupersededbythewatershedsegmentationfor more complicated backgrounds. Assume a gray-level image and a structuring element. The residue of openingascomparedtooriginalimage ( )constitutesanew useful operation called a Top hat transformation. Thetophattransformationisagoodtoolforextractinglightobjectsona dark but slowly changing background. Those parts of the image that cannot fit into structuring element are removed by opening. Subtractingtheopenedimagefromtheoriginalprovidesanimagewhere removed objects stand out. The actual segmentation can be performed by simple thresholding (Fig 18). If an image were a hat, the transformation would extract only the top of it, provided that the structuring element is larger than the hole in the hat. Fig 18:The top hat transform permits the extractionof light objects from an uneven background. Example from visual industrial inspection Afactoryproducingglasscapillariesformercurymaximalthermometers had the following problem: Thethinglasstubeshouldbenarrowedinoneparticularplacetoprevent mercuryfallingbackwhenthetemperaturedecreasesfromthemaximal value.Thisisdonebyusinganarrowgasflameandlowpressureinthe capillary. Thecapillaryisilluminatedbyacollimatedlightbeamwhenthecapillary wallcollapsesduetoheatandlowpressure,aninstantspecularreflection is observed and serves as a trigger to cover the gas flame. Originally the machine was controlled by a human operator who looked at thetubeimageprojectedopticallyonthescreen;thegasflamewas covered when the specular reflection was observed. Thistaskhadtobeautomatedandthetriggersignallearnedfroma digitizedimage.Thespecularreflectionisdetectedbyamorphological procedure (Fig 19). Fig 19: An industrial example of gray-scale opening and top hat segmentation, i.e., image based control of glass tube narrowing by gas flame. (a)Original image of the glass tube, 512x256 pixels. (b)Erosion by a one-pixel-wideverticalstructuringelement20pixelslong.(c)Openingwiththesameelement.(d)Final specular reflection segementation by the top hat transformation. Morphology segmentation and watersheds Particles segmentation, marking, watersheds: Finding objects of interest in the image is called as segmentation. Mathematicalmorphologyhelpsmainlytosegmentimagesoftextureor imagesofparticlesinwhichtheinputimagecanbeeitherbinaryorgray-scale. In the binary case, the task is to segment overlapping particles; in the gray-scale case, the segmentation is the same as object contour extraction. Morphologicalparticle segmentation is performed in two basic steps: Location of particle markers Watersheds used for particle reconstructionThemarkerofanobjectorsetXisasetMthatisincludedinX.Markers havethesamehomotopyasthesetX,andaretypicallylocatedinthe central part of the object.Applicationspecificknowledgeshouldbeusedformarker-finding technique. Object marking in many cases left to user, who marks objects manually on the screen. Whentheobjectsaremarked,theycanbegrownfromthemarkers,e.g., usingwatershedtransformation,whichismotivatedbythetopographic view of images. Considertheanalogyofalandscapeandrain;waterwillfindtheswiftest descent path until it reaches some lake or sea.Thelandscapecanbeentirelypartitionedintoregionswhichattractwater to a particular sea or lakethese will be called catchment basins. Theseregionsareinfluencezonesoftheregionalminimaintheimage. Watersheds,alsocalledwatershedlines,separatecatchmentbasins. Watersheds and catchment basins are illustrated in the below figure: Fig 20: Illustration of catchment basins and watersheds in a 3D landscape view. Binary morphological segmentationThe top hat transformation method is used to find the objects that differ in brightness from an uneven background. The top hat approach just finds peaks in the image function that differ from the local background. The gray-level shape of the peaks does not play any role, but the shape of the structuring element does. Watershed segmentation takes into account both sources of information and supersedes the top hat method. Morphologicalsegmentationinbinaryimagesaimstofindregions corresponding to individual overlapping objects. Eachparticleismarkedfirstultimateerosionmaybeusedforthis purpose, or markers may be placed manually.Thenexttaskistogrowobjectsfromthemarkersprovidedtheyarekept within the limits of the original set and parts of objects are not joined when they come close to each other. The oldest technique for this purpose is called conditional dilation. Ordinary dilationisusedforgrowing,andtheresultisconstrainedbythetwo conditions. Geodesicreconstructionismoresophisticatedandperformsmuchfaster thanconditionaldilation.Thestructuringelementadaptsaccordingtothe neighborhood of the processed pixel. Geodesic influence zones are sometimes used for segmenting particles. Fig 21: Segmentation by geodesic influence zones(SKIZ) need not lead to correct result. Theoriginalbinaryimageisconvertedintograyscaleusingthenegative distance transform. If a drop of waterfalls onto a topographic surface of the distimage,itfollowsthesteepestslopetowardsaregionalminimum(Fig 22). Fig22:Segmentationofbinaryparticles.(a)Inputbinaryimages.(b)Grayscaleimagecreatedfrom(a) usingthedistfunction.(c)Topographicnotionofthecatchmentbasin.(d)Correctlysegmented particles using watersheds of image (b). ApplicationofwatershedparticlesegmentationisshowninFig23.We selected an image of a few touching particles as an input Fig 23(a). Thedistancefunctioncalculatedfromthebackgroundisvisualizedusing contours in Fig 23(b). Theregionalmaximaofthedistancefunctionserveasmarkersofthe individual particles (Fig 23(c)).Inpreparationforwatershedsegmentation,thedistancefunctionis negated, and is shown together with the dilated markers (Fig 23(d)). ab c d Fig 23: (a) Original binary image (b) level lines distance function (c) maxima of distance function (d) Watersheds of inverted distance function Gray-scale segmentation, watersheds Themarkersandwatershedsmethodcanalsobeappliedtogray-scale segmentation.Watershedsarealsousedascrest-lineextractorsingray-scale images. Thecontourofaregioninagray-levelimagecorrespondstopointsinthe imagewheregray-levelschangemostquicklythisisanalogoustoedge based segmentation. a dc b Thewatershedtransformationisappliedtothegradientmagnitude(Fig 24). Algebraicdifferenceofunit-sizedilationandunit-sizeerosionoftheinput image X () = ( ) ( ).

Fig 24:Segmentation in gray-scale images using gradient magnitude. The main problem with segmentation via gradient images without markers is over-segmentation, meaning that the image is partitioned into too many regions. Thewatershedsegmentationmethodsdonotsufferfromover-segmentation. The below Fig will illustrate the application of watershed segmentation.

Fig 25: Watershed segmentation (a) Original image. (b) Dots are superimposed markers found by non morphological methods. (c) Modified gradient. (d) Watersheds from markers (b). Example Image: Fig 26: Illustration of segmentation method