Medical Image Analysis Image Representation and Analysis Figures come from the textbook: Medical Image Analysis, by Atam P. Dhawan, IEEE Press, 2003.

Medical Image AnalysisMedical Image AnalysisImage Representation and Analysis

Figures come from the textbook: Medical Image Analysis, by Atam P. Dhawan, IEEE Press, 2003.

Image Representation and Image Representation and AnalysisAnalysis

Figures come from the textbook: Medical Image Analysis, by Atam P. Dhawan, IEEE Press, 2003.

A hierarchical framework of processing steps representing the image (data) and knowledge (model) domains

Scenes of specific objectsSurface regions (S-regions)RegionContours and edgesPixels

Bottom-Up

Scenario

Scene-1 Scene-I

Object-1 Object-J

S-Region-1 S-Region-K

Region-1 Region-L

Pixel (i,j)

Edge-MEdge-1

Pixel (k,l)

Top-Down

Figure 8.1. A hierarchical representation of image features.

Image Reconstruction

ImageSegmentation

(Edge and Region)

Feature Extractionand

Representation

Classificationand

Object Identification

Analysisof Classified Objects

Multi-Modality/Multi-Subject/Multi-Dimensional

Registration, Visualization and Analysis

Raw Data from Imaging System

Single ImageUnderstanding

Multi-Modality/ Multi-Subject/Multi-Dimensional

Image Understanding

Scene Representation

Models

Object Representation

Models

Feature Representation

Models

Edge/Region Representation

Models

Physical Property/Constraint

Models

Knowledge Domain

DataDomain

Figure 8.2. A hierarchical structure of medical image analysis.

Feature Extraction and Feature Extraction and RepresentationRepresentation

Figures come from the textbook: Medical Image Analysis, by Atam P. Dhawan, IEEE Press, 2003.

Statistical pixel-level (SPL) features◦Mean, variance, histogram, area,

contrast of pixels within the region, edge gradient of boundary pixels

Shape feature◦Circularity, compactness, moments,

chain-codes and Hough transform, morphological processing methods

Feature Extraction and Feature Extraction and RepresentationRepresentation

Figures come from the textbook: Medical Image Analysis, by Atam P. Dhawan, IEEE Press, 2003.

Texture features◦Second-order histogram statistics or

co-occurrence matrices, wavelet processing methods for spatio-frequency analysis

Relational features◦Relational and hierarchical structure

of the regions associated with a single or a group of objects

Statistical Pixel-Level (SPL) Statistical Pixel-Level (SPL) FeaturesFeaturesHistogram

Mean

Variance and central moments

n

rnrp ii

)()(

1

0

)(1 L

iii rpr

nm

1

0

))((L

i

niin mrrp

Statistical Pixel-Level (SPL) Statistical Pixel-Level (SPL) FeaturesFeatures

◦The third central moment is a measure of noncentrality

◦The fourth central moment is a measure of flatness of the histogram

Energy

1

0

2)]([L

iirpE

Statistical Pixel-Level (SPL) Statistical Pixel-Level (SPL) FeaturesFeaturesEntropy

◦The entropy Ent is a measure of information represented by the distribution of gray-values in the region

1

02 )(log)(

L

iii rrpEnt

Statistical Pixel-Level (SPL) Statistical Pixel-Level (SPL) FeaturesFeaturesLocal contrast

Maximum, minimumThe mean, variance, energy and

entropy of contrast valuesGradient information for the

boundary pixels

),(),,(max

),(),(),(

yxPyxP

yxPyxPyxC

sc

sc

Shape FeaturesShape FeaturesLongest axis GEShortest axis HFPerimeter and area of the minimum

bounded rectangle ABCDElongation ratio: GE/HFPerimeter and the area of the

segmented regionHough transform of the region using

the gradient information of the boundary pixels of the region

p A

Shape FeaturesShape FeaturesCircularity ( = 1 for a circle) of

the region computed as

Compactness of the region computed as

C

2

4

p

AC

pC

A

pC p

2

Shape FeaturesShape FeaturesChain code for boundary contour

◦Obtained using a set of orientation primitives on the boundary segments derived from a piecewise linear approximation

Fourier descriptor of boundary contours◦Obtained using the Fourier transform

of the sequence of boundary segments derived from a piecewise linear approximation

Shape FeaturesShape FeaturesCentral moments based shape

features for the segmented region

Morphological shape descriptors◦Obtained through the morphological

processing on the segmented region

Boundary Encoding: Chain Boundary Encoding: Chain CodeCodeOrientation primitives

◦8-connected neighborhoodDivide-and-conquer

◦Curve approximationMaximum-deviation criterion

◦Perpendicular distance between any point on the original curve segment between the selected vertices and the corresponding approximated straight-line segment

Figures come from the textbook: Medical Image Analysis, by Atam P. Dhawan, IEEE Press, 2003.

04

23 1

5 6 7

xc 04

23 1

5 6 7

Figure 8.4. The 8-connected neighborhood codes (left) and the orientation directions (right) with respect to the center pixel xc.

FA D

C

E

B

A D

C

E

B

A D

C

E

B

A D

C

E

B

A

B C

D Chain Code: 110000554455533

Figure 8.5. A schematic example of developing chain code for a region with boundary contour ABCDE. From top left to bottom right: the original boundary contour, two points A and C with maximum vertical distance parameter BF, two segments AB and BC approximating the contour ABC, five segments approximating the entire contour ABCDE, contour approximation represented in terms of orientation primitives, and the respective chain code of the boundary contour.

Boundary Encoding: Fourier Boundary Encoding: Fourier DescriptorDescriptorClosed boundary of a region

Discrete Fourier transform (DFT) of the sequence

Rigid geometric transformation of a boundary◦Translation, rotation, scaling

)()()( niynxnu 1,...,2,1,0 Nn

1

0

/ 2)(1

][N

n

Nind enu

Nn F

Moments for Shape Moments for Shape DescriptionDescriptionCentral moments of a segmented

image

Invariant moments◦Shape matching, pattern recognition

L

i

L

j

qj

pipq yxfyyxx

1 1

),()()(

L

i

L

jiii yxfxx

1 1

),(

L

i

L

jiii yxfyy

1 1

),(

Figures come from the textbook: Medical Image Analysis, by Atam P. Dhawan, IEEE Press, 2003.

Set A

Set B

Figure 8.6. A large region with square shape representing the set A and a small region with rectangular shape representing the structuring element set B.

: Dilation of A by B

A B: Erosion of A by B

( A B) B

A A

BA BA

A B

Figure 8.7: The dilation of set A by the structuring element set B (top left), the erosion of set A by the structuring element set B (top right) and the result of two successive erosions of set A by the structuring element set B (bottom).

A

B

BA

BAFigure 8.8. Dilation and erosion of an arbitrary shape region A (top left) by a circular structuring element B (top right): dilation of A by B (bottom left) and erosion of A by B (bottom right).

Figure comes from the Wikipedia, www.wikipedia.org.

Dilation

Figure comes from the Wikipedia, www.wikipedia.org.

Erosion

Morphological Processing for Morphological Processing for Shape DescriptionShape DescriptionOpening

Closing

BBABA )(

BBABA )(

AB

BA BA Figure 8.9. The morphological opening and closing of set A (top left) by the structuring element set B (top right): opening of A by B (bottom left) and closing of A by B (bottom right).

Figure comes from the Wikipedia, www.wikipedia.org.

Opening

Figure comes from the Wikipedia, www.wikipedia.org.

Closing

Morphological Processing for Morphological Processing for Shape DescriptionShape DescriptionSkeleton

Image processing◦Erosion can reduce the background

noise◦Opening can remove the speckle noise

and provide smooth contours

N

nn AKAK

0

)()(

BnBAnBAAKn )()()(

Morphological Processing for Morphological Processing for Shape DescriptionShape DescriptionImage processing

◦Closing preserves the peaks and reduces the sharp variations in the signal such as dark artifacts

◦Opening followed by closing can reduce the bright and dark artifacts and noise

◦The morphological gradient image can be obtained by subtracting the eroded image from the dilated image

◦Edges can also be detected by subtracting the eroded image from the original image

Figure 8.10. Example of morphological operations on MR brain image using a structuring element of

(a) the original MR brain image; (b) the thresholded MR brain image for morphological operations; (c) dilation of the thesholded MR brain image; (d) resultant image after 5 successive dilations of the thresholded brain image; (e) erosion of the thresholded MR brain image; (f) closing of the thesholded MR brain image; (g) opening of the thresholded MR brain image; and (h) morphological boundary detection on the thresholded MR brain image.

10

01

(b)(a)

(c) (d)

(f)(e)

Figures come from the textbook: Medical Image Analysis, by Atam P. Dhawan, IEEE Press, 2003.

(g) (h)

Texture FeaturesTexture FeaturesTexture

◦Statistical◦Structural

A repetitive arrangement of square and triangular shapes

◦Spectral Fourier and wavelet transforms

Gray-level co-occurrence matrix (GLCM)◦ is the distribution of the number of

occurrences of a pair of gray values and separated by a distance vector

),( jipi j

],[ dydxd

2 2 2 0 10 2 2 1 10 1 1 2 01 2 2 0 12 1 0 1 1

0 3 1 02 1 0 11 4 3 2 i0 1 2

j

(a)

(b)

Figure 8.11. (a) A matrix representation of a 5x5 pixel image with three gray values; (b) the GLCM P(i,j) for d=[1,1].

Texture FeaturesTexture Features

◦The probability of occurrence of a pair of gray values and separated by a distance vector

,◦The probability that a difference in

gray-levels exists between two distinct pixels

),,( drq yyH

qy ryd),( dsd yH

rqs yyy

Second-Order Histogram Second-Order Histogram StatisticsStatisticsEntropy of

Angular second moment of

),,( drq yyH

t

q

t

r

y

yy

y

yyrqrqH yyHyyHS

1 1

)],,([log),,( 10 dd

),,( drq yyH

t

q

t

r

y

yy

y

yyrqH yyHASM

1 1

2)],,([ d

Second-Order Histogram Second-Order Histogram StatisticsStatisticsContrast of

Inverse difference moment of

),,( drq yyH

t

q

t

r

t

q

t

r

y

yy

y

yyrqrq

y

yy

y

yyrqrq yyHyyyyHyy

1 11 1

),,()(),,(),( 2 dd

),,( drq yyH

t

q

t

r

y

yy

y

yy rq

rqH yy

yyHIDM

1 1),(1

),,( d

Second-Order Histogram Second-Order Histogram StatisticsStatisticsCorrelation of ),,( drq yyH

t

q

t

r

rq

rq

y

yy

y

yyrqyryq

yyH yyHyyCor

1 1

),,())((1

d

t

r

y

yyrqqm yyHyH

1

),,(),( dd

t

q

y

yyrqrm yyHyH

1

),,(),( dd

Second-Order Histogram Second-Order Histogram StatisticsStatisticsMean of

Deviation of

),,( drq yyH

),( dqm yH

t

r

m

y

yyqmqH yHy

1

),( d

t

q

t

r

m

y

yyqm

y

yyrmrqH yHyHyy

1 1

),(),(

2

dd

Second-Order Histogram Second-Order Histogram StatisticsStatisticsEntropy of

Angular second moment of

),( dsd yH

t

s

sd

y

yysdsdyH yHyHS

1

)],([log),( 10),( ddd

),( dsd yH

t

s

sd

y

yysdyH yHASM

1

2),( )],([ dd

Second-Order Histogram Second-Order Histogram StatisticsStatisticsMean of ),( dsd yH

t

s

sd

y

yysdsyH yHy

1

),(),( dd

Figures 8.12 (a) A part of a digitized X-ray mammogram showing a region of benign lesion (b) a part of a digitized X-ray mammogram showing a region of malignant cancer of the breast (c). A second-order histograms of (a) computed from the gray-level co-occurrence matrices with a distance vector of [1,1] and (d) A second-order histogram of (b) computed from the gray-level co-occurrence matrices with a distance vector of [1,1] .

(a) (b)

(c)

(d)

Relational FeaturesRelational FeaturesRelational features

◦Information about adjacencies, repetitive patterns and geometrical relationships among regions of an object

Quad-tree representationTree and graph structures

Figures come from the textbook: Medical Image Analysis, by Atam P. Dhawan, IEEE Press, 2003.

R1

R21 R22

R23

R41

R43

R24

R42

R44

R3

Figure 8.13: A block representation of an image with major quad partitions (top) and its quad-tree representation.

Figures come from the textbook: Medical Image Analysis, by Atam P. Dhawan, IEEE Press, 2003.

R

R4

R3

R2

R1

R24

R23

R22

R21

R44

R43

R42

R41

R14

R13

R12

R11

R34

R33

R32

R31

A

C

B

D

F

I

EB

C

A

I

ED

F

Figure 8.14. A 2-D brain ventricles and skull model (top) and region-based tree representation.

Feature and Image Feature and Image ClassificationClassificationStatistical classification methods

◦Unsupervised: k-means, fuzzy clustering

◦SupervisedNearest neighbor classifier

◦Assigned to the class if

jjD uff )(

jcf

jj

j Nfu

1

Cj ,...,2,1ic

)(min)( 1 ff jCji DD

Feature and Image Feature and Image ClassificationClassificationBayes classifier

◦Risk of wrong classification for assigning the feature vector to the class

◦Assigned to the class if

C

kkkjj cpZr

1

)|()( ff

Cj ,...,2,1

jc

ic

C

kkkj

C

kkki cpZcpZ

11

)|()|( ff

Feature and Image Feature and Image ClassificationClassificationRule-based systems

◦Analyze the feature vector using multiple sets of rules that are designed to check specific conditions in the database of feature vectors to initiate an action

Strategy RulesA priori

knowledgeor models

Focus of Attention Rules

Knowledge Rules

ActivityCenter

InputDatabase

OutputDatabase

Figure 8.15. A schematic diagram of a rule-based system for image analysis.

Feature and Image Feature and Image ClassificationClassificationImage and feature classification:

neural networks◦Backpropagation◦Radial basis function◦Associative memories◦Self-organizing

Neuro-fuzzy pattern classification

Figures come from the textbook: Medical Image Analysis, by Atam P. Dhawan, IEEE Press, 2003.

X f()

1

w2

w0

w1

wd

f():

Y

Figure 8.16. A computational neuron model with linear synapses.

Figures come from the textbook: Medical Image Analysis, by Atam P. Dhawan, IEEE Press, 2003.

M1

winner-take-alloutput layer

L

1

fuzzy membershipfunction layer

x1

xi

xd

hyperplanelayer

inputlayer

max

M2

MK

C

Figure 8.17. The architecture of the Neuro-Fuzzy Pattern Classifier.

Figures come from the textbook: Medical Image Analysis, by Atam P. Dhawan, IEEE Press, 2003.

L

input fromhyperplane

layer

2

1

scaling

1f

2f

Lf

f

multiplication

Mf

outputfuzzy

function

Figure 8.18. The structure of the fuzzy membership function.

Figures come from the textbook: Medical Image Analysis, by Atam P. Dhawan, IEEE Press, 2003.

1 2 3 4 5 6 7 8

1

2

3

4

5

6

7

8

Figure 8.19. Convex set-based separation of two categories.

Figure 8.20. (a). Fuzzy membership function M1(x) for the subset #1 of the black category. (b). Fuzzy membership function M2(x) for the subset #2 of the black category.

(a)

Figures come from the textbook: Medical Image Analysis, by Atam P. Dhawan, IEEE Press, 2003.

(b)

Figure 8.21. Fuzzy membership function M3(x) (decision surface) for the white category membership.

Figure 8.22. Resulting decision surface Mblack(x) for the black category membership function.

Image Analysis Example: Image Analysis Example: Analysis of Difficult-to-Analysis of Difficult-to-Diagnose Mammographic Diagnose Mammographic MicrocalcificationMicrocalcification

Features◦Number of microcalcification◦Average number of pixels per

microcalcification◦…◦Entropy of◦…◦Energy fro the wavelet packet at

Level 0◦…

),,( drq yyH

6D