Regions Machine Vision Course Presented by: Dr. Fariborz Mahmoudi
2
Overview
Introduction to Regions Region Segmentation – Approaches Thresholding - Automatic Region Representation Data Structures Split and Merge
3
What is a Region? Basic definition :- A group of connected pixels with
similar properties.
Important in interpreting an image because they may correspond to objects in a scene.
For correct interpretation, image must be partitioned into regions that correspond to objects or parts of an object.
4
Partitioning – How? Partitioning into regions done often by using gray
values of the image pixels.
Two general approaches :- Region-based segmentation Boundary estimation using edge detection
5
Region-based Approach Pixels corresponding to an object grouped
together and marked.
Important principles:- 1. Value similarity Gray value differences Gray value variance
6
Region-based Approach (continued)
2. Spatial Proximity Euclidean distance Compactness of a region
Assumption: Points on same object will projectto spatially close pixels on the image with similar gray values.
7
Limitation The assumption does not hold true in all cases.
To overcome this, group pixels using given principles and use domain-dependent knowledge.
Match regions to object models.
8
Edge Detection Approach Segmentation by finding pixels on a region
boundary.
Edges found by looking at neighboring pixels.
Region boundary formed by measuring gray value differences between neighboring pixels
9
Segmentation versus Edge Detection
Closed boundaries
Multi-spectral images improve segmentation
Computation based on similarity
Boundaries formed not necessarily closed
No significant improvement for multi-spectral images
Computation based on difference
10
Region Segmentation Problem Statement:- Given a set of image pixels I and a homogeneity
predicate P(.), find a partition S of the image Iinto a set of n regions Ri such that
n
ii IR
1
11
Region Segmentation – Problem Statement (Continued)
P(Ri) = True , for all i
i.e any region satisfies the homogeneity predicate
Any two adjacent regions cannot be merged into a single region
FalseRRP ji
13
Thresholding – A Key Aspect Most algorithms involve establishing a threshold
level of certain parameter.
Correct thresholding leads to better segmentation.
Using samples of image intensity available, appropriate threshold should be set automatically in a robust algorithm i.e. no hard-wiring of gray values
14
Automatic Thresholding Use of one or more of the following knowledge: 1. Intensity characteristics of objects 2. Sizes of objects 3. Fractions of image occupied by objects 4. Number of different types of objects
Size and probability of occurrence – most popular
Intensity distributions estimate by histogram computation.
15
Automatic Thresholding Methods
Some automatic thresholding schemes: 1. P-tile method 2. Mode method 3. Iterative threshold selection 4. Adaptive thresholding 5. Variable thresholding 6. Double thresholding
16
P-Tile Method Uses knowledge about the area or size of the
desired object to threshold an image
objects occupy p% of the image area
17
Mode Method The objects in an image have the same gray
value, the background has a different gray value, and the image pixels are affected by zero-mean Gaussian noise
(a) Ideal case (b) Practice case
18
Mode Method(Cont.)
Algorithm: Peakiness detection for Appropriate Threshold Selection
1. Find the two highest local maxima in the histogram. Suppose these are gi, gj
2. Find Lowest point gk in the histogram between gi, gj
3. Find Peakiness, defined as Min(H(gi),H(gj))/H(gk)
4. Use the combination (gi, gj , gk )with highest peakiness to threshold the image. The value gk is a good threshold to separate objects corresponding to gi, gj
19
Iterative Threshold Selection1. Select an initial estimate of the Threshold, T. such
as average intensity of the image
2. Partition the image into 2 groups: R1 and R2, using the threshold T
3. Calculate the mean gray values μ1 and μ2 of the partitions R1 and R2.
4. Select a new threshold: T= (μ1 + μ2)/2
5. Repeat steps 2 to 4 until the mean value μ1 and μ2 do not change in successive iterations
20
Adaptive (Local) Thresholding Methods to deal with uneven illumination or uneven
distribution of gray values in the background
partition the image into m×m subimages and thresholding each subimage based on the histogram of the subimage
The final segmentation of the image is the union of the regions of its subimages.
23
Variable Thresholding Another useful technique in the case of uneven
illumination
Approximate the intensity values of the image by a simple function such as a plane or biquadratic.
The fit function is determined by the gray value of the background.
Histogramming and thresholding can be done relative to the base level determined by the fitted function.
24
a: original imageb: 3-D plot c: histogramd: T = 85e: T= 165f: planar function( fit
between box & Background)
g: difference plot (a-f)h: the normalized
imagei: histogram of
normalized imagej: result T=110
25
Double Thresholding(1) Select two thresholds T1 and T2.(2) Partition the image into three regions :R1, containing all pixels with gray values below T1.R2 , containing all pixels with gray values between T1 and
T2 , inclusive.R3 , containing all pixels with gray values above T2. where R1=core region, R2 = fringe (intermediate,
transition) region, R3 =background.(3) Visit each pixel in region R2 . If the pixel has a neighbor
in region R1, then reassign the pixel to region R1.(4) Repeat step 3 until no pixels are reassigned.(5) Reassign any pixels left in region R2 to R3 .
26
(a). Original image(b). Histogram
(c). Core T1=70(d). Result after growing the
core using T2=155
(e).Regular Method T= T2
(f). Regular Method T1≤T≤T2
27
Limitation of Histograms
They are applied globally to the entire image. This can be a drawback in complex scenes.
Very useful in initial segmentation on the basis of gray levels.
Spatial information about intensity values is thrown away.
Very different scenes may give a strikingly similar
histogram representation.
Histogram for both images
28
Region Representation Different representations suitable to different
applications. Three general classes:- 1. Array representation - same size arrays, membership arrays 2. Hierarchical representation - pyramid, quad tree 3. Symbolic representation
29
Array representation Uses array of same size as original image with
entries indicating the region to which a pixel belongs.
Image Data
30
Membership Arrays Membership arrays (images) commonly
called masks. Each region has a mask that indicates which pixels belong to which region.
Advantage – a pixel can be allowed to be a member of more than one region.
31
• Preserves all details of regions required in most applications
• Very popular method with lots of hardware support available
• Symbolic information is not explicitly represented.
Characteristics of array representation method
32
Hierarchical Representation Images represented at many resolutions
As resolution , array size and some data is lost. More difficult to recover information.
But, memory and computation requirements are also decreased.
Used to accommodate various segmentation algorithms which operate at different resolutions.
33
Pyramids
An ‘n x n’ image represented by the image and ‘k’ reduced versions of the image.
Pixel at level ‘l’ has combined information from several pixels at level ‘l+1’
Top level – level 0 – single pixel
Bottom level – original image
Simplest method for resolution reduction is averaging.
35
Quad Trees Extension of pyramids for binary images.
Three types of nodes – white, black, gray. White or black node – no splitting.
Gray node – split into 4 sub-regions.
Each node is either a leaf or has 4 children.
37
Quad Tree ExampleNon-binary image
1122 3344
1122 3344
1122 3344
Not uniformNot uniform
uniformuniform
38
Symbolic Representations Commonly used symbolic characteristics: 1. Enclosing rectangle 2. Centroid 3. Moments 4. Euler number
5. Others: Mean, variance of intensity values
39
Data Structures for Segmentation
Data structures used for storing region information during operations like splitting and merging.
Commonly used data structures:- 1. RAG – Region Adjacency Graph 2. Picture trees 3. Super grid
40
RAG (Region Adjacency Graph) Nodes represent regions; arcs between
nodes represent a common boundary between regions.
Different properties of regions stored in node data structure.
After initial segmentation, results stored in RAG and regions combined to obtain a better segmentation.
42
Algorithm: Region Adjacency Graph
(1) Scan the membership array a and perform the following steps at each pixel index [i,j]
(2) Let r1 = a[i,j]
(3) Visit the neighbors [k,l] of the pixel at [i,j]. For each neighbor, perform the following:
(4) Let r2 = a[k,l] , if r1 != r2 , add an arc between nodes r1 and r2 in the region adjacency graph
43
Picture Trees Emphasize inclusion of a region within
another region.
Recursive splitting of an image into component parts. Splitting stops with constant characteristic conditions.
Quad tree is a special case.
45
Super Grid Boundaries should be located between the pixels of
two adjacent regions.
However, often boundaries become actual pixels in image representations.
For NxN image, super grid will be (2N+1) x (2N+1).
Each pixel surrounded by 8 non-pixel points which are used to indicate boundary.
Simplifies merge and split operations.
47
Split and Merge After segmentation the regions may need to be
refined or reformed. Split operation adds missing boundaries by splitting
regions that contain part of different objects. Merge operation eliminates false boundaries and
spurious regions by merging adjacent regions that belong to the same object.
48
Algorithm: Region Merging(1) Form initial regions in the image using thresholding
(or a similar approach) followed by component labeling.
(2) Prepare a region adjacency graph (RAG) for the image.
(3) For each region in an image, perform the following steps: (a) Consider its adjacent region and test to see if they
are similar. (b) For regions that are similar, merge them and modify
the RAG. (4) Repeat step 3 until no regions are merged.
49
similarity Approaches Determining similarity between two regions is most
important step.
Approaches for judging similarity based on:• Gray values
• Have the same mean intensity value• Have the same statistical distribution of intensity value
• Color• Texture• Size• Shape• Spatial proximity and connectedness
50
Region Merging A Statistical Approach
Assumption:- Regions have constant gray values corrupted by statistically independent, zero-mean Gaussian noise
Gray values are now taken from normal distributions. Suppose two adjacent regions R1 and R2 contain m1 and m2
points respectively. Possible hypotheses are: H0 : Both regions belong to same object. Intensities are all drawn
from a single Gaussian distribution with parameters H1: Regions belong to different objects. Intensities are drawn
from separate Gaussian distributions with parameters
These parameters are generally not know, but estimated using the available samples.
2211 ,,,
51
Statistical Approach (Continued) Assume a region contains n pixels having gray levels, gi,
i =1,2,….n with a normal distribution:
The Maximum Likelihood Estimation can be used to determine parameters like mean and variance.
2
2
2
2
1)(
gi
egip
53
Statistical Approach (Continued) Similarly, under H1, assuming normal
distributions for regions R1 and R2, the joint probability density can be obtained.
54
Statistical Approach (Another slide!!) The likelihood ratio, L, is the defined as the ratio of the probability densities under the two
hypotheses:
If L < threshold T, strong likelihood that there is only one region and the two regions may be merged.
NOTE: Can also be applied for edge detection.
Modifications: Possible to assume planar or quadratic intensity distributions instead of constant gray values.
22
11
210
021
121
|,....,
)|,....,(mm
mm
Hggp
HggpL
55
Removing Weak Edgesfor Region Merging
Merge regions if boundary is weak.
Length of common boundary is considered.
Weak boundary – intensities of either side differ by less than amount T
Imp:- Consider relative lengths of the weak and complete boundaries between regions.
56
Removing Weak EdgesTwo main approaches
Approach 1. Merge adjacent regions R1 and R2 if W = length of weak part of boundary S = min(S1,S2) is minimum of two perimeters = a threshold If is too small too many region merges is big very conservative algorithm; valid candidates for
merging might be ignored. Good heuristic is 0.5
S
W
57
Removing edges (Continued)Approach 2. Merge adjacent regions R1 and R2 if
where S = common boundary W = length of weak part of boundary = threshold Good heuristic is 0.75
S
W
58
Region Splitting Property of a region is not constant - SPLIT Segmentation starts with large regions. Whole image can be
used to start with. Important decisions before splitting:- 1. Is a property not constant over a region? It is application dependant and requires knowledge about regions
like: - Variance of intensity values - Function fitting and error calculation
2. Where to split a region? - Finding best boundary for dividing (Measures of edge
strength within region). - Regular decomposition methods (divide in fixed equal region like
quad-tree)
59
Algorithm: Split and Merge Region Segmentation Start with the entire image as a single region
Split: Pick a region R. If H(R) is false, then split the region into four sub regions. H(R)=1 if the variance is small, 0 otherwise
Merge: Consider any two or more neighboring sub regions, R1, R2,…, Rn, in the image. If H(R1UR2U…Rn) is true, merge the n regions into a single region
Repeat these steps until no further splits or merges.
60
Region Growing
Group pixels or subregions into larger regions based on pre-defined data
Start with a set of ‘seed’ points
Check the neighboring pixels and add them to the region if they are similar to the seed
61
Application dependent for example: If targets need to be
detected using infrared images, choose the brightest pixel(s)
Without a priori knowledge, for example: compute the histogram and
choose the gray-level values corresponding to the strongest peaks
How to Choose Seeds
62
similarity criteria
based on any characteristic of the region in the image: Average intensity Variance Color Texture Motion Shape Size
63
Similarity Criteria for Region Growing
Satellite imagery – color
Monochrome images – Descriptors : Gray levels, moments, texture Connectivity or adjacency information
64
Example – Region Growing Fig. shows x-ray image of
a weld with defects and a histogram of the image.
Defects have max. value (255). All these point are used as seed regions.