Chapter 4 Edge and Texture Edges describe the spatial differences across an image. These differences form boundaries that allow the human visual system to distinguish between homogeneous colour regions in an image. Simi- larly, content-based image retrieval systems use low-level edges in higher level feature extraction techniques such as contour extraction and texture analysis to differentiate between regions within an image. Edges have been used extensively for content-based image retrieval and much research has been conducted [13, 60, 19, 4, 104]. However, many of the techniques proposed in the literature only use very simple edge detectors [27, 19, 4]. By themselves simple edge detectors perform well against complex edge detectors, however they perform poorly when used for higher level feature extraction such as contour extraction and texture analysis. Specific edge detectors have been designed to extract features required by texture analysis [60, 105, 39] but few edge detectors have been designed with the intent of accurate contour following. Contours are important in content-based retrieval systems as they are one of the high-level structural representations within an image. Since the performance of contour following is intrinsically dependent on edge detection the primary purpose of this chapter is to investigate edge detection techniques for contour following and to build upon these techniques to produce an edge detector tuned for contour following that can also be used for texture analysis. The resulting edge detector, called the Asymmetry edge detector, is able to provide the best single pixel responses across multiple orientations compared with existing techniques. Section 4.1 discusses the limitations of existing edge detection techniques. Section 4.2 identifies the requirements of an edge detector within the context of this research. Section 4.3 analyses and compares existing multi-orientation operators. Section 4.4 presents the new Asymmetry edge detector. Section 4.5 presents a new technique for thinning multi-orientation edge responses. Section 4.6 discusses the Asymmetry detector as a computational model of the visual cortex. Section 4.7 presents a new technique for inhibiting textures in edge detection. Section 4.8 presents conclusions drawn from the findings of this chapter. 75
34
Embed
Edge and Texture - pub.ro › _curs › cm › proiecte › Content_Based... · very simple edge detectors [27, 19, 4]. By themselves simple edge detectors perform well against complex
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Chapter 4
Edge and Texture
Edges describe the spatial differences across an image. These differences form boundaries that allow
the human visual system to distinguish between homogeneous colour regions in an image. Simi-
larly, content-based image retrieval systems use low-level edges in higher level feature extraction
techniques such as contour extraction and texture analysis to differentiate between regions within
an image.
Edges have been used extensively for content-based image retrieval and much research has been
conducted [13, 60, 19, 4, 104]. However, many of the techniques proposed in the literature only use
very simple edge detectors [27, 19, 4]. By themselves simple edge detectors perform well against
complex edge detectors, however they perform poorly when used for higher level feature extraction
such as contour extraction and texture analysis. Specific edge detectors have been designed to
extract features required by texture analysis [60, 105, 39] but few edge detectors have been designed
with the intent of accurate contour following. Contours are important in content-based retrieval
systems as they are one of the high-level structural representations within an image. Since the
performance of contour following is intrinsically dependent on edge detection the primary purpose
of this chapter is to investigate edge detection techniques for contour following and to build upon
these techniques to produce an edge detector tuned for contour following that can also be used for
texture analysis. The resulting edge detector, called the Asymmetry edge detector, is able to provide
the best single pixel responses across multiple orientations compared with existing techniques.
Section 4.1 discusses the limitations of existing edge detection techniques. Section 4.2 identifies
the requirements of an edge detector within the context of this research. Section 4.3 analyses
and compares existing multi-orientation operators. Section 4.4 presents the new Asymmetry edge
detector. Section 4.5 presents a new technique for thinning multi-orientation edge responses. Section
4.6 discusses the Asymmetry detector as a computational model of the visual cortex. Section 4.7
presents a new technique for inhibiting textures in edge detection. Section 4.8 presents conclusions
drawn from the findings of this chapter.
75
(a) Original image (b) Simple difference operator (c) Laplacian
(d) Roberts (e) Prewitt (f) Sobel
(g) Frei-Chen (h) Kirsch (i) Robinson
Figure 4.1: Some common edge detectors applied to image (a). Each result image represents the
absolute maximum magnitude at each pixel after the individual masks have been applied.
4.1 Edge Detection
Edges form where the pixel intensity changes rapidly over a small area. Edges are detected by
centring a window over a pixel and detecting the strength of edge within the window. The result is
stored at the same pixel location. Edge responses produced by a number of common edge detectors
are shown in Figure 4.1.
Edge detection techniques often use a mask that is convolved with the pixels in the window.
A simple difference mask is shown in Figure 4.2 (a). The difference mask is directional. An edge
detector can also be non-directional such as the Laplacian or difference of Gaussians as shown in
Figure 4.2 (b). Since edges are directional and contours consist of oriented edges, we are primarily
interested in directional edge detectors.
Other simple, but extensively used edge detectors, include the Sobel, Roberts, and Prewitt
76
-1 1
(a) Simple Difference Operator 0
-1
0
-1
0
0 -1
-1
4
(b) Laplacian
1 0
0 -1 -1
10
0
(c) Roberts
-1 -1 -1
000
1 1 1 1
1
1
-1
-1
-1 0
0
0
(d) Prewitt
-1 -2 -1
000
1 2 1 1
2
1
-2
-1
-1 0
0
0
(e) Sobel
Figure 4.2: Some common edge detectors.
operators (Figure 4.2) [69]. Such operators are directional and can be used to detect orientations at
90 ◦ intervals. Other operators such as the Frei-Chen [58], Kirsch [57], and Robinson [59] operators
can also be oriented at 45 ◦ intervals allowing up to 4 orientations to be detected (Figure 4.3 (a)).
In contrast, the human vision system detects 18 different orientations at 10 ◦ intervals [10].
Operators that are specified by a continuous function rather than a fixed mask can be rotated
to any arbitrary orientation. The Gabor filter [60] and the Canny operator [13] (Figure 4.4) can
both be described mathematically and are two of the most advanced edge detectors as they have
a similar receptive field to the edge detectors of the human vision system [12].
Figure 4.1 shows the output of the various edge detectors discussed in this section applied to a
test image. However, it is not possible to determine a good edge detector simply by looking at its
output. Instead we must look at the design and features of an edge detector with respect to the
requirements of contour following.
4.2 Edge Detector Requirements
For each pixel, contour following requires the orientation and strength of each edge. Contour
following also requires highly tuned edge responses. Tuning can occur across orientations and also
across spatial locations. Figure A.8 shows the orientation tuning response curve for a simple cell
in human vision. Likewise, oriented edge detectors produce different edge responses depending on
the orientation of the edge input. The output will peak when the orientation of the edge and the
77
3 3 3
303
-5 -5 -5 3
3
3
-5
3
-5 -5
3
0
-5 3 3
30-5
-5 3 3 3
3
3
-5
-5
3 3
-5
0
(a) Kirsch masks
(b) Kirsch mask results
Figure 4.3: The Kirsch mask [57] applied to the image in Figure 4.1(a). The masks detect edges at
0 ◦, −45 ◦, −90 ◦, and −135 ◦.
78
detector are aligned and will fall off as their orientations change. Since contour following will follow
the orientation with the largest strength it is important that the edge detectors are tuned tightly so
that the contour following algorithm doesn’t inadvertently follow the wrong orientation. However,
the tuning can not be too tight as responses by two edge detectors with adjacent orientations can
be used to determine the exact orientation of an edge that lies between the two orientations.
Position tuning is also important as a contour following algorithm will also consider a neigh-
bourhood of pixels to determine the next pixel to include in the contour. If two adjacent pixels
produce a strong response then the contour following algorithm may unnecessarily create two
contours at that point rather than following the pixel that the edge is truly aligned to.
Adjacent orientation responses are used to determine the exact orientation of an edge. In the
same manner it is possible to use adjacent position responses to determine the exact position of
an edge. This process is called subpixel edge detection [106], however subpixel edge detection is
beyond the scope of this research, primarily because each stage of edge and contour processing
assumes that each edge is aligned with the centre of a pixel.
In summary, the edge detector must satisfy the following requirements:
• Produce multi-orientation output
• Orientation-tuned with only two adjacent responses generated
• Position-tuned with only one adjacent response generated
• Efficient, small window, convolution-style operator
4.3 Multi-orientation Operators
The Gabor and Canny operators are the most suitable operators for multi-orientation edge de-
tection as they are described by a continuous function (and therefore can be used to construct
multi-orientation detectors), resemble edge detectors in the human vision system, and have been
extensively investigated [60, 13, 107]. Other fixed mask operators such as the Laplacian, difference
of Gaussians, Roberts, Prewitt, and Sobel operators are not suitable because they only support 1
to 4 orientations. An additional benefit of the Gabor and Canny operators is that they are scalable
and can be used to identify edges of different resolutions.
In this research we have decided to use the S-Gabor filter proposed by Heitger et al. [12] over the
standard Gabor filter. The standard Gabor filter modulates a sine or cosine wave with a Gaussian
envelope:
Godd(x) = e−x2/2σ2sin[2πv0x] (4.1)
Geven(x) = e−x2/2σ2cos[2πv0x] (4.2)
79
where σ is the bandwidth of the Gaussian envelope and v0 is the wavelength of the sine wave.
The odd Gabor filter is used for edge detection whilst the even Gabor filter can be used for line
detection. The Gaussian envelope of the Gabor filter is not able to curtail the periodic nature of the
sine or cosine wave and therefore additional fluctuations of the wave may appear at the extremities
of the filter. Since edges are a local phenomenon there is no need for a periodic wave and the
S-Gabor filter reduces the frequency of the sine wave as x increases so that only one wavelength is
present:
Sodd(x) = e−x2/2σ2sin[2πv0xξ(x)] (4.3)
Seven(x) = e−x2/2σ2cos[2πv0xξ(x)] (4.4)
ξ(x) = ke−λx2/σ2+ (1− k) (4.5)
where k determines the change of wavelength. The Canny operator is simpler as it does not use
periodic functions:
C(x) =xe−x2/2σ2
σ2(4.6)
When a multi-orientation operator is applied to an image multiple edge images are generated.
Therefore the greater the number of orientations per edge detector the greater the amount of
memory is required to store the result images and also the longer it will take to generate the
images. For the purposes of optimisation it is beneficial for the number of orientations to be as
small as possible. We have decided to use 12 orientations at 15 ◦ intervals as a compromise between
the 18 orientations of human vision and the 1 to 4 orientations offered by the fixed mask operators.
4.3.1 Multi-orientation Experiments
The S-Gabor and Canny operators were chosen because they can be used at any orientation and
resemble the receptive fields of visual cortex simple cells. The odd S-Gabor filter was constructed
in two dimensions using the following formulae:
Sodd(x′, y′) = e−(x′2+y′2)/2σ2sin[2πv0y
′ξ(x′, y′)] (4.7)
ξ(x′, y′) = ke−λ(x′2+y′2)/σ2+ (1− k) (4.8)
where x′ and y′ are the rotated and scaled pixel co-ordinates defined below in Equations 4.10 and
4.11. The remaining parameters were adjusted to provide a filter that produces only one period of
the sine wave under the Gaussian envelope with a wavelength of 2 pixels, resulting in σ = 0.646,
v0 = 0.5, λ = 0.3, and k = 0.5.
The Canny filter was constructed in two dimensions using the following formula:
C(x) =−y′e−(x′2+y′2)/2σ2
σ2(4.9)
where σ = 0.35 to also provide a separation of one pixel between lobe peaks.
80
The filters were rotated and scaled by pre-rotating and scaling the x and y pixel co-ordinates:
x′ =x cos(−θ)− y sin(−θ)
sx(4.10)
y′ =x sin(−θ) + y cos(−θ)
sy(4.11)
where θ = n π12 , n = (0→ 11), sy = 1, and sx determines the elongated aspect ratio of the filter.
The S-Gabor and Canny operators are very similar in shape and the similarity is shown in
their respective tuning response curves. The tuning response curves display the magnitude of the
response of the operator at different lateral positions and orientations to the edge stimulus. A
vertical black and white edge was used as the stimulus and 12 orientations of the operator were
convolved with the stimulus. Position response values were taken from the few pixels either side of
the edge whilst orientation responses were taken from each of the 12 resulting images.
In our analysis we are primarily interested in the highest frequency edges representable by the
image. These edges are formed between two adjacent pixels. Therefore the filters have a width of 2
pixels with each lobe centred on a pixel. The length of the filter must be greater than one pixel and
should be less than 10 pixels so that curves are detectable. A longer filter is desirable to filter out
noisy edges. Because there is no exact restriction on filter length we will first analyse the tuning
response curves at different lengths to determine the best length.
4.3.2 Multi-orientation Results
Figure 4.4 shows the aspect ratios of the S-Gabor and Canny operators tested. Figures 4.5 and 4.6
show the tuning response curves for the S-Gabor and Canny operators respectively at the different
aspect ratios. By comparing the graphs it can be seen that the Canny and S-Gabor operators
show very similar results (although at different aspect ratios). This can be explained by the Canny
operator being shorter than the S-Gabor operator. Since there is no difference in orientation and
position tuning between the two operators either one may be used. We have selected the Canny
operator because it requires fewer parameters.
The tuning response curves show that shorter filters provide very good position tuning but
poor orientation tuning, whilst the longer filters provide good orientation tuning but poor position
tuning at orientations slightly different to that of the edge. These tuning response curves can be
explained by visualising the overlapping of the operator lobes over a test edge (see Figure 4.7 (a)
to (d)). These scenarios indicate that whenever the edge stimulus is asymmetrical over the length
of the filter a response shouldn’t be generated. What is required is an asymmetry detector whose
response is negated from the response of the edge detector.
81
S Gabor
Canny
Canny asymmetry
1:11.5:12:13:14:1
1:11.5:12:13:14:16:1
1.33:12:12.67:14:1
Figure 4.4: Filters tested.
4.4 New Asymmetry Detector
A simple approach to identify asymmetry of edge response along the length of an edge detector
could be to simply use the same edge detector but at a 90 ◦ orientation. However, such a filter
would give the same tuning responses as those in Figure 4.6 but shifted 90 ◦ and wouldn’t be
sufficient to nullify erroneous responses. What is required is a filter which is the same shape as the
edge detector but at a 90 ◦ orientation (see Figure 4.7).
The same formula for constructing the Canny edge detector in Equation 4.9 is used for the
asymmetry filter (σ = 0.5) however the rotation and scaling equations are modified to allow for an
orthogonal orientation and aspect ratio:
x′ =3[x cos(π
2 − θ)− y sin(π2 − θ)]
2sx(4.12)
y′ =x sin(π
2 − θ) + y cos(π2 − θ)
sy(4.13)
The direction of asymmetry is not relevant so the absolute asymmetry response is subtracted
from the Canny edge detector modulated by a tuning factor t:
EA = |C| − t|A| (4.14)
where C is the response of the Canny edge detector, A is the response from the asymmetry filter,
and EA is the final edge response.
82
0.00E+00
1.00E+02
2.00E+02
3.00E+02
4.00E+02
5.00E+02
6.00E+02
Response
0 15 30 45 60 75 90 105 120 135 150 165
S1
S2
S3
S4
S5
S6
S7
Orientation (degrees)
Position
S Gabor 1:1
0.00E+00
1.00E+02
2.00E+02
3.00E+02
4.00E+02
5.00E+02
6.00E+02
Response
0 15 30 45 60 75 90 105 120 135 150 165
S1
S2
S3
S4
S5
S6
S7
Orientation (degrees)
Position
S Gabor 1.5:1
0.00E+00
1.00E+02
2.00E+02
3.00E+02
4.00E+02
5.00E+02
6.00E+02
Response
0 15 30 45 60 75 90 105 120 135 150 165
S1
S2
S3
S4
S5
S6
S7
Orientation (degrees)
Position
S Gabor 2:1
0.00E+00
1.00E+02
2.00E+02
3.00E+02
4.00E+02
5.00E+02
6.00E+02
Response
0 15 30 45 60 75 90 105 120 135 150 165
S1
S2
S3
S4
S5
S6
S7
Orientation (degrees)
Position
S Gabor 3:1
0.00E+00
1.00E+02
2.00E+02
3.00E+02
4.00E+02
5.00E+02
6.00E+02
Response
0 15 30 45 60 75 90 105 120 135 150 165
S1
S2
S3
S4
S5
S6
S7
Orientation (degrees)
Position
S Gabor 4:1
Figure 4.5: Gabor tuning response curves
83
0.00E+00
1.00E+02
2.00E+02
3.00E+02
4.00E+02
5.00E+02
6.00E+02
Response
0 15 30 45 60 75 90 105 120 135 150 165
S1
S2
S3
S4
S5
S6
S7
Orientation (degrees)
Position
Canny 4:1
0.00E+00
1.00E+02
2.00E+02
3.00E+02
4.00E+02
5.00E+02
6.00E+02
Response
0 15 30 45 60 75 90 105 120 135 150 165
S1
S2
S3
S4
S5
S6
S7
Orientation (degrees)
Position
Canny 6:1
0.00E+00
1.00E+02
2.00E+02
3.00E+02
4.00E+02
5.00E+02
6.00E+02
Response
0 15 30 45 60 75 90 105 120 135 150 165
S1
S2
S3
S4
S5
S6
S7
Orientation (degrees)
Position
Canny 2:1
0.00E+00
1.00E+02
2.00E+02
3.00E+02
4.00E+02
5.00E+02
6.00E+02
Response
0 15 30 45 60 75 90 105 120 135 150 165
S1
S2
S3
S4
S5
S6
S7
Orientation (degrees)
Position
Canny 3:1
0.00E+00
1.00E+02
2.00E+02
3.00E+02
4.00E+02
5.00E+02
6.00E+02
Response
0 15 30 45 60 75 90 105 120 135 150 165
S1
S2
S3
S4
S5
S6
S7
Orientation (degrees)
Position
Canny 1:1
0.00E+00
1.00E+02
2.00E+02
3.00E+02
4.00E+02
5.00E+02
6.00E+02
Response
0 15 30 45 60 75 90 105 120 135 150 165
S1
S2
S3
S4
S5
S6
S7
Orientation (degrees)
Position
Canny 1.5:1
Figure 4.6: Canny tuning response curves
84
Asymmetry detector
Edge Detector
(a) (b)
(c) (d) (e)
Figure 4.7: (a) to (d) Edge operator scenarios. (a) Alignment of operator with edge; (b) orientation
misalignment; (c) orientation and position misalignment; (d) position misalignment. (e) Asymmetry
detector overlaid on edge detector.
4.4.1 Asymmetry Detector Results
The tuning curves of asymmetry filters for the 3:1, 4:1, and 6:1 Canny edge detectors are shown in
Figure 4.8. The tuning curves are sufficient to nullify erroneous responses (however, shorter aspect
ratios below 3:1 were not sufficient). The result of the asymmetry edge detector with tuning t = 1
is shown in Figure 4.9 (a). With a tighter tuning parameter of t = 2 the result is a perfectly tuned
edge detector in both orientation and position (Figure 4.9 (b)).
The edge stimulus used is a perfect vertical edge aligned to one of the edge detector orientations.
To test whether the Asymmetry detector performs as well with edge orientations which are not
aligned with one of the edge detector orientations the same vertical edge was tested with edge
orientations at a 7.5 ◦ offset which is half way between the usual 15 ◦ interval between edge detector
orientations. Figure 4.10 shows the results for the 7.5 ◦ offset edge detector. The Asymmetry edge
detector successfully provides two identical responses for each adjacent orientation indicating that
the orientation of the edge lies exactly halfway between the two orientations.
The tuned operator appears to work well for any aspect ratio greater than or equal to 3:1.
However, because the tuned operator is inhibited by asymmetrical stimulus it may have problems
at corners (Figure 4.11). The tuning curves at corners for the three aspect ratios are shown in Figure
4.12. Figure 4.12 shows that there is no response for the edge as the edge detector approaches the
corner. Larger aspect ratio operators fall off early whilst the 3:1 aspect ratio operator falls off only
one pixel before the end of the contour. Therefore, the best operator for all scenarios is the 3:1
aspect ratio operator.
85
0
100
200
300
400
500
600
Response
0 15 30 45 60 75 90 105 120 135 150 165
S1
S2
S3
S4
S5
S6
S7
Orientation (degrees)
Position
Canny Asymmetry 2.67:1 (matches 4:1)
0
100
200
300
400
500
600
Response
0 15 30 45 60 75 90 105 120 135 150 165
S1
S2
S3
S4
S5
S6
S7
Orientation (degrees)
Position
Canny Asymmetry 4:1 (matches 6:1)
0
100
200
300
400
500
600
Response
0 15 30 45 60 75 90 105 120 135 150 165
S1
S2
S3
S4
S5
S6
S7
Orientation (degrees)
Position
Canny Asymmetry 2:1 (matches 3:1)
0
100
200
300
400
500
600
Response
0 15 30 45 60 75 90 105 120 135 150 165
S1
S2
S3
S4
S5
S6
S7
Orientation (degrees)
Position
Canny Asymmetry 1.33:1 (matches 2:1)
Figure 4.8: Asymmetry tuning curves.
0
100
200
300
400
500
600
Response
0 15 30 45 60 75 90 105 120 135 150 165
S1
S2
S3
S4
S5
S6
S7
Orientation (degrees)
Position
Canny Tuned 3:1
0
100
200
300
400
500
600
Response
0 15 30 45 60 75 90 105 120 135 150 165
S1
S2
S3
S4
S5
S6
S7
Orientation (degrees)
Position
Canny Tuned 3:1 minus 2x Asymmetry
Figure 4.9: Combined edge detector and asymmetry inhibitor at 3:1 aspect ratio. (a) t = 1, (b)
t = 2.
86
0
50
100
150
200
250
Response
0 15 30 45 60 75 90 105 120 135 150 165
S1
S2
S3
S4
S5
S6
S7
Orientation (degrees)
Position
Canny Tuned 3:1 minus 2x Asymmetry, 7.5˚ offset
Figure 4.10: Tuned edge detector at 7.5 ◦ orientation offset.
The one pixel fall off in edge response before the end of the contour may affect contour extraction
as the contours extracted will not include the last pixel of the corner. However, losing one pixel
before the end of a contour appears to be a fair trade off for the improved orientation and position
tuning gained. In addition, contour-end detection and vertex extraction which are investigated in
the following chapter would be able to identify the corner and higher level processing stages would
be able to link the vertex to the edges.
Figure 4.13 shows how the Asymmetry detector compares with the standard Canny detector
for sample test images. The Asymmetry edge detector results of Figure 4.13 (c) show tighter
positional tuning than the Canny edge detector results of Figure 4.13 (b). The orientation tuning
performance is not as easily seen in a single aggregate image however the impact of improved
orientation tuning in the Asymmetry detector can be seen in later stages of processing which is
indicated by the thinned Canny and Asymmetry responses of Figure 4.13 (d) and (e) respectively.
The thinned Asymmetry edges using the thinning technique discussed in the next section contain
fewer spurious responses than the thinned Canny edges.
4.5 Thinning
Using the Asymmetry edge detector developed in the previous sections the edge responses should
be tightly tuned in both orientation and position. However, it is still possible that an edge may
generate responses over a number of positions because it’s wavelength is greater than that of the
edge detector. Therefore it is still necessary to perform some thinning on the edge responses to
reduce contours to 1 pixel thickness, which is required by the contour following algorithm.
We are only interested in thinning along the direction of a contour. Current thinning techniques
such as morphological thinning and skeletonisation ignore the direction of a contour. As a result
thinning will occur in all directions. Figure 4.15 (a)-(d) shows the results of thinning the cube
87
Figure 4.11: Possible problem when tuned edge detector is placed over a corner.
0
100
200
300
400
500
Response
1 2 3 4 5 6 7 8 9 10 11 12
S1
S2
S3
S4
S5
S6
S7
Orientation (degrees)
Position
Canny Tuned 4:1 Corner
0
100
200
300
400
500
600
Response
1 2 3 4 5 6 7 8 9 10 11 12
S1
S2
S3
S4
S5
S6
S7
Orientation (degrees)
Position
Canny Tuned 3:1 Corner
0
100
200
300
400
500
600
Response
1 2 3 4 5 6 7 8 9 10 11 12
S1
S2
S3
S4
S5
S6
S7
Orientation (degrees)
Position
Canny Tuned 2:1 Corner
0
50
100
150
200
250
300
350
Response
1 2 3 4 5 6 7 8 9 10 11 12
S1
S2
S3
S4
S5
S6
S7
Orientation (degrees)
Position
Canny Tuned 6:1 Corner
Figure 4.12: Corner tuning curves
88
(e)
(d)
(c)
(b)
(a)
Figure 4.13: Canny and Asymmetry edge responses for the Chapel, Plane, and Claire images. (a)
Sample images; (b) Results after applying the Canny edge detector; (c) Results after applying the