06507646

3204 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 22, NO. 8, AUGUST 2013

Corner Detection and Classification UsingAnisotropic Directional Derivative Representations

Peng-Lang Shui, Member, IEEE, and Wei-Chuan Zhang

Abstract— This paper proposes a corner detector and classifierusing anisotropic directional derivative (ANDD) representations.The ANDD representation at a pixel is a function of the orientedangle and characterizes the local directional grayscale variationaround the pixel. The proposed corner detector fuses the ideasof the contour- and intensity-based detection. It consists of threecascaded blocks. First, the edge map of an image is obtained bythe Canny detector and from which contours are extracted andpatched. Next, the ANDD representation at each pixel on contoursis calculated and normalized by its maximal magnitude. Thearea surrounded by the normalized ANDD representation formsa new corner measure. Finally, the nonmaximum suppressionand thresholding are operated on each contour to find cornersin terms of the corner measure. Moreover, a corner classifierbased on the peak number of the ANDD representation is given.Experiments are made to evaluate the proposed detector andclassifier. The proposed detector is competitive with the tworecent state-of-the-art corner detectors, the He & Yung detectorand CPDA detector, in detection capability and attains higherrepeatability under affine transforms. The proposed classifiercan discriminate effectively simple corners, Y-type corners, andhigher order corners.

Index Terms— Anisotropic Gaussian directional derivative,corner detection.

LIST OF ACRONYMSANDD Anisotropic directional derivative.CSS Curvature scale space.CPDA Chord to point distance accumulation.UCM Universal corner model.BCF Basic corner function.GT Ground truth.ROC Receiver operating characteristic.

I. INTRODUCTION

CORNER detection, as a front-end processing in a feature-based image understanding system, is rather important

in computer vision and image processing. Corner detectionyields great effect on sequent tasks, such as object trackingand recognition in images, image registration, etc. There haveexisted considerable corner detectors and these can be broadlyclassified into three groups: intensity-based detectors [1]–[8],contour-based detectors [9]–[15], and model-based detectors

Manuscript received March 8, 2012; revised April 9, 2013; accepted April16, 2013. Date of publication April 24, 2013; date of current versionJune 4, 2013. This work was supported by the National Natural Sciencefoundation of China under Grant 61271295. The associate editor coordi-nating the review of this manuscript and approving it for publication wasProf. Marios S. Pattichis.

The authors are with the National Lab of Radar Signal Processing,Xidian University, Xiàn 710071, China (e-mail: [email protected];[email protected]).

Digital Object Identifier 10.1109/TIP.2013.2259834

[16], [17]. Different types of detectors have individual mer-its and defects. The model-based detectors find corners bymatching image patches to the predefined models. Often, thepredefined models are difficult to cover all corners in realimages, limiting their performance in applications. Contour-based and intensity-based detectors attract more attention.

Intensity-based detectors use the first-order or second-orderderivatives of images for corner detection. Early detectorsusing the second-order derivatives are found to be noise-sensitive and are seldom used later. By developing theMoravec’s idea [1], Harris and Stephens proposed the famousHarris detector [2] and the corner measure is constructedfrom the local autocorrelation matrix of the gradients, whichis the base of many later studies. The local autocorrelationmatrix of the gradients is not enough to characterize high-ordercorners. Harris corner detector can only detect L-type corners(simple corners) and lacks the capability to classify high-ordercorners or junctions [3]. The steerable-scalable kernels thatare generated by linear interpolation and finite combinationof several basic kernels were presented for edge detectionand junction analysis [4]. The Harris-Laplace corner detectorinvariant to scale and the Harris-affine corner detector invariantto affine transforms were established in [5], [6]. Multiscaleintensity variations improve corner detection, localization, andrepeatability but are insufficient in corner classification. Later,multiscale and multidirectional intensity variations were usedin corner detection. The steerable filters were designed tocapture multidirectional intensity variations [7]. Multiscale andmultidirectional transforms, such as the Log-Gabor wavelet[8], the shearlet [18], and the simplified Gabor wavelet [19],were used to extract fine local intensity variations for edgeand corner detection. Fine directional intensity variations arerather important in feature detection and classification andhave been utilized to successfully classify pixels into fourgroups: regular pixels, edge pixels, pixels near edges, andcorners [18].

The contour-based corner detectors consist of three cascadedbasic blocks: edge detection, contour extraction, and cornerdecision on contours [9]–[15]. The curvature scale space (CSS)method is widely used in the third block. Each contour isrepresented as a planar curve parameterized by arc-length.It is smoothed by a set of multiscale Gaussian functions tocalculate multiscale local curvatures. The absolute curvaturemaximum points constitute the candidate corners and fromwhich weak and false corners are removed by thresholding.Also, some detectors use the multiscale tracking [10] andmultiscale curvature product [11]. The CSS corner detectors

1057-7149/$31.00 © 2013 IEEE

SHUI AND ZHANG: CORNER DETECTION AND CLASSIFICATION USING ANDD REPRESENTATIONS 3205

were mentioned to suffer from two defects at least [11], [14].The curvature estimator is sensitive to the local variation ornoise on contours. Large-scale Gaussian functions suppressnoise but degrade corner localization while small-scale onespreserve high precision in localization but are noise-sensitive.Scale selection is in a dilemma. In order to mend the twodefects, the detector using the chord to point distance accu-mulation (CPDA) [11], [14] and the adaptive threshold method[13] are developed. In fact, except for these two defects,the another defect in the contour-based corner detectors isthat intensity variation information is ineffective in the cornerdecision and corner classification cannot but depend uponfinding junctions of contours.

In this paper, the anisotropic Gaussian directional derivativekernels [20], [21] are used to extract fine multidirectionalintensity variations of a grayscale image for corner detectionand classification. A universal corner model (UCM) is usedto describe simple (L-type) corners and composite or high-order corners, including Y-type, X-type, and star-like corners.The anisotropic directional derivative (ANDD) representationof the UCM is derived analytically. It is shown that the ANDDrepresentation of a corner is expressed as the sum of severalbasic components and whose number indicates the type of thecorner. Each ANDD filter is an anisotropic Gaussian convolu-tion kernel followed by a directional derivative operator, whichcan conciliate the conflict between noise robustness and detailpreservation [21]. Moreover, the analytic expression of theANDD representation makes it easier to analyze its propertiessuch as noise-robustness and corner resolution, to constructcorner measure for detection, and to design corner classifier.

From the difference between the ANDD representations ofcorners and smooth edge pixels, a new corner measure isconstructed to pick out corners from contours. The new cornermeasure is embedded into the routine of the contour -basedcorner detection. A new corner detector is developed, whichconsists of edge detection, extraction and patching of contours,and corner decision on contours via the corner measure.Different from the existing contour-based corner detectors,corner decision on contours uses the directional intensityvariations around an edge pixel rather than the local curvatureor other geometric feature of the planar contours, whichbrings improvement in corner detection. The proposed detectorembeds the directional intensity variations into the frameworkof the contour-based detection and fuses the merits of contour-based detection and intensity-based detection. Moreover, asimple but efficient corner classifier is presented by countingthe peak number of the ANDD representation of a corner.

This paper is organized as follows. In Section II, anisotropicdirectional derivatives and the UCM are introduced. Theanalytic ANDD representation of the UCM is presented and itsproperties are analyzed. A new corner measure on contours isconstructed. In Section III, the discrete ANDD representationsand the computation of the corner measure in the discrete caseare given. Moreover, the empirical cumulative distributionsof the corner measure on non-ideal edges are discussed. Thenew corner detector and classifier are proposed in Section IV.Experiments and performance comparison are reported inSection V. Finally, we conclude our paper in Section VI.

II. ANISTROPIC DIRECTIONAL DERIVATIVE

REPRESENTATIONS OF CORNERS

In this section, the ANDD filters based on the anisotropicGaussian kernels are introduced. The UCM and its ANDDrepresentation are presented. The corner resolution of theANDD representation is analyzed and its relation with theanisotropic factor is discussed. Moreover, a new corner mea-sure on contours is constructed.

A. Anisotropic Directional Derivative Filters

A two-variable elongated Gaussian function is written as[20], [21]

gσ,ρ(x) = 1

2πσ 2 exp

(− 1

2σ 2 xT[ρ−2 00 ρ2

]x),

ρ ≥ 1, σ > 0, x = [x, y]T (1)

where the superscript T denotes the transpose, σ is the scale,ρ ≥ 1 is the anisotropic factor, and x = [x, y]T stands forthe planar coordinate. Its partial derivative with respect to thesecond variable is

∂gσ,ρ∂y

(x) = −ρ2 y

σ 2 gσ,ρ(x) (2)

Rotating the argument in (1) at an angle θ yields an anisotropicGaussian kernel

ϕσ,ρ,θ (x) = gσ,ρ(Rθx),Rθ =[

cos θ sin θ− sin θ cos θ

](3)

where θ is the rotation angle and Rθ is the rotation matrix.In the same way, an ANDD filter at the orientation π/2 + θis derived as follows:

ψσ,ρ,θ (x) = ∂gσ,ρ∂y

(Rθx). (4)

The ANDD of an image I (x) at the orientation π/2 + θ is

∂ Iσ,ρ(x; θ) ≡ ∂

∂θ

(I ∗ ϕσ,ρ,θ (x)

) = I ∗ ψσ,ρ,θ (x) (5)

The function ∂ Iσ,ρ(x;θ ), θ ∈ [0, 2π] characterizes the inten-sity variations around a pixel x. Fig. 1 plots the anisotropicGaussian kernels and ANDD filters at eight orientations.

Some intensity-based corner detectors use the gradient oper-ator and the gradients are calculated from smoothed image byan isotropic Gaussian kernel. Isotropic Gaussian smoothingassures that the image is differentiable at edges and cornersthat are often singularities of an image function. However, thesmoothing also blurs local fine structures on edges and at cor-ners. As a result, the directional derivatives at all orientationsare determined by two partial derivatives and two partial deriv-atives are difficult to distinguish different types of corners. Therecent edge or corner detectors use the multiscale directionalfilter banks, such as the steerable filters [7], Log-Gabor wavelet[8], shearlet [18], and simplified Gabor wavelet [19], instead ofthe gradient operator, to characterize anisotropic local struc-tures at edges and corners. Similarly, the ANND filters canalso be interpreted as a directional filter bank and can extractanisotropic intensity variations around edge pixels and corners.


Fig. 1. (a) Illustration of anisotropic Gaussian kernels at eight orientations.(b) Illustration of ANDD filters at eight orientations.

The ANDD representations possess several additional mer-its. First, each orientation uses individual smoothing kernel.The anisotropic smoothing suppresses noise but also preservesanisotropic structures of corners and edges well. Second, theanalytic ANDD filters bring convenient in analysis. Third, theadjustable scale of the ANDDs allows multiscale and multi-directional intensity variation descriptions as the Log-Gaborwavelet [8], shearlet [18], and simplified Gabor wavelet [19].

B. Universal Corner Model and ANDD Representation

A grayscale image can be interpreted as a two-variablepiecewise smooth function. The boundaries of smooth regionsform edges of the image. A simple corner can be informallycharacterized as a point at which only two smooth regionsmeet, with one region subtending a smaller angle than theother [22], [23]. A high-order corner or junction is a pointat which three or more smooth regions meet. A junction atwhich three regions meet is a Y(T)-type corner, a junction atwhich four regions meet is an X-type corner, and a junctionat which more than four regions meet is a star-like corner.

A universal corner model (UCM) is used to describe dif-ferent types of corners and it consists of several disjointwedge-shaped regions sharing a vertex. In the polar coordinatesystem, a basic corner function (BCF) is defined by

BCFβ1,β2(r, α)

={

1, 0 ≤ r < +∞, β1 ≤ α ≤ β2, β2 − β1 �= π0, otherwise

(6)

where r is the radius, α is the polar angle, β1 is the lowerbound of the polar angle, and β2 is the upper bound of the

Fig. 2. Illustrative diagram of a BCF and it takes the value of one insidethe gray region and takes the value of zero outside the gray region.

polar angle. As shown in Fig. 2, the BCF takes the value ofone in the gray region and the value of zero outside the grayregion.

For convenience in analysis, a corner is always put atthe origin. Simple corners and high-order corners such asthe Y-junctions, X-junctions and star-like junctions, can berepresented as a sum of several BCFs. A UCM is defined asfollows:

UCM(Ti ,βi ),i=1,...,s(r, α)

=s∑

i=1

Ti BCFβi ,βi+1(r, α)(Note:βs+1 = β1) (7)

where Ti is the gray value at individual wedge-shapedregions and is referred as the intensity of the componentBCFβi ,βi+1(r, α). The similar model also appears in the model-based junction detection [17].

Fig. 3(a) illustrates a step edge, a simple corner, aY-type corner, an X-type corner, and a star-like corner. Eachcorresponds to individual parameter setting in the UCM.A step edge corresponds to s = 2 and β2 − β1 = ±π , asimple corner to s = 2 and β2 − β1 �= ±π , a Y-type cornerto s = 3, an X-type corner to s = 4, and a star-like tos ≥ 5. Fig. 3(b) shows the magnitudes of their ANDD repre-sentations. As comparison, Fig. 3(c) exhibits the magnitudesof their isotropic directional derivatives, corresponding to theANDD representations with σ = ρ = 1. It can be seen thatthe step edge and four corners have quite different ANDDrepresentations. However, the isotropic directional derivativesof the step edge and four corners have a same shape, meaningthat the isotropic directional derivatives are difficult to tellcorners from edge pixels.

The ANDD representation of the UCM is the sum of theANDD representations of several BCFs. To obtain the analyticexpression of the ANDD representation of the UCM, we needto derive the ANDD representation of a BCF. Transferring theANDD filters in (4) to the polar coordinate system, the ANDD


Fig. 3. (a) Illustration of a step edge model, a simple corner model, a Y-type corner model, an X-type corner model, and a star-like corner model. (b) ANDDrepresentations of the five models. (c) Isotropic directional derivatives of the five models.

representation of a BCF has an analytic expression (8).

∂∇BCFβ1,β2

(θ) = ∫ ∫R2 BCFβ1,β2(r, α)ψσ,ρ,θ (−r,−α)rdrdα

= ρ

2√

2πσ

{cos(θ−β1)√

cos2(θ−β1)+ρ4 sin2(θ−β1)

− cos(θ−β2)√cos2(θ−β2)+ρ4 sin2(θ−β2)

}. (8)

As a result, the ANDD representation of the UCM is

∂∇UCMTi ,βi ,s(θ) = ρ

2√

2πσ

s∑i=1

(Ti − Ti−1) cos(θ − βi )√cos2(θ − βi )+ ρ4 sin2(θ − βi )

(9)where Ts = T0.

Step edge, simple corner, and high-order corners matchthe ANDD representations with different parameter settings.Substituting β1 = β, β2 = β + π , s = 2 into (9) yields theANDD representation of a step edge,

∂∇Eβ (θ) = ρ√

2πσ

(T1 − T2) cos(θ − β)√cos2(θ − β)+ ρ4 sin2(θ − β)

(10)

It has two zeros at θ = β ± π/2 and two extremes atθ = β, π + β. The magnitude at the two extremes is directlyproportional to the grayscale difference on the two sides of theedge and the anisotropic factor while inversely proportional to

the scale. The maximal magnitude of the ANDD representationhas been used in the noise-robust edge detector [21].

Each term in the ANDD representation

ξβ,ρ (θ) ≡ cos(θ − β)√cos2(θ − β)+ ρ4 sin2(θ − β)

(11)

is referred to as a basic component specified by the angle βand anisotropic factor ρ. In terms of the basic component, theANDD representation in (9) can be rewritten as

∂∇UCMTi ,βi ,s(θ) = ρ

2√

2πσ

s∑i=1

aiξβi ,ρ(θ), �i = Ti − Ti−1 (12)

where �i is the strength of the i -th basic component. Thenumber of the basic components in (12) indicates the typeof a corner. A step edge has one single basic component, asimple corner has two components, a Y-type corner has threecomponents, an X-type corner has four components, and a star-like corner has at least five components. Classifying corner isequivalent to counting the number of the components in theANDD representation of a corner.

As shown in the Fig. 3 (b), the number of the basiccomponents in (12) is relevant to the number of the maximaof the magnitudes of the ANDD representation. A step edgehas only a pair of maxima symmetric with respect to theorigin, a simple corner has two pairs of maxima, a Y-type


corner has three pairs of maxima, an X-type corner has fourpairs of maxima, and a star-like corner has at least five pairsof maxima. The number of the magnitude maxima in theANDD representation of a corner can be utilized for cornerclassification. In the Fig. 3 (c), all the graphs have only a pairof maxima symmetric with respect to the origin, implying thatthe isotropic directional derivatives are unable to tell cornersfrom edge pixels. When the anisotropic factor ρ = 1, theANDD representation of the UCM degenerates to a singleform

∂∇UCMTi ,βi ,s(θ) = 1

2√

2πσ

s∑i=1

(Ti − Ti−1) cos(θ − βi )

= �

2√

2πσcos(θ − β) (13)

Because isotropic Gaussian kernels smooth out the detailsof corners, isotropic directional derivatives are unable todiscriminate the types of corners.

C. Corner Resolution and Anisotropic Factor

The ability of the ANDD representation to classify cornersdepends on whether the magnitude maxima related to eachbasic component can be resolved. In terms of the ANDDrepresentation of the UCM, the resolution to corner types isrelevant to the strengths, the distribution of the angles in theUCM, and the forms of the basic components. The last termis uniquely determined by the anisotropic factor. Generally,the ANDD representation with a large anisotropic factor hasa strong ability to classify corners.

In what follows, we quantitatively analyze the relationshipbetween the corner resolution and anisotropic factor. As doesin the edge resolution of the ANDDs [21], consider the sumof two basic components with the same strength

β1,β2(θ) = ξβ1,ρ (θ)+ ξβ2,ρ (θ) . (14)

For a given anisotropic factor, we find out the condition ofthe angle difference �β = β2 − β1 under which the peaksrelated to the two basic components can be resolved from(14). Each component has a pair of peaks at θ = βi , βi + π .It depends upon the property of (14) at the midpoint β = (β2+β1)/2 whether the function (14) has two pairs of resolvablepeaks. It has two pairs of resolvable peaks if the midpointis a minimum. It has only a pair of peaks at β and β + πif the midpoint is a maximum. The midpoint is a stationarypoint of the function (14) and its first-order derivative is zeroat the midpoint. Its second-order derivative at the midpoint isderived as follows:

′′ρ,β1,β2

(β) = 2ρ4 cos (�β/2)((ρ4 − 1)(1 − cos(�β))− 1

)(cos2 (�β/2)+ ρ4 sin2 (�β/2)

)5/2(15)

The midpoint is a minimum if

(ρ4 − 1) cos�β < ρ4 − 2 (16)

It implies two important facts. On the one hand, for a givenangle difference �β = β2 − β1 ≤ π , the two pairs of peaks

Fig. 4. (a) Traces of the three peaks of the ANDD representations of theY-type corners and (b) traces of the three peaks of the ANDD representationsof a Y-type corner as ρ varies from 1.8 to 8.

related to the two components can be resolved only when theanisotropic factor satisfies

ρ4 > 1 + 1

1 − cos�β≥ 2 (17)

Moreover, when the anisotropic factor ρ4 < 2, the ANDDrepresentation does not have ability to discriminate the typeof a corner. On the other hand, for a given anisotropic factorρ4 ≥ 2, the two pairs of peaks can be resolved only when theangle difference �β satisfies

2arc sin

(1√

2(ρ4 − 1)

)< �β < π−2arc sin

(1√

2(ρ4 − 1)

).

(18)The ANDD representation with a large anisotropic factor

has a strong ability to classify corners and the constant2arc sin

(1/√

2(ρ4 − 1))

is referred to as the corner resolutionconstant of the ANDD representation. Generally, when all theangle differences �βi = βi+1 − βi of a high-order cornersatisfy (18), the peaks related to all the basic components canbe resolved from the ANDD representtation and the type ofthe corner can be determined.

Two examples are illustrated to explain the corner resolutionconstant. An example uses a family of Y-type corners withβ1 = 90°, β2 = 5°, β3 = α, T1 = 2, T2 = 3, and T3 = 1.As α varies from 90° to 150°, the traces of the three peaksof the ANDD representations with σ 2 = ρ2 = 6 are shownin Fig. 4 (a). Only when α − β1 > 16°, the peaks of thefirst two basic components are resolvable. The result basicallyaccords with the resolution constant 2arc sin

(1/

√70)

≈ 14°.Another example is a Y-type corner with β1 = 5°, β2 = 90°,β3 = 105°, T1 = 2, T2 = 3, and T3 = 1. Fig. 4 (b) illustratesthe traces of the three peaks of the ANDD representationsas ρ varies from 1.8 to 8. The peaks of the first two basiccomponents are resolvable only when ρ > 2.4, basicallyaccording with ρ > 2.3471 from (17). The corner resolutionconstant gives a good evaluation of the ability for the ANDDrepresentation to classify corners.


D. New Corner Measure: Residual Area

Corner detection is to pick out corners from edge pixelsand normal pixels. When detection is made on the extractedcontours of an image, corner detection is to pick out cornersfrom edge pixels on contours. A corner measure needs tobe constructed from the ANDD representation to tell cornersfrom edge pixels. In terms of the previous analysis, theANDD representation of an edge pixel contains a singlebasic component while that of a corner contains at least twobasic components. Our idea to construct corner measure isfirst removing a maximal basic component from the ANDDrepresentation of a pixel under test and then constructing acorner measure from the residual part. Based on the factthat the magnitudes of the ANDD representation depend onthe grayscale differences around an edge pixel or corner, theANDD representation ∂∇(θ ) is first normalized by its maximalmagnitude to get a normalized ANDD representation,

∂∇(θ) = ∂∇(θ)maxθ

{|∂∇(θ)|} . (19)

The normalized representation is only related to the relativestrengths and angles of the UCM. In terms of (10), thenormalized representation of a step edge is written as

∂∇Eβ(θ) = sign(T1 − T2)ξβ,ρ (θ) (20)

From the normalized representation, the maximal basiccomponent is removed by

β = arg maxθ

{∣∣∂∇(θ)∣∣},R−∂∇(θ) ≡ ∂∇(θ)− sign{∂∇(β)}ξβ,ρ (θ) (21)

The function R−∂∇(θ) is referred to as the residual ANDDrepresentation. In (21), the maximal peak in the normalizedrepresentation is first localized and then the maximal basiccomponent is removed. When ∂∇(θ ) is the ANDD represen-tation of an edge pixel, the residual ANDD representationapproximates to zero. When ∂∇(θ ) is the ANDD representa-tion of a corner, the residual ANDD representation consists ofseveral basic components. In terms of the difference betweencorners and edge pixels at the residual ANDD representations,the area surrounded by the closed curve

∣∣R−∂∇(θ)∣∣ in thepolar coordinate system, defined by

R − area = 1

2

∫ 2π

0

(R−∂∇(θ))2 dθ (22)

is used as a corner measure to decide whether the pixel undertest is an edge pixel or a corner. This new measure is namedthe residual area. It takes small values at edge pixels whiletakes large values at corners.

III. DISCRETE ANDD FILTERS AND DISTRIBUTION OF

RESIDUAL AREA

In this section, the discrete ANDD filters and the formulaeto compute the corner measure in the discrete case are given.Moreover, the empirical probability cumulative distributionsof the corner measure on the edges of two test images arecomputed, by which the threshold for corner decision can beempirically determined.

A. Discrete ANDD Filters

Due to discrete image signals, the anisotropic Gaussiankernels and ANDD filters must be discretized in applica-tions. Sampling the continuous functions with a scale σ andanisotropic factor ρ on the 2D integer lattice Z

2 yields discreteANDD filters as follows:

ϕσ,ρ,k(n) = gσ,ρ(Rkn), ψσ,ρ,k (n) = ∂gσ,ρ∂y

(Rkn)

Rk =[

cos θk sin θk

− sin θk cos θk

],n =

[nx

ny

]∈ Z

2

θk = 2π(k − 1)

K, k = 1, 2, . . . , K (23)

where the integer vector n = [nx , ny]T denotes the coordinateon the 2D integer lattice Z

2. In order to make that the discreteversion inherits the properties of the continuous kernels asmuch as possible, the scale σ is required to be not less thanthe anisotropic factor ρ such that the scales of the anisotropicGaussian kernels at all the orientations are no less than one.In this paper, we take σ = ρ. In fact, it is also allowableto take σ ≥ ρ in applications. The sampling interval of theoriented angle depends upon the corner resolution constant.The number of the orientations, K , is required to satisfy

K ≥ 3π

arc sin

(1√

2(ρ4−1)

) (24)

It means that there are at least three samples within themainlobe of each basic component. For instance, when σ 2 =ρ2 = 6, the corner resolution constant is 14° and the samplinginterval should be less than 4.3° or K ≥ 360/4.3 ≈ 84.

From the discrete ANDD filters, the discrete ANDD repre-sentation is computed by

∂∇(n; k) =∑mx

∑my

I (n − m)ψσ,ρ,k(m),

n = [nx , ny]T ∈ Z2,m = [mx ,my]T ∈ Z

2 (25)

where n and m represent the coordinates on the 2D integerlattice Z

2.The discrete normalized ANDD representation, residual

ANDD representation, and residual area are computed by

∂∇(n; k) = ∂∇(n; k)

maxk

{|∂∇(n; k)|} , k0 = arg maxk

{|∂∇(n; k)|},

R−∂∇(n; k) ≡ ∂∇(n; k)− sign{∂∇(n; k0)}ξk0�θ,ρ (k�θ) ,

R − area(n) = �θ

2

P∑k=1

(R−∂∇(n; k)

)2, �θ = 2π

P. (26)

B. Residual Area Distribution on Non-Ideal Edges

On ideal step edges, the residual area is zero, a deterministicquantity. Real images contain considerable non-idea edges,such as ramp edges, pulse edges, and stair edges. These edgesdo not follow the ideal step edge model. Moreover, the discreteANDD filters and the localization error in edge extractionmake the residual area apart from zero. For real images, theresidual area on extracted edges is not a deterministic quantity


Fig. 5. Empirical cumulative distributions of the residual area on the edgesof the two test images: (a)-(b) test images ‘Block’ and ‘Lab’, (c)-(d) edgesextracted by the Canny edge detector, (e)-(f) empirical cumulative distributionsof the residual area on the edges.

any more. Several uncertain factors make that the values of theresidual area at edge pixels spread in a wide range. Here, weregard the residual area as a random variable on edge pixels inorder to empirically determine the threshold in corner decisionby its empirical probability cumulative distribution.

Two grayscale images, 256 × 256 ‘Block’ and 512 × 512‘Lab’, are used in experiments. Their edges are extracted bythe Canny edge detector [24] and the residual areas at theedge pixels are computed in terms of (26). The empiricalcumulative distributions of the residual area on the edgesof the two images are shown in Fig. 5. When the simplethreshold decision is applied on edge pixels to find corners,it can be observed that the probability of edge pixels tobe falsely declared as corners is about 0.2 if the decisionthreshold is taken as 0.12 and the probability is about 0.02if the decision threshold is taken as 0.40. Thus, the deci-sion threshold in corner detection is taken from the interval[0.12, 0.40]. In corner detection, the non-maximum suppres-

Fig. 6. Flowchart of the proposed corner detector.

sion and thresholding are combined to pick out corners fromedge pixels. Since non-maximum suppression removes quite afew false corners, the probability of edge pixels to be falselydeclared as corners is less than the estimate from the empiricalcumulative distributions in Fig. 5. The decision thresholdselection is a compromise. A small threshold is apt to yield ahigh detection probability but a large number of false cornerswhile a large threshold misses more corners but yields a smallnumber of false corners.

IV. CORNER DETECTOR AND CLASSIFIER VIA

ANDD REPRESENTATIONS

By means of the ANDD representation and the new cornermeasure, a corner detector and a corner classifier are proposed.The corner detector employs the course of the contour-baseddetection scheme equipped with the residual area as the cornermeasure. It consists of three cascaded blocks: edge detection,contour extraction and patching, and corner decision. In thefirst block, edges of single-pixel width are extracted from animage by the Canny edge detector [24]. The Canny detectorcan be also replaced by other edge detectors. As shown in theFig. 3 (c), the gradients at corners or junctions are sometimessmall. As a result, some corners are excluded in the detectededges. The second block includes contour extraction, fillingsmall gaps between contours, and contour stretch, by whichthe missed corners in edge detection are gotten back. In thethird block, the residual area is used as the corner measureto pick out corners from contours by the non-maximumsuppression and thresholding. The flowchart of the proposedcorner detector is shown in Fig. 6.

The edge map of an image is extracted by the Canny edgedetector [24]. The edge map is a binary image where the pixelswith value one label edge pixels. The Canny edge detector hasthree adjustable parameters, the Gaussian smooth scale, lowthreshold, and high threshold for edge decision. Selecting thethree parameters can achieve the tradeoff between missed edgepixels and false edge pixels and between noise-robustness andedge resolution [21]. Below, all the blocks are introduced indetails.

A. Contour Extraction and Patching

Contour extraction: From the edge map, the contoursare extracted. Each contour is represented by a chain codeC = P1, P2, . . . , Pn , where Pi = (xi , yi ), i = 1, 2, . . . , n isthe position of the i -th pixel on the chain code. P1 = Pn


Fig. 7. Demonstration of the full procedure of contour extraction andpatching.

for a loop contour and P1 �= Pn for an open contour and nocontour is self-intersection. The contour extraction uses thesame algorithm as in [12]–[14]. � stands for the set of theends of all the open contours.

Filling and connecting: For an end P in �, if its neigh-borhood of a specified size (for example, 5×5) contains otherends, we select the nearest one from P as Q and fill the gapbetween P and Q. When P and Q are on the same contour,label the filled contour as a loop contour and remove P andQ from �. When P and Q are on two contours, connect theminto an open contour and remove P and Q from �. If thereare more than one nearest ends from P , we select the end bywhich the connected contour has minimal curvature increment.Repeat the procedure till the neighborhood of each end in �does not contain other ends.

Stretching open contours: When the neighborhood of theend of one open contour contains edge pixels on other contour,stretch the end of the open contour to the nearest edge pixelon the other contour.

Fig. 7 illustrates a full procedure from edge detection tocontour extraction and patching. It can be seen that most ofthe missed corners in the edge detection are picked up by thecontour extraction and patching. Fig. 7 (a) is the edge mapextracted by the Canny edge detector, where some cornersare left out because of not enough large magnitudes of thegradients. Fig. 7 (b) plots the contours extracted from the edgemap. Fig. 7 (c) illustrates the contours after filling small gapsand connection. Some omitted edge pixels are picked up. Forinstance, the broken contours ‘63’ and ‘157’ are recovered intoa loop contour, the contours ‘86’ and ‘87’, the contours ‘27’

and ‘36’ are connected into two longer contours. These filledgaps contain corners of interest. Fig. 7 (d) is the contours afterend stretch, where some ends of the contours ‘80’ ‘86’ ‘16’and ‘102’ are stretched. Stretch picks up some omitted edgepixels in the edge detection.

B. Corner Decision Via Residual Area

The proposed corner detector uses the residual area as thecorner measure to pick out corners from each contour bythree steps: computation of the corner measure, non-maximumsuppression and thresholding.

Computation of corner measure: Let {C =P1, P2, . . . , Pn} be an extracted contour. At each pixel on thecontour C , calculate the normalized ANDD representationand corner measure, residual area R-area (Pi ) by (26).

Non-maximum suppression: As do in the contour-basedcorner detection [9]–[15], only the local maxima of the cornermeasure on a contour are indicated as candidate corners. Letthe width of the window for the non-maximum suppressionbe 2L + 1. Then, a pixel Pi on the contour C is a candidatecorner if its corner measure satisfies:

R − area(Pi ) > max{R − area(Pi±l ), l = 1, 2, . . . , L} (27)

where Pi±l are replaced by P(i±l)modn+1 when i ± l > n ori ± l < 1 for a loop contour and R-area(Pi±l ) = 0 wheni ± l > n or i ± l < 1 for an open contour. Running (27) onall the contours, we obtain the set of the candidate corners,denoted as �C .

Thresholding: Let η be a predefined threshold for cornerdecision, which is empirically selected in terms of the residualarea distributions on non-ideal edges in Section III(B). A pixelP ∈ �C is decided as a corner only when it satisfies:

R − area(P) ≥ η (28)

The proposed corner detector combines the contour-basedframework with the intensity-based decision. It inherits themerits of the contour-based detection and intensity-baseddetection and overcomes their some defects. The intensity-based detection easily yields false corners in smooth regions[1]–[7]. The contour extraction before detection in the pro-posed detector reduces the occurrence of false corners insmooth regions. Moreover, fine local intensity variations incorner decision are exploited to improve the capability ofcorner detection.

C. Corner Classifier Via ANDD Representations

In feature matching of images, types of corners are asimportant as their positions. Most of corner detectors cannotclassify corners. Corner classification needs finer descriptionof intensity variations around corners. Multi-scale multidi-rectional intensity variations via the Log–Gabor wavelet [8]and shearlet [18] provide potential for corner classification.The shearlet-based edge detector can classify pixels into fourgroups: regular pixels, smooth edge pixels, pixels near edges,and corners. In [25], a simple corner classifier based on theHarris detection was proposed and it can classify corners intotwo groups: simple corners and high-order corners.


Here, the normalized ANDD representations of corners areused to classify corners. In the ANDD representation of acorner, the number of the peaks or the basic componentsindicates its type. Let � be the set of detected corners. Thenormalized ANDD representation of a non-ideal corner P ∈ �satisfies:

∂∇(P, k) ≈κ(P)∑i=1

aiξβi ,ρ (k�θ) (29)

where κ(P) is the number of the basic components and isreferred to as the type-index of a corner. Classifying corneris to estimate the type-index of a corner from its ANDDrepresentations, which can be realized by two approaches. Oneis finding the peak number of the ANDD representation. Theother is estimating the number of the basic components. Weadopt the first approach. Let τ > 0 be a predefined threshold,empirically taken as 0.12.

κ(P) = #{k:∣∣∂∇(P, k)

∣∣> max

{τ,∣∣∂∇(P, k + 1)

∣∣ , ∣∣∂∇(P, k − 1)∣∣}} (30)

where #A denotes the number of the entries in a finite set A.The index κ(P) of a corner stands for the number of peakswith magnitude more than the threshold τ in the normalizedANDD representation. The proposed corner classifier is asfollows:

κ(P) =⎧⎨⎩

2, P is a simple corner,3, P is a Y - type corner,

≥ 4, P is an X - type or star - like corner.(31)

Like in corner detection, the threshold τ is a compromise.A large threshold is easy to falsely declare high-order cornersinto low-order ones while a small threshold is apt to falselydeclare low-order corners into high-order ones.

V. EXPERIMENTS AND PERFORMANCE EVALUATION

This section focuses on experiments and performance eval-uation. The proposed detector is compared with three state-of-the-art detectors, the He & Yung detector [13], CPDAdetector [14], and classical Harris detector [2], from detectionperformance, localization error, and repeatability under imageaffine transforms. Two images with GTs are used to comparedetection capabilty and localization precision. Nine imageswithout GTs are used to compare the repeatabilty under imageaffine transforms. Two images with GTs of types are used toevaluate corner classification.

Two commonly-used 8-bit grayscale images with GT, 256×256 ‘Block’ and 512×512 ‘Lab’ [26], are used in experimentsto evaluate detection ability and localization precision. Fig. 8illustrates them and their GTs, where the ‘Block’ has 59strong corners and the ‘Lab’ has 249 strong corners. Eachdetector includes adjustable parameters and their settings affectdetection results. The results of the detectors under individualparameter settings are difficult to give a fair evaluation.Comparison of edge detectors also meets the same problem.The ROC curves are introduced for a fair comparison [27].Its essential idea is to compare the results of detectors underindividual optimal parameter settings. Here, the ROC curvesare borrowed to evaluate the corner detectors.

Fig. 8. Test images (a) ’Block’ and (b) ’Lab’ and their GTs where cornersare labeled by small squares.

For a test image with the GT that contains NG cornersnecessary to be detected, if a detector at a given parametersetting reports ND corners including NT correctly detectedcorners and NF false corners, then the point (NG − NT , NF )is a sample point of the performance test. A corner in theGT is correctly detected only when there exist at least onedetected corner in its 9 × 9 neighborhood. The missed rateis difined as (NG − NT )/NG and the fasle rate is defined asNF /NG . When the parameter setting runs in a wide range, amass of sample points ((NG − NT )/NG , NF /NG ) are collected.The low envelop of these sample points is the ROC curve ofthe corner detector. Corner detection is desired to have missedcorners and false corners as few as possible. Each point on theROC curve can be interpreted as the result under the optimalparameter setting at a given number of missed corners. TheROC curve does not depend on a specific parameter setting.For two detectors, a detector is better when its ROC curvelocates on the left-bottom side of that of the other detector.

Besides detection performance, the localization error isanother index to evaluate corner detectors. Let the GT ofa test image be {(xi , yi ), i = 1, 2, . . . , N} and the detectedcorner set be {(xm, ym),m = 1, 2, . . . ,M} by a detectorat a given parameter setting. The matched corner pairs are{(xi , yi ), (xm(i), ym(i)) : i = 1, 2, . . . , NT }. The localizationerror is defined by [15]

EL =√√√√ 1

NT

NT∑i=1

{(xi − xm(i)

)2 + (yi − ym(i))2} (32)

The third index to evaluate a corner detector is the repeata-bility under geometric transforms. It is important in applica-tions such as image feature matching and image registration.

A. ROC Curve Evaluation of Detection Performance

The He & Yung detector [13], CPDA detector [14] andthe proposed detector realize corner detection on contoursand their performance depends upon edge detection. Edgedetection results are relevant to the parameter setting ofthe Canny edge detector [24]. Thus, the Gaussian smoothscale σs , low threshold TL , and high threshold TH are takenas adjustable parameters in the ROC curve evaluation of thethree contour-based detectors. Besides the three adjustable


Fig. 9. ROC curve comparison of the proposed detector, He & Yung detector [13], CPDA detector, and Harris detector, where the optimal parameter settingsof the detectors are labeled for (a) ‘Block’ when three corners are missed and (b) ‘Lab’ when sixty corners are missed.

parameters, the scale and anisotropic factor in the ANDDrepresentation, σ = ρ, and the decision threshold η are alsoadjustable for the proposed detector. It uses five adjustableparameters and the parameter settings are taken in the set⎧⎪⎪⎨

⎪⎪⎩

σs = 1, 2, . . . , 5; TL = 0, 0.05, 0.1, 0.15;TH = 0.20, 0.25, . . . , 0.40;σ 2 = ρ2 = 2, 4, 6, 8;

η = 0.11, 0.12, . . . , 0.40.

⎫⎪⎪⎬⎪⎪⎭

(33)

The Harris detector has three adjustable parameters, theGaussian smooth scale σs , the sensitivity factor κ in the Harriscorner measure, and the decision threshold η (where the cornermeasure is normalized by its maximal value on the wholeimage). The Harris corner detector uses the standard programin the Matlab software of 2010 version, where the defaultparameter setting is (σs, κ, η) = (1.5, 0.04, 0.01). In the ROCcurve computation, the parameter settings are taken in the set⎧⎨

⎩σs = 1, 1.5, 2, 2.5, 3κ = 0.01, 0.02, . . . , 0.1η = 0.002, 0.004, . . . , 0.1

⎫⎬⎭ (34)

The Harris corner detector uses 5 × 5 window in the non-maximum suppression.

In the He & Yung detector [13], except for the threeparameters in the Canny edge detector, the coefficient R inthe adaptive threshold and angle-threshold δ are adjustable.The experiments use the Matlab code of the algorithm openedin the website by the authors [28]. In the ROC curve compu-tation, the default parameter setting is replaced by adjustableparameter settings, which are taken in the set⎧⎪⎪⎨

⎪⎪⎩

σs = 1, 2, . . . , 5; TL = 0, 0.05, 0.1, 0.15TH = 0.20, 0.25, . . . , 0.40R = 1.05, 1.06, . . . , 1.80δ = 140°, 141°, . . . , 178°

⎫⎪⎪⎬⎪⎪⎭

(35)

In the CPDA detector [14], except for the three parametersin the Canny edge detector, the curvature threshold η andthe angle-threshold δ are adjustable. The experiments use theoriginal code of the algorithm provided by the authors. Theparameter settings are taken in the set⎧⎪⎪⎨

⎪⎪⎩

σs = 1, 2, . . . , 5; TL = 0, 0.05, 0.1, 0.15TH = 0.20, 0.25, . . . , 0.40η = 0.01, 0.02, . . . , 0.35δ = 140°, 141°, . . . , 178°

⎫⎪⎪⎬⎪⎪⎭

(36)

The ROC curves of the four detectors to the two test imagesare shown in Fig. 9. For the ‘Block’ with a simple scene,the proposed detector and CPDA detector attain comparabledetection performance, which is better than the He & Yungdetector and Harris detector. For the ‘Lab’ with a complicatedscene, when missed corners are no more than sixty out of249 labeled corners in the GT, the proposed detector andHe & Yung detector attain comparable detection performance,superior to that of the CPAD and Harris detectors. The Harrisdetector behaves much poorer than the three contour-baseddetectors in performance. The reason is that the Harris detectordirectly operates on all the pixels of an image and lacks enoughability to exclude false corners. The contour extraction canexclude quite a few false corners.

Noise-robustness is important in corner detection, becauseimages are often corrupted by noise in acquisition and trans-mission. Here, zero-mean white Gaussian noise of variance100 is added to the test images to verify the noise-robustness.In the noisy cases, the ROC curves of the four detectors areshown in Fig. 10. The contour-based detectors are more robustto noise than the Harris detector. In fact, the contour extractionwith a large-scale Gaussian smooth kernel can effectivelyreduce noise influence. The contour extraction and noise-robust ANDD representation make the proposed detector more


Fig. 10. ROC curve comparison of the four detectors for the test images (a) Block and (b) Lab corrupted by additive zero-mean white Gaussian noise of avariance 100.

robust to noise than other three detectors. Experiments withother nosie levels are made, showing that the ROC curvesof the proposed detector are almost invariant when noisevariance is not more than 30 for the two test images. Moreover,contrasting the optimal parameter settings of the four detectorsin Figs. 9 and 10, it is found that in the noisy case the smoothscales and decision thresholds are mostly lifted to counterworknoise. Hence, when images are corrupted by noise, increasingsmooth scale of edge detection and lifting the decisionthreshold of corner decision are appropriate tactics.

By examining the parameter settings of the sample pointsaround the ROC curves, we find that the performance ofthe proposed detector is not sensitive to low threshold andGaussian smooth scale in edge extraction and is sensitiveto high threshold in edge extraction, scale and anisotropicfactor in the ANDD representation, and decision threshold.Particularly, the test sample points when σ 2 = ρ2 = 6 arealways close to the ROC curves. Therefore, the scale andanisotropic factor can be fixed at

√6 in applications. The

default parameter setting is recommended as (1.0, 0.1, 0.35,6, 0.12) in the noiseless case.

Below, we examine the localization error of the proposeddetector in terms of (32). For a fair comparison, the optimalparameter settings of the four detectors labeled in Fig. 9 areused. For the test image ‘Block’, the number of missed cornersis three for all the four detectors and for the test image ‘Lab’the number of missed corners is sixty for all the four detectors.In the noisy case, the optimal parameter settings for the fourdetectors are used when six corners are missed for the testimage ‘Block’ and seventy-five corners are missed for the testimage ‘Lab’. The localization errors of the four detectors arelisted in Table I. It can be observed that noise of low leveldoes not incur a significant change in localization error for allthe four detectors, for the corners in the GTs are strong local

TABLE I

LOCALIZATION ERRORS OF THE DETECTORS FOR THE NOISELESS AND

NOISY IMAGES (UNIT IS A PIXEL)

Noiseless Test Images

Test MissedHarris CPDA [14]

He &YungProposed

image corners [13]

Block 3 1.932 1.528 1.603 1.597

Lab 60 1.950 1.460 1.492 1.436

Noisy test images and noise variance 100

Block 6 1.981 1.607 1.643 1.641

Lab 75 2.115 1.526 1.558 1.483

feature. The CPDA detector attains the smallest localizationerror for the ‘Block’ and the proposed detector attains thesmallest localization error for the ‘Lab’. The Harris detectoris the poorest in corner localization.

The localization error of corners is affected by many factors,for example, the strength of corners, smoothing of contours,noise, and so on. Generally, small localization errors happen atstrong corners while large localization errors appear at weakcorners for a detector. Smoothing contours perhaps incursbias in localization. It is diffucult to thoroughly analyze thereasons behind the localization error of each detector. Fromthe experiments, the contour-based detectors have smallerlocalization errors than the intensity-based Harris detector.The CPDA detector attains the highest localization precisionfor the test image ‘Block’. The proposed detector attainsthe highest localization precision for the test image ‘Lab’.In the noisy case, the proposed detector attains the highestlocalization precision for the two test images. Due to thenoise, the localization errors of the four detector increaseby 2.048%, 1.556%, 1.601%, and 1.562%, respectively. The


Fig. 11. Nine test images to compute the average repeatablity under affinetransforms.

CPDA detector and the proposed detector are robust to noise,owing to the scale smoothing of contours in the CPDA andthe noise-robust ANDD in the proposed detector.

B. Repeatability Under Affine Transforms

Let N0 and NT be the numbers of corners detected from anoriginal image and transformed image by a detector, respec-tively. Nr is the number of repeated corners in the originalimage and transformed image. The repeatability of a detectorunder a given transform on a test image is defined as [15]

Ra = Nr

2

(1

N0+ 1

NT

)(37)

The average repeatability of a detector at a given transformon a group of test images is the average value of the repeatabil-ity of all the test images at the given transform. Here, nine testimages in Fig.11, often appearing in corner detection literature,are used to evaluate the repeatability. Its computation needn’tthe GTs of test images.

The four detectors use the parameter settings labeled inFig. 9(b) for the noiseless test image ‘Lab’. The four kindsof basic single-parameter transforms are considered.

1) Rotations at 18 different angles in [−90°, 90°] at 10°apart;

2) Uniform scaling transforms with x ′ = xsx , y′ = ysy ,sx = sy in [0.5, 2.0] at 0.1 apart;

3) Nonuniform scaling transforms sx = 1, sy �= 1 in[0.5,0.9] ∪ [1.1,2.0] at 0.1 apart;

4) Shear transforms, where the shear factor c in [−1,1] at0.1 apart. [

x ′y ′]

=[

1 c0 1

] [xy

]

Any affine transform without translation can be decomposedinto rotation, uniform scaling, nonuniform scaling, shear, andrefection. The average repeatabilities of a corner detector inthese four kinds of basic transforms reflect its repeatabilityunder affine transforms. The average repeatabilities of the fourdetectors are shown in Fig. 12.

The repeatabilities of the four detectors exhibit differentbehavior. For rotation and uniform scale transforms, all thefour detectors preserve high repeatability, owing to the cornermeasures of the detectors are invariant to rotation and uniformscale transforms in the continuous form. The influence ofrotation and uniform scale transforms on corner detectionresults mainly comes from resampling of transformed images.In rotation and uniform scale transforms, the Harris detectorand the proposed detector have higher repeatability than theHe & Yung detector and the CPDA detector, which is becausethat the intensity-based corner measures are more robust toresampling than the contour-based corner measures. For non-uniform and shear transforms, the corner measures of the fourdetectors vary with the transforms in the continuous form.Hence, the repeatabilities of all the four detectors are low. Itcan be seen that the Harris detector and the proposed detectorhave higher repeatability than the He & Yung detector and theCPDA detector. The latter two detectors are based upon thelocal curvature of contours. The local curvature of contoursis sensitive to non-uniform scale and shear transforms. Asa whole, the proposed detector achieves higher repeatabil-ity under affine transforms. Note that the Harris detectorhas too low repeatabilty under shear transforms though ithas high repeatabilities under the first three kinds of basictransforms.

C. Experiments of Corner Classification

It is important in image feature matching and object recog-nition to know corner’s types. Generally, simple corners arefrom the edge vertex of object outlines while high-ordercorners, such as Y-type and X-type, are from overlaps ofobjects in images. Most of corner detectors detect and localizecorners but cannot report their types. A simple corner classifieralong the Harris corner detection idea is given in [25]. It canclassify corners into two groups: simple corners and high-order corners. The proposed corner classifier can classifycorners into simple corners, Y-type corners, and X-type or star-like corners. The previous experiments show that the ANDDrepresentation with the scale and anisotropic factor σ 2 =ρ2 = 6 is suitable for corner detection. Corner classificationneeds higher resolution ANDD representation to attain finerdirectional intensity variations at corners. Here, the scale factorand anisotropic factor are taken as σ = 4 and ρ2 = 10 forclassification. In terms of (25), the number of orientations is384. For the two test images ‘Block’ and ‘Christopher’, theresults of the proposed corner classifier and classifier in [25]are demonstrated in Table II and Fig. 13.

The image ‘Block’ has twenty-eight simple corners andthirty-one Y-type corners and the image ‘Christopher’ hasnineteen simple corners, thirty-six Y-type corners, and thirtyhigher-order corners. The correct rate of a corner classifier is


Fig. 12. Average repeatabilities of the four detectors under (a) rotation, (b) uniform scaling, (c) nonuniform scaling, and (d) shear transforms.

TABLE II

CLASSIFICATION RESULTS OF THE TEST IMAGES ‘BLOCK’ AND ‘CHRISTOPHER’

Test images Classifier Types of Corners SimpleY- X-Type or

Type Star-Like

Block

Ground truth with types 28 31 0

Proposed classifierDetected corners 26 31 0

Type-matched corners 19 21 0

Classifier in [25]Detected corners 25 31

Type-matched corners 17 18

Christopher

Ground truth with types 19 36 30

Proposed classifierDetected corners 14 31 25

Type-matched corners 13 21 18

Classifier in [25]Detected corners 12 57

Type-matched corners 10 35

defined as the ratio of the number of type-matched cornersto the number of correctly detected corners. When correctlydetected corners are classified into two groups as in [25]. Theproposed classifier achieves 70.2% correct rate for the ‘Block’and 74.2% correct rate for the ‘Christopher’. The classifierin [25] has the correct rate 62.5% for the ‘Block’ and thecorrect rate 65.5% for the ‘Christopher’. The proposed cornerclassifier achieves higher correct rate. Moreover, the proposedclassifier provides fair ability to discriminate Y-type cornersand higer order corners. In the test image ‘Christopher’,twenty-one out of the thirty-one detected Y-type corners are

correctly classified and eighteen out of the twenty-five detectedhigher order corners are correctly classified.

Computational cost of a corner detector and classifier isa key factor to limit its applications. In our experiments,the detector uses the ANDD filters with 84 orientations andthe classifier uses the ANDD filters with 384 orientations.The 2D convolution computation of the ANDD representationsat all the pixels of an image is time-consuming. Because thedetection is operated on contours and the classification on thedetected corners and the number of the pixels on the contoursand the number of the detected corners are much smaller than


Fig. 13. Corner classification results of the test images ‘Block’ and ‘Christo-pher’: (a)-(b) test images ‘Block’ and ‘Christopher’, (c)-(d) classificationresults by the classifier in [25], (e)-(f) classification results by the proposedclassifier.

the size of an image, the proposed detector and classifier areof high computational efficiency.

In fact, the ANDD representation also suffers from a lim-itation. All the ANDD filters are antisymmetric with respectto their centers and thus the ANDD representation at a pixelsatisfies: ∂∇(θ + π) = −∂∇(θ). For some special high-order corners, such as an X-type corner with β3 = β1 + π ,β4 = β2 + π , T1 = T3, and T2 = T4 in the UCM,its ANDD representation vanishes to zero, meaning that theANDD representations fail to detect and classify such specialhigh-order corners. Since such special corners rarely appearin real images, this defect does not yield significant effect onapplications of the proposed detector and classifier.

VI. CONCLUSION

In this paper, we have proposed the corner detector andclassifier based on the ANDD representations on contours. Thedetector is evaluated from detection capability, localizationerror, and repeatability under image affine transforms. Theexperimental results show that it is very completitive withthe recent state-of-the-art corner detectors. Its good detectionperformance is owing to the reasonable fusion of the contour-based detection with intensity-based detection. Contour extrac-tion reduces the occurence probability of false corners andthe fine intensity variation information on contours enhancesthe ability to capture true corners. Corners are anisotropiclocal features in images. The fine description of local intensityvariations of corners becomes the key of corner detection andclassification. The ANDD representations provide a powerfultool of the fine description. Starting from the analytic ANDDrepresentation of the UCM, the corner resultion can be quan-titively analyzed and the corner measure can be analyticallydefined. Moreover, based on the characteristics of the ANDDrepresentations of different kinds of corners, the simple butefficient corner classifier is developed, which can classifycorners into simple corners, Y-type corners, and higher ordercorners.

As a tool to represent anisotropic local features in images,the ANDD representations have some potential worthy tofurther exploit. Color images are used in most of visualsystems and it is an interesting issue to extend the ANDDrepresentations for edge and corner detection as well as cornerclassification of color images. The antisymmetry of the ANDDfilters brings the limitation in detecting and classifying somespecial high-order corners. Developing asymmetric ANDDfilters is a possible way to overcome the limitation. Cornerclassification is an important issue in image feature matchingbut receives rather little attention. The proposed corner classi-fier provides a fair ability to classify corners. However, fromdemands of applications, corner classification ability needs tobe improved further.

REFERENCES

[1] H. P. Moravec, “Toward automatic visual obstacle avoidance,” in Proc.5th Int. Joint Conf. Artif. Intell., Cambridge, MA, USA, 1977, p. 584.

[2] C. Harris and M. Stephens, “A combined corner and edge detector,” inProc. 4th Alvey Vision Conf., 1988, pp. 147–151.

[3] J. A. Noble, “Finding corners,” Image Vis. Comput., vol. 6, no. 2,pp. 121–128, 1988.

[4] P. Perona, “Steerable-scalable kernels for edge detection and junctionanalysis,” Image Vis. Comput., vol. 10, no. 10, pp. 663–672, 1992.

[5] L. Alvarez and F. Morales, “Affine morphological multiscale analysisof corners and multiple junctions,” Int. J. Comput. Vis., vol. 25, no. 2,pp. 95–107, 1997.

[6] K. Mikolajczyk and C. Schmid, “Scale and affine invariant interest pointdetectors,” Int. J. Comput. Vis., vol. 60, no. 1, pp. 63–86, 2004.

[7] M. Jacob and M. Unser, “Design of steerable filters for feature detectionusing canny-like criteria,” IEEE Trans. Pattern Anal. Mach. Intell.,vol. 26, no. 8, pp. 1007–1019, Aug. 2004.

[8] X.-T. Gao, F. Sattar, and R. Venkateswarlu, “Multiscale corner detectionof gray level images based on log-Gabor wavelet transform,” IEEETrans. Circuits Syst. Video Technol., vol. 17, no. 7, pp. 868–875,Jul. 2007.

[9] A. Rattarangsi and R. T. Chin, “Scale-based detection of corners ofplanar curves,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 14, no. 4,pp. 430–449, Apr. 1992.


[10] F. Mokhtarian and R. Suomela, “Robust image corner detection throughcurvature scale space,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 20,no. 12, pp. 1376–1381, Dec. 1998.

[11] J. H. Han and T. T. Poston, “Chord-to-point distance accumulationand planar curvature: A new approach to discrete curvature,” PatternRecognit. Lett., vol. 22, no. 10, pp. 1133–1144, 2001.

[12] X. Zhang, M. Lei, D. Yang, Y. Wang, and L. Ma, “Multiscale curvatureproduct for robust image corner detection in curvature scale space,”Pattern Recognit. Lett., vol. 28, no. 5, pp. 545–554, 2007.

[13] X. C. He and H. C. Yung, “Corner detector based on global localcurvature properties,” Opt. Eng., vol. 47, no. 5, p. 057008, 2008.

[14] M. Awrangjeb and G. Lu, “Robust image corner detection basedon the chord-to-point distance accumulation technique,” IEEE Trans.Multimedia, vol. 10, no. 6, pp. 1059–1072, Oct. 2008.

[15] M. Awrangjeb and G. Lu, “An improved curvature scale-space cor-ner detector and a robust corner matching approach for transformedimage identification,” IEEE Trans. Image Process., vol. 17, no. 12,pp. 2425–2441, Dec. 2008.

[16] K. Rangarajan, M. Shah, and D. V. Brackle, “Optimal corner detector,”Comp. Vis., Graph., Image Process., vol. 48, no. 2, pp. 230–245, 1989.

[17] E. D. Sinzinger, “A model-based approach to junction detection usingradial energy,” Pattern Recognit., vol. 41, no. 2, pp. 494–505, 2008.

[18] S. Yi, D. Labate, G. R. Easley, and H. Krim, “A shearlet approach toedge analysis and detection,” IEEE Trans. Image Process., vol. 18, no. 5,pp. 929–941, May 2009.

[19] W. Jiang, K.-M. Lam, and T.-Z. Shen, “Efficient edge detection usingsimplified Gabor wavelets,” IEEE Trans. Syst., Man, Cybern. B, vol. 39,no. 4, pp. 1036–1047, Aug. 2009.

[20] J.-M. Geusebroek, A. W. M. Smeulder, and J. van de Weijer, “Fastanisotropic Gauss filtering,” IEEE Trans. Image Process., vol. 12, no. 8,pp. 938–943, Aug. 2003.

[21] P.-L. Shui and W.-C. Zhang, “Noise-robust edge detector combiningisotropic and anisotropic Gaussian kernels,” Pattern Recognit., vol. 45,no. 2, pp. 812–820, 2012.

[22] L. Parida and D. Geiger, “Junction: Detection classification and recon-struction,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 20, no. 7,pp. 687–698, Jul. 1998.

[23] M. A. Ruzon and C. Tomasi, “Edge, junction, and corner detection usingcolor distributions,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 23,no. 11, pp. 1281–1295, Nov. 2001.

[24] J. Canny, “A computational approach to edge detection,” IEEE Trans.Pattern Anal. Mach. Intell., vol. 8, no. 6, pp. 679–698, Jun. 1986.

[25] J.-B. Ryu, H.-H. Park, and J. Park, “Corner classification using Harrisalgorithm,” Electron. Lett., vol. 47, no. 9, pp. 536–538, 2011.

[26] F. Mokhtarian. Image Corner Detection Through Curvature Scale Space[Online]. Available: http://www.ee.surrey.ac.uk/CVSSP/demos/corners

[27] K. Bowyer, C. Kranenburg, and A. Dougherty, “Edge detector evaluationusing empirical ROC curves,” Comput. Vis. Image Understand, vol. 84,no. 1, pp. 77–103, 2001.

[28] (2009). A Corner Detector Based on Global and Local CurvatureProperties [Online]. Available: http://www.mathworks.cn/matlabcentral/fileexchange/7652

Peng-Lang Shui (M’02) was born in Xi’an, China,in 1967. He received the B.S. degree in appliedmathematics from Xidian University, Xi’an, the M.S.degree in mathematics from Nanjing University,Nanjing, China, and the Ph.D. degree in electricalengineering from Xidian University, in 1989, 1992,and 1999, respectively.

He is currently a Professor with the National Lab-oratory of Radar Signal Processing, Xidian Univer-sity. His current research interests include multiratefilter bank theory and applications, image process-

ing, and radar target detection and tracking.

Wei-Chuan Zhang received the M.S. degree inelectronic engineering from Southwest Jiaotong Uni-versity, Chengdu, China, and the Ph.D. degreein electronic engineering from Xidian University,Xi’an, China, in 2006 and 2013, respectively.

He is currently a Lecturer of Xi’an PolytechnicUniversity. His current research interests includepattern recognition and image processing.

06507646

Documents

proposed corner detector

universal corner

basic corner

contourbased detectors

intensitybased detectors

intensitybased detection

image processing

featurebased image