This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Research ArticleResearch on the Fundamental Principles and Characteristics ofCorrespondence Function
Xiangru Li12 Guanghui Wang3 and Q M Jonathan Wu1
1Department of Electrical and Computer Engineering University of Windsor Windsor ON Canada N9B 3P42School of Mathematical Sciences South China Normal University Guangzhou 510631 China3Department of Electrical Engineering amp Computer Science University of Kansas Lawrence KS 66045 USA
Correspondence should be addressed to Xiangru Li xiangruligmailcom
Received 17 April 2015 Revised 19 August 2015 Accepted 20 August 2015
Academic Editor Erik Cuevas
Copyright copy 2015 Xiangru Li et al This is an open access article distributed under the Creative Commons Attribution Licensewhich permits unrestricted use distribution and reproduction in any medium provided the original work is properly cited
The correspondence function (CF) is a concept recently introduced to reject the mismatches from given putative correspondencesThe fundamental idea of the CF is that the relationship of some corresponding points between two images to be registered canbe described by a pair of vector-valued functions estimated by a nonparametric regression method with more flexibility than thenormal parametricmodel for example homographymatrix similarity transformation and projective transformationsMismatchesare rejected by checking their consistencywith theCFThis paper proposes a visual scheme to investigate the fundamental principlesof the CF and studies its characteristics by experimentally comparing it with the widely used parametric model epipolar geometry(EG) It is shown that the CF describes the mapping from the points in one image to their corresponding points in another imagewhich enables a direct estimation of the positions of the corresponding points In contrast the EG acts by reducing the search spacefor corresponding points from a two-dimensional space to a line which is a problem in one-dimensional space As a result theundetected mismatches of the CF are usually near the correct corresponding points but many of the undetected mismatches of theEG are far from the correct point
1 Introduction
Finding point correspondences between two images is afundamental problem in computer vision [1 2] In two givenimages the corresponding points (CPs) are the projections ofthe same point in a scene Many computer vision algorithmsand applications rely on the successful identification of pointcorrespondences between two images for example trackingstereo vision motion analysis object recognition remotesensing image mosaicing and automatic quality controlamong others [3ndash7]
Point correspondences are usually established by thefollowing procedures extracting salient points calculatingtheir descriptors based on a small and local area aroundthem establishing putative correspondences by comparingthe descriptors and refining the putative correspondencesCompared with the representations based on a large spatialarea local feature descriptors are usually more robust to
brightness variation deformation and occlusion but haveless distinctivenessThis typically results in a high percentageof mismatchesoutliers among the computed putative cor-respondences which are very likely to ruin traditional esti-mation methods [8ndash10] Therefore an essential problem incomputer vision is rejecting mismatches from given putativecorrespondences in a refining stage [11ndash13]
Correspondence function (CF) is a model recently intro-duced based on point set mapping theory to reject mis-matches fromputative correspondences [2]The fundamentalidea of CF is that for two given images 119868 and 119868
1015840 of a scenethe relationships between their CPs can be described bya pair of vector-valued functions which are estimated bya nonparametric regression method Mismatches are thendetected by checking whether they are consistent with theCFs The key of the model is that the relationships betweenCPs are represented with more flexibility than the usualparametric model for example homography matrix [14]
Hindawi Publishing CorporationMathematical Problems in EngineeringVolume 2015 Article ID 721842 15 pageshttpdxdoiorg1011552015721842
2 Mathematical Problems in Engineering
A
B
C
D E
F
G
I998400
p998400F
p998400DE
p998400C
p998400B
p998400A
O pD pEF pG I
p998400G
pA pBC
(a)
I998400
p998400Fp998400
DEp998400Cp998400
Bp998400A
pA pBC pEF
I
middot middot middot
middot middot middot
pD(b)
Figure 1 A sketch map of the relationships between CPs (a) Two axes are a sketch map of two images 119868 and 1198681015840 of a scene to be matched that
curve is a sketch map of the correspondence manifold formed by CPs between 119868 and 1198681015840 [17] (b) a sketch map of the relationship between
CPs presented to emphasize the mapping from one image point to its CP(s)
p998400Fp998400
DEp998400Cp998400
Bp998400A
pA pBC pD pEF
I998400
I
middot middot middot
middot middot middot
(a)
p998400Fp998400
DEp998400Cp998400
Bp998400A
pA pBC pD pEF
I998400
I
middot middot middot
middot middot middot
(b)
Figure 2 Convert the relationship between CPs into two function mappings The red dotted lines represent the deleted correspondingconnections and the original corresponding relationship in Figure 1(b) can be recovered from (a) and (b) by collecting all of the remainingconnections and deleting the directions (a) CR
119868rarr1198681015840 these relationships between CPs can be depicted by a function from 119868 to 119868
1015840 (b) CR1198681015840rarr119868
these relationships between CPs can be depicted by a function from 119868
1015840 to 119868
similarity transformation [15] and projective transforma-tions [15] The flexibility is to account for nonrigidity ofobjects or to reduce the undue influence from outliers
In this work we propose a visual scheme to investigate thefundamental principles of CF (Figures 1 and 2) and to studyits characteristics by experimentally comparing it with thewidely used parametricmodel epipolar geometry (EG) (somerudimentary comparisons were made between CF and EG ina conference paper ([16])) It is shown that the CF describes
themapping from a point in one image to its CP(s) in anotherimage which enables us to directly estimate the positions ofthe CPs However the EG acts by reducing the search spacefor CPs from a two-dimensional space to a line which is aproblem in one-dimensional space In applications the resultof the difference between EG and CF is that the undetectedmismatches by CF are usually near the correct CPs but manyof the undetected mismatches of EG are far from the correctcorrespondence
2are two camera centres P is a scene point 119868 and 119868
1015840 are two images of a scene eand e1015840 are two epipolar points in the images and l and l1015840 are two epipolar lines determined by two points p1015840 and p respectively (b) A sketchmap of image 119868
1015840 in (a)
2 Related Research
The mismatch-rejecting problem has been investigated inmany studies [14 18 19] and the EG constraint is a widelyused model in resolving the problem
Suppose that 119868 and 1198681015840 are two images of a scene F is
the fundamental matrix and 119878 sub 119868 times 1198681015840 is a set of putative
point correspondences between them The EG constraintsays
p119879Fp1015840 = 0 (1)
where (p p1015840) isin 119868 times 1198681015840 is a pair of CPs and a superscript 119879
denotes the transpose of a matrix or vector The principleof the mismatch-rejecting methods based on EG is that thecorresponding point p1015840 of p isin 119868 is on a line l1015840 = F119879p in 119868
1015840 andvice versa (Figure 3(a)) where l and l1015840 are called two epipolarlines defined by points p1015840 and p respectively [3] Thereforeif F is known then we can reject some mismatches from 119878 bychecking whether they are consistent with the EG constraint(1) [1 3 4]
However as shown in Figure 3(b) if p10158401
isin 1198681015840 is another
point on the epipolar line l1015840 the mismatch (p p10158401) cannot
be detected by this constraint Therefore the EG constraintis a necessary but insufficient condition by which themismatches consistent with (1) cannot be detected Unfor-tunately the undue influence of these kinds of mismatchesmay be large enough to severely distort the final resultsin applications for example in image mosaicing and 3-dimensional reconstruction
CF is a concept recently introduced to reject mismatchesfromputative correspondences between images [2] Its poten-tial superiority has been shown experimentally on efficiencyand accuracy From the point set mapping perspective CF
depicts the relationships between corresponding points bymapping a point in one image to its corresponding point inanother [2]
For two images of a general scene the relationshipbetween some corresponding points (CPs) can be classifiedinto three types one-one one-many and many-one [2 4]The fundamental idea of the CF is to decompose the rela-tionships between the CPs into two subsets The two subsetsconsist of one-one and many-one or one-one and one-manykinds of corresponding relationships The two subsets ofcorresponding relationships can be described by two vector-valued functions 119891 119868 rarr 119868
1015840 and 1198911015840
1198681015840
rarr 119868Suppose 119878 = (p
119894 p1015840119894) 119894 = 1 119899 sub 119868 times 119868
1015840 are aset of putative correspondences CF for example 119891 can berepresented by
119891 (p) =
119899
sum
119894=1
w119894119896 (p p1015840
119894) (2)
where 119896(sdot sdot) is kernel function and w119894
= (1199081
119894 1199082
119894) are para-
meters to be estimated Based on this representation andGaussian kernel Li and Hu [2] studied the estimation of CFbased on Support Vector Machine
Actually the vector-valued function interpolationregres-sion with similar idea has been extensively investigated formodelling the transformations between two images in sce-narios of image registration and computer graphics Forexample Goshtasby [20] used a vector-valued function froma reference image to a target image for remote sensingimage registration based on surface spline Bookstein [21]studied the registration of medical images based on the thin-plate spline (TPS) Scholkopf et al [22] proposed a machinelearningmethod to establish dense point correspondences by
4 Mathematical Problems in Engineering
B2
B1
A1(u1 1)
Image I
Left camera Right cameraOL OR
A998400i(u
998400i
998400i )
Image I998400
(a)
Left camera Right camera
DC2
C1
AImage I Image I998400
OL OR
A9984001 A998400
2
(b)
Figure 4 Exception of uniqueness in correspondence problem (a) Many points A1015840119894 sub 119868
1015840 share a correspondence point A1
isin 119868 (b) Twopoints A1015840
1and A1015840
2in image 119868
1015840 correspond to one common point A isin 119868 and there are no CPs in image 119868 for the points in (A10158401A10158402) sub 1198681015840
estimating the deformation fields that transform two givenobjects into each other
In these scenarios the focus is to transform an image intoa coordinate systemof another image therefore two images tobe processed are usually called a reference image and a targetimage respectively and only one vector-valued functionis needed to describe the relationship from the referenceimage to the target image A good review can be found in[15 23] for the works on image registration based on thisidea
Letp isin 119868 andp1015840 isin 1198681015840 be two image points119892(p) = 119891(p)minusp
and y = p1015840 minus p The CF for example 119891(p) = p1015840 119868 rarr 1198681015840 can
be expressed by
y = 119892 (p) (3)
where y = p1015840 minus p The mapping y = 119892(p) 119868 rarr 1198681015840
minus 119868
is a special expression of CF and referred to as a vector field[24 25] Zhao et al [24] and Ma et al [25] investigated thevector field learning problem using a Bayesian frameworkand studied its application in outlier rejection Ma et al[26 27] proposed an estimation method 119871
2119864 for the vector
field and investigated its application in point set registrationbetween images of nonrigid object Ma et al [28] investi-gated the point correspondence problem by interpolatingthe vector field and formulated it a maximum a posteriori(MAP) estimation of a Bayesian model with some hiddenvariables
In literature related researches are usually focused onlearning and application of the correspondence function ortransformation function based on a flexible function Weinvestigate the principles and characteristics of it in thiswork
3 Principles of Correspondence Function
Suppose 119868 and 1198681015840 are two images to be registered The CPs
between 119868 and 1198681015840 form a surface in the joint-image space
119868 times 1198681015840 [17] which is illustrated schematically in Figure 1(a)
In this figure images 119868 and 1198681015840 are illustrated as a hori-
zontal axis and a vertical axis respectively And the sur-face is referred to as correspondence manifold (CM) in[17]
In Figure 1(a) (p119860
p1015840119860
) is a pair of CPs between images 119868
and 1198681015840 and 119860 is a point in the joint-image space 119868 times 119868
1015840 formedby CPs (p
119860 p1015840119860
) and the other points on the curves from 119860
to 119864 and from 119865 to 119866 have similar meanings Additionallycurves 119860119861 119862119863 and 119865119866 correspond to the image pointsprojected from the scene surface which can be distinguishedby both cameras of 119868 and 119868
1015840 curves 119861119862 and 119863119864 and points119864 119865 correspond to the scene portions which can only bedistinguished by one camera as illustrated in Figure 4(a)The gap between 119864 and 119865 corresponds to the scene portionbetween C
1and C
2in Figure 4(b) that can be observed by
one camera but not by the other one This sketch map showsthat the relationship between corresponding points can beclassified into three types one-one one-many and many-one This characteristic is independent of the camera modelof the images to be registered and the complexity of scenesurface
To highlight the mapping between CPs we move theorigin to an infinite point and connect the CPs by a dottedline then the sketch map in Figure 1(a) becomes Figure 1(b)It is well known that a function is a mapping that candepict two types of relationshipsmdashmany-one and one-onemdashin mathematics Therefore the relationship between CPs istoo complex to be described by a function
The fundamental idea of CF is to decompose the cor-responding relationships (we name the mapping relations
Mathematical Problems in Engineering 5
between CPs as ldquocorresponding relationshipsrdquo) into twodirectional relationships that can be depicted by two func-tions For convenience these directional relationships arereferred to as ldquocomponent relationsrdquo (CRs) and labeled asCR119868rarr1198681015840 andCR
1198681015840rarr119868
respectivelyTheCR consists of one-onemapping and many-one mapping For example Figures 2(a)and 2(b) are a pair of CRs of Figure 1
The component relation for example CR119868rarr1198681015840 can be
constructed as follows for every point p isin 119868 if there isa unique corresponding point p1015840 isin 119868
1015840 the correspond-ing relationship (p p1015840) should be one member of CR
119868rarr1198681015840
Otherwise if p has more than one corresponding pointp10158401 p1015840
119896119896gt0
isin 1198681015840 we have two choices either ignore all
of the corresponding relationships (p p119894) 1 le 119894 le 119896 or
select any one of the CPs p10158401198940
insert (p p10158401198940
) into CR1198681015840rarr119868
andignore the other corresponding relationships (p p
119894) 1 le 119894 le
119896 119894 = 1198940 Component relation CR
1198681015840rarr119868
can be constructedsimilarly by checking every potential point p1015840 isin 119868
1015840 Forexample in Figures 1 and 2 point p
119860isin 119868 has a unique
corresponding point p1015840119860in 1198681015840 therefore the corresponding
relationship (p119860
p1015840119860
) is in component relation CR119868rarr1198681015840 In
particular the corresponding point of p119863
isin 119868 is also uniquein 1198681015840 Therefore (p
119863 p1015840119863119864
) is one member of the componentrelation CR
119868rarr1198681015840 even though the CPs of p1015840
119863119864are not unique
in image 119868Therefore CR
119868rarr1198681015840 is a one-to-one or many-to-one map-
ping from 119868 to 1198681015840 and can be depicted by a vector-valued
function 119891 119868 rarr 1198681015840 Similarly CR
1198681015840rarr119868
is also a one-to-oneor many-to-one mapping from 119868
1015840 to 119868 and can be depicted bya vector-valued function 119891
1015840 1198681015840
rarr 119868
On the other hand for any given corresponding pointpair (p p1015840) isin 119868 times 119868
1015840 if it is of the one-one type thecorresponding relationship expressed by it can be depictednot only by a function from 119868 to 119868
1015840 but also by a functionfrom 119868
1015840 to 119868 Therefore the corresponding relationships ofthe one-one type can be found in both CR
119868rarr1198681015840 and CR
1198681015840rarr119868
such as the corresponding point pair (p
119860 p1015840119860
) in Figures1(b) 2(a) and 2(b) Otherwise if one given correspondingpoint pair (p p1015840) isin 119868 times 119868
1015840 is of the many-one type phas a unique corresponding image (If (p p1015840) are a pair ofCPs p is called the corresponding image of p1015840 and viceversa) in 119868
1015840 based on the definition of CPs of the many-onetypeTherefore the corresponding relationships of themany-one type can also be depicted by a function mapping andare members of CR
119868rarr1198681015840 like the corresponding point pairs
(p119863
p1015840119863119864
) (p119864119865
p1015840119863119864
) in Figures 1(b) and 2(a) Similarlythe corresponding relationships of one-many can be depictedby a function mapping from 119868
1015840 to 119868 and are members ofCR1198681015840rarr119868
such as the CPs (p119861119862
p1015840119861) (p
119861119862 p1015840119862) in Figures
1(b) and 2(b)In conclusion for a pair of images 119868 and 119868
1015840 to be registeredof a scene the relationships between CPs can be decomposedinto a pair of component relations CR
119868rarr1198681015840 and CR
1198681015840rarr119868
which consist of a number of one-to-one and many-to-onemappingsTheCR
119868rarr1198681015840 andCR
1198681015840rarr119868
can be described by a pair
of vector-valued functions as 119891 119868 rarr 1198681015840 and 119891
1015840 1198681015840
rarr 119868The (119891 119891
1015840) are named as correspondence functions (CFs) in
[2]In image registration 119868 and 119868
1015840 are referred to as areference image and a target image respectively the objectiveis to transform the target image into the coordinate systemof the reference image Therefore only one vector-valuedfunction for example 119891 119868 rarr 119868
1015840 is needed to describethe corresponding relationship and named transformationfunction in [23]
Therefore CF is a pair of functions (119891 1198911015840) demonstrating
that for any one pair of CPs p isin 119868 and p1015840 isin 1198681015840 (p p1015840) are
consistent with at least one of two functions 119891 or 1198911015840 which
can be estimated from a given set of putative correspondencesbased on robust regression method [2] Also for any givenpoint p isin 119868 its corresponding point can be uniquelyestimated by the CF and vice versa (Figure 2)
In practice putative correspondences are usually cor-rupted by noise and outliers and the number of putativecorrespondences is limited Therefore only an estimation ofCF with a certain accuracy can be estimated and the actualresults may be not strictly consistent with the ldquonecessary andsufficient conditionrdquo
Therefore the characteristic is that CF model can reducethe searching space of CPs into an elliptical area and the mis-matches of CF are usually near the correct CPs (Section 5)
4 Learning of CF and Its Application inMismatch Rejection
Based on the discussions in Section 3 for a pair of images1198681and 1198682of a scene the relationship of some corresponding
points between 1198681and 119868
2can be decomposed into a pair
of component relations (CRs) CR119868rarr1198681015840 and CR
1198681015840rarr119868
A CRconsists of one-one mapping and many-one mapping
Therefore CR119868rarr1198681015840 and CR
1198681015840rarr119868
can be described by twovector-valued functions 119891 119868 rarr 119868
1015840 and 1198911015840
1198681015840
rarr 119868
119891 (p) = p1015840
1198911015840(p1015840) = p
(4)
where p isin 119868 p1015840 isin 1198681015840 and (119891 119891
putative correspondences between 1198681and 1198682 If we treat y =
p1015840 as an output of a system and x = p as an input theCF 119891 can be estimated from 119878 = (x y) = (p p1015840) p isin
119868 p1015840 isin 1198681015840 using a vector-valued regression method for
example vector field consensus (VFC) method [24 28]and IECF (Iteratively Estimate Correspondence Function)algorithm [2] The fundamental ideas of VFC and IECF areas follows approximately represent the unknown CF 119891 as aweighted sumof kernel functions (2) and estimate theweights
6 Mathematical Problems in Engineering
iteratively by gradually reducing the undue effects fromoutliers The objective of this work is to investigate thefundamental principles and characteristics of CF Thereforewe do not compare the various implementation of CF anddo experiments by IECF The CF 119891
1015840 can be estimatedsimilarly
Theoretically the correctness of a pair of putative corre-spondences (p p1015840) can be determined by checking whetherit is consistent with one of the estimated correspondencefunctions 119891 or 119891
1015840 However due to the influence of imagenoise the observed corresponding points usually are notstrictly consistent with the estimated CFs 119891 and 119891
1015840 and atolerance parameter 120572
119868is needed to determine whether they
are consistent It is shown that the consistency of (p p1015840) with119891 can be determined by
119888119891
(p p1015840) = (119891 (p) minus p1015840) Σminus1
(119891 (p) minus p1015840)119879
(5)
where the superscript119879 represents a transpose of a vector andΣ is the covariance matrix of random variable 119891(p) minus p1015840 andcan be estimated using IECF algorithm [2]The consistency of(p p1015840)with119891
1015840 ismeasured similarly and denoted by 1198881198911015840(p p1015840)
Then some mismatches can be rejected by the constraints119888119891(p p1015840) gt 120572
In this section we experimentally investigate the characteris-tics of theCF by comparing it with thewidely used parametricmodel epipolar geometry (EG) ((1) Figure 3) in the contextof rejecting mismatches
51 Experimental Configuration The mismatch-rejectingmethods based onCF and EG are implemented by algorithmsICF (identifying point correspondences by correspondencefunction) [2] and RANSAC (Random Sample Consensus)[11 29 30] respectively in this study Since the coordinatesof the observed putative CPs are usually corrupted by noisethey are not strictly consistent with the CF and EG equationsTherefore two tolerance parameters are needed for thecomputation we name the two parameters 120572
119877in EG +
RANSAC and 120572119868in CF + ICF
In evaluating the performance of local descriptors andpoints matching models there are several schemes forexample recall precision [31] and ROC curve [14] Howeverthe objective of this workrsquos experiments is to show thecharacteristic of fail-detected mismatches of CF Thereforeto make a fair comparison we adjusted the values of 120572
119877and
120572119868such that a same number of putative correspondences are
identified as possible correct matches in every experiment
and the RANSAC are implemented without a limit oniteration number
Suppose 119868 and 1198681015840 are two images of a scene to be matched
and suppose 119878 is a set of putative correspondences betweenthem A mismatch-rejecting method will partition 119878 intotwo subsets 119878
119888and 119878
119898 as possible correct correspondences
and possible mismatches For readability we denote the twosubsets by 119878
EG119888
and 119878EG119898
in the method based on EG constraintand 119878
CF119888
and 119878CF119898
in the method based on CF Thus set 119878 canbe divided into four subsets 119878
EG119888
cap 119878CF119888 119878
EG119888
cap 119878CF119888 119878
EG119888
cap 119878CF119888
and 119878EG119888
cap 119878CF119888 where 119878
EG119888
= 119878 minus 119878EG119888
= 119878EG119898
and 119878CF119888
=
119878 minus 119878CF119888
= 119878CF119898
The meanings of the above four subsets are asfollows
(1) 119878EG119888
cap 119878CF119888
= 119878EG119898
cap 119878CF119898 the putative correspondences
that are regarded as possible correct matches by themethods based on EG and CF
(2) 119878EG119888
cap 119878CF119888
= 119878EG119898
cap 119878CF119898 the putative correspondences
that are regarded as possible mismatches by EGmethod and possible correct matches by CF
(3) 119878EG119888
cap 119878CF119888
= 119878EG119898
cap 119878CF119898 the putative correspondences
that are regarded as possible correct matches by theEG method and possible mismatches by CF
(4) 119878EG119888
cap 119878CF119888
= 119878EG119898
cap 119878CF119898 the putative correspondences
that are regarded as possible mismatches by themethods based on EG and CF
On subsets 119878EG119888
cap119878CF119888
and 119878EG119888
cap119878CF119888 the twomethods based on
CF and EG constraint are consistentTherefore it is sufficientto focus on 119878
EG119888
cap 119878CFc and 119878
EG119888
cap 119878CF119888
to compare the CFmodeland the EG constraint
For convenience in comparing the results of EG and CFwe manually removed some of the correct and near-correctputative correspondences from 119878
EG119888
cap 119878CF119888
and 119878EG119888
cap 119878CF119888
in some experiments and the two sets composed of theremaining putative correspondences are denoted by 119879(119878
EG119888
cap
119878CF119888
) and 119879(119878EG119888
cap 119878CF119888
) respectively
(i) 119879(119878EG119888
cap 119878CF119888
) the typical putative correspondencesthat are classified as possible correct matches by EGand as possible mismatches by CF
(ii) 119879(119878EG119888
cap 119878CF119888
) the typical putative correspondencesthat are classified as possible mismatches by EG andas possible correct correspondences by CF
52 Experiments
521 Main Results One of the used image pairs is pre-sented in Figure 5 There are 1278 and 1207 SIFT featurepoints extracted from two images respectively (the featurepoints are detected by the multiscale DoG (Difference ofGaussian) scheme and described by the orientation his-togram technique [32]) and 448 putative correspondences
Mathematical Problems in Engineering 7
(a) 119878EG119888cap 119878
CF119888
(b) 119878EG119888cap 119878
CF119888
(c) 119879(119878EG119888cap 119878
CF119888) (d) 119879(119878EG
119888cap 119878
CF119888)
Figure 5 An image pair of a toy bear (a) 26 EG potential mismatches (PMs) that are accepted by CF as matches (b) 26 CF PMs that areaccepted by EG as matches (c) some typical EG PMs that are accepted by CF (d) some typical CF PMs that are accepted by EG 120572
119868= 10597
and 120572119877
= 000019
(a) 119878EG119888cap 119878
CF119888
(b) 119878EG119888cap 119878
CF119888
(c) 119878EG119888cap 119878
CF119888
(d) 119878EG119888cap 119878
CF119888
Figure 6 An image pair of a corridor (a) 25 EG PMs that are accepted by CF as matches (b) 25 CF PMs that are accepted by EG as matches(c) 31 EG PMs that are accepted by CF as matches (d) 31 CF PMs that are accepted by EG as matches In (a)-(b) 120572
119868= 5991 and 120572
119877= 000048
and in (c)-(d) 120572119868
= 110 and 120572119877
= 005
computed from the feature points by the NNDR (Near-est Neighbor and Distance Ratio) method [32] Due tothe ambiguity of local information some of the putativecorrespondences are incorrect and need to be rejected Inthis experiment the two methods based on EG and CFare inconsistent on 52 putative correspondences (Figures5(a) and 5(b)) For example for every putative correspon-dence (p p1015840) in Figure 5(a) the CF method identified it as
a possible correct match but the EG method identified it asa possible mismatch Therefore based on the definition of119878EG119888
cap 119878CF119888
and 119878EG119888
cap 119878CF119888 we can regard 119878
EG119888
cap 119878CF119888
as a setof potential mismatches that the EG constraint fails to detectand 119878
EG119888
cap 119878CF119888
as a set of potential mismatches that the CFmodel fails to reject More results are presented in Figures 67 and 8
8 Mathematical Problems in Engineering
(a) 119878EG119888cap 119878
CF119888
(b) 119878EG119888cap 119878
CF119888
(c) 119879(119878EG119888cap 119878
CF119888) (d) 119879(119878EG
119888cap 119878
CF119888)
Figure 7 An image pair of Baxia (a) Some EG PMs that are accepted by CF asmatches (b) some CF PMs that are accepted by EG asmatches(c) some typical EG PMs that are accepted by CF (d) some typical CF PMs that are accepted by EG 120572
119868= 10597 and 120572
119877= 005
(a) 119878EG119888cap 119878
CF119888
(b) 119878EG119888cap 119878
CF119888
(c) 119879(119878EGc cap 119878CF119888) (d) 119879(119878EG
119888cap 119878
CF119888)
Figure 8 An image pair of temple Fayu (a) 204 EG PMs that are accepted by CF as matches (b) 204 CF PMs that are accepted by EG asmatches (c) some typical EG PMs that are accepted by CF (d) some typical CF PMs that are accepted by EG 120572
119868= 31 and 120572
119877= 00001515
(for visibility only 100 randomly chosen PMs are presented in (a) and (b))
522 Comparing the CF and EG Constraint by Analyzing theUndetected Mismatches For ease of explanation we intro-duce the following two definitions type-I mismatches andtype-II mismatches Suppose (p p1015840) is a mismatch If thereexists a correct corresponding point pair (pp1015840) satisfying thatthe points p and p1015840 are in the neighborhood of p and p1015840respectively then (p p1015840) is defined as a mismatch of type-IOtherwise we call (p p1015840) a type-II mismatch For example
the mismatches in Figure 7(d) and (p p10158403) in Figure 3(b)
belong to type-II and (p p10158402) in Figure 3(b) belong to type-
I By the EG constraint we can only determine whether aputative correspondence is in a band around the epipolar line(Figures 3(b) 9(b) and 9(d)) but not whether it is close to thecorrect one Therefore the undetected mismatches usuallybelong to type-II in the methods based on the EG constraint(Figures 3 5(b) 5(d) 6(b) 6(d) 7(b) 7(d) 8(b) and 8(d))
Mathematical Problems in Engineering 9
l1
l2
p1
l3p3 p2
l4p4
l9984001 p9984001
l9984002 p9984002
l9984003 p9984003
l9984004p9984004
(a)
l1
l3
p1
p3
l2 p2
l9984001p9984001
l9984002p9984002
l9984003p9984003
(b)
l2p2
l1 p1l3p3
l9984001p9984001
l9984002p9984002
l9984003 p9984003
(c)
l1
l6
p1
p6
l3
l4 p4
p3
l2
l5
p2
p5
l9984001p9984001
l9984002 p9984002
l9984003
l9984006
p9984003
p9984006
l9984004 p9984004
l9984005p9984005
(d)
Figure 9 Some PMs and their corresponding epipolar lines The EG model cannot detect the mismatches that are near the epipolar line butfar from the correct CPs The undetected mismatches by CF are usually near both the correct CPs and the epipolar line (a) and (c) presentthe EG PMs that are accepted by CF as matches in the experiments of Figures 7(c) and 8(c) respectively (b) and (d) present the CF PMs thatare accepted by EG as matches in the experiments of Figures 7(d) and 8(d) respectively In these experiments (p
119894 p1015840119894) is a pair of putative
CPs and l119894is the epipolar line corresponding to p1015840
119894and l1015840119894is the epipolar line corresponding to p
119894
The CF constraint determines the mapping relationshipsbetween two CPs and directly estimates the position of thecorresponding point Therefore by the mismatch-rejectingmethods based on CF we can restrict the error bound ofthe putative correspondences by a tolerance parameter 120572
119868
the undetected mismatches by CF usually belong to type-I To show the difference between the CF model and EGconstraints a typical result is presented in Figure 6 in whichwe relax the tolerance parameters 120572
119868and 120572119877to ensure the two
sets 119878EG119888
cap 119878CF119888
and 119878EG119888
cap 119878CF119888
change at equal speedHowever themismatches of type-I and type-II are usually
dramatically different from each other in usefulness andundue influence From the mismatches of type-I we cansearch for the correct CPs by including more informationOn the contrary other than ruining the traditional estimationmethods and distorting the final application results (egimage mosaicing and three-dimensional reconstruction) themismatches of the type-II provide less useful information
53 Further Evaluation on Some Popular Benchmark DataThe characteristics of the CF are also investigated on twoseries of popular benchmark data sets widely used to compareand evaluate related algorithms [33 34]
To evaluate the two schemes quantitatively this workconducted experiments on a series of image sets with ground-truth correspondences [31 35] The images were capturedunder special consideration to enable the encoding of theground-truth correspondences with homographies between
the reference image and other images The ground-truthhomographies are computed based on the following stepsfirst a set of putative correspondences are manually selectedand an initial estimation of the homography is computedbased on the selected putative correspondences second theimages are approximately aligned with the rough estimationof homography third reliable interest points are detectedand matched automatically and an accurate homography iscomputed from them
The experimental results are presented in Figures 11 12 13and 14 In every experiment the upper image and the lowerimage are called the reference image and the target imagerespectively In addition to the computed putative correspon-dences we also present the ground-truth CPs and ground-truth correspondence relationships in these experimentsTheground-truth CPs in the target images are computed for all ofthe marked interest points in reference images based on theground-truth homographies and denoted by red circles Thedetected interest points are denoted by black plus signs Theground-truth correspondence relationships are displayed asblue dashed lines and the computed putative correspondencerelationships are shown by a solid line with red or green forvisibility
Suppose that 119878 = (p1 p10158401) (p
119899 p1015840119899) are a set of
putative correspondences and 119878 = (p1 p10158401) (p
119899 p1015840119899) are
the ground-truth correspondences of 119878 We quantitativelyevaluate the quality of 119878 by the root-mean-square deviation(RMSD) RMSD(119878) = radicsum
Figure 10The possible correct correspondences identified by CF in the experiment of Figure 8 are presented based on depth (a) 34 putativecorrespondences (PCs) over the first crossbeam (b) 324 PCs over the second crossbeam (c) 214 PCs over the third crossbeam (d) 20 PCsaround the fourth crossbeam (e) 2 PCs on the wall (for clarity only 50 randomly chosen PCs are presented in (b) and (c))
The second series of evaluation data are the Middle-bury Stereo Datasets [36 37] (httpvisionmiddleburyedustereodata) These data sets were created using an auto-mated version of structured-lighting technique [38] Eachdataset consists of seven rectified views This work used thefirst and the fifth views for evaluation The experimentalresults are presented in Figures 15 16 17 18 and 19
54 CF and EG in Practice
541 Necessity and Sufficiency As analyzed the EG con-straint is a necessary but insufficient condition in the corre-spondence problem and theCFmodel describes themappingbetween CPs directly However in applications some correctCPsmay be inconsistent with the estimated EGmodel andorthe estimated CF model there may be some mismatchesthat are ldquoconsistentrdquo with the estimated CF model there alsomay be some of the putative correspondences which areldquoconsistentrdquo with the estimated CF but ldquoinconsistentrdquo withthe estimated EG for example the experimental results inFigures 5(a) 5(b) 7(a) 7(b) 8(a) and 8(b) The reasons forthe above results are as follows
(i) The coordinates of the obtained putative CPs areusually corrupted by noise
(ii) There are usually some errors in the estimation of theCF and EG
(iii) As pointed out in the first paragraph of Section 5a preset tolerance parameter is needed to define theldquoconsistentrdquo and ldquoinconsistentrdquo in the applicationsof CF and EG However the noise in the obtainedputative CPs is random
542 Characteristics and Limitations of the CF Model andEG Model In the two models there are no restrictionson the depth range of the imaged scene However theEG is a global describing model and the CF is a localdescribing model In the EG model any pair of putativeCPs can influence the overall model estimation whereas inthe CF model a pair of putative CPs only influence themodel estimation within a local area in their vicinity Theshortcoming of the global model EG is that the overallmodel estimation can be degraded or ruined by PCs withconsiderable noise or mismatches in any area of the imagesThe shortcoming of the local model CF is that a pair of goodCPs cannot improve the estimation of CF outside of a localarea around itself Thus the uneven distribution of putativecorrespondences with different quality results in the fact thatthe estimated CF may be of high quality in some areas butof low quality in others for example in the experiment ofFigure 10 the qualities of the CF estimations over the second-nearest-to-the-cameras crossbeam and the third-nearest-to-the-cameras crossbeam are better than estimation over thefirst-nearest-to-the-cameras crossbeam The ldquolocal areardquo isdetermined by the CF estimation method and its chosenparameters for example in the CF estimation method SP(SVM) [2] the local area is determined by the chosen kerneland the scale parameter The above limitations of EG and CFcan be alleviated by investigating robust estimation methodsfor example the RANSAC M-estimators [8] LMedS (LeastMedian of Squares) [9] and MLESAC [10 12] for EGIn future work we will investigate further the estimationmethod for CF to improve its robustness
6 Conclusion
The CF is a recently introduced nonparametric model forrejecting mismatchesoutliers in image point matching In
Mathematical Problems in Engineering 11
(a) RMSD = 1386 (b) RMSD = 22707 (c) RMSD = 384 (d) RMSD = 23626Figure 11 Quantitative evaluation on two popular benchmark sets bikes and leuven The ground-truth CPs are presented by red circles thedetected interest points by black plus signs The ground-truth correspondences are displayed as blue dashed lines and the computed putativecorrespondences are shown by a solid green solid line (a) and (c) present some EG PMs that are accepted by CF as matches (b) and (d)present some CF PMs that are accepted by EG as matches 120572
119868= 10597 120572
119877= 21435 times 10
minus5 in (b) and 120572119877
= 58 times 10minus5 in (d)
(a) RMSD = 1631 (b) RMSD = 40098 (c) RMSD = 358 (d) RMSD = 32288Figure 12 Quantitative evaluation on two popular benchmark sets wall and bark ((a) and (c)) The EG PMs that are accepted by CF asmatches ((b) and (d)) the CF PMs that are accepted by EG as matches 120572
119868= 10597 120572
119877= 2029 times 10
minus4 in (b) and 120572119877
= 105 times 10minus4 in (d)
(a) RMSD = 1017 (b) RMSD = 42639 (c) RMSD = 329 (d) RMSD = 17646Figure 13 Quantitative evaluation on two popular benchmark sets graf and boat The ground-truth CPs are presented by red circles and thedetected interest points by black plus signs The ground-truth correspondences are displayed as blue dashed lines and the computed putativecorrespondences are shown by a solid green solid line (a) and (c) present some EG PMs that are accepted by CF as matches (b) and (d)present some CF PMs that are accepted by EG as matches 120572
Figure 14 Quantitative evaluation on two popular benchmark sets trees and ubc The ground-truth correspondences are displayed as bluedashed lines and the computed putative correspondences are shown by a yellow solid line (a) and (c) present some EG PMs that are acceptedbyCF asmatches (b) and (d) present someCFPMs that are accepted by EG asmatches120572
119868= 10597120572
119877= 59times10
minus5 in (b) and120572119877
= 185times10minus4
in (d)
(a) 119878EG119888cap 119878
CF119888
(b) 119878EG119888cap 119878
CF119888
(c) 119878EG119888cap 119878
CF119888
(d) 119878EG119888cap 119878
CF119888
Figure 15 Evaluation on Middlebury Stereo Datasets art and books ((a) and (c)) Some potential EG mismatches that are accepted by CF asmatches ((b) and (d)) some potential CF mismatches that are accepted by EG as matches 120572
119877= 0013 in (a) and (b) 120572
119877= 000018 in (c) and
(d) and 120572119868
= 10597
this study we investigated the principles of the CF andstudied its characteristics by comparing it with the widelyused parametric model epipolar geometry (EG) constraint
It is shown that the CF describes the mapping relation-ships between two CPs and should be able to estimate theposition of the corresponding point Therefore in additionto mismatch rejection a potential application of the CF is to
guide the point matching process by incorporating it into thecorrespondence propagation [39]
In practice putative correspondences are usually cor-rupted by noise and outliers and the number of putativecorrespondences is limited Therefore only an estimationof the CF with a certain accuracy can be estimated andthe actual results will violate the ldquonecessary and sufficient
Mathematical Problems in Engineering 13
(a) 119878EG119888cap SCF119888
(b) 119878EG119888cap 119878
CF119888
(c) 119878EG119888cap 119878
CF119888
(d) 119878EG119888cap 119878
CF119888
Figure 16 Evaluation on Middlebury Stereo Datasets computer and dolls ((a) and (c)) Some potential EG mismatches that are accepted byCF as matches ((b) and (d)) some potential CF mismatches that are accepted by EG as matches 120572
119877= 000018 in (a) and (b) 120572
119877= 000048 in
(c) and (d) and 120572119868
= 10597
(a) 119878EG119888cap 119878
CF119888
(b) 119878EG119888cap 119878
CF119888
(c) 119878EG119888cap 119878
CF119888
(d) 119878EG119888cap 119878
CF119888
Figure 17 Evaluation onMiddlebury Stereo Datasets drumsticks and dwarves ((a) and (c)) Some potential EGmismatches that are acceptedby CF as matches ((b) and (d)) some potential CF mismatches that are accepted by EG as matches 120572
119877= 000078 in (a) and (b) 120572
119877= 00035
in (c) and (d) and 120572119868
= 10597
conditionrdquoThe characteristic is that theCFmodel can reducethe searching space of CPs into an elliptical area and themismatches of the CF are usually near the correct CPs
Conflict of Interests
The authors declare that there is no conflict of interestsregarding the publication of this paper
Acknowledgments
The authors are grateful for valuable advice from DrDibyendu Mukherjee and the anonymous reviewers andwould like to thank the National Natural Science Foundationof China (Grant nos 61075033 61005033 and 61273248)the Natural Science Foundation of Guangdong Province(2014A030313425 and S2011010003348) and theOpenProject
14 Mathematical Problems in Engineering
(a) 119878EG119888cap 119878
CF119888
(b) 119878EG119888cap 119878
CF119888
(c) 119878EG119888cap 119878
CF119888
(d) 119878EG119888cap 119878
CF119888
Figure 18 Evaluation onMiddlebury Stereo Datasets laundry andmoebius ((a) and (c)) Some potential EGmismatches that are accepted byCF asmatches ((b) and (d)) some potential CFmismatches that are accepted by EG asmatches 120572
119877= 000000275 in (a) and (b) 120572
119877= 000035
in (c) and (d) and 120572119868
= 10597
(a) 119878EG119888cap 119878
CF119888
(b) 119878EG119888cap 119878
CF119888
Figure 19 Evaluation on Middlebury Stereo Dataset reindeer (a)Two potential EG mismatches that are accepted by CF as matches(b) two potential CFmismatches that are accepted by EG asmatches120572119877
= 0005 and 120572119868
= 30
Program of the National Laboratory of Pattern Recognition(NLPR) (201001060) for their support
References
[1] J P Monaco A C Bovik and L K Cormack ldquoActive foveateduncalibrated stereovisionrdquo International Journal of ComputerVision vol 85 no 2 pp 192ndash207 2009
[2] X Li and Z Hu ldquoRejecting mismatches by correspondencefunctionrdquo International Journal of Computer Vision vol 89 no1 pp 1ndash17 2010
[3] R Hartley and A Zisserman Multiple View Geometry inComputer Vision Cambridge University Press 2nd edition2003
[4] M Sonka V Hlavac and R Boyle Image Processing Analysisand Machine Vision Thomson-Engineering 3rd edition 2007
[5] G Carneiro and A D Jepson ldquoFlexible spatial configuration oflocal image featuresrdquo IEEE Transactions on Pattern Analysis andMachine Intelligence vol 29 no 12 pp 2089ndash2104 2007
[6] W Aguilar Y Frauel F Escolano M E Martinez-Perez AEspinosa-Romero andM A Lozano ldquoA robust graph transfor-mation matching for non-rigid registrationrdquo Image and VisionComputing vol 27 no 7 pp 897ndash910 2009
[7] L Wang F Duan and K Lv ldquoFisheye-lens-based visual suncompass for perception of spatial orientationrdquo MathematicalProblems in Engineering vol 2012 Article ID 460430 p 15 2012
[8] P Huber Robust Statistics Wiley New York NY USA 1981[9] P J Rousseeuw ldquoLeast median of squares regressionrdquo Journal
of the American Statistical Association vol 79 no 388 pp 871ndash880 1984
[10] P H S Torr and A Zisserman ldquoMLESAC a new robustestimator with application to estimating image geometryrdquoComputer Vision and Image Understanding vol 78 no 1 pp138ndash156 2000
[11] M A Fischler and R C Bolles ldquoRandom sample consensus aparadigm for model fitting with applications to image analysisand automated cartographyrdquo Communications of the ACM vol24 no 6 pp 381ndash395 1981
[12] B J Tordoff and D W Murray ldquoGuided-MLESAC fasterimage transform estimation by using matching priorsrdquo IEEETransactions on Pattern Analysis and Machine Intelligence vol27 no 10 pp 1523ndash1535 2005
Mathematical Problems in Engineering 15
[13] D Sidibe P Montesinos and S Janaqi ldquoMatching localinvariant features with contextual information an experimentalevaluationrdquo Electronic Letters on Computer Vision and ImageAnalysis vol 7 no 1 pp 26ndash39 2008
[14] Q-H Tran T-J Chin G Carneiro M S Brown and D SuterldquoIn defence of RANSAC for outlier rejection in deformableregistrationrdquo Lecture Notes in Computer Science vol 7575 no4 pp 274ndash287 2012
[15] A Goshtasby ldquoTransformation functions for image registra-tionrdquo in Proceedings of the Computer Vision and Pattern Recog-nition (CVPR rsquo11) June 2011
[16] X Li Y Wang J Zhou and X Li ldquoComparative research oncorrespondence functionrdquo in Proceedings of the InternationalConference on Intelligent System Design and Engineering Appli-cation (ISDEA rsquo10) pp 444ndash447 October 2010
[17] X-R Li X-M Li H-L Li and M-Y Cao ldquoRejecting outliersbased on correspondence manifoldrdquo Acta Automatica Sinicavol 35 no 1 pp 17ndash22 2009
[18] D Pizarro and A Bartoli ldquoFeature-based deformable surfacedetection with self-occlusion reasoningrdquo International Journalof Computer Vision vol 97 no 1 pp 54ndash70 2012
[19] J Yu A Eriksson T-J Chin and D Suter ldquoAn adversarialoptimization approach to efficient outlier removalrdquo Journal ofMathematical Imaging and Vision vol 48 no 3 pp 451ndash4662014
[20] A Goshtasby ldquoRegistration of images with geometric distor-tionsrdquo IEEETransactions onGeoscience andRemote Sensing vol26 no 1 pp 60ndash64 1988
[21] F L Bookstein ldquoPrincipal warps thin-plate splines and thedecomposition of deformationsrdquo IEEE Transactions on PatternAnalysis and Machine Intelligence vol 11 no 6 pp 567ndash5851992
[22] B Scholkopf F Steinke and V Blanz ldquoObject correspondenceas a machine learning problemrdquo in Proceedings of the 22ndInternational Conference on Machine Learning (ICML rsquo05) pp776ndash783 2005
[23] L Zagorchev and A Goshtasby ldquoA comparative study oftransformation functions for nonrigid image registrationrdquo IEEETransactions on Image Processing vol 15 no 3 pp 529ndash5382006
[24] J Zhao JMa J Tian J Ma andD Zhang ldquoA robustmethod forvector field learning with application to mismatch removingrdquoin Proceedings of the IEEE Conference on Computer Vision andPattern Recognition (CVPR rsquo11) Providence RI USA June 2011
[25] JMa J Zhao J Tian X Bai and Z Tu ldquoRegularized vector fieldlearning with sparse approximation for mismatch removalrdquoPattern Recognition vol 46 no 12 pp 3519ndash3532 2013
[26] J Ma J Zhao J Tian Z Tu and A L Yuille ldquoRobust estimationof nonrigid transformation for point set registrationrdquo in Pro-ceedings of the 26th IEEE Conference on Computer Vision andPattern Recognition (CVPR rsquo13) pp 2147ndash2154 June 2013
[27] J MaW Qiu J Zhao Y Ma A L Yuille and Z Tu ldquoRobust l2e
estimation of transformation for non-rigid registrationrdquo IEEETransactions on Signal Processing vol 63 no 5 pp 1115ndash11292015
[28] J Ma J Zhao J Tian A L Yuille and Z Tu ldquoRobust pointmatching via vector field consensusrdquo IEEE Transactions onImage Processing vol 23 no 4 pp 1706ndash1721 2014
[29] L Goshen and I Shimshoni ldquoBalanced exploration andexploitation model search for efficient epipolar geometry esti-mationrdquo IEEE Transactions on Pattern Analysis and MachineIntelligence vol 30 no 7 pp 1230ndash1242 2008
[30] O Chum and J Matas ldquoMatching with prosacmdashprogressivesample consensusrdquo in Proceedings of the IEEE Computer SocietyConference on Computer Vision and Pattern Recognition (CVPRrsquo05) vol 1 pp 220ndash226 IEEE June 2005
[31] K Mikolajczyk and C Schmid ldquoA performance evaluation oflocal descriptorsrdquo IEEE Transactions on Pattern Analysis andMachine Intelligence vol 27 no 10 pp 1615ndash1630 2005
[32] D G Lowe ldquoDistinctive image features from scale-invariantkeypointsrdquo International Journal of Computer Vision vol 60 no2 pp 91ndash110 2004
[33] S Gauglitz T Hollerer and M Turk ldquoEvaluation of interestpoint detectors and feature descriptors for visual trackingrdquoInternational Journal of Computer Vision vol 94 no 3 pp 335ndash360 2011
[34] Y Yu K Huang W Chen and T Tan ldquoA novel algorithmfor view and illumination invariant image matchingrdquo IEEETransactions on Image Processing vol 21 no 1 pp 229ndash2402012
[35] K Mikolajczyk T Tuytelaars C Schmid et al ldquoA comparisonof affine region detectorsrdquo International Journal of ComputerVision vol 65 no 1-2 pp 43ndash72 2005
[36] H Hirschmuller and D Scharstein ldquoEvaluation of cost func-tions for stereo matchingrdquo in Proceedings of the IEEE ComputerSociety Conference on Computer Vision and Pattern Recognition(CVPR rsquo07) June 2007
[37] D Scharstein and C Pal ldquoLearning conditional random fieldsfor stereordquo in Proceedings of the IEEE Computer Society Confer-ence on Computer Vision and Pattern Recognition (CVPR rsquo07)IEEE Minneapolis Minn USA June 2007
[38] D Scharstein andR Szeliski ldquoHigh-accuracy stereo depthmapsusing structured lightrdquo in Proceedings of the IEEE ComputerSociety Conference on Computer Vision and Pattern Recognition(CVPR rsquo03) vol 1 pp I-195ndashI-202 IEEE June 2003
[39] H Wang S Yan J Liu X Tang and T S Huang ldquoCorre-spondence propagationwithweak priorsrdquo IEEETransactions onImage Processing vol 18 no 1 pp 140ndash150 2009
Figure 1 A sketch map of the relationships between CPs (a) Two axes are a sketch map of two images 119868 and 1198681015840 of a scene to be matched that
curve is a sketch map of the correspondence manifold formed by CPs between 119868 and 1198681015840 [17] (b) a sketch map of the relationship between
CPs presented to emphasize the mapping from one image point to its CP(s)
p998400Fp998400
DEp998400Cp998400
Bp998400A
pA pBC pD pEF
I998400
I
middot middot middot
middot middot middot
(a)
p998400Fp998400
DEp998400Cp998400
Bp998400A
pA pBC pD pEF
I998400
I
middot middot middot
middot middot middot
(b)
Figure 2 Convert the relationship between CPs into two function mappings The red dotted lines represent the deleted correspondingconnections and the original corresponding relationship in Figure 1(b) can be recovered from (a) and (b) by collecting all of the remainingconnections and deleting the directions (a) CR
119868rarr1198681015840 these relationships between CPs can be depicted by a function from 119868 to 119868
1015840 (b) CR1198681015840rarr119868
these relationships between CPs can be depicted by a function from 119868
1015840 to 119868
similarity transformation [15] and projective transforma-tions [15] The flexibility is to account for nonrigidity ofobjects or to reduce the undue influence from outliers
In this work we propose a visual scheme to investigate thefundamental principles of CF (Figures 1 and 2) and to studyits characteristics by experimentally comparing it with thewidely used parametricmodel epipolar geometry (EG) (somerudimentary comparisons were made between CF and EG ina conference paper ([16])) It is shown that the CF describes
themapping from a point in one image to its CP(s) in anotherimage which enables us to directly estimate the positions ofthe CPs However the EG acts by reducing the search spacefor CPs from a two-dimensional space to a line which is aproblem in one-dimensional space In applications the resultof the difference between EG and CF is that the undetectedmismatches by CF are usually near the correct CPs but manyof the undetected mismatches of EG are far from the correctcorrespondence
2are two camera centres P is a scene point 119868 and 119868
1015840 are two images of a scene eand e1015840 are two epipolar points in the images and l and l1015840 are two epipolar lines determined by two points p1015840 and p respectively (b) A sketchmap of image 119868
1015840 in (a)
2 Related Research
The mismatch-rejecting problem has been investigated inmany studies [14 18 19] and the EG constraint is a widelyused model in resolving the problem
Suppose that 119868 and 1198681015840 are two images of a scene F is
the fundamental matrix and 119878 sub 119868 times 1198681015840 is a set of putative
point correspondences between them The EG constraintsays
p119879Fp1015840 = 0 (1)
where (p p1015840) isin 119868 times 1198681015840 is a pair of CPs and a superscript 119879
denotes the transpose of a matrix or vector The principleof the mismatch-rejecting methods based on EG is that thecorresponding point p1015840 of p isin 119868 is on a line l1015840 = F119879p in 119868
1015840 andvice versa (Figure 3(a)) where l and l1015840 are called two epipolarlines defined by points p1015840 and p respectively [3] Thereforeif F is known then we can reject some mismatches from 119878 bychecking whether they are consistent with the EG constraint(1) [1 3 4]
However as shown in Figure 3(b) if p10158401
isin 1198681015840 is another
point on the epipolar line l1015840 the mismatch (p p10158401) cannot
be detected by this constraint Therefore the EG constraintis a necessary but insufficient condition by which themismatches consistent with (1) cannot be detected Unfor-tunately the undue influence of these kinds of mismatchesmay be large enough to severely distort the final resultsin applications for example in image mosaicing and 3-dimensional reconstruction
CF is a concept recently introduced to reject mismatchesfromputative correspondences between images [2] Its poten-tial superiority has been shown experimentally on efficiencyand accuracy From the point set mapping perspective CF
depicts the relationships between corresponding points bymapping a point in one image to its corresponding point inanother [2]
For two images of a general scene the relationshipbetween some corresponding points (CPs) can be classifiedinto three types one-one one-many and many-one [2 4]The fundamental idea of the CF is to decompose the rela-tionships between the CPs into two subsets The two subsetsconsist of one-one and many-one or one-one and one-manykinds of corresponding relationships The two subsets ofcorresponding relationships can be described by two vector-valued functions 119891 119868 rarr 119868
1015840 and 1198911015840
1198681015840
rarr 119868Suppose 119878 = (p
119894 p1015840119894) 119894 = 1 119899 sub 119868 times 119868
1015840 are aset of putative correspondences CF for example 119891 can berepresented by
119891 (p) =
119899
sum
119894=1
w119894119896 (p p1015840
119894) (2)
where 119896(sdot sdot) is kernel function and w119894
= (1199081
119894 1199082
119894) are para-
meters to be estimated Based on this representation andGaussian kernel Li and Hu [2] studied the estimation of CFbased on Support Vector Machine
Actually the vector-valued function interpolationregres-sion with similar idea has been extensively investigated formodelling the transformations between two images in sce-narios of image registration and computer graphics Forexample Goshtasby [20] used a vector-valued function froma reference image to a target image for remote sensingimage registration based on surface spline Bookstein [21]studied the registration of medical images based on the thin-plate spline (TPS) Scholkopf et al [22] proposed a machinelearningmethod to establish dense point correspondences by
4 Mathematical Problems in Engineering
B2
B1
A1(u1 1)
Image I
Left camera Right cameraOL OR
A998400i(u
998400i
998400i )
Image I998400
(a)
Left camera Right camera
DC2
C1
AImage I Image I998400
OL OR
A9984001 A998400
2
(b)
Figure 4 Exception of uniqueness in correspondence problem (a) Many points A1015840119894 sub 119868
1015840 share a correspondence point A1
isin 119868 (b) Twopoints A1015840
1and A1015840
2in image 119868
1015840 correspond to one common point A isin 119868 and there are no CPs in image 119868 for the points in (A10158401A10158402) sub 1198681015840
estimating the deformation fields that transform two givenobjects into each other
In these scenarios the focus is to transform an image intoa coordinate systemof another image therefore two images tobe processed are usually called a reference image and a targetimage respectively and only one vector-valued functionis needed to describe the relationship from the referenceimage to the target image A good review can be found in[15 23] for the works on image registration based on thisidea
Letp isin 119868 andp1015840 isin 1198681015840 be two image points119892(p) = 119891(p)minusp
and y = p1015840 minus p The CF for example 119891(p) = p1015840 119868 rarr 1198681015840 can
be expressed by
y = 119892 (p) (3)
where y = p1015840 minus p The mapping y = 119892(p) 119868 rarr 1198681015840
minus 119868
is a special expression of CF and referred to as a vector field[24 25] Zhao et al [24] and Ma et al [25] investigated thevector field learning problem using a Bayesian frameworkand studied its application in outlier rejection Ma et al[26 27] proposed an estimation method 119871
2119864 for the vector
field and investigated its application in point set registrationbetween images of nonrigid object Ma et al [28] investi-gated the point correspondence problem by interpolatingthe vector field and formulated it a maximum a posteriori(MAP) estimation of a Bayesian model with some hiddenvariables
In literature related researches are usually focused onlearning and application of the correspondence function ortransformation function based on a flexible function Weinvestigate the principles and characteristics of it in thiswork
3 Principles of Correspondence Function
Suppose 119868 and 1198681015840 are two images to be registered The CPs
between 119868 and 1198681015840 form a surface in the joint-image space
119868 times 1198681015840 [17] which is illustrated schematically in Figure 1(a)
In this figure images 119868 and 1198681015840 are illustrated as a hori-
zontal axis and a vertical axis respectively And the sur-face is referred to as correspondence manifold (CM) in[17]
In Figure 1(a) (p119860
p1015840119860
) is a pair of CPs between images 119868
and 1198681015840 and 119860 is a point in the joint-image space 119868 times 119868
1015840 formedby CPs (p
119860 p1015840119860
) and the other points on the curves from 119860
to 119864 and from 119865 to 119866 have similar meanings Additionallycurves 119860119861 119862119863 and 119865119866 correspond to the image pointsprojected from the scene surface which can be distinguishedby both cameras of 119868 and 119868
1015840 curves 119861119862 and 119863119864 and points119864 119865 correspond to the scene portions which can only bedistinguished by one camera as illustrated in Figure 4(a)The gap between 119864 and 119865 corresponds to the scene portionbetween C
1and C
2in Figure 4(b) that can be observed by
one camera but not by the other one This sketch map showsthat the relationship between corresponding points can beclassified into three types one-one one-many and many-one This characteristic is independent of the camera modelof the images to be registered and the complexity of scenesurface
To highlight the mapping between CPs we move theorigin to an infinite point and connect the CPs by a dottedline then the sketch map in Figure 1(a) becomes Figure 1(b)It is well known that a function is a mapping that candepict two types of relationshipsmdashmany-one and one-onemdashin mathematics Therefore the relationship between CPs istoo complex to be described by a function
The fundamental idea of CF is to decompose the cor-responding relationships (we name the mapping relations
Mathematical Problems in Engineering 5
between CPs as ldquocorresponding relationshipsrdquo) into twodirectional relationships that can be depicted by two func-tions For convenience these directional relationships arereferred to as ldquocomponent relationsrdquo (CRs) and labeled asCR119868rarr1198681015840 andCR
1198681015840rarr119868
respectivelyTheCR consists of one-onemapping and many-one mapping For example Figures 2(a)and 2(b) are a pair of CRs of Figure 1
The component relation for example CR119868rarr1198681015840 can be
constructed as follows for every point p isin 119868 if there isa unique corresponding point p1015840 isin 119868
1015840 the correspond-ing relationship (p p1015840) should be one member of CR
119868rarr1198681015840
Otherwise if p has more than one corresponding pointp10158401 p1015840
119896119896gt0
isin 1198681015840 we have two choices either ignore all
of the corresponding relationships (p p119894) 1 le 119894 le 119896 or
select any one of the CPs p10158401198940
insert (p p10158401198940
) into CR1198681015840rarr119868
andignore the other corresponding relationships (p p
119894) 1 le 119894 le
119896 119894 = 1198940 Component relation CR
1198681015840rarr119868
can be constructedsimilarly by checking every potential point p1015840 isin 119868
1015840 Forexample in Figures 1 and 2 point p
119860isin 119868 has a unique
corresponding point p1015840119860in 1198681015840 therefore the corresponding
relationship (p119860
p1015840119860
) is in component relation CR119868rarr1198681015840 In
particular the corresponding point of p119863
isin 119868 is also uniquein 1198681015840 Therefore (p
119863 p1015840119863119864
) is one member of the componentrelation CR
119868rarr1198681015840 even though the CPs of p1015840
119863119864are not unique
in image 119868Therefore CR
119868rarr1198681015840 is a one-to-one or many-to-one map-
ping from 119868 to 1198681015840 and can be depicted by a vector-valued
function 119891 119868 rarr 1198681015840 Similarly CR
1198681015840rarr119868
is also a one-to-oneor many-to-one mapping from 119868
1015840 to 119868 and can be depicted bya vector-valued function 119891
1015840 1198681015840
rarr 119868
On the other hand for any given corresponding pointpair (p p1015840) isin 119868 times 119868
1015840 if it is of the one-one type thecorresponding relationship expressed by it can be depictednot only by a function from 119868 to 119868
1015840 but also by a functionfrom 119868
1015840 to 119868 Therefore the corresponding relationships ofthe one-one type can be found in both CR
119868rarr1198681015840 and CR
1198681015840rarr119868
such as the corresponding point pair (p
119860 p1015840119860
) in Figures1(b) 2(a) and 2(b) Otherwise if one given correspondingpoint pair (p p1015840) isin 119868 times 119868
1015840 is of the many-one type phas a unique corresponding image (If (p p1015840) are a pair ofCPs p is called the corresponding image of p1015840 and viceversa) in 119868
1015840 based on the definition of CPs of the many-onetypeTherefore the corresponding relationships of themany-one type can also be depicted by a function mapping andare members of CR
119868rarr1198681015840 like the corresponding point pairs
(p119863
p1015840119863119864
) (p119864119865
p1015840119863119864
) in Figures 1(b) and 2(a) Similarlythe corresponding relationships of one-many can be depictedby a function mapping from 119868
1015840 to 119868 and are members ofCR1198681015840rarr119868
such as the CPs (p119861119862
p1015840119861) (p
119861119862 p1015840119862) in Figures
1(b) and 2(b)In conclusion for a pair of images 119868 and 119868
1015840 to be registeredof a scene the relationships between CPs can be decomposedinto a pair of component relations CR
119868rarr1198681015840 and CR
1198681015840rarr119868
which consist of a number of one-to-one and many-to-onemappingsTheCR
119868rarr1198681015840 andCR
1198681015840rarr119868
can be described by a pair
of vector-valued functions as 119891 119868 rarr 1198681015840 and 119891
1015840 1198681015840
rarr 119868The (119891 119891
1015840) are named as correspondence functions (CFs) in
[2]In image registration 119868 and 119868
1015840 are referred to as areference image and a target image respectively the objectiveis to transform the target image into the coordinate systemof the reference image Therefore only one vector-valuedfunction for example 119891 119868 rarr 119868
1015840 is needed to describethe corresponding relationship and named transformationfunction in [23]
Therefore CF is a pair of functions (119891 1198911015840) demonstrating
that for any one pair of CPs p isin 119868 and p1015840 isin 1198681015840 (p p1015840) are
consistent with at least one of two functions 119891 or 1198911015840 which
can be estimated from a given set of putative correspondencesbased on robust regression method [2] Also for any givenpoint p isin 119868 its corresponding point can be uniquelyestimated by the CF and vice versa (Figure 2)
In practice putative correspondences are usually cor-rupted by noise and outliers and the number of putativecorrespondences is limited Therefore only an estimation ofCF with a certain accuracy can be estimated and the actualresults may be not strictly consistent with the ldquonecessary andsufficient conditionrdquo
Therefore the characteristic is that CF model can reducethe searching space of CPs into an elliptical area and the mis-matches of CF are usually near the correct CPs (Section 5)
4 Learning of CF and Its Application inMismatch Rejection
Based on the discussions in Section 3 for a pair of images1198681and 1198682of a scene the relationship of some corresponding
points between 1198681and 119868
2can be decomposed into a pair
of component relations (CRs) CR119868rarr1198681015840 and CR
1198681015840rarr119868
A CRconsists of one-one mapping and many-one mapping
Therefore CR119868rarr1198681015840 and CR
1198681015840rarr119868
can be described by twovector-valued functions 119891 119868 rarr 119868
1015840 and 1198911015840
1198681015840
rarr 119868
119891 (p) = p1015840
1198911015840(p1015840) = p
(4)
where p isin 119868 p1015840 isin 1198681015840 and (119891 119891
putative correspondences between 1198681and 1198682 If we treat y =
p1015840 as an output of a system and x = p as an input theCF 119891 can be estimated from 119878 = (x y) = (p p1015840) p isin
119868 p1015840 isin 1198681015840 using a vector-valued regression method for
example vector field consensus (VFC) method [24 28]and IECF (Iteratively Estimate Correspondence Function)algorithm [2] The fundamental ideas of VFC and IECF areas follows approximately represent the unknown CF 119891 as aweighted sumof kernel functions (2) and estimate theweights
6 Mathematical Problems in Engineering
iteratively by gradually reducing the undue effects fromoutliers The objective of this work is to investigate thefundamental principles and characteristics of CF Thereforewe do not compare the various implementation of CF anddo experiments by IECF The CF 119891
1015840 can be estimatedsimilarly
Theoretically the correctness of a pair of putative corre-spondences (p p1015840) can be determined by checking whetherit is consistent with one of the estimated correspondencefunctions 119891 or 119891
1015840 However due to the influence of imagenoise the observed corresponding points usually are notstrictly consistent with the estimated CFs 119891 and 119891
1015840 and atolerance parameter 120572
119868is needed to determine whether they
are consistent It is shown that the consistency of (p p1015840) with119891 can be determined by
119888119891
(p p1015840) = (119891 (p) minus p1015840) Σminus1
(119891 (p) minus p1015840)119879
(5)
where the superscript119879 represents a transpose of a vector andΣ is the covariance matrix of random variable 119891(p) minus p1015840 andcan be estimated using IECF algorithm [2]The consistency of(p p1015840)with119891
1015840 ismeasured similarly and denoted by 1198881198911015840(p p1015840)
Then some mismatches can be rejected by the constraints119888119891(p p1015840) gt 120572
In this section we experimentally investigate the characteris-tics of theCF by comparing it with thewidely used parametricmodel epipolar geometry (EG) ((1) Figure 3) in the contextof rejecting mismatches
51 Experimental Configuration The mismatch-rejectingmethods based onCF and EG are implemented by algorithmsICF (identifying point correspondences by correspondencefunction) [2] and RANSAC (Random Sample Consensus)[11 29 30] respectively in this study Since the coordinatesof the observed putative CPs are usually corrupted by noisethey are not strictly consistent with the CF and EG equationsTherefore two tolerance parameters are needed for thecomputation we name the two parameters 120572
119877in EG +
RANSAC and 120572119868in CF + ICF
In evaluating the performance of local descriptors andpoints matching models there are several schemes forexample recall precision [31] and ROC curve [14] Howeverthe objective of this workrsquos experiments is to show thecharacteristic of fail-detected mismatches of CF Thereforeto make a fair comparison we adjusted the values of 120572
119877and
120572119868such that a same number of putative correspondences are
identified as possible correct matches in every experiment
and the RANSAC are implemented without a limit oniteration number
Suppose 119868 and 1198681015840 are two images of a scene to be matched
and suppose 119878 is a set of putative correspondences betweenthem A mismatch-rejecting method will partition 119878 intotwo subsets 119878
119888and 119878
119898 as possible correct correspondences
and possible mismatches For readability we denote the twosubsets by 119878
EG119888
and 119878EG119898
in the method based on EG constraintand 119878
CF119888
and 119878CF119898
in the method based on CF Thus set 119878 canbe divided into four subsets 119878
EG119888
cap 119878CF119888 119878
EG119888
cap 119878CF119888 119878
EG119888
cap 119878CF119888
and 119878EG119888
cap 119878CF119888 where 119878
EG119888
= 119878 minus 119878EG119888
= 119878EG119898
and 119878CF119888
=
119878 minus 119878CF119888
= 119878CF119898
The meanings of the above four subsets are asfollows
(1) 119878EG119888
cap 119878CF119888
= 119878EG119898
cap 119878CF119898 the putative correspondences
that are regarded as possible correct matches by themethods based on EG and CF
(2) 119878EG119888
cap 119878CF119888
= 119878EG119898
cap 119878CF119898 the putative correspondences
that are regarded as possible mismatches by EGmethod and possible correct matches by CF
(3) 119878EG119888
cap 119878CF119888
= 119878EG119898
cap 119878CF119898 the putative correspondences
that are regarded as possible correct matches by theEG method and possible mismatches by CF
(4) 119878EG119888
cap 119878CF119888
= 119878EG119898
cap 119878CF119898 the putative correspondences
that are regarded as possible mismatches by themethods based on EG and CF
On subsets 119878EG119888
cap119878CF119888
and 119878EG119888
cap119878CF119888 the twomethods based on
CF and EG constraint are consistentTherefore it is sufficientto focus on 119878
EG119888
cap 119878CFc and 119878
EG119888
cap 119878CF119888
to compare the CFmodeland the EG constraint
For convenience in comparing the results of EG and CFwe manually removed some of the correct and near-correctputative correspondences from 119878
EG119888
cap 119878CF119888
and 119878EG119888
cap 119878CF119888
in some experiments and the two sets composed of theremaining putative correspondences are denoted by 119879(119878
EG119888
cap
119878CF119888
) and 119879(119878EG119888
cap 119878CF119888
) respectively
(i) 119879(119878EG119888
cap 119878CF119888
) the typical putative correspondencesthat are classified as possible correct matches by EGand as possible mismatches by CF
(ii) 119879(119878EG119888
cap 119878CF119888
) the typical putative correspondencesthat are classified as possible mismatches by EG andas possible correct correspondences by CF
52 Experiments
521 Main Results One of the used image pairs is pre-sented in Figure 5 There are 1278 and 1207 SIFT featurepoints extracted from two images respectively (the featurepoints are detected by the multiscale DoG (Difference ofGaussian) scheme and described by the orientation his-togram technique [32]) and 448 putative correspondences
Mathematical Problems in Engineering 7
(a) 119878EG119888cap 119878
CF119888
(b) 119878EG119888cap 119878
CF119888
(c) 119879(119878EG119888cap 119878
CF119888) (d) 119879(119878EG
119888cap 119878
CF119888)
Figure 5 An image pair of a toy bear (a) 26 EG potential mismatches (PMs) that are accepted by CF as matches (b) 26 CF PMs that areaccepted by EG as matches (c) some typical EG PMs that are accepted by CF (d) some typical CF PMs that are accepted by EG 120572
119868= 10597
and 120572119877
= 000019
(a) 119878EG119888cap 119878
CF119888
(b) 119878EG119888cap 119878
CF119888
(c) 119878EG119888cap 119878
CF119888
(d) 119878EG119888cap 119878
CF119888
Figure 6 An image pair of a corridor (a) 25 EG PMs that are accepted by CF as matches (b) 25 CF PMs that are accepted by EG as matches(c) 31 EG PMs that are accepted by CF as matches (d) 31 CF PMs that are accepted by EG as matches In (a)-(b) 120572
119868= 5991 and 120572
119877= 000048
and in (c)-(d) 120572119868
= 110 and 120572119877
= 005
computed from the feature points by the NNDR (Near-est Neighbor and Distance Ratio) method [32] Due tothe ambiguity of local information some of the putativecorrespondences are incorrect and need to be rejected Inthis experiment the two methods based on EG and CFare inconsistent on 52 putative correspondences (Figures5(a) and 5(b)) For example for every putative correspon-dence (p p1015840) in Figure 5(a) the CF method identified it as
a possible correct match but the EG method identified it asa possible mismatch Therefore based on the definition of119878EG119888
cap 119878CF119888
and 119878EG119888
cap 119878CF119888 we can regard 119878
EG119888
cap 119878CF119888
as a setof potential mismatches that the EG constraint fails to detectand 119878
EG119888
cap 119878CF119888
as a set of potential mismatches that the CFmodel fails to reject More results are presented in Figures 67 and 8
8 Mathematical Problems in Engineering
(a) 119878EG119888cap 119878
CF119888
(b) 119878EG119888cap 119878
CF119888
(c) 119879(119878EG119888cap 119878
CF119888) (d) 119879(119878EG
119888cap 119878
CF119888)
Figure 7 An image pair of Baxia (a) Some EG PMs that are accepted by CF asmatches (b) some CF PMs that are accepted by EG asmatches(c) some typical EG PMs that are accepted by CF (d) some typical CF PMs that are accepted by EG 120572
119868= 10597 and 120572
119877= 005
(a) 119878EG119888cap 119878
CF119888
(b) 119878EG119888cap 119878
CF119888
(c) 119879(119878EGc cap 119878CF119888) (d) 119879(119878EG
119888cap 119878
CF119888)
Figure 8 An image pair of temple Fayu (a) 204 EG PMs that are accepted by CF as matches (b) 204 CF PMs that are accepted by EG asmatches (c) some typical EG PMs that are accepted by CF (d) some typical CF PMs that are accepted by EG 120572
119868= 31 and 120572
119877= 00001515
(for visibility only 100 randomly chosen PMs are presented in (a) and (b))
522 Comparing the CF and EG Constraint by Analyzing theUndetected Mismatches For ease of explanation we intro-duce the following two definitions type-I mismatches andtype-II mismatches Suppose (p p1015840) is a mismatch If thereexists a correct corresponding point pair (pp1015840) satisfying thatthe points p and p1015840 are in the neighborhood of p and p1015840respectively then (p p1015840) is defined as a mismatch of type-IOtherwise we call (p p1015840) a type-II mismatch For example
the mismatches in Figure 7(d) and (p p10158403) in Figure 3(b)
belong to type-II and (p p10158402) in Figure 3(b) belong to type-
I By the EG constraint we can only determine whether aputative correspondence is in a band around the epipolar line(Figures 3(b) 9(b) and 9(d)) but not whether it is close to thecorrect one Therefore the undetected mismatches usuallybelong to type-II in the methods based on the EG constraint(Figures 3 5(b) 5(d) 6(b) 6(d) 7(b) 7(d) 8(b) and 8(d))
Mathematical Problems in Engineering 9
l1
l2
p1
l3p3 p2
l4p4
l9984001 p9984001
l9984002 p9984002
l9984003 p9984003
l9984004p9984004
(a)
l1
l3
p1
p3
l2 p2
l9984001p9984001
l9984002p9984002
l9984003p9984003
(b)
l2p2
l1 p1l3p3
l9984001p9984001
l9984002p9984002
l9984003 p9984003
(c)
l1
l6
p1
p6
l3
l4 p4
p3
l2
l5
p2
p5
l9984001p9984001
l9984002 p9984002
l9984003
l9984006
p9984003
p9984006
l9984004 p9984004
l9984005p9984005
(d)
Figure 9 Some PMs and their corresponding epipolar lines The EG model cannot detect the mismatches that are near the epipolar line butfar from the correct CPs The undetected mismatches by CF are usually near both the correct CPs and the epipolar line (a) and (c) presentthe EG PMs that are accepted by CF as matches in the experiments of Figures 7(c) and 8(c) respectively (b) and (d) present the CF PMs thatare accepted by EG as matches in the experiments of Figures 7(d) and 8(d) respectively In these experiments (p
119894 p1015840119894) is a pair of putative
CPs and l119894is the epipolar line corresponding to p1015840
119894and l1015840119894is the epipolar line corresponding to p
119894
The CF constraint determines the mapping relationshipsbetween two CPs and directly estimates the position of thecorresponding point Therefore by the mismatch-rejectingmethods based on CF we can restrict the error bound ofthe putative correspondences by a tolerance parameter 120572
119868
the undetected mismatches by CF usually belong to type-I To show the difference between the CF model and EGconstraints a typical result is presented in Figure 6 in whichwe relax the tolerance parameters 120572
119868and 120572119877to ensure the two
sets 119878EG119888
cap 119878CF119888
and 119878EG119888
cap 119878CF119888
change at equal speedHowever themismatches of type-I and type-II are usually
dramatically different from each other in usefulness andundue influence From the mismatches of type-I we cansearch for the correct CPs by including more informationOn the contrary other than ruining the traditional estimationmethods and distorting the final application results (egimage mosaicing and three-dimensional reconstruction) themismatches of the type-II provide less useful information
53 Further Evaluation on Some Popular Benchmark DataThe characteristics of the CF are also investigated on twoseries of popular benchmark data sets widely used to compareand evaluate related algorithms [33 34]
To evaluate the two schemes quantitatively this workconducted experiments on a series of image sets with ground-truth correspondences [31 35] The images were capturedunder special consideration to enable the encoding of theground-truth correspondences with homographies between
the reference image and other images The ground-truthhomographies are computed based on the following stepsfirst a set of putative correspondences are manually selectedand an initial estimation of the homography is computedbased on the selected putative correspondences second theimages are approximately aligned with the rough estimationof homography third reliable interest points are detectedand matched automatically and an accurate homography iscomputed from them
The experimental results are presented in Figures 11 12 13and 14 In every experiment the upper image and the lowerimage are called the reference image and the target imagerespectively In addition to the computed putative correspon-dences we also present the ground-truth CPs and ground-truth correspondence relationships in these experimentsTheground-truth CPs in the target images are computed for all ofthe marked interest points in reference images based on theground-truth homographies and denoted by red circles Thedetected interest points are denoted by black plus signs Theground-truth correspondence relationships are displayed asblue dashed lines and the computed putative correspondencerelationships are shown by a solid line with red or green forvisibility
Suppose that 119878 = (p1 p10158401) (p
119899 p1015840119899) are a set of
putative correspondences and 119878 = (p1 p10158401) (p
119899 p1015840119899) are
the ground-truth correspondences of 119878 We quantitativelyevaluate the quality of 119878 by the root-mean-square deviation(RMSD) RMSD(119878) = radicsum
Figure 10The possible correct correspondences identified by CF in the experiment of Figure 8 are presented based on depth (a) 34 putativecorrespondences (PCs) over the first crossbeam (b) 324 PCs over the second crossbeam (c) 214 PCs over the third crossbeam (d) 20 PCsaround the fourth crossbeam (e) 2 PCs on the wall (for clarity only 50 randomly chosen PCs are presented in (b) and (c))
The second series of evaluation data are the Middle-bury Stereo Datasets [36 37] (httpvisionmiddleburyedustereodata) These data sets were created using an auto-mated version of structured-lighting technique [38] Eachdataset consists of seven rectified views This work used thefirst and the fifth views for evaluation The experimentalresults are presented in Figures 15 16 17 18 and 19
54 CF and EG in Practice
541 Necessity and Sufficiency As analyzed the EG con-straint is a necessary but insufficient condition in the corre-spondence problem and theCFmodel describes themappingbetween CPs directly However in applications some correctCPsmay be inconsistent with the estimated EGmodel andorthe estimated CF model there may be some mismatchesthat are ldquoconsistentrdquo with the estimated CF model there alsomay be some of the putative correspondences which areldquoconsistentrdquo with the estimated CF but ldquoinconsistentrdquo withthe estimated EG for example the experimental results inFigures 5(a) 5(b) 7(a) 7(b) 8(a) and 8(b) The reasons forthe above results are as follows
(i) The coordinates of the obtained putative CPs areusually corrupted by noise
(ii) There are usually some errors in the estimation of theCF and EG
(iii) As pointed out in the first paragraph of Section 5a preset tolerance parameter is needed to define theldquoconsistentrdquo and ldquoinconsistentrdquo in the applicationsof CF and EG However the noise in the obtainedputative CPs is random
542 Characteristics and Limitations of the CF Model andEG Model In the two models there are no restrictionson the depth range of the imaged scene However theEG is a global describing model and the CF is a localdescribing model In the EG model any pair of putativeCPs can influence the overall model estimation whereas inthe CF model a pair of putative CPs only influence themodel estimation within a local area in their vicinity Theshortcoming of the global model EG is that the overallmodel estimation can be degraded or ruined by PCs withconsiderable noise or mismatches in any area of the imagesThe shortcoming of the local model CF is that a pair of goodCPs cannot improve the estimation of CF outside of a localarea around itself Thus the uneven distribution of putativecorrespondences with different quality results in the fact thatthe estimated CF may be of high quality in some areas butof low quality in others for example in the experiment ofFigure 10 the qualities of the CF estimations over the second-nearest-to-the-cameras crossbeam and the third-nearest-to-the-cameras crossbeam are better than estimation over thefirst-nearest-to-the-cameras crossbeam The ldquolocal areardquo isdetermined by the CF estimation method and its chosenparameters for example in the CF estimation method SP(SVM) [2] the local area is determined by the chosen kerneland the scale parameter The above limitations of EG and CFcan be alleviated by investigating robust estimation methodsfor example the RANSAC M-estimators [8] LMedS (LeastMedian of Squares) [9] and MLESAC [10 12] for EGIn future work we will investigate further the estimationmethod for CF to improve its robustness
6 Conclusion
The CF is a recently introduced nonparametric model forrejecting mismatchesoutliers in image point matching In
Mathematical Problems in Engineering 11
(a) RMSD = 1386 (b) RMSD = 22707 (c) RMSD = 384 (d) RMSD = 23626Figure 11 Quantitative evaluation on two popular benchmark sets bikes and leuven The ground-truth CPs are presented by red circles thedetected interest points by black plus signs The ground-truth correspondences are displayed as blue dashed lines and the computed putativecorrespondences are shown by a solid green solid line (a) and (c) present some EG PMs that are accepted by CF as matches (b) and (d)present some CF PMs that are accepted by EG as matches 120572
119868= 10597 120572
119877= 21435 times 10
minus5 in (b) and 120572119877
= 58 times 10minus5 in (d)
(a) RMSD = 1631 (b) RMSD = 40098 (c) RMSD = 358 (d) RMSD = 32288Figure 12 Quantitative evaluation on two popular benchmark sets wall and bark ((a) and (c)) The EG PMs that are accepted by CF asmatches ((b) and (d)) the CF PMs that are accepted by EG as matches 120572
119868= 10597 120572
119877= 2029 times 10
minus4 in (b) and 120572119877
= 105 times 10minus4 in (d)
(a) RMSD = 1017 (b) RMSD = 42639 (c) RMSD = 329 (d) RMSD = 17646Figure 13 Quantitative evaluation on two popular benchmark sets graf and boat The ground-truth CPs are presented by red circles and thedetected interest points by black plus signs The ground-truth correspondences are displayed as blue dashed lines and the computed putativecorrespondences are shown by a solid green solid line (a) and (c) present some EG PMs that are accepted by CF as matches (b) and (d)present some CF PMs that are accepted by EG as matches 120572
Figure 14 Quantitative evaluation on two popular benchmark sets trees and ubc The ground-truth correspondences are displayed as bluedashed lines and the computed putative correspondences are shown by a yellow solid line (a) and (c) present some EG PMs that are acceptedbyCF asmatches (b) and (d) present someCFPMs that are accepted by EG asmatches120572
119868= 10597120572
119877= 59times10
minus5 in (b) and120572119877
= 185times10minus4
in (d)
(a) 119878EG119888cap 119878
CF119888
(b) 119878EG119888cap 119878
CF119888
(c) 119878EG119888cap 119878
CF119888
(d) 119878EG119888cap 119878
CF119888
Figure 15 Evaluation on Middlebury Stereo Datasets art and books ((a) and (c)) Some potential EG mismatches that are accepted by CF asmatches ((b) and (d)) some potential CF mismatches that are accepted by EG as matches 120572
119877= 0013 in (a) and (b) 120572
119877= 000018 in (c) and
(d) and 120572119868
= 10597
this study we investigated the principles of the CF andstudied its characteristics by comparing it with the widelyused parametric model epipolar geometry (EG) constraint
It is shown that the CF describes the mapping relation-ships between two CPs and should be able to estimate theposition of the corresponding point Therefore in additionto mismatch rejection a potential application of the CF is to
guide the point matching process by incorporating it into thecorrespondence propagation [39]
In practice putative correspondences are usually cor-rupted by noise and outliers and the number of putativecorrespondences is limited Therefore only an estimationof the CF with a certain accuracy can be estimated andthe actual results will violate the ldquonecessary and sufficient
Mathematical Problems in Engineering 13
(a) 119878EG119888cap SCF119888
(b) 119878EG119888cap 119878
CF119888
(c) 119878EG119888cap 119878
CF119888
(d) 119878EG119888cap 119878
CF119888
Figure 16 Evaluation on Middlebury Stereo Datasets computer and dolls ((a) and (c)) Some potential EG mismatches that are accepted byCF as matches ((b) and (d)) some potential CF mismatches that are accepted by EG as matches 120572
119877= 000018 in (a) and (b) 120572
119877= 000048 in
(c) and (d) and 120572119868
= 10597
(a) 119878EG119888cap 119878
CF119888
(b) 119878EG119888cap 119878
CF119888
(c) 119878EG119888cap 119878
CF119888
(d) 119878EG119888cap 119878
CF119888
Figure 17 Evaluation onMiddlebury Stereo Datasets drumsticks and dwarves ((a) and (c)) Some potential EGmismatches that are acceptedby CF as matches ((b) and (d)) some potential CF mismatches that are accepted by EG as matches 120572
119877= 000078 in (a) and (b) 120572
119877= 00035
in (c) and (d) and 120572119868
= 10597
conditionrdquoThe characteristic is that theCFmodel can reducethe searching space of CPs into an elliptical area and themismatches of the CF are usually near the correct CPs
Conflict of Interests
The authors declare that there is no conflict of interestsregarding the publication of this paper
Acknowledgments
The authors are grateful for valuable advice from DrDibyendu Mukherjee and the anonymous reviewers andwould like to thank the National Natural Science Foundationof China (Grant nos 61075033 61005033 and 61273248)the Natural Science Foundation of Guangdong Province(2014A030313425 and S2011010003348) and theOpenProject
14 Mathematical Problems in Engineering
(a) 119878EG119888cap 119878
CF119888
(b) 119878EG119888cap 119878
CF119888
(c) 119878EG119888cap 119878
CF119888
(d) 119878EG119888cap 119878
CF119888
Figure 18 Evaluation onMiddlebury Stereo Datasets laundry andmoebius ((a) and (c)) Some potential EGmismatches that are accepted byCF asmatches ((b) and (d)) some potential CFmismatches that are accepted by EG asmatches 120572
119877= 000000275 in (a) and (b) 120572
119877= 000035
in (c) and (d) and 120572119868
= 10597
(a) 119878EG119888cap 119878
CF119888
(b) 119878EG119888cap 119878
CF119888
Figure 19 Evaluation on Middlebury Stereo Dataset reindeer (a)Two potential EG mismatches that are accepted by CF as matches(b) two potential CFmismatches that are accepted by EG asmatches120572119877
= 0005 and 120572119868
= 30
Program of the National Laboratory of Pattern Recognition(NLPR) (201001060) for their support
References
[1] J P Monaco A C Bovik and L K Cormack ldquoActive foveateduncalibrated stereovisionrdquo International Journal of ComputerVision vol 85 no 2 pp 192ndash207 2009
[2] X Li and Z Hu ldquoRejecting mismatches by correspondencefunctionrdquo International Journal of Computer Vision vol 89 no1 pp 1ndash17 2010
[3] R Hartley and A Zisserman Multiple View Geometry inComputer Vision Cambridge University Press 2nd edition2003
[4] M Sonka V Hlavac and R Boyle Image Processing Analysisand Machine Vision Thomson-Engineering 3rd edition 2007
[5] G Carneiro and A D Jepson ldquoFlexible spatial configuration oflocal image featuresrdquo IEEE Transactions on Pattern Analysis andMachine Intelligence vol 29 no 12 pp 2089ndash2104 2007
[6] W Aguilar Y Frauel F Escolano M E Martinez-Perez AEspinosa-Romero andM A Lozano ldquoA robust graph transfor-mation matching for non-rigid registrationrdquo Image and VisionComputing vol 27 no 7 pp 897ndash910 2009
[7] L Wang F Duan and K Lv ldquoFisheye-lens-based visual suncompass for perception of spatial orientationrdquo MathematicalProblems in Engineering vol 2012 Article ID 460430 p 15 2012
[8] P Huber Robust Statistics Wiley New York NY USA 1981[9] P J Rousseeuw ldquoLeast median of squares regressionrdquo Journal
of the American Statistical Association vol 79 no 388 pp 871ndash880 1984
[10] P H S Torr and A Zisserman ldquoMLESAC a new robustestimator with application to estimating image geometryrdquoComputer Vision and Image Understanding vol 78 no 1 pp138ndash156 2000
[11] M A Fischler and R C Bolles ldquoRandom sample consensus aparadigm for model fitting with applications to image analysisand automated cartographyrdquo Communications of the ACM vol24 no 6 pp 381ndash395 1981
[12] B J Tordoff and D W Murray ldquoGuided-MLESAC fasterimage transform estimation by using matching priorsrdquo IEEETransactions on Pattern Analysis and Machine Intelligence vol27 no 10 pp 1523ndash1535 2005
Mathematical Problems in Engineering 15
[13] D Sidibe P Montesinos and S Janaqi ldquoMatching localinvariant features with contextual information an experimentalevaluationrdquo Electronic Letters on Computer Vision and ImageAnalysis vol 7 no 1 pp 26ndash39 2008
[14] Q-H Tran T-J Chin G Carneiro M S Brown and D SuterldquoIn defence of RANSAC for outlier rejection in deformableregistrationrdquo Lecture Notes in Computer Science vol 7575 no4 pp 274ndash287 2012
[15] A Goshtasby ldquoTransformation functions for image registra-tionrdquo in Proceedings of the Computer Vision and Pattern Recog-nition (CVPR rsquo11) June 2011
[16] X Li Y Wang J Zhou and X Li ldquoComparative research oncorrespondence functionrdquo in Proceedings of the InternationalConference on Intelligent System Design and Engineering Appli-cation (ISDEA rsquo10) pp 444ndash447 October 2010
[17] X-R Li X-M Li H-L Li and M-Y Cao ldquoRejecting outliersbased on correspondence manifoldrdquo Acta Automatica Sinicavol 35 no 1 pp 17ndash22 2009
[18] D Pizarro and A Bartoli ldquoFeature-based deformable surfacedetection with self-occlusion reasoningrdquo International Journalof Computer Vision vol 97 no 1 pp 54ndash70 2012
[19] J Yu A Eriksson T-J Chin and D Suter ldquoAn adversarialoptimization approach to efficient outlier removalrdquo Journal ofMathematical Imaging and Vision vol 48 no 3 pp 451ndash4662014
[20] A Goshtasby ldquoRegistration of images with geometric distor-tionsrdquo IEEETransactions onGeoscience andRemote Sensing vol26 no 1 pp 60ndash64 1988
[21] F L Bookstein ldquoPrincipal warps thin-plate splines and thedecomposition of deformationsrdquo IEEE Transactions on PatternAnalysis and Machine Intelligence vol 11 no 6 pp 567ndash5851992
[22] B Scholkopf F Steinke and V Blanz ldquoObject correspondenceas a machine learning problemrdquo in Proceedings of the 22ndInternational Conference on Machine Learning (ICML rsquo05) pp776ndash783 2005
[23] L Zagorchev and A Goshtasby ldquoA comparative study oftransformation functions for nonrigid image registrationrdquo IEEETransactions on Image Processing vol 15 no 3 pp 529ndash5382006
[24] J Zhao JMa J Tian J Ma andD Zhang ldquoA robustmethod forvector field learning with application to mismatch removingrdquoin Proceedings of the IEEE Conference on Computer Vision andPattern Recognition (CVPR rsquo11) Providence RI USA June 2011
[25] JMa J Zhao J Tian X Bai and Z Tu ldquoRegularized vector fieldlearning with sparse approximation for mismatch removalrdquoPattern Recognition vol 46 no 12 pp 3519ndash3532 2013
[26] J Ma J Zhao J Tian Z Tu and A L Yuille ldquoRobust estimationof nonrigid transformation for point set registrationrdquo in Pro-ceedings of the 26th IEEE Conference on Computer Vision andPattern Recognition (CVPR rsquo13) pp 2147ndash2154 June 2013
[27] J MaW Qiu J Zhao Y Ma A L Yuille and Z Tu ldquoRobust l2e
estimation of transformation for non-rigid registrationrdquo IEEETransactions on Signal Processing vol 63 no 5 pp 1115ndash11292015
[28] J Ma J Zhao J Tian A L Yuille and Z Tu ldquoRobust pointmatching via vector field consensusrdquo IEEE Transactions onImage Processing vol 23 no 4 pp 1706ndash1721 2014
[29] L Goshen and I Shimshoni ldquoBalanced exploration andexploitation model search for efficient epipolar geometry esti-mationrdquo IEEE Transactions on Pattern Analysis and MachineIntelligence vol 30 no 7 pp 1230ndash1242 2008
[30] O Chum and J Matas ldquoMatching with prosacmdashprogressivesample consensusrdquo in Proceedings of the IEEE Computer SocietyConference on Computer Vision and Pattern Recognition (CVPRrsquo05) vol 1 pp 220ndash226 IEEE June 2005
[31] K Mikolajczyk and C Schmid ldquoA performance evaluation oflocal descriptorsrdquo IEEE Transactions on Pattern Analysis andMachine Intelligence vol 27 no 10 pp 1615ndash1630 2005
[32] D G Lowe ldquoDistinctive image features from scale-invariantkeypointsrdquo International Journal of Computer Vision vol 60 no2 pp 91ndash110 2004
[33] S Gauglitz T Hollerer and M Turk ldquoEvaluation of interestpoint detectors and feature descriptors for visual trackingrdquoInternational Journal of Computer Vision vol 94 no 3 pp 335ndash360 2011
[34] Y Yu K Huang W Chen and T Tan ldquoA novel algorithmfor view and illumination invariant image matchingrdquo IEEETransactions on Image Processing vol 21 no 1 pp 229ndash2402012
[35] K Mikolajczyk T Tuytelaars C Schmid et al ldquoA comparisonof affine region detectorsrdquo International Journal of ComputerVision vol 65 no 1-2 pp 43ndash72 2005
[36] H Hirschmuller and D Scharstein ldquoEvaluation of cost func-tions for stereo matchingrdquo in Proceedings of the IEEE ComputerSociety Conference on Computer Vision and Pattern Recognition(CVPR rsquo07) June 2007
[37] D Scharstein and C Pal ldquoLearning conditional random fieldsfor stereordquo in Proceedings of the IEEE Computer Society Confer-ence on Computer Vision and Pattern Recognition (CVPR rsquo07)IEEE Minneapolis Minn USA June 2007
[38] D Scharstein andR Szeliski ldquoHigh-accuracy stereo depthmapsusing structured lightrdquo in Proceedings of the IEEE ComputerSociety Conference on Computer Vision and Pattern Recognition(CVPR rsquo03) vol 1 pp I-195ndashI-202 IEEE June 2003
[39] H Wang S Yan J Liu X Tang and T S Huang ldquoCorre-spondence propagationwithweak priorsrdquo IEEETransactions onImage Processing vol 18 no 1 pp 140ndash150 2009
2are two camera centres P is a scene point 119868 and 119868
1015840 are two images of a scene eand e1015840 are two epipolar points in the images and l and l1015840 are two epipolar lines determined by two points p1015840 and p respectively (b) A sketchmap of image 119868
1015840 in (a)
2 Related Research
The mismatch-rejecting problem has been investigated inmany studies [14 18 19] and the EG constraint is a widelyused model in resolving the problem
Suppose that 119868 and 1198681015840 are two images of a scene F is
the fundamental matrix and 119878 sub 119868 times 1198681015840 is a set of putative
point correspondences between them The EG constraintsays
p119879Fp1015840 = 0 (1)
where (p p1015840) isin 119868 times 1198681015840 is a pair of CPs and a superscript 119879
denotes the transpose of a matrix or vector The principleof the mismatch-rejecting methods based on EG is that thecorresponding point p1015840 of p isin 119868 is on a line l1015840 = F119879p in 119868
1015840 andvice versa (Figure 3(a)) where l and l1015840 are called two epipolarlines defined by points p1015840 and p respectively [3] Thereforeif F is known then we can reject some mismatches from 119878 bychecking whether they are consistent with the EG constraint(1) [1 3 4]
However as shown in Figure 3(b) if p10158401
isin 1198681015840 is another
point on the epipolar line l1015840 the mismatch (p p10158401) cannot
be detected by this constraint Therefore the EG constraintis a necessary but insufficient condition by which themismatches consistent with (1) cannot be detected Unfor-tunately the undue influence of these kinds of mismatchesmay be large enough to severely distort the final resultsin applications for example in image mosaicing and 3-dimensional reconstruction
CF is a concept recently introduced to reject mismatchesfromputative correspondences between images [2] Its poten-tial superiority has been shown experimentally on efficiencyand accuracy From the point set mapping perspective CF
depicts the relationships between corresponding points bymapping a point in one image to its corresponding point inanother [2]
For two images of a general scene the relationshipbetween some corresponding points (CPs) can be classifiedinto three types one-one one-many and many-one [2 4]The fundamental idea of the CF is to decompose the rela-tionships between the CPs into two subsets The two subsetsconsist of one-one and many-one or one-one and one-manykinds of corresponding relationships The two subsets ofcorresponding relationships can be described by two vector-valued functions 119891 119868 rarr 119868
1015840 and 1198911015840
1198681015840
rarr 119868Suppose 119878 = (p
119894 p1015840119894) 119894 = 1 119899 sub 119868 times 119868
1015840 are aset of putative correspondences CF for example 119891 can berepresented by
119891 (p) =
119899
sum
119894=1
w119894119896 (p p1015840
119894) (2)
where 119896(sdot sdot) is kernel function and w119894
= (1199081
119894 1199082
119894) are para-
meters to be estimated Based on this representation andGaussian kernel Li and Hu [2] studied the estimation of CFbased on Support Vector Machine
Actually the vector-valued function interpolationregres-sion with similar idea has been extensively investigated formodelling the transformations between two images in sce-narios of image registration and computer graphics Forexample Goshtasby [20] used a vector-valued function froma reference image to a target image for remote sensingimage registration based on surface spline Bookstein [21]studied the registration of medical images based on the thin-plate spline (TPS) Scholkopf et al [22] proposed a machinelearningmethod to establish dense point correspondences by
4 Mathematical Problems in Engineering
B2
B1
A1(u1 1)
Image I
Left camera Right cameraOL OR
A998400i(u
998400i
998400i )
Image I998400
(a)
Left camera Right camera
DC2
C1
AImage I Image I998400
OL OR
A9984001 A998400
2
(b)
Figure 4 Exception of uniqueness in correspondence problem (a) Many points A1015840119894 sub 119868
1015840 share a correspondence point A1
isin 119868 (b) Twopoints A1015840
1and A1015840
2in image 119868
1015840 correspond to one common point A isin 119868 and there are no CPs in image 119868 for the points in (A10158401A10158402) sub 1198681015840
estimating the deformation fields that transform two givenobjects into each other
In these scenarios the focus is to transform an image intoa coordinate systemof another image therefore two images tobe processed are usually called a reference image and a targetimage respectively and only one vector-valued functionis needed to describe the relationship from the referenceimage to the target image A good review can be found in[15 23] for the works on image registration based on thisidea
Letp isin 119868 andp1015840 isin 1198681015840 be two image points119892(p) = 119891(p)minusp
and y = p1015840 minus p The CF for example 119891(p) = p1015840 119868 rarr 1198681015840 can
be expressed by
y = 119892 (p) (3)
where y = p1015840 minus p The mapping y = 119892(p) 119868 rarr 1198681015840
minus 119868
is a special expression of CF and referred to as a vector field[24 25] Zhao et al [24] and Ma et al [25] investigated thevector field learning problem using a Bayesian frameworkand studied its application in outlier rejection Ma et al[26 27] proposed an estimation method 119871
2119864 for the vector
field and investigated its application in point set registrationbetween images of nonrigid object Ma et al [28] investi-gated the point correspondence problem by interpolatingthe vector field and formulated it a maximum a posteriori(MAP) estimation of a Bayesian model with some hiddenvariables
In literature related researches are usually focused onlearning and application of the correspondence function ortransformation function based on a flexible function Weinvestigate the principles and characteristics of it in thiswork
3 Principles of Correspondence Function
Suppose 119868 and 1198681015840 are two images to be registered The CPs
between 119868 and 1198681015840 form a surface in the joint-image space
119868 times 1198681015840 [17] which is illustrated schematically in Figure 1(a)
In this figure images 119868 and 1198681015840 are illustrated as a hori-
zontal axis and a vertical axis respectively And the sur-face is referred to as correspondence manifold (CM) in[17]
In Figure 1(a) (p119860
p1015840119860
) is a pair of CPs between images 119868
and 1198681015840 and 119860 is a point in the joint-image space 119868 times 119868
1015840 formedby CPs (p
119860 p1015840119860
) and the other points on the curves from 119860
to 119864 and from 119865 to 119866 have similar meanings Additionallycurves 119860119861 119862119863 and 119865119866 correspond to the image pointsprojected from the scene surface which can be distinguishedby both cameras of 119868 and 119868
1015840 curves 119861119862 and 119863119864 and points119864 119865 correspond to the scene portions which can only bedistinguished by one camera as illustrated in Figure 4(a)The gap between 119864 and 119865 corresponds to the scene portionbetween C
1and C
2in Figure 4(b) that can be observed by
one camera but not by the other one This sketch map showsthat the relationship between corresponding points can beclassified into three types one-one one-many and many-one This characteristic is independent of the camera modelof the images to be registered and the complexity of scenesurface
To highlight the mapping between CPs we move theorigin to an infinite point and connect the CPs by a dottedline then the sketch map in Figure 1(a) becomes Figure 1(b)It is well known that a function is a mapping that candepict two types of relationshipsmdashmany-one and one-onemdashin mathematics Therefore the relationship between CPs istoo complex to be described by a function
The fundamental idea of CF is to decompose the cor-responding relationships (we name the mapping relations
Mathematical Problems in Engineering 5
between CPs as ldquocorresponding relationshipsrdquo) into twodirectional relationships that can be depicted by two func-tions For convenience these directional relationships arereferred to as ldquocomponent relationsrdquo (CRs) and labeled asCR119868rarr1198681015840 andCR
1198681015840rarr119868
respectivelyTheCR consists of one-onemapping and many-one mapping For example Figures 2(a)and 2(b) are a pair of CRs of Figure 1
The component relation for example CR119868rarr1198681015840 can be
constructed as follows for every point p isin 119868 if there isa unique corresponding point p1015840 isin 119868
1015840 the correspond-ing relationship (p p1015840) should be one member of CR
119868rarr1198681015840
Otherwise if p has more than one corresponding pointp10158401 p1015840
119896119896gt0
isin 1198681015840 we have two choices either ignore all
of the corresponding relationships (p p119894) 1 le 119894 le 119896 or
select any one of the CPs p10158401198940
insert (p p10158401198940
) into CR1198681015840rarr119868
andignore the other corresponding relationships (p p
119894) 1 le 119894 le
119896 119894 = 1198940 Component relation CR
1198681015840rarr119868
can be constructedsimilarly by checking every potential point p1015840 isin 119868
1015840 Forexample in Figures 1 and 2 point p
119860isin 119868 has a unique
corresponding point p1015840119860in 1198681015840 therefore the corresponding
relationship (p119860
p1015840119860
) is in component relation CR119868rarr1198681015840 In
particular the corresponding point of p119863
isin 119868 is also uniquein 1198681015840 Therefore (p
119863 p1015840119863119864
) is one member of the componentrelation CR
119868rarr1198681015840 even though the CPs of p1015840
119863119864are not unique
in image 119868Therefore CR
119868rarr1198681015840 is a one-to-one or many-to-one map-
ping from 119868 to 1198681015840 and can be depicted by a vector-valued
function 119891 119868 rarr 1198681015840 Similarly CR
1198681015840rarr119868
is also a one-to-oneor many-to-one mapping from 119868
1015840 to 119868 and can be depicted bya vector-valued function 119891
1015840 1198681015840
rarr 119868
On the other hand for any given corresponding pointpair (p p1015840) isin 119868 times 119868
1015840 if it is of the one-one type thecorresponding relationship expressed by it can be depictednot only by a function from 119868 to 119868
1015840 but also by a functionfrom 119868
1015840 to 119868 Therefore the corresponding relationships ofthe one-one type can be found in both CR
119868rarr1198681015840 and CR
1198681015840rarr119868
such as the corresponding point pair (p
119860 p1015840119860
) in Figures1(b) 2(a) and 2(b) Otherwise if one given correspondingpoint pair (p p1015840) isin 119868 times 119868
1015840 is of the many-one type phas a unique corresponding image (If (p p1015840) are a pair ofCPs p is called the corresponding image of p1015840 and viceversa) in 119868
1015840 based on the definition of CPs of the many-onetypeTherefore the corresponding relationships of themany-one type can also be depicted by a function mapping andare members of CR
119868rarr1198681015840 like the corresponding point pairs
(p119863
p1015840119863119864
) (p119864119865
p1015840119863119864
) in Figures 1(b) and 2(a) Similarlythe corresponding relationships of one-many can be depictedby a function mapping from 119868
1015840 to 119868 and are members ofCR1198681015840rarr119868
such as the CPs (p119861119862
p1015840119861) (p
119861119862 p1015840119862) in Figures
1(b) and 2(b)In conclusion for a pair of images 119868 and 119868
1015840 to be registeredof a scene the relationships between CPs can be decomposedinto a pair of component relations CR
119868rarr1198681015840 and CR
1198681015840rarr119868
which consist of a number of one-to-one and many-to-onemappingsTheCR
119868rarr1198681015840 andCR
1198681015840rarr119868
can be described by a pair
of vector-valued functions as 119891 119868 rarr 1198681015840 and 119891
1015840 1198681015840
rarr 119868The (119891 119891
1015840) are named as correspondence functions (CFs) in
[2]In image registration 119868 and 119868
1015840 are referred to as areference image and a target image respectively the objectiveis to transform the target image into the coordinate systemof the reference image Therefore only one vector-valuedfunction for example 119891 119868 rarr 119868
1015840 is needed to describethe corresponding relationship and named transformationfunction in [23]
Therefore CF is a pair of functions (119891 1198911015840) demonstrating
that for any one pair of CPs p isin 119868 and p1015840 isin 1198681015840 (p p1015840) are
consistent with at least one of two functions 119891 or 1198911015840 which
can be estimated from a given set of putative correspondencesbased on robust regression method [2] Also for any givenpoint p isin 119868 its corresponding point can be uniquelyestimated by the CF and vice versa (Figure 2)
In practice putative correspondences are usually cor-rupted by noise and outliers and the number of putativecorrespondences is limited Therefore only an estimation ofCF with a certain accuracy can be estimated and the actualresults may be not strictly consistent with the ldquonecessary andsufficient conditionrdquo
Therefore the characteristic is that CF model can reducethe searching space of CPs into an elliptical area and the mis-matches of CF are usually near the correct CPs (Section 5)
4 Learning of CF and Its Application inMismatch Rejection
Based on the discussions in Section 3 for a pair of images1198681and 1198682of a scene the relationship of some corresponding
points between 1198681and 119868
2can be decomposed into a pair
of component relations (CRs) CR119868rarr1198681015840 and CR
1198681015840rarr119868
A CRconsists of one-one mapping and many-one mapping
Therefore CR119868rarr1198681015840 and CR
1198681015840rarr119868
can be described by twovector-valued functions 119891 119868 rarr 119868
1015840 and 1198911015840
1198681015840
rarr 119868
119891 (p) = p1015840
1198911015840(p1015840) = p
(4)
where p isin 119868 p1015840 isin 1198681015840 and (119891 119891
putative correspondences between 1198681and 1198682 If we treat y =
p1015840 as an output of a system and x = p as an input theCF 119891 can be estimated from 119878 = (x y) = (p p1015840) p isin
119868 p1015840 isin 1198681015840 using a vector-valued regression method for
example vector field consensus (VFC) method [24 28]and IECF (Iteratively Estimate Correspondence Function)algorithm [2] The fundamental ideas of VFC and IECF areas follows approximately represent the unknown CF 119891 as aweighted sumof kernel functions (2) and estimate theweights
6 Mathematical Problems in Engineering
iteratively by gradually reducing the undue effects fromoutliers The objective of this work is to investigate thefundamental principles and characteristics of CF Thereforewe do not compare the various implementation of CF anddo experiments by IECF The CF 119891
1015840 can be estimatedsimilarly
Theoretically the correctness of a pair of putative corre-spondences (p p1015840) can be determined by checking whetherit is consistent with one of the estimated correspondencefunctions 119891 or 119891
1015840 However due to the influence of imagenoise the observed corresponding points usually are notstrictly consistent with the estimated CFs 119891 and 119891
1015840 and atolerance parameter 120572
119868is needed to determine whether they
are consistent It is shown that the consistency of (p p1015840) with119891 can be determined by
119888119891
(p p1015840) = (119891 (p) minus p1015840) Σminus1
(119891 (p) minus p1015840)119879
(5)
where the superscript119879 represents a transpose of a vector andΣ is the covariance matrix of random variable 119891(p) minus p1015840 andcan be estimated using IECF algorithm [2]The consistency of(p p1015840)with119891
1015840 ismeasured similarly and denoted by 1198881198911015840(p p1015840)
Then some mismatches can be rejected by the constraints119888119891(p p1015840) gt 120572
In this section we experimentally investigate the characteris-tics of theCF by comparing it with thewidely used parametricmodel epipolar geometry (EG) ((1) Figure 3) in the contextof rejecting mismatches
51 Experimental Configuration The mismatch-rejectingmethods based onCF and EG are implemented by algorithmsICF (identifying point correspondences by correspondencefunction) [2] and RANSAC (Random Sample Consensus)[11 29 30] respectively in this study Since the coordinatesof the observed putative CPs are usually corrupted by noisethey are not strictly consistent with the CF and EG equationsTherefore two tolerance parameters are needed for thecomputation we name the two parameters 120572
119877in EG +
RANSAC and 120572119868in CF + ICF
In evaluating the performance of local descriptors andpoints matching models there are several schemes forexample recall precision [31] and ROC curve [14] Howeverthe objective of this workrsquos experiments is to show thecharacteristic of fail-detected mismatches of CF Thereforeto make a fair comparison we adjusted the values of 120572
119877and
120572119868such that a same number of putative correspondences are
identified as possible correct matches in every experiment
and the RANSAC are implemented without a limit oniteration number
Suppose 119868 and 1198681015840 are two images of a scene to be matched
and suppose 119878 is a set of putative correspondences betweenthem A mismatch-rejecting method will partition 119878 intotwo subsets 119878
119888and 119878
119898 as possible correct correspondences
and possible mismatches For readability we denote the twosubsets by 119878
EG119888
and 119878EG119898
in the method based on EG constraintand 119878
CF119888
and 119878CF119898
in the method based on CF Thus set 119878 canbe divided into four subsets 119878
EG119888
cap 119878CF119888 119878
EG119888
cap 119878CF119888 119878
EG119888
cap 119878CF119888
and 119878EG119888
cap 119878CF119888 where 119878
EG119888
= 119878 minus 119878EG119888
= 119878EG119898
and 119878CF119888
=
119878 minus 119878CF119888
= 119878CF119898
The meanings of the above four subsets are asfollows
(1) 119878EG119888
cap 119878CF119888
= 119878EG119898
cap 119878CF119898 the putative correspondences
that are regarded as possible correct matches by themethods based on EG and CF
(2) 119878EG119888
cap 119878CF119888
= 119878EG119898
cap 119878CF119898 the putative correspondences
that are regarded as possible mismatches by EGmethod and possible correct matches by CF
(3) 119878EG119888
cap 119878CF119888
= 119878EG119898
cap 119878CF119898 the putative correspondences
that are regarded as possible correct matches by theEG method and possible mismatches by CF
(4) 119878EG119888
cap 119878CF119888
= 119878EG119898
cap 119878CF119898 the putative correspondences
that are regarded as possible mismatches by themethods based on EG and CF
On subsets 119878EG119888
cap119878CF119888
and 119878EG119888
cap119878CF119888 the twomethods based on
CF and EG constraint are consistentTherefore it is sufficientto focus on 119878
EG119888
cap 119878CFc and 119878
EG119888
cap 119878CF119888
to compare the CFmodeland the EG constraint
For convenience in comparing the results of EG and CFwe manually removed some of the correct and near-correctputative correspondences from 119878
EG119888
cap 119878CF119888
and 119878EG119888
cap 119878CF119888
in some experiments and the two sets composed of theremaining putative correspondences are denoted by 119879(119878
EG119888
cap
119878CF119888
) and 119879(119878EG119888
cap 119878CF119888
) respectively
(i) 119879(119878EG119888
cap 119878CF119888
) the typical putative correspondencesthat are classified as possible correct matches by EGand as possible mismatches by CF
(ii) 119879(119878EG119888
cap 119878CF119888
) the typical putative correspondencesthat are classified as possible mismatches by EG andas possible correct correspondences by CF
52 Experiments
521 Main Results One of the used image pairs is pre-sented in Figure 5 There are 1278 and 1207 SIFT featurepoints extracted from two images respectively (the featurepoints are detected by the multiscale DoG (Difference ofGaussian) scheme and described by the orientation his-togram technique [32]) and 448 putative correspondences
Mathematical Problems in Engineering 7
(a) 119878EG119888cap 119878
CF119888
(b) 119878EG119888cap 119878
CF119888
(c) 119879(119878EG119888cap 119878
CF119888) (d) 119879(119878EG
119888cap 119878
CF119888)
Figure 5 An image pair of a toy bear (a) 26 EG potential mismatches (PMs) that are accepted by CF as matches (b) 26 CF PMs that areaccepted by EG as matches (c) some typical EG PMs that are accepted by CF (d) some typical CF PMs that are accepted by EG 120572
119868= 10597
and 120572119877
= 000019
(a) 119878EG119888cap 119878
CF119888
(b) 119878EG119888cap 119878
CF119888
(c) 119878EG119888cap 119878
CF119888
(d) 119878EG119888cap 119878
CF119888
Figure 6 An image pair of a corridor (a) 25 EG PMs that are accepted by CF as matches (b) 25 CF PMs that are accepted by EG as matches(c) 31 EG PMs that are accepted by CF as matches (d) 31 CF PMs that are accepted by EG as matches In (a)-(b) 120572
119868= 5991 and 120572
119877= 000048
and in (c)-(d) 120572119868
= 110 and 120572119877
= 005
computed from the feature points by the NNDR (Near-est Neighbor and Distance Ratio) method [32] Due tothe ambiguity of local information some of the putativecorrespondences are incorrect and need to be rejected Inthis experiment the two methods based on EG and CFare inconsistent on 52 putative correspondences (Figures5(a) and 5(b)) For example for every putative correspon-dence (p p1015840) in Figure 5(a) the CF method identified it as
a possible correct match but the EG method identified it asa possible mismatch Therefore based on the definition of119878EG119888
cap 119878CF119888
and 119878EG119888
cap 119878CF119888 we can regard 119878
EG119888
cap 119878CF119888
as a setof potential mismatches that the EG constraint fails to detectand 119878
EG119888
cap 119878CF119888
as a set of potential mismatches that the CFmodel fails to reject More results are presented in Figures 67 and 8
8 Mathematical Problems in Engineering
(a) 119878EG119888cap 119878
CF119888
(b) 119878EG119888cap 119878
CF119888
(c) 119879(119878EG119888cap 119878
CF119888) (d) 119879(119878EG
119888cap 119878
CF119888)
Figure 7 An image pair of Baxia (a) Some EG PMs that are accepted by CF asmatches (b) some CF PMs that are accepted by EG asmatches(c) some typical EG PMs that are accepted by CF (d) some typical CF PMs that are accepted by EG 120572
119868= 10597 and 120572
119877= 005
(a) 119878EG119888cap 119878
CF119888
(b) 119878EG119888cap 119878
CF119888
(c) 119879(119878EGc cap 119878CF119888) (d) 119879(119878EG
119888cap 119878
CF119888)
Figure 8 An image pair of temple Fayu (a) 204 EG PMs that are accepted by CF as matches (b) 204 CF PMs that are accepted by EG asmatches (c) some typical EG PMs that are accepted by CF (d) some typical CF PMs that are accepted by EG 120572
119868= 31 and 120572
119877= 00001515
(for visibility only 100 randomly chosen PMs are presented in (a) and (b))
522 Comparing the CF and EG Constraint by Analyzing theUndetected Mismatches For ease of explanation we intro-duce the following two definitions type-I mismatches andtype-II mismatches Suppose (p p1015840) is a mismatch If thereexists a correct corresponding point pair (pp1015840) satisfying thatthe points p and p1015840 are in the neighborhood of p and p1015840respectively then (p p1015840) is defined as a mismatch of type-IOtherwise we call (p p1015840) a type-II mismatch For example
the mismatches in Figure 7(d) and (p p10158403) in Figure 3(b)
belong to type-II and (p p10158402) in Figure 3(b) belong to type-
I By the EG constraint we can only determine whether aputative correspondence is in a band around the epipolar line(Figures 3(b) 9(b) and 9(d)) but not whether it is close to thecorrect one Therefore the undetected mismatches usuallybelong to type-II in the methods based on the EG constraint(Figures 3 5(b) 5(d) 6(b) 6(d) 7(b) 7(d) 8(b) and 8(d))
Mathematical Problems in Engineering 9
l1
l2
p1
l3p3 p2
l4p4
l9984001 p9984001
l9984002 p9984002
l9984003 p9984003
l9984004p9984004
(a)
l1
l3
p1
p3
l2 p2
l9984001p9984001
l9984002p9984002
l9984003p9984003
(b)
l2p2
l1 p1l3p3
l9984001p9984001
l9984002p9984002
l9984003 p9984003
(c)
l1
l6
p1
p6
l3
l4 p4
p3
l2
l5
p2
p5
l9984001p9984001
l9984002 p9984002
l9984003
l9984006
p9984003
p9984006
l9984004 p9984004
l9984005p9984005
(d)
Figure 9 Some PMs and their corresponding epipolar lines The EG model cannot detect the mismatches that are near the epipolar line butfar from the correct CPs The undetected mismatches by CF are usually near both the correct CPs and the epipolar line (a) and (c) presentthe EG PMs that are accepted by CF as matches in the experiments of Figures 7(c) and 8(c) respectively (b) and (d) present the CF PMs thatare accepted by EG as matches in the experiments of Figures 7(d) and 8(d) respectively In these experiments (p
119894 p1015840119894) is a pair of putative
CPs and l119894is the epipolar line corresponding to p1015840
119894and l1015840119894is the epipolar line corresponding to p
119894
The CF constraint determines the mapping relationshipsbetween two CPs and directly estimates the position of thecorresponding point Therefore by the mismatch-rejectingmethods based on CF we can restrict the error bound ofthe putative correspondences by a tolerance parameter 120572
119868
the undetected mismatches by CF usually belong to type-I To show the difference between the CF model and EGconstraints a typical result is presented in Figure 6 in whichwe relax the tolerance parameters 120572
119868and 120572119877to ensure the two
sets 119878EG119888
cap 119878CF119888
and 119878EG119888
cap 119878CF119888
change at equal speedHowever themismatches of type-I and type-II are usually
dramatically different from each other in usefulness andundue influence From the mismatches of type-I we cansearch for the correct CPs by including more informationOn the contrary other than ruining the traditional estimationmethods and distorting the final application results (egimage mosaicing and three-dimensional reconstruction) themismatches of the type-II provide less useful information
53 Further Evaluation on Some Popular Benchmark DataThe characteristics of the CF are also investigated on twoseries of popular benchmark data sets widely used to compareand evaluate related algorithms [33 34]
To evaluate the two schemes quantitatively this workconducted experiments on a series of image sets with ground-truth correspondences [31 35] The images were capturedunder special consideration to enable the encoding of theground-truth correspondences with homographies between
the reference image and other images The ground-truthhomographies are computed based on the following stepsfirst a set of putative correspondences are manually selectedand an initial estimation of the homography is computedbased on the selected putative correspondences second theimages are approximately aligned with the rough estimationof homography third reliable interest points are detectedand matched automatically and an accurate homography iscomputed from them
The experimental results are presented in Figures 11 12 13and 14 In every experiment the upper image and the lowerimage are called the reference image and the target imagerespectively In addition to the computed putative correspon-dences we also present the ground-truth CPs and ground-truth correspondence relationships in these experimentsTheground-truth CPs in the target images are computed for all ofthe marked interest points in reference images based on theground-truth homographies and denoted by red circles Thedetected interest points are denoted by black plus signs Theground-truth correspondence relationships are displayed asblue dashed lines and the computed putative correspondencerelationships are shown by a solid line with red or green forvisibility
Suppose that 119878 = (p1 p10158401) (p
119899 p1015840119899) are a set of
putative correspondences and 119878 = (p1 p10158401) (p
119899 p1015840119899) are
the ground-truth correspondences of 119878 We quantitativelyevaluate the quality of 119878 by the root-mean-square deviation(RMSD) RMSD(119878) = radicsum
Figure 10The possible correct correspondences identified by CF in the experiment of Figure 8 are presented based on depth (a) 34 putativecorrespondences (PCs) over the first crossbeam (b) 324 PCs over the second crossbeam (c) 214 PCs over the third crossbeam (d) 20 PCsaround the fourth crossbeam (e) 2 PCs on the wall (for clarity only 50 randomly chosen PCs are presented in (b) and (c))
The second series of evaluation data are the Middle-bury Stereo Datasets [36 37] (httpvisionmiddleburyedustereodata) These data sets were created using an auto-mated version of structured-lighting technique [38] Eachdataset consists of seven rectified views This work used thefirst and the fifth views for evaluation The experimentalresults are presented in Figures 15 16 17 18 and 19
54 CF and EG in Practice
541 Necessity and Sufficiency As analyzed the EG con-straint is a necessary but insufficient condition in the corre-spondence problem and theCFmodel describes themappingbetween CPs directly However in applications some correctCPsmay be inconsistent with the estimated EGmodel andorthe estimated CF model there may be some mismatchesthat are ldquoconsistentrdquo with the estimated CF model there alsomay be some of the putative correspondences which areldquoconsistentrdquo with the estimated CF but ldquoinconsistentrdquo withthe estimated EG for example the experimental results inFigures 5(a) 5(b) 7(a) 7(b) 8(a) and 8(b) The reasons forthe above results are as follows
(i) The coordinates of the obtained putative CPs areusually corrupted by noise
(ii) There are usually some errors in the estimation of theCF and EG
(iii) As pointed out in the first paragraph of Section 5a preset tolerance parameter is needed to define theldquoconsistentrdquo and ldquoinconsistentrdquo in the applicationsof CF and EG However the noise in the obtainedputative CPs is random
542 Characteristics and Limitations of the CF Model andEG Model In the two models there are no restrictionson the depth range of the imaged scene However theEG is a global describing model and the CF is a localdescribing model In the EG model any pair of putativeCPs can influence the overall model estimation whereas inthe CF model a pair of putative CPs only influence themodel estimation within a local area in their vicinity Theshortcoming of the global model EG is that the overallmodel estimation can be degraded or ruined by PCs withconsiderable noise or mismatches in any area of the imagesThe shortcoming of the local model CF is that a pair of goodCPs cannot improve the estimation of CF outside of a localarea around itself Thus the uneven distribution of putativecorrespondences with different quality results in the fact thatthe estimated CF may be of high quality in some areas butof low quality in others for example in the experiment ofFigure 10 the qualities of the CF estimations over the second-nearest-to-the-cameras crossbeam and the third-nearest-to-the-cameras crossbeam are better than estimation over thefirst-nearest-to-the-cameras crossbeam The ldquolocal areardquo isdetermined by the CF estimation method and its chosenparameters for example in the CF estimation method SP(SVM) [2] the local area is determined by the chosen kerneland the scale parameter The above limitations of EG and CFcan be alleviated by investigating robust estimation methodsfor example the RANSAC M-estimators [8] LMedS (LeastMedian of Squares) [9] and MLESAC [10 12] for EGIn future work we will investigate further the estimationmethod for CF to improve its robustness
6 Conclusion
The CF is a recently introduced nonparametric model forrejecting mismatchesoutliers in image point matching In
Mathematical Problems in Engineering 11
(a) RMSD = 1386 (b) RMSD = 22707 (c) RMSD = 384 (d) RMSD = 23626Figure 11 Quantitative evaluation on two popular benchmark sets bikes and leuven The ground-truth CPs are presented by red circles thedetected interest points by black plus signs The ground-truth correspondences are displayed as blue dashed lines and the computed putativecorrespondences are shown by a solid green solid line (a) and (c) present some EG PMs that are accepted by CF as matches (b) and (d)present some CF PMs that are accepted by EG as matches 120572
119868= 10597 120572
119877= 21435 times 10
minus5 in (b) and 120572119877
= 58 times 10minus5 in (d)
(a) RMSD = 1631 (b) RMSD = 40098 (c) RMSD = 358 (d) RMSD = 32288Figure 12 Quantitative evaluation on two popular benchmark sets wall and bark ((a) and (c)) The EG PMs that are accepted by CF asmatches ((b) and (d)) the CF PMs that are accepted by EG as matches 120572
119868= 10597 120572
119877= 2029 times 10
minus4 in (b) and 120572119877
= 105 times 10minus4 in (d)
(a) RMSD = 1017 (b) RMSD = 42639 (c) RMSD = 329 (d) RMSD = 17646Figure 13 Quantitative evaluation on two popular benchmark sets graf and boat The ground-truth CPs are presented by red circles and thedetected interest points by black plus signs The ground-truth correspondences are displayed as blue dashed lines and the computed putativecorrespondences are shown by a solid green solid line (a) and (c) present some EG PMs that are accepted by CF as matches (b) and (d)present some CF PMs that are accepted by EG as matches 120572
Figure 14 Quantitative evaluation on two popular benchmark sets trees and ubc The ground-truth correspondences are displayed as bluedashed lines and the computed putative correspondences are shown by a yellow solid line (a) and (c) present some EG PMs that are acceptedbyCF asmatches (b) and (d) present someCFPMs that are accepted by EG asmatches120572
119868= 10597120572
119877= 59times10
minus5 in (b) and120572119877
= 185times10minus4
in (d)
(a) 119878EG119888cap 119878
CF119888
(b) 119878EG119888cap 119878
CF119888
(c) 119878EG119888cap 119878
CF119888
(d) 119878EG119888cap 119878
CF119888
Figure 15 Evaluation on Middlebury Stereo Datasets art and books ((a) and (c)) Some potential EG mismatches that are accepted by CF asmatches ((b) and (d)) some potential CF mismatches that are accepted by EG as matches 120572
119877= 0013 in (a) and (b) 120572
119877= 000018 in (c) and
(d) and 120572119868
= 10597
this study we investigated the principles of the CF andstudied its characteristics by comparing it with the widelyused parametric model epipolar geometry (EG) constraint
It is shown that the CF describes the mapping relation-ships between two CPs and should be able to estimate theposition of the corresponding point Therefore in additionto mismatch rejection a potential application of the CF is to
guide the point matching process by incorporating it into thecorrespondence propagation [39]
In practice putative correspondences are usually cor-rupted by noise and outliers and the number of putativecorrespondences is limited Therefore only an estimationof the CF with a certain accuracy can be estimated andthe actual results will violate the ldquonecessary and sufficient
Mathematical Problems in Engineering 13
(a) 119878EG119888cap SCF119888
(b) 119878EG119888cap 119878
CF119888
(c) 119878EG119888cap 119878
CF119888
(d) 119878EG119888cap 119878
CF119888
Figure 16 Evaluation on Middlebury Stereo Datasets computer and dolls ((a) and (c)) Some potential EG mismatches that are accepted byCF as matches ((b) and (d)) some potential CF mismatches that are accepted by EG as matches 120572
119877= 000018 in (a) and (b) 120572
119877= 000048 in
(c) and (d) and 120572119868
= 10597
(a) 119878EG119888cap 119878
CF119888
(b) 119878EG119888cap 119878
CF119888
(c) 119878EG119888cap 119878
CF119888
(d) 119878EG119888cap 119878
CF119888
Figure 17 Evaluation onMiddlebury Stereo Datasets drumsticks and dwarves ((a) and (c)) Some potential EGmismatches that are acceptedby CF as matches ((b) and (d)) some potential CF mismatches that are accepted by EG as matches 120572
119877= 000078 in (a) and (b) 120572
119877= 00035
in (c) and (d) and 120572119868
= 10597
conditionrdquoThe characteristic is that theCFmodel can reducethe searching space of CPs into an elliptical area and themismatches of the CF are usually near the correct CPs
Conflict of Interests
The authors declare that there is no conflict of interestsregarding the publication of this paper
Acknowledgments
The authors are grateful for valuable advice from DrDibyendu Mukherjee and the anonymous reviewers andwould like to thank the National Natural Science Foundationof China (Grant nos 61075033 61005033 and 61273248)the Natural Science Foundation of Guangdong Province(2014A030313425 and S2011010003348) and theOpenProject
14 Mathematical Problems in Engineering
(a) 119878EG119888cap 119878
CF119888
(b) 119878EG119888cap 119878
CF119888
(c) 119878EG119888cap 119878
CF119888
(d) 119878EG119888cap 119878
CF119888
Figure 18 Evaluation onMiddlebury Stereo Datasets laundry andmoebius ((a) and (c)) Some potential EGmismatches that are accepted byCF asmatches ((b) and (d)) some potential CFmismatches that are accepted by EG asmatches 120572
119877= 000000275 in (a) and (b) 120572
119877= 000035
in (c) and (d) and 120572119868
= 10597
(a) 119878EG119888cap 119878
CF119888
(b) 119878EG119888cap 119878
CF119888
Figure 19 Evaluation on Middlebury Stereo Dataset reindeer (a)Two potential EG mismatches that are accepted by CF as matches(b) two potential CFmismatches that are accepted by EG asmatches120572119877
= 0005 and 120572119868
= 30
Program of the National Laboratory of Pattern Recognition(NLPR) (201001060) for their support
References
[1] J P Monaco A C Bovik and L K Cormack ldquoActive foveateduncalibrated stereovisionrdquo International Journal of ComputerVision vol 85 no 2 pp 192ndash207 2009
[2] X Li and Z Hu ldquoRejecting mismatches by correspondencefunctionrdquo International Journal of Computer Vision vol 89 no1 pp 1ndash17 2010
[3] R Hartley and A Zisserman Multiple View Geometry inComputer Vision Cambridge University Press 2nd edition2003
[4] M Sonka V Hlavac and R Boyle Image Processing Analysisand Machine Vision Thomson-Engineering 3rd edition 2007
[5] G Carneiro and A D Jepson ldquoFlexible spatial configuration oflocal image featuresrdquo IEEE Transactions on Pattern Analysis andMachine Intelligence vol 29 no 12 pp 2089ndash2104 2007
[6] W Aguilar Y Frauel F Escolano M E Martinez-Perez AEspinosa-Romero andM A Lozano ldquoA robust graph transfor-mation matching for non-rigid registrationrdquo Image and VisionComputing vol 27 no 7 pp 897ndash910 2009
[7] L Wang F Duan and K Lv ldquoFisheye-lens-based visual suncompass for perception of spatial orientationrdquo MathematicalProblems in Engineering vol 2012 Article ID 460430 p 15 2012
[8] P Huber Robust Statistics Wiley New York NY USA 1981[9] P J Rousseeuw ldquoLeast median of squares regressionrdquo Journal
of the American Statistical Association vol 79 no 388 pp 871ndash880 1984
[10] P H S Torr and A Zisserman ldquoMLESAC a new robustestimator with application to estimating image geometryrdquoComputer Vision and Image Understanding vol 78 no 1 pp138ndash156 2000
[11] M A Fischler and R C Bolles ldquoRandom sample consensus aparadigm for model fitting with applications to image analysisand automated cartographyrdquo Communications of the ACM vol24 no 6 pp 381ndash395 1981
[12] B J Tordoff and D W Murray ldquoGuided-MLESAC fasterimage transform estimation by using matching priorsrdquo IEEETransactions on Pattern Analysis and Machine Intelligence vol27 no 10 pp 1523ndash1535 2005
Mathematical Problems in Engineering 15
[13] D Sidibe P Montesinos and S Janaqi ldquoMatching localinvariant features with contextual information an experimentalevaluationrdquo Electronic Letters on Computer Vision and ImageAnalysis vol 7 no 1 pp 26ndash39 2008
[14] Q-H Tran T-J Chin G Carneiro M S Brown and D SuterldquoIn defence of RANSAC for outlier rejection in deformableregistrationrdquo Lecture Notes in Computer Science vol 7575 no4 pp 274ndash287 2012
[15] A Goshtasby ldquoTransformation functions for image registra-tionrdquo in Proceedings of the Computer Vision and Pattern Recog-nition (CVPR rsquo11) June 2011
[16] X Li Y Wang J Zhou and X Li ldquoComparative research oncorrespondence functionrdquo in Proceedings of the InternationalConference on Intelligent System Design and Engineering Appli-cation (ISDEA rsquo10) pp 444ndash447 October 2010
[17] X-R Li X-M Li H-L Li and M-Y Cao ldquoRejecting outliersbased on correspondence manifoldrdquo Acta Automatica Sinicavol 35 no 1 pp 17ndash22 2009
[18] D Pizarro and A Bartoli ldquoFeature-based deformable surfacedetection with self-occlusion reasoningrdquo International Journalof Computer Vision vol 97 no 1 pp 54ndash70 2012
[19] J Yu A Eriksson T-J Chin and D Suter ldquoAn adversarialoptimization approach to efficient outlier removalrdquo Journal ofMathematical Imaging and Vision vol 48 no 3 pp 451ndash4662014
[20] A Goshtasby ldquoRegistration of images with geometric distor-tionsrdquo IEEETransactions onGeoscience andRemote Sensing vol26 no 1 pp 60ndash64 1988
[21] F L Bookstein ldquoPrincipal warps thin-plate splines and thedecomposition of deformationsrdquo IEEE Transactions on PatternAnalysis and Machine Intelligence vol 11 no 6 pp 567ndash5851992
[22] B Scholkopf F Steinke and V Blanz ldquoObject correspondenceas a machine learning problemrdquo in Proceedings of the 22ndInternational Conference on Machine Learning (ICML rsquo05) pp776ndash783 2005
[23] L Zagorchev and A Goshtasby ldquoA comparative study oftransformation functions for nonrigid image registrationrdquo IEEETransactions on Image Processing vol 15 no 3 pp 529ndash5382006
[24] J Zhao JMa J Tian J Ma andD Zhang ldquoA robustmethod forvector field learning with application to mismatch removingrdquoin Proceedings of the IEEE Conference on Computer Vision andPattern Recognition (CVPR rsquo11) Providence RI USA June 2011
[25] JMa J Zhao J Tian X Bai and Z Tu ldquoRegularized vector fieldlearning with sparse approximation for mismatch removalrdquoPattern Recognition vol 46 no 12 pp 3519ndash3532 2013
[26] J Ma J Zhao J Tian Z Tu and A L Yuille ldquoRobust estimationof nonrigid transformation for point set registrationrdquo in Pro-ceedings of the 26th IEEE Conference on Computer Vision andPattern Recognition (CVPR rsquo13) pp 2147ndash2154 June 2013
[27] J MaW Qiu J Zhao Y Ma A L Yuille and Z Tu ldquoRobust l2e
estimation of transformation for non-rigid registrationrdquo IEEETransactions on Signal Processing vol 63 no 5 pp 1115ndash11292015
[28] J Ma J Zhao J Tian A L Yuille and Z Tu ldquoRobust pointmatching via vector field consensusrdquo IEEE Transactions onImage Processing vol 23 no 4 pp 1706ndash1721 2014
[29] L Goshen and I Shimshoni ldquoBalanced exploration andexploitation model search for efficient epipolar geometry esti-mationrdquo IEEE Transactions on Pattern Analysis and MachineIntelligence vol 30 no 7 pp 1230ndash1242 2008
[30] O Chum and J Matas ldquoMatching with prosacmdashprogressivesample consensusrdquo in Proceedings of the IEEE Computer SocietyConference on Computer Vision and Pattern Recognition (CVPRrsquo05) vol 1 pp 220ndash226 IEEE June 2005
[31] K Mikolajczyk and C Schmid ldquoA performance evaluation oflocal descriptorsrdquo IEEE Transactions on Pattern Analysis andMachine Intelligence vol 27 no 10 pp 1615ndash1630 2005
[32] D G Lowe ldquoDistinctive image features from scale-invariantkeypointsrdquo International Journal of Computer Vision vol 60 no2 pp 91ndash110 2004
[33] S Gauglitz T Hollerer and M Turk ldquoEvaluation of interestpoint detectors and feature descriptors for visual trackingrdquoInternational Journal of Computer Vision vol 94 no 3 pp 335ndash360 2011
[34] Y Yu K Huang W Chen and T Tan ldquoA novel algorithmfor view and illumination invariant image matchingrdquo IEEETransactions on Image Processing vol 21 no 1 pp 229ndash2402012
[35] K Mikolajczyk T Tuytelaars C Schmid et al ldquoA comparisonof affine region detectorsrdquo International Journal of ComputerVision vol 65 no 1-2 pp 43ndash72 2005
[36] H Hirschmuller and D Scharstein ldquoEvaluation of cost func-tions for stereo matchingrdquo in Proceedings of the IEEE ComputerSociety Conference on Computer Vision and Pattern Recognition(CVPR rsquo07) June 2007
[37] D Scharstein and C Pal ldquoLearning conditional random fieldsfor stereordquo in Proceedings of the IEEE Computer Society Confer-ence on Computer Vision and Pattern Recognition (CVPR rsquo07)IEEE Minneapolis Minn USA June 2007
[38] D Scharstein andR Szeliski ldquoHigh-accuracy stereo depthmapsusing structured lightrdquo in Proceedings of the IEEE ComputerSociety Conference on Computer Vision and Pattern Recognition(CVPR rsquo03) vol 1 pp I-195ndashI-202 IEEE June 2003
[39] H Wang S Yan J Liu X Tang and T S Huang ldquoCorre-spondence propagationwithweak priorsrdquo IEEETransactions onImage Processing vol 18 no 1 pp 140ndash150 2009
Figure 4 Exception of uniqueness in correspondence problem (a) Many points A1015840119894 sub 119868
1015840 share a correspondence point A1
isin 119868 (b) Twopoints A1015840
1and A1015840
2in image 119868
1015840 correspond to one common point A isin 119868 and there are no CPs in image 119868 for the points in (A10158401A10158402) sub 1198681015840
estimating the deformation fields that transform two givenobjects into each other
In these scenarios the focus is to transform an image intoa coordinate systemof another image therefore two images tobe processed are usually called a reference image and a targetimage respectively and only one vector-valued functionis needed to describe the relationship from the referenceimage to the target image A good review can be found in[15 23] for the works on image registration based on thisidea
Letp isin 119868 andp1015840 isin 1198681015840 be two image points119892(p) = 119891(p)minusp
and y = p1015840 minus p The CF for example 119891(p) = p1015840 119868 rarr 1198681015840 can
be expressed by
y = 119892 (p) (3)
where y = p1015840 minus p The mapping y = 119892(p) 119868 rarr 1198681015840
minus 119868
is a special expression of CF and referred to as a vector field[24 25] Zhao et al [24] and Ma et al [25] investigated thevector field learning problem using a Bayesian frameworkand studied its application in outlier rejection Ma et al[26 27] proposed an estimation method 119871
2119864 for the vector
field and investigated its application in point set registrationbetween images of nonrigid object Ma et al [28] investi-gated the point correspondence problem by interpolatingthe vector field and formulated it a maximum a posteriori(MAP) estimation of a Bayesian model with some hiddenvariables
In literature related researches are usually focused onlearning and application of the correspondence function ortransformation function based on a flexible function Weinvestigate the principles and characteristics of it in thiswork
3 Principles of Correspondence Function
Suppose 119868 and 1198681015840 are two images to be registered The CPs
between 119868 and 1198681015840 form a surface in the joint-image space
119868 times 1198681015840 [17] which is illustrated schematically in Figure 1(a)
In this figure images 119868 and 1198681015840 are illustrated as a hori-
zontal axis and a vertical axis respectively And the sur-face is referred to as correspondence manifold (CM) in[17]
In Figure 1(a) (p119860
p1015840119860
) is a pair of CPs between images 119868
and 1198681015840 and 119860 is a point in the joint-image space 119868 times 119868
1015840 formedby CPs (p
119860 p1015840119860
) and the other points on the curves from 119860
to 119864 and from 119865 to 119866 have similar meanings Additionallycurves 119860119861 119862119863 and 119865119866 correspond to the image pointsprojected from the scene surface which can be distinguishedby both cameras of 119868 and 119868
1015840 curves 119861119862 and 119863119864 and points119864 119865 correspond to the scene portions which can only bedistinguished by one camera as illustrated in Figure 4(a)The gap between 119864 and 119865 corresponds to the scene portionbetween C
1and C
2in Figure 4(b) that can be observed by
one camera but not by the other one This sketch map showsthat the relationship between corresponding points can beclassified into three types one-one one-many and many-one This characteristic is independent of the camera modelof the images to be registered and the complexity of scenesurface
To highlight the mapping between CPs we move theorigin to an infinite point and connect the CPs by a dottedline then the sketch map in Figure 1(a) becomes Figure 1(b)It is well known that a function is a mapping that candepict two types of relationshipsmdashmany-one and one-onemdashin mathematics Therefore the relationship between CPs istoo complex to be described by a function
The fundamental idea of CF is to decompose the cor-responding relationships (we name the mapping relations
Mathematical Problems in Engineering 5
between CPs as ldquocorresponding relationshipsrdquo) into twodirectional relationships that can be depicted by two func-tions For convenience these directional relationships arereferred to as ldquocomponent relationsrdquo (CRs) and labeled asCR119868rarr1198681015840 andCR
1198681015840rarr119868
respectivelyTheCR consists of one-onemapping and many-one mapping For example Figures 2(a)and 2(b) are a pair of CRs of Figure 1
The component relation for example CR119868rarr1198681015840 can be
constructed as follows for every point p isin 119868 if there isa unique corresponding point p1015840 isin 119868
1015840 the correspond-ing relationship (p p1015840) should be one member of CR
119868rarr1198681015840
Otherwise if p has more than one corresponding pointp10158401 p1015840
119896119896gt0
isin 1198681015840 we have two choices either ignore all
of the corresponding relationships (p p119894) 1 le 119894 le 119896 or
select any one of the CPs p10158401198940
insert (p p10158401198940
) into CR1198681015840rarr119868
andignore the other corresponding relationships (p p
119894) 1 le 119894 le
119896 119894 = 1198940 Component relation CR
1198681015840rarr119868
can be constructedsimilarly by checking every potential point p1015840 isin 119868
1015840 Forexample in Figures 1 and 2 point p
119860isin 119868 has a unique
corresponding point p1015840119860in 1198681015840 therefore the corresponding
relationship (p119860
p1015840119860
) is in component relation CR119868rarr1198681015840 In
particular the corresponding point of p119863
isin 119868 is also uniquein 1198681015840 Therefore (p
119863 p1015840119863119864
) is one member of the componentrelation CR
119868rarr1198681015840 even though the CPs of p1015840
119863119864are not unique
in image 119868Therefore CR
119868rarr1198681015840 is a one-to-one or many-to-one map-
ping from 119868 to 1198681015840 and can be depicted by a vector-valued
function 119891 119868 rarr 1198681015840 Similarly CR
1198681015840rarr119868
is also a one-to-oneor many-to-one mapping from 119868
1015840 to 119868 and can be depicted bya vector-valued function 119891
1015840 1198681015840
rarr 119868
On the other hand for any given corresponding pointpair (p p1015840) isin 119868 times 119868
1015840 if it is of the one-one type thecorresponding relationship expressed by it can be depictednot only by a function from 119868 to 119868
1015840 but also by a functionfrom 119868
1015840 to 119868 Therefore the corresponding relationships ofthe one-one type can be found in both CR
119868rarr1198681015840 and CR
1198681015840rarr119868
such as the corresponding point pair (p
119860 p1015840119860
) in Figures1(b) 2(a) and 2(b) Otherwise if one given correspondingpoint pair (p p1015840) isin 119868 times 119868
1015840 is of the many-one type phas a unique corresponding image (If (p p1015840) are a pair ofCPs p is called the corresponding image of p1015840 and viceversa) in 119868
1015840 based on the definition of CPs of the many-onetypeTherefore the corresponding relationships of themany-one type can also be depicted by a function mapping andare members of CR
119868rarr1198681015840 like the corresponding point pairs
(p119863
p1015840119863119864
) (p119864119865
p1015840119863119864
) in Figures 1(b) and 2(a) Similarlythe corresponding relationships of one-many can be depictedby a function mapping from 119868
1015840 to 119868 and are members ofCR1198681015840rarr119868
such as the CPs (p119861119862
p1015840119861) (p
119861119862 p1015840119862) in Figures
1(b) and 2(b)In conclusion for a pair of images 119868 and 119868
1015840 to be registeredof a scene the relationships between CPs can be decomposedinto a pair of component relations CR
119868rarr1198681015840 and CR
1198681015840rarr119868
which consist of a number of one-to-one and many-to-onemappingsTheCR
119868rarr1198681015840 andCR
1198681015840rarr119868
can be described by a pair
of vector-valued functions as 119891 119868 rarr 1198681015840 and 119891
1015840 1198681015840
rarr 119868The (119891 119891
1015840) are named as correspondence functions (CFs) in
[2]In image registration 119868 and 119868
1015840 are referred to as areference image and a target image respectively the objectiveis to transform the target image into the coordinate systemof the reference image Therefore only one vector-valuedfunction for example 119891 119868 rarr 119868
1015840 is needed to describethe corresponding relationship and named transformationfunction in [23]
Therefore CF is a pair of functions (119891 1198911015840) demonstrating
that for any one pair of CPs p isin 119868 and p1015840 isin 1198681015840 (p p1015840) are
consistent with at least one of two functions 119891 or 1198911015840 which
can be estimated from a given set of putative correspondencesbased on robust regression method [2] Also for any givenpoint p isin 119868 its corresponding point can be uniquelyestimated by the CF and vice versa (Figure 2)
In practice putative correspondences are usually cor-rupted by noise and outliers and the number of putativecorrespondences is limited Therefore only an estimation ofCF with a certain accuracy can be estimated and the actualresults may be not strictly consistent with the ldquonecessary andsufficient conditionrdquo
Therefore the characteristic is that CF model can reducethe searching space of CPs into an elliptical area and the mis-matches of CF are usually near the correct CPs (Section 5)
4 Learning of CF and Its Application inMismatch Rejection
Based on the discussions in Section 3 for a pair of images1198681and 1198682of a scene the relationship of some corresponding
points between 1198681and 119868
2can be decomposed into a pair
of component relations (CRs) CR119868rarr1198681015840 and CR
1198681015840rarr119868
A CRconsists of one-one mapping and many-one mapping
Therefore CR119868rarr1198681015840 and CR
1198681015840rarr119868
can be described by twovector-valued functions 119891 119868 rarr 119868
1015840 and 1198911015840
1198681015840
rarr 119868
119891 (p) = p1015840
1198911015840(p1015840) = p
(4)
where p isin 119868 p1015840 isin 1198681015840 and (119891 119891
putative correspondences between 1198681and 1198682 If we treat y =
p1015840 as an output of a system and x = p as an input theCF 119891 can be estimated from 119878 = (x y) = (p p1015840) p isin
119868 p1015840 isin 1198681015840 using a vector-valued regression method for
example vector field consensus (VFC) method [24 28]and IECF (Iteratively Estimate Correspondence Function)algorithm [2] The fundamental ideas of VFC and IECF areas follows approximately represent the unknown CF 119891 as aweighted sumof kernel functions (2) and estimate theweights
6 Mathematical Problems in Engineering
iteratively by gradually reducing the undue effects fromoutliers The objective of this work is to investigate thefundamental principles and characteristics of CF Thereforewe do not compare the various implementation of CF anddo experiments by IECF The CF 119891
1015840 can be estimatedsimilarly
Theoretically the correctness of a pair of putative corre-spondences (p p1015840) can be determined by checking whetherit is consistent with one of the estimated correspondencefunctions 119891 or 119891
1015840 However due to the influence of imagenoise the observed corresponding points usually are notstrictly consistent with the estimated CFs 119891 and 119891
1015840 and atolerance parameter 120572
119868is needed to determine whether they
are consistent It is shown that the consistency of (p p1015840) with119891 can be determined by
119888119891
(p p1015840) = (119891 (p) minus p1015840) Σminus1
(119891 (p) minus p1015840)119879
(5)
where the superscript119879 represents a transpose of a vector andΣ is the covariance matrix of random variable 119891(p) minus p1015840 andcan be estimated using IECF algorithm [2]The consistency of(p p1015840)with119891
1015840 ismeasured similarly and denoted by 1198881198911015840(p p1015840)
Then some mismatches can be rejected by the constraints119888119891(p p1015840) gt 120572
In this section we experimentally investigate the characteris-tics of theCF by comparing it with thewidely used parametricmodel epipolar geometry (EG) ((1) Figure 3) in the contextof rejecting mismatches
51 Experimental Configuration The mismatch-rejectingmethods based onCF and EG are implemented by algorithmsICF (identifying point correspondences by correspondencefunction) [2] and RANSAC (Random Sample Consensus)[11 29 30] respectively in this study Since the coordinatesof the observed putative CPs are usually corrupted by noisethey are not strictly consistent with the CF and EG equationsTherefore two tolerance parameters are needed for thecomputation we name the two parameters 120572
119877in EG +
RANSAC and 120572119868in CF + ICF
In evaluating the performance of local descriptors andpoints matching models there are several schemes forexample recall precision [31] and ROC curve [14] Howeverthe objective of this workrsquos experiments is to show thecharacteristic of fail-detected mismatches of CF Thereforeto make a fair comparison we adjusted the values of 120572
119877and
120572119868such that a same number of putative correspondences are
identified as possible correct matches in every experiment
and the RANSAC are implemented without a limit oniteration number
Suppose 119868 and 1198681015840 are two images of a scene to be matched
and suppose 119878 is a set of putative correspondences betweenthem A mismatch-rejecting method will partition 119878 intotwo subsets 119878
119888and 119878
119898 as possible correct correspondences
and possible mismatches For readability we denote the twosubsets by 119878
EG119888
and 119878EG119898
in the method based on EG constraintand 119878
CF119888
and 119878CF119898
in the method based on CF Thus set 119878 canbe divided into four subsets 119878
EG119888
cap 119878CF119888 119878
EG119888
cap 119878CF119888 119878
EG119888
cap 119878CF119888
and 119878EG119888
cap 119878CF119888 where 119878
EG119888
= 119878 minus 119878EG119888
= 119878EG119898
and 119878CF119888
=
119878 minus 119878CF119888
= 119878CF119898
The meanings of the above four subsets are asfollows
(1) 119878EG119888
cap 119878CF119888
= 119878EG119898
cap 119878CF119898 the putative correspondences
that are regarded as possible correct matches by themethods based on EG and CF
(2) 119878EG119888
cap 119878CF119888
= 119878EG119898
cap 119878CF119898 the putative correspondences
that are regarded as possible mismatches by EGmethod and possible correct matches by CF
(3) 119878EG119888
cap 119878CF119888
= 119878EG119898
cap 119878CF119898 the putative correspondences
that are regarded as possible correct matches by theEG method and possible mismatches by CF
(4) 119878EG119888
cap 119878CF119888
= 119878EG119898
cap 119878CF119898 the putative correspondences
that are regarded as possible mismatches by themethods based on EG and CF
On subsets 119878EG119888
cap119878CF119888
and 119878EG119888
cap119878CF119888 the twomethods based on
CF and EG constraint are consistentTherefore it is sufficientto focus on 119878
EG119888
cap 119878CFc and 119878
EG119888
cap 119878CF119888
to compare the CFmodeland the EG constraint
For convenience in comparing the results of EG and CFwe manually removed some of the correct and near-correctputative correspondences from 119878
EG119888
cap 119878CF119888
and 119878EG119888
cap 119878CF119888
in some experiments and the two sets composed of theremaining putative correspondences are denoted by 119879(119878
EG119888
cap
119878CF119888
) and 119879(119878EG119888
cap 119878CF119888
) respectively
(i) 119879(119878EG119888
cap 119878CF119888
) the typical putative correspondencesthat are classified as possible correct matches by EGand as possible mismatches by CF
(ii) 119879(119878EG119888
cap 119878CF119888
) the typical putative correspondencesthat are classified as possible mismatches by EG andas possible correct correspondences by CF
52 Experiments
521 Main Results One of the used image pairs is pre-sented in Figure 5 There are 1278 and 1207 SIFT featurepoints extracted from two images respectively (the featurepoints are detected by the multiscale DoG (Difference ofGaussian) scheme and described by the orientation his-togram technique [32]) and 448 putative correspondences
Mathematical Problems in Engineering 7
(a) 119878EG119888cap 119878
CF119888
(b) 119878EG119888cap 119878
CF119888
(c) 119879(119878EG119888cap 119878
CF119888) (d) 119879(119878EG
119888cap 119878
CF119888)
Figure 5 An image pair of a toy bear (a) 26 EG potential mismatches (PMs) that are accepted by CF as matches (b) 26 CF PMs that areaccepted by EG as matches (c) some typical EG PMs that are accepted by CF (d) some typical CF PMs that are accepted by EG 120572
119868= 10597
and 120572119877
= 000019
(a) 119878EG119888cap 119878
CF119888
(b) 119878EG119888cap 119878
CF119888
(c) 119878EG119888cap 119878
CF119888
(d) 119878EG119888cap 119878
CF119888
Figure 6 An image pair of a corridor (a) 25 EG PMs that are accepted by CF as matches (b) 25 CF PMs that are accepted by EG as matches(c) 31 EG PMs that are accepted by CF as matches (d) 31 CF PMs that are accepted by EG as matches In (a)-(b) 120572
119868= 5991 and 120572
119877= 000048
and in (c)-(d) 120572119868
= 110 and 120572119877
= 005
computed from the feature points by the NNDR (Near-est Neighbor and Distance Ratio) method [32] Due tothe ambiguity of local information some of the putativecorrespondences are incorrect and need to be rejected Inthis experiment the two methods based on EG and CFare inconsistent on 52 putative correspondences (Figures5(a) and 5(b)) For example for every putative correspon-dence (p p1015840) in Figure 5(a) the CF method identified it as
a possible correct match but the EG method identified it asa possible mismatch Therefore based on the definition of119878EG119888
cap 119878CF119888
and 119878EG119888
cap 119878CF119888 we can regard 119878
EG119888
cap 119878CF119888
as a setof potential mismatches that the EG constraint fails to detectand 119878
EG119888
cap 119878CF119888
as a set of potential mismatches that the CFmodel fails to reject More results are presented in Figures 67 and 8
8 Mathematical Problems in Engineering
(a) 119878EG119888cap 119878
CF119888
(b) 119878EG119888cap 119878
CF119888
(c) 119879(119878EG119888cap 119878
CF119888) (d) 119879(119878EG
119888cap 119878
CF119888)
Figure 7 An image pair of Baxia (a) Some EG PMs that are accepted by CF asmatches (b) some CF PMs that are accepted by EG asmatches(c) some typical EG PMs that are accepted by CF (d) some typical CF PMs that are accepted by EG 120572
119868= 10597 and 120572
119877= 005
(a) 119878EG119888cap 119878
CF119888
(b) 119878EG119888cap 119878
CF119888
(c) 119879(119878EGc cap 119878CF119888) (d) 119879(119878EG
119888cap 119878
CF119888)
Figure 8 An image pair of temple Fayu (a) 204 EG PMs that are accepted by CF as matches (b) 204 CF PMs that are accepted by EG asmatches (c) some typical EG PMs that are accepted by CF (d) some typical CF PMs that are accepted by EG 120572
119868= 31 and 120572
119877= 00001515
(for visibility only 100 randomly chosen PMs are presented in (a) and (b))
522 Comparing the CF and EG Constraint by Analyzing theUndetected Mismatches For ease of explanation we intro-duce the following two definitions type-I mismatches andtype-II mismatches Suppose (p p1015840) is a mismatch If thereexists a correct corresponding point pair (pp1015840) satisfying thatthe points p and p1015840 are in the neighborhood of p and p1015840respectively then (p p1015840) is defined as a mismatch of type-IOtherwise we call (p p1015840) a type-II mismatch For example
the mismatches in Figure 7(d) and (p p10158403) in Figure 3(b)
belong to type-II and (p p10158402) in Figure 3(b) belong to type-
I By the EG constraint we can only determine whether aputative correspondence is in a band around the epipolar line(Figures 3(b) 9(b) and 9(d)) but not whether it is close to thecorrect one Therefore the undetected mismatches usuallybelong to type-II in the methods based on the EG constraint(Figures 3 5(b) 5(d) 6(b) 6(d) 7(b) 7(d) 8(b) and 8(d))
Mathematical Problems in Engineering 9
l1
l2
p1
l3p3 p2
l4p4
l9984001 p9984001
l9984002 p9984002
l9984003 p9984003
l9984004p9984004
(a)
l1
l3
p1
p3
l2 p2
l9984001p9984001
l9984002p9984002
l9984003p9984003
(b)
l2p2
l1 p1l3p3
l9984001p9984001
l9984002p9984002
l9984003 p9984003
(c)
l1
l6
p1
p6
l3
l4 p4
p3
l2
l5
p2
p5
l9984001p9984001
l9984002 p9984002
l9984003
l9984006
p9984003
p9984006
l9984004 p9984004
l9984005p9984005
(d)
Figure 9 Some PMs and their corresponding epipolar lines The EG model cannot detect the mismatches that are near the epipolar line butfar from the correct CPs The undetected mismatches by CF are usually near both the correct CPs and the epipolar line (a) and (c) presentthe EG PMs that are accepted by CF as matches in the experiments of Figures 7(c) and 8(c) respectively (b) and (d) present the CF PMs thatare accepted by EG as matches in the experiments of Figures 7(d) and 8(d) respectively In these experiments (p
119894 p1015840119894) is a pair of putative
CPs and l119894is the epipolar line corresponding to p1015840
119894and l1015840119894is the epipolar line corresponding to p
119894
The CF constraint determines the mapping relationshipsbetween two CPs and directly estimates the position of thecorresponding point Therefore by the mismatch-rejectingmethods based on CF we can restrict the error bound ofthe putative correspondences by a tolerance parameter 120572
119868
the undetected mismatches by CF usually belong to type-I To show the difference between the CF model and EGconstraints a typical result is presented in Figure 6 in whichwe relax the tolerance parameters 120572
119868and 120572119877to ensure the two
sets 119878EG119888
cap 119878CF119888
and 119878EG119888
cap 119878CF119888
change at equal speedHowever themismatches of type-I and type-II are usually
dramatically different from each other in usefulness andundue influence From the mismatches of type-I we cansearch for the correct CPs by including more informationOn the contrary other than ruining the traditional estimationmethods and distorting the final application results (egimage mosaicing and three-dimensional reconstruction) themismatches of the type-II provide less useful information
53 Further Evaluation on Some Popular Benchmark DataThe characteristics of the CF are also investigated on twoseries of popular benchmark data sets widely used to compareand evaluate related algorithms [33 34]
To evaluate the two schemes quantitatively this workconducted experiments on a series of image sets with ground-truth correspondences [31 35] The images were capturedunder special consideration to enable the encoding of theground-truth correspondences with homographies between
the reference image and other images The ground-truthhomographies are computed based on the following stepsfirst a set of putative correspondences are manually selectedand an initial estimation of the homography is computedbased on the selected putative correspondences second theimages are approximately aligned with the rough estimationof homography third reliable interest points are detectedand matched automatically and an accurate homography iscomputed from them
The experimental results are presented in Figures 11 12 13and 14 In every experiment the upper image and the lowerimage are called the reference image and the target imagerespectively In addition to the computed putative correspon-dences we also present the ground-truth CPs and ground-truth correspondence relationships in these experimentsTheground-truth CPs in the target images are computed for all ofthe marked interest points in reference images based on theground-truth homographies and denoted by red circles Thedetected interest points are denoted by black plus signs Theground-truth correspondence relationships are displayed asblue dashed lines and the computed putative correspondencerelationships are shown by a solid line with red or green forvisibility
Suppose that 119878 = (p1 p10158401) (p
119899 p1015840119899) are a set of
putative correspondences and 119878 = (p1 p10158401) (p
119899 p1015840119899) are
the ground-truth correspondences of 119878 We quantitativelyevaluate the quality of 119878 by the root-mean-square deviation(RMSD) RMSD(119878) = radicsum
Figure 10The possible correct correspondences identified by CF in the experiment of Figure 8 are presented based on depth (a) 34 putativecorrespondences (PCs) over the first crossbeam (b) 324 PCs over the second crossbeam (c) 214 PCs over the third crossbeam (d) 20 PCsaround the fourth crossbeam (e) 2 PCs on the wall (for clarity only 50 randomly chosen PCs are presented in (b) and (c))
The second series of evaluation data are the Middle-bury Stereo Datasets [36 37] (httpvisionmiddleburyedustereodata) These data sets were created using an auto-mated version of structured-lighting technique [38] Eachdataset consists of seven rectified views This work used thefirst and the fifth views for evaluation The experimentalresults are presented in Figures 15 16 17 18 and 19
54 CF and EG in Practice
541 Necessity and Sufficiency As analyzed the EG con-straint is a necessary but insufficient condition in the corre-spondence problem and theCFmodel describes themappingbetween CPs directly However in applications some correctCPsmay be inconsistent with the estimated EGmodel andorthe estimated CF model there may be some mismatchesthat are ldquoconsistentrdquo with the estimated CF model there alsomay be some of the putative correspondences which areldquoconsistentrdquo with the estimated CF but ldquoinconsistentrdquo withthe estimated EG for example the experimental results inFigures 5(a) 5(b) 7(a) 7(b) 8(a) and 8(b) The reasons forthe above results are as follows
(i) The coordinates of the obtained putative CPs areusually corrupted by noise
(ii) There are usually some errors in the estimation of theCF and EG
(iii) As pointed out in the first paragraph of Section 5a preset tolerance parameter is needed to define theldquoconsistentrdquo and ldquoinconsistentrdquo in the applicationsof CF and EG However the noise in the obtainedputative CPs is random
542 Characteristics and Limitations of the CF Model andEG Model In the two models there are no restrictionson the depth range of the imaged scene However theEG is a global describing model and the CF is a localdescribing model In the EG model any pair of putativeCPs can influence the overall model estimation whereas inthe CF model a pair of putative CPs only influence themodel estimation within a local area in their vicinity Theshortcoming of the global model EG is that the overallmodel estimation can be degraded or ruined by PCs withconsiderable noise or mismatches in any area of the imagesThe shortcoming of the local model CF is that a pair of goodCPs cannot improve the estimation of CF outside of a localarea around itself Thus the uneven distribution of putativecorrespondences with different quality results in the fact thatthe estimated CF may be of high quality in some areas butof low quality in others for example in the experiment ofFigure 10 the qualities of the CF estimations over the second-nearest-to-the-cameras crossbeam and the third-nearest-to-the-cameras crossbeam are better than estimation over thefirst-nearest-to-the-cameras crossbeam The ldquolocal areardquo isdetermined by the CF estimation method and its chosenparameters for example in the CF estimation method SP(SVM) [2] the local area is determined by the chosen kerneland the scale parameter The above limitations of EG and CFcan be alleviated by investigating robust estimation methodsfor example the RANSAC M-estimators [8] LMedS (LeastMedian of Squares) [9] and MLESAC [10 12] for EGIn future work we will investigate further the estimationmethod for CF to improve its robustness
6 Conclusion
The CF is a recently introduced nonparametric model forrejecting mismatchesoutliers in image point matching In
Mathematical Problems in Engineering 11
(a) RMSD = 1386 (b) RMSD = 22707 (c) RMSD = 384 (d) RMSD = 23626Figure 11 Quantitative evaluation on two popular benchmark sets bikes and leuven The ground-truth CPs are presented by red circles thedetected interest points by black plus signs The ground-truth correspondences are displayed as blue dashed lines and the computed putativecorrespondences are shown by a solid green solid line (a) and (c) present some EG PMs that are accepted by CF as matches (b) and (d)present some CF PMs that are accepted by EG as matches 120572
119868= 10597 120572
119877= 21435 times 10
minus5 in (b) and 120572119877
= 58 times 10minus5 in (d)
(a) RMSD = 1631 (b) RMSD = 40098 (c) RMSD = 358 (d) RMSD = 32288Figure 12 Quantitative evaluation on two popular benchmark sets wall and bark ((a) and (c)) The EG PMs that are accepted by CF asmatches ((b) and (d)) the CF PMs that are accepted by EG as matches 120572
119868= 10597 120572
119877= 2029 times 10
minus4 in (b) and 120572119877
= 105 times 10minus4 in (d)
(a) RMSD = 1017 (b) RMSD = 42639 (c) RMSD = 329 (d) RMSD = 17646Figure 13 Quantitative evaluation on two popular benchmark sets graf and boat The ground-truth CPs are presented by red circles and thedetected interest points by black plus signs The ground-truth correspondences are displayed as blue dashed lines and the computed putativecorrespondences are shown by a solid green solid line (a) and (c) present some EG PMs that are accepted by CF as matches (b) and (d)present some CF PMs that are accepted by EG as matches 120572
Figure 14 Quantitative evaluation on two popular benchmark sets trees and ubc The ground-truth correspondences are displayed as bluedashed lines and the computed putative correspondences are shown by a yellow solid line (a) and (c) present some EG PMs that are acceptedbyCF asmatches (b) and (d) present someCFPMs that are accepted by EG asmatches120572
119868= 10597120572
119877= 59times10
minus5 in (b) and120572119877
= 185times10minus4
in (d)
(a) 119878EG119888cap 119878
CF119888
(b) 119878EG119888cap 119878
CF119888
(c) 119878EG119888cap 119878
CF119888
(d) 119878EG119888cap 119878
CF119888
Figure 15 Evaluation on Middlebury Stereo Datasets art and books ((a) and (c)) Some potential EG mismatches that are accepted by CF asmatches ((b) and (d)) some potential CF mismatches that are accepted by EG as matches 120572
119877= 0013 in (a) and (b) 120572
119877= 000018 in (c) and
(d) and 120572119868
= 10597
this study we investigated the principles of the CF andstudied its characteristics by comparing it with the widelyused parametric model epipolar geometry (EG) constraint
It is shown that the CF describes the mapping relation-ships between two CPs and should be able to estimate theposition of the corresponding point Therefore in additionto mismatch rejection a potential application of the CF is to
guide the point matching process by incorporating it into thecorrespondence propagation [39]
In practice putative correspondences are usually cor-rupted by noise and outliers and the number of putativecorrespondences is limited Therefore only an estimationof the CF with a certain accuracy can be estimated andthe actual results will violate the ldquonecessary and sufficient
Mathematical Problems in Engineering 13
(a) 119878EG119888cap SCF119888
(b) 119878EG119888cap 119878
CF119888
(c) 119878EG119888cap 119878
CF119888
(d) 119878EG119888cap 119878
CF119888
Figure 16 Evaluation on Middlebury Stereo Datasets computer and dolls ((a) and (c)) Some potential EG mismatches that are accepted byCF as matches ((b) and (d)) some potential CF mismatches that are accepted by EG as matches 120572
119877= 000018 in (a) and (b) 120572
119877= 000048 in
(c) and (d) and 120572119868
= 10597
(a) 119878EG119888cap 119878
CF119888
(b) 119878EG119888cap 119878
CF119888
(c) 119878EG119888cap 119878
CF119888
(d) 119878EG119888cap 119878
CF119888
Figure 17 Evaluation onMiddlebury Stereo Datasets drumsticks and dwarves ((a) and (c)) Some potential EGmismatches that are acceptedby CF as matches ((b) and (d)) some potential CF mismatches that are accepted by EG as matches 120572
119877= 000078 in (a) and (b) 120572
119877= 00035
in (c) and (d) and 120572119868
= 10597
conditionrdquoThe characteristic is that theCFmodel can reducethe searching space of CPs into an elliptical area and themismatches of the CF are usually near the correct CPs
Conflict of Interests
The authors declare that there is no conflict of interestsregarding the publication of this paper
Acknowledgments
The authors are grateful for valuable advice from DrDibyendu Mukherjee and the anonymous reviewers andwould like to thank the National Natural Science Foundationof China (Grant nos 61075033 61005033 and 61273248)the Natural Science Foundation of Guangdong Province(2014A030313425 and S2011010003348) and theOpenProject
14 Mathematical Problems in Engineering
(a) 119878EG119888cap 119878
CF119888
(b) 119878EG119888cap 119878
CF119888
(c) 119878EG119888cap 119878
CF119888
(d) 119878EG119888cap 119878
CF119888
Figure 18 Evaluation onMiddlebury Stereo Datasets laundry andmoebius ((a) and (c)) Some potential EGmismatches that are accepted byCF asmatches ((b) and (d)) some potential CFmismatches that are accepted by EG asmatches 120572
119877= 000000275 in (a) and (b) 120572
119877= 000035
in (c) and (d) and 120572119868
= 10597
(a) 119878EG119888cap 119878
CF119888
(b) 119878EG119888cap 119878
CF119888
Figure 19 Evaluation on Middlebury Stereo Dataset reindeer (a)Two potential EG mismatches that are accepted by CF as matches(b) two potential CFmismatches that are accepted by EG asmatches120572119877
= 0005 and 120572119868
= 30
Program of the National Laboratory of Pattern Recognition(NLPR) (201001060) for their support
References
[1] J P Monaco A C Bovik and L K Cormack ldquoActive foveateduncalibrated stereovisionrdquo International Journal of ComputerVision vol 85 no 2 pp 192ndash207 2009
[2] X Li and Z Hu ldquoRejecting mismatches by correspondencefunctionrdquo International Journal of Computer Vision vol 89 no1 pp 1ndash17 2010
[3] R Hartley and A Zisserman Multiple View Geometry inComputer Vision Cambridge University Press 2nd edition2003
[4] M Sonka V Hlavac and R Boyle Image Processing Analysisand Machine Vision Thomson-Engineering 3rd edition 2007
[5] G Carneiro and A D Jepson ldquoFlexible spatial configuration oflocal image featuresrdquo IEEE Transactions on Pattern Analysis andMachine Intelligence vol 29 no 12 pp 2089ndash2104 2007
[6] W Aguilar Y Frauel F Escolano M E Martinez-Perez AEspinosa-Romero andM A Lozano ldquoA robust graph transfor-mation matching for non-rigid registrationrdquo Image and VisionComputing vol 27 no 7 pp 897ndash910 2009
[7] L Wang F Duan and K Lv ldquoFisheye-lens-based visual suncompass for perception of spatial orientationrdquo MathematicalProblems in Engineering vol 2012 Article ID 460430 p 15 2012
[8] P Huber Robust Statistics Wiley New York NY USA 1981[9] P J Rousseeuw ldquoLeast median of squares regressionrdquo Journal
of the American Statistical Association vol 79 no 388 pp 871ndash880 1984
[10] P H S Torr and A Zisserman ldquoMLESAC a new robustestimator with application to estimating image geometryrdquoComputer Vision and Image Understanding vol 78 no 1 pp138ndash156 2000
[11] M A Fischler and R C Bolles ldquoRandom sample consensus aparadigm for model fitting with applications to image analysisand automated cartographyrdquo Communications of the ACM vol24 no 6 pp 381ndash395 1981
[12] B J Tordoff and D W Murray ldquoGuided-MLESAC fasterimage transform estimation by using matching priorsrdquo IEEETransactions on Pattern Analysis and Machine Intelligence vol27 no 10 pp 1523ndash1535 2005
Mathematical Problems in Engineering 15
[13] D Sidibe P Montesinos and S Janaqi ldquoMatching localinvariant features with contextual information an experimentalevaluationrdquo Electronic Letters on Computer Vision and ImageAnalysis vol 7 no 1 pp 26ndash39 2008
[14] Q-H Tran T-J Chin G Carneiro M S Brown and D SuterldquoIn defence of RANSAC for outlier rejection in deformableregistrationrdquo Lecture Notes in Computer Science vol 7575 no4 pp 274ndash287 2012
[15] A Goshtasby ldquoTransformation functions for image registra-tionrdquo in Proceedings of the Computer Vision and Pattern Recog-nition (CVPR rsquo11) June 2011
[16] X Li Y Wang J Zhou and X Li ldquoComparative research oncorrespondence functionrdquo in Proceedings of the InternationalConference on Intelligent System Design and Engineering Appli-cation (ISDEA rsquo10) pp 444ndash447 October 2010
[17] X-R Li X-M Li H-L Li and M-Y Cao ldquoRejecting outliersbased on correspondence manifoldrdquo Acta Automatica Sinicavol 35 no 1 pp 17ndash22 2009
[18] D Pizarro and A Bartoli ldquoFeature-based deformable surfacedetection with self-occlusion reasoningrdquo International Journalof Computer Vision vol 97 no 1 pp 54ndash70 2012
[19] J Yu A Eriksson T-J Chin and D Suter ldquoAn adversarialoptimization approach to efficient outlier removalrdquo Journal ofMathematical Imaging and Vision vol 48 no 3 pp 451ndash4662014
[20] A Goshtasby ldquoRegistration of images with geometric distor-tionsrdquo IEEETransactions onGeoscience andRemote Sensing vol26 no 1 pp 60ndash64 1988
[21] F L Bookstein ldquoPrincipal warps thin-plate splines and thedecomposition of deformationsrdquo IEEE Transactions on PatternAnalysis and Machine Intelligence vol 11 no 6 pp 567ndash5851992
[22] B Scholkopf F Steinke and V Blanz ldquoObject correspondenceas a machine learning problemrdquo in Proceedings of the 22ndInternational Conference on Machine Learning (ICML rsquo05) pp776ndash783 2005
[23] L Zagorchev and A Goshtasby ldquoA comparative study oftransformation functions for nonrigid image registrationrdquo IEEETransactions on Image Processing vol 15 no 3 pp 529ndash5382006
[24] J Zhao JMa J Tian J Ma andD Zhang ldquoA robustmethod forvector field learning with application to mismatch removingrdquoin Proceedings of the IEEE Conference on Computer Vision andPattern Recognition (CVPR rsquo11) Providence RI USA June 2011
[25] JMa J Zhao J Tian X Bai and Z Tu ldquoRegularized vector fieldlearning with sparse approximation for mismatch removalrdquoPattern Recognition vol 46 no 12 pp 3519ndash3532 2013
[26] J Ma J Zhao J Tian Z Tu and A L Yuille ldquoRobust estimationof nonrigid transformation for point set registrationrdquo in Pro-ceedings of the 26th IEEE Conference on Computer Vision andPattern Recognition (CVPR rsquo13) pp 2147ndash2154 June 2013
[27] J MaW Qiu J Zhao Y Ma A L Yuille and Z Tu ldquoRobust l2e
estimation of transformation for non-rigid registrationrdquo IEEETransactions on Signal Processing vol 63 no 5 pp 1115ndash11292015
[28] J Ma J Zhao J Tian A L Yuille and Z Tu ldquoRobust pointmatching via vector field consensusrdquo IEEE Transactions onImage Processing vol 23 no 4 pp 1706ndash1721 2014
[29] L Goshen and I Shimshoni ldquoBalanced exploration andexploitation model search for efficient epipolar geometry esti-mationrdquo IEEE Transactions on Pattern Analysis and MachineIntelligence vol 30 no 7 pp 1230ndash1242 2008
[30] O Chum and J Matas ldquoMatching with prosacmdashprogressivesample consensusrdquo in Proceedings of the IEEE Computer SocietyConference on Computer Vision and Pattern Recognition (CVPRrsquo05) vol 1 pp 220ndash226 IEEE June 2005
[31] K Mikolajczyk and C Schmid ldquoA performance evaluation oflocal descriptorsrdquo IEEE Transactions on Pattern Analysis andMachine Intelligence vol 27 no 10 pp 1615ndash1630 2005
[32] D G Lowe ldquoDistinctive image features from scale-invariantkeypointsrdquo International Journal of Computer Vision vol 60 no2 pp 91ndash110 2004
[33] S Gauglitz T Hollerer and M Turk ldquoEvaluation of interestpoint detectors and feature descriptors for visual trackingrdquoInternational Journal of Computer Vision vol 94 no 3 pp 335ndash360 2011
[34] Y Yu K Huang W Chen and T Tan ldquoA novel algorithmfor view and illumination invariant image matchingrdquo IEEETransactions on Image Processing vol 21 no 1 pp 229ndash2402012
[35] K Mikolajczyk T Tuytelaars C Schmid et al ldquoA comparisonof affine region detectorsrdquo International Journal of ComputerVision vol 65 no 1-2 pp 43ndash72 2005
[36] H Hirschmuller and D Scharstein ldquoEvaluation of cost func-tions for stereo matchingrdquo in Proceedings of the IEEE ComputerSociety Conference on Computer Vision and Pattern Recognition(CVPR rsquo07) June 2007
[37] D Scharstein and C Pal ldquoLearning conditional random fieldsfor stereordquo in Proceedings of the IEEE Computer Society Confer-ence on Computer Vision and Pattern Recognition (CVPR rsquo07)IEEE Minneapolis Minn USA June 2007
[38] D Scharstein andR Szeliski ldquoHigh-accuracy stereo depthmapsusing structured lightrdquo in Proceedings of the IEEE ComputerSociety Conference on Computer Vision and Pattern Recognition(CVPR rsquo03) vol 1 pp I-195ndashI-202 IEEE June 2003
[39] H Wang S Yan J Liu X Tang and T S Huang ldquoCorre-spondence propagationwithweak priorsrdquo IEEETransactions onImage Processing vol 18 no 1 pp 140ndash150 2009
between CPs as ldquocorresponding relationshipsrdquo) into twodirectional relationships that can be depicted by two func-tions For convenience these directional relationships arereferred to as ldquocomponent relationsrdquo (CRs) and labeled asCR119868rarr1198681015840 andCR
1198681015840rarr119868
respectivelyTheCR consists of one-onemapping and many-one mapping For example Figures 2(a)and 2(b) are a pair of CRs of Figure 1
The component relation for example CR119868rarr1198681015840 can be
constructed as follows for every point p isin 119868 if there isa unique corresponding point p1015840 isin 119868
1015840 the correspond-ing relationship (p p1015840) should be one member of CR
119868rarr1198681015840
Otherwise if p has more than one corresponding pointp10158401 p1015840
119896119896gt0
isin 1198681015840 we have two choices either ignore all
of the corresponding relationships (p p119894) 1 le 119894 le 119896 or
select any one of the CPs p10158401198940
insert (p p10158401198940
) into CR1198681015840rarr119868
andignore the other corresponding relationships (p p
119894) 1 le 119894 le
119896 119894 = 1198940 Component relation CR
1198681015840rarr119868
can be constructedsimilarly by checking every potential point p1015840 isin 119868
1015840 Forexample in Figures 1 and 2 point p
119860isin 119868 has a unique
corresponding point p1015840119860in 1198681015840 therefore the corresponding
relationship (p119860
p1015840119860
) is in component relation CR119868rarr1198681015840 In
particular the corresponding point of p119863
isin 119868 is also uniquein 1198681015840 Therefore (p
119863 p1015840119863119864
) is one member of the componentrelation CR
119868rarr1198681015840 even though the CPs of p1015840
119863119864are not unique
in image 119868Therefore CR
119868rarr1198681015840 is a one-to-one or many-to-one map-
ping from 119868 to 1198681015840 and can be depicted by a vector-valued
function 119891 119868 rarr 1198681015840 Similarly CR
1198681015840rarr119868
is also a one-to-oneor many-to-one mapping from 119868
1015840 to 119868 and can be depicted bya vector-valued function 119891
1015840 1198681015840
rarr 119868
On the other hand for any given corresponding pointpair (p p1015840) isin 119868 times 119868
1015840 if it is of the one-one type thecorresponding relationship expressed by it can be depictednot only by a function from 119868 to 119868
1015840 but also by a functionfrom 119868
1015840 to 119868 Therefore the corresponding relationships ofthe one-one type can be found in both CR
119868rarr1198681015840 and CR
1198681015840rarr119868
such as the corresponding point pair (p
119860 p1015840119860
) in Figures1(b) 2(a) and 2(b) Otherwise if one given correspondingpoint pair (p p1015840) isin 119868 times 119868
1015840 is of the many-one type phas a unique corresponding image (If (p p1015840) are a pair ofCPs p is called the corresponding image of p1015840 and viceversa) in 119868
1015840 based on the definition of CPs of the many-onetypeTherefore the corresponding relationships of themany-one type can also be depicted by a function mapping andare members of CR
119868rarr1198681015840 like the corresponding point pairs
(p119863
p1015840119863119864
) (p119864119865
p1015840119863119864
) in Figures 1(b) and 2(a) Similarlythe corresponding relationships of one-many can be depictedby a function mapping from 119868
1015840 to 119868 and are members ofCR1198681015840rarr119868
such as the CPs (p119861119862
p1015840119861) (p
119861119862 p1015840119862) in Figures
1(b) and 2(b)In conclusion for a pair of images 119868 and 119868
1015840 to be registeredof a scene the relationships between CPs can be decomposedinto a pair of component relations CR
119868rarr1198681015840 and CR
1198681015840rarr119868
which consist of a number of one-to-one and many-to-onemappingsTheCR
119868rarr1198681015840 andCR
1198681015840rarr119868
can be described by a pair
of vector-valued functions as 119891 119868 rarr 1198681015840 and 119891
1015840 1198681015840
rarr 119868The (119891 119891
1015840) are named as correspondence functions (CFs) in
[2]In image registration 119868 and 119868
1015840 are referred to as areference image and a target image respectively the objectiveis to transform the target image into the coordinate systemof the reference image Therefore only one vector-valuedfunction for example 119891 119868 rarr 119868
1015840 is needed to describethe corresponding relationship and named transformationfunction in [23]
Therefore CF is a pair of functions (119891 1198911015840) demonstrating
that for any one pair of CPs p isin 119868 and p1015840 isin 1198681015840 (p p1015840) are
consistent with at least one of two functions 119891 or 1198911015840 which
can be estimated from a given set of putative correspondencesbased on robust regression method [2] Also for any givenpoint p isin 119868 its corresponding point can be uniquelyestimated by the CF and vice versa (Figure 2)
In practice putative correspondences are usually cor-rupted by noise and outliers and the number of putativecorrespondences is limited Therefore only an estimation ofCF with a certain accuracy can be estimated and the actualresults may be not strictly consistent with the ldquonecessary andsufficient conditionrdquo
Therefore the characteristic is that CF model can reducethe searching space of CPs into an elliptical area and the mis-matches of CF are usually near the correct CPs (Section 5)
4 Learning of CF and Its Application inMismatch Rejection
Based on the discussions in Section 3 for a pair of images1198681and 1198682of a scene the relationship of some corresponding
points between 1198681and 119868
2can be decomposed into a pair
of component relations (CRs) CR119868rarr1198681015840 and CR
1198681015840rarr119868
A CRconsists of one-one mapping and many-one mapping
Therefore CR119868rarr1198681015840 and CR
1198681015840rarr119868
can be described by twovector-valued functions 119891 119868 rarr 119868
1015840 and 1198911015840
1198681015840
rarr 119868
119891 (p) = p1015840
1198911015840(p1015840) = p
(4)
where p isin 119868 p1015840 isin 1198681015840 and (119891 119891
putative correspondences between 1198681and 1198682 If we treat y =
p1015840 as an output of a system and x = p as an input theCF 119891 can be estimated from 119878 = (x y) = (p p1015840) p isin
119868 p1015840 isin 1198681015840 using a vector-valued regression method for
example vector field consensus (VFC) method [24 28]and IECF (Iteratively Estimate Correspondence Function)algorithm [2] The fundamental ideas of VFC and IECF areas follows approximately represent the unknown CF 119891 as aweighted sumof kernel functions (2) and estimate theweights
6 Mathematical Problems in Engineering
iteratively by gradually reducing the undue effects fromoutliers The objective of this work is to investigate thefundamental principles and characteristics of CF Thereforewe do not compare the various implementation of CF anddo experiments by IECF The CF 119891
1015840 can be estimatedsimilarly
Theoretically the correctness of a pair of putative corre-spondences (p p1015840) can be determined by checking whetherit is consistent with one of the estimated correspondencefunctions 119891 or 119891
1015840 However due to the influence of imagenoise the observed corresponding points usually are notstrictly consistent with the estimated CFs 119891 and 119891
1015840 and atolerance parameter 120572
119868is needed to determine whether they
are consistent It is shown that the consistency of (p p1015840) with119891 can be determined by
119888119891
(p p1015840) = (119891 (p) minus p1015840) Σminus1
(119891 (p) minus p1015840)119879
(5)
where the superscript119879 represents a transpose of a vector andΣ is the covariance matrix of random variable 119891(p) minus p1015840 andcan be estimated using IECF algorithm [2]The consistency of(p p1015840)with119891
1015840 ismeasured similarly and denoted by 1198881198911015840(p p1015840)
Then some mismatches can be rejected by the constraints119888119891(p p1015840) gt 120572
In this section we experimentally investigate the characteris-tics of theCF by comparing it with thewidely used parametricmodel epipolar geometry (EG) ((1) Figure 3) in the contextof rejecting mismatches
51 Experimental Configuration The mismatch-rejectingmethods based onCF and EG are implemented by algorithmsICF (identifying point correspondences by correspondencefunction) [2] and RANSAC (Random Sample Consensus)[11 29 30] respectively in this study Since the coordinatesof the observed putative CPs are usually corrupted by noisethey are not strictly consistent with the CF and EG equationsTherefore two tolerance parameters are needed for thecomputation we name the two parameters 120572
119877in EG +
RANSAC and 120572119868in CF + ICF
In evaluating the performance of local descriptors andpoints matching models there are several schemes forexample recall precision [31] and ROC curve [14] Howeverthe objective of this workrsquos experiments is to show thecharacteristic of fail-detected mismatches of CF Thereforeto make a fair comparison we adjusted the values of 120572
119877and
120572119868such that a same number of putative correspondences are
identified as possible correct matches in every experiment
and the RANSAC are implemented without a limit oniteration number
Suppose 119868 and 1198681015840 are two images of a scene to be matched
and suppose 119878 is a set of putative correspondences betweenthem A mismatch-rejecting method will partition 119878 intotwo subsets 119878
119888and 119878
119898 as possible correct correspondences
and possible mismatches For readability we denote the twosubsets by 119878
EG119888
and 119878EG119898
in the method based on EG constraintand 119878
CF119888
and 119878CF119898
in the method based on CF Thus set 119878 canbe divided into four subsets 119878
EG119888
cap 119878CF119888 119878
EG119888
cap 119878CF119888 119878
EG119888
cap 119878CF119888
and 119878EG119888
cap 119878CF119888 where 119878
EG119888
= 119878 minus 119878EG119888
= 119878EG119898
and 119878CF119888
=
119878 minus 119878CF119888
= 119878CF119898
The meanings of the above four subsets are asfollows
(1) 119878EG119888
cap 119878CF119888
= 119878EG119898
cap 119878CF119898 the putative correspondences
that are regarded as possible correct matches by themethods based on EG and CF
(2) 119878EG119888
cap 119878CF119888
= 119878EG119898
cap 119878CF119898 the putative correspondences
that are regarded as possible mismatches by EGmethod and possible correct matches by CF
(3) 119878EG119888
cap 119878CF119888
= 119878EG119898
cap 119878CF119898 the putative correspondences
that are regarded as possible correct matches by theEG method and possible mismatches by CF
(4) 119878EG119888
cap 119878CF119888
= 119878EG119898
cap 119878CF119898 the putative correspondences
that are regarded as possible mismatches by themethods based on EG and CF
On subsets 119878EG119888
cap119878CF119888
and 119878EG119888
cap119878CF119888 the twomethods based on
CF and EG constraint are consistentTherefore it is sufficientto focus on 119878
EG119888
cap 119878CFc and 119878
EG119888
cap 119878CF119888
to compare the CFmodeland the EG constraint
For convenience in comparing the results of EG and CFwe manually removed some of the correct and near-correctputative correspondences from 119878
EG119888
cap 119878CF119888
and 119878EG119888
cap 119878CF119888
in some experiments and the two sets composed of theremaining putative correspondences are denoted by 119879(119878
EG119888
cap
119878CF119888
) and 119879(119878EG119888
cap 119878CF119888
) respectively
(i) 119879(119878EG119888
cap 119878CF119888
) the typical putative correspondencesthat are classified as possible correct matches by EGand as possible mismatches by CF
(ii) 119879(119878EG119888
cap 119878CF119888
) the typical putative correspondencesthat are classified as possible mismatches by EG andas possible correct correspondences by CF
52 Experiments
521 Main Results One of the used image pairs is pre-sented in Figure 5 There are 1278 and 1207 SIFT featurepoints extracted from two images respectively (the featurepoints are detected by the multiscale DoG (Difference ofGaussian) scheme and described by the orientation his-togram technique [32]) and 448 putative correspondences
Mathematical Problems in Engineering 7
(a) 119878EG119888cap 119878
CF119888
(b) 119878EG119888cap 119878
CF119888
(c) 119879(119878EG119888cap 119878
CF119888) (d) 119879(119878EG
119888cap 119878
CF119888)
Figure 5 An image pair of a toy bear (a) 26 EG potential mismatches (PMs) that are accepted by CF as matches (b) 26 CF PMs that areaccepted by EG as matches (c) some typical EG PMs that are accepted by CF (d) some typical CF PMs that are accepted by EG 120572
119868= 10597
and 120572119877
= 000019
(a) 119878EG119888cap 119878
CF119888
(b) 119878EG119888cap 119878
CF119888
(c) 119878EG119888cap 119878
CF119888
(d) 119878EG119888cap 119878
CF119888
Figure 6 An image pair of a corridor (a) 25 EG PMs that are accepted by CF as matches (b) 25 CF PMs that are accepted by EG as matches(c) 31 EG PMs that are accepted by CF as matches (d) 31 CF PMs that are accepted by EG as matches In (a)-(b) 120572
119868= 5991 and 120572
119877= 000048
and in (c)-(d) 120572119868
= 110 and 120572119877
= 005
computed from the feature points by the NNDR (Near-est Neighbor and Distance Ratio) method [32] Due tothe ambiguity of local information some of the putativecorrespondences are incorrect and need to be rejected Inthis experiment the two methods based on EG and CFare inconsistent on 52 putative correspondences (Figures5(a) and 5(b)) For example for every putative correspon-dence (p p1015840) in Figure 5(a) the CF method identified it as
a possible correct match but the EG method identified it asa possible mismatch Therefore based on the definition of119878EG119888
cap 119878CF119888
and 119878EG119888
cap 119878CF119888 we can regard 119878
EG119888
cap 119878CF119888
as a setof potential mismatches that the EG constraint fails to detectand 119878
EG119888
cap 119878CF119888
as a set of potential mismatches that the CFmodel fails to reject More results are presented in Figures 67 and 8
8 Mathematical Problems in Engineering
(a) 119878EG119888cap 119878
CF119888
(b) 119878EG119888cap 119878
CF119888
(c) 119879(119878EG119888cap 119878
CF119888) (d) 119879(119878EG
119888cap 119878
CF119888)
Figure 7 An image pair of Baxia (a) Some EG PMs that are accepted by CF asmatches (b) some CF PMs that are accepted by EG asmatches(c) some typical EG PMs that are accepted by CF (d) some typical CF PMs that are accepted by EG 120572
119868= 10597 and 120572
119877= 005
(a) 119878EG119888cap 119878
CF119888
(b) 119878EG119888cap 119878
CF119888
(c) 119879(119878EGc cap 119878CF119888) (d) 119879(119878EG
119888cap 119878
CF119888)
Figure 8 An image pair of temple Fayu (a) 204 EG PMs that are accepted by CF as matches (b) 204 CF PMs that are accepted by EG asmatches (c) some typical EG PMs that are accepted by CF (d) some typical CF PMs that are accepted by EG 120572
119868= 31 and 120572
119877= 00001515
(for visibility only 100 randomly chosen PMs are presented in (a) and (b))
522 Comparing the CF and EG Constraint by Analyzing theUndetected Mismatches For ease of explanation we intro-duce the following two definitions type-I mismatches andtype-II mismatches Suppose (p p1015840) is a mismatch If thereexists a correct corresponding point pair (pp1015840) satisfying thatthe points p and p1015840 are in the neighborhood of p and p1015840respectively then (p p1015840) is defined as a mismatch of type-IOtherwise we call (p p1015840) a type-II mismatch For example
the mismatches in Figure 7(d) and (p p10158403) in Figure 3(b)
belong to type-II and (p p10158402) in Figure 3(b) belong to type-
I By the EG constraint we can only determine whether aputative correspondence is in a band around the epipolar line(Figures 3(b) 9(b) and 9(d)) but not whether it is close to thecorrect one Therefore the undetected mismatches usuallybelong to type-II in the methods based on the EG constraint(Figures 3 5(b) 5(d) 6(b) 6(d) 7(b) 7(d) 8(b) and 8(d))
Mathematical Problems in Engineering 9
l1
l2
p1
l3p3 p2
l4p4
l9984001 p9984001
l9984002 p9984002
l9984003 p9984003
l9984004p9984004
(a)
l1
l3
p1
p3
l2 p2
l9984001p9984001
l9984002p9984002
l9984003p9984003
(b)
l2p2
l1 p1l3p3
l9984001p9984001
l9984002p9984002
l9984003 p9984003
(c)
l1
l6
p1
p6
l3
l4 p4
p3
l2
l5
p2
p5
l9984001p9984001
l9984002 p9984002
l9984003
l9984006
p9984003
p9984006
l9984004 p9984004
l9984005p9984005
(d)
Figure 9 Some PMs and their corresponding epipolar lines The EG model cannot detect the mismatches that are near the epipolar line butfar from the correct CPs The undetected mismatches by CF are usually near both the correct CPs and the epipolar line (a) and (c) presentthe EG PMs that are accepted by CF as matches in the experiments of Figures 7(c) and 8(c) respectively (b) and (d) present the CF PMs thatare accepted by EG as matches in the experiments of Figures 7(d) and 8(d) respectively In these experiments (p
119894 p1015840119894) is a pair of putative
CPs and l119894is the epipolar line corresponding to p1015840
119894and l1015840119894is the epipolar line corresponding to p
119894
The CF constraint determines the mapping relationshipsbetween two CPs and directly estimates the position of thecorresponding point Therefore by the mismatch-rejectingmethods based on CF we can restrict the error bound ofthe putative correspondences by a tolerance parameter 120572
119868
the undetected mismatches by CF usually belong to type-I To show the difference between the CF model and EGconstraints a typical result is presented in Figure 6 in whichwe relax the tolerance parameters 120572
119868and 120572119877to ensure the two
sets 119878EG119888
cap 119878CF119888
and 119878EG119888
cap 119878CF119888
change at equal speedHowever themismatches of type-I and type-II are usually
dramatically different from each other in usefulness andundue influence From the mismatches of type-I we cansearch for the correct CPs by including more informationOn the contrary other than ruining the traditional estimationmethods and distorting the final application results (egimage mosaicing and three-dimensional reconstruction) themismatches of the type-II provide less useful information
53 Further Evaluation on Some Popular Benchmark DataThe characteristics of the CF are also investigated on twoseries of popular benchmark data sets widely used to compareand evaluate related algorithms [33 34]
To evaluate the two schemes quantitatively this workconducted experiments on a series of image sets with ground-truth correspondences [31 35] The images were capturedunder special consideration to enable the encoding of theground-truth correspondences with homographies between
the reference image and other images The ground-truthhomographies are computed based on the following stepsfirst a set of putative correspondences are manually selectedand an initial estimation of the homography is computedbased on the selected putative correspondences second theimages are approximately aligned with the rough estimationof homography third reliable interest points are detectedand matched automatically and an accurate homography iscomputed from them
The experimental results are presented in Figures 11 12 13and 14 In every experiment the upper image and the lowerimage are called the reference image and the target imagerespectively In addition to the computed putative correspon-dences we also present the ground-truth CPs and ground-truth correspondence relationships in these experimentsTheground-truth CPs in the target images are computed for all ofthe marked interest points in reference images based on theground-truth homographies and denoted by red circles Thedetected interest points are denoted by black plus signs Theground-truth correspondence relationships are displayed asblue dashed lines and the computed putative correspondencerelationships are shown by a solid line with red or green forvisibility
Suppose that 119878 = (p1 p10158401) (p
119899 p1015840119899) are a set of
putative correspondences and 119878 = (p1 p10158401) (p
119899 p1015840119899) are
the ground-truth correspondences of 119878 We quantitativelyevaluate the quality of 119878 by the root-mean-square deviation(RMSD) RMSD(119878) = radicsum
Figure 10The possible correct correspondences identified by CF in the experiment of Figure 8 are presented based on depth (a) 34 putativecorrespondences (PCs) over the first crossbeam (b) 324 PCs over the second crossbeam (c) 214 PCs over the third crossbeam (d) 20 PCsaround the fourth crossbeam (e) 2 PCs on the wall (for clarity only 50 randomly chosen PCs are presented in (b) and (c))
The second series of evaluation data are the Middle-bury Stereo Datasets [36 37] (httpvisionmiddleburyedustereodata) These data sets were created using an auto-mated version of structured-lighting technique [38] Eachdataset consists of seven rectified views This work used thefirst and the fifth views for evaluation The experimentalresults are presented in Figures 15 16 17 18 and 19
54 CF and EG in Practice
541 Necessity and Sufficiency As analyzed the EG con-straint is a necessary but insufficient condition in the corre-spondence problem and theCFmodel describes themappingbetween CPs directly However in applications some correctCPsmay be inconsistent with the estimated EGmodel andorthe estimated CF model there may be some mismatchesthat are ldquoconsistentrdquo with the estimated CF model there alsomay be some of the putative correspondences which areldquoconsistentrdquo with the estimated CF but ldquoinconsistentrdquo withthe estimated EG for example the experimental results inFigures 5(a) 5(b) 7(a) 7(b) 8(a) and 8(b) The reasons forthe above results are as follows
(i) The coordinates of the obtained putative CPs areusually corrupted by noise
(ii) There are usually some errors in the estimation of theCF and EG
(iii) As pointed out in the first paragraph of Section 5a preset tolerance parameter is needed to define theldquoconsistentrdquo and ldquoinconsistentrdquo in the applicationsof CF and EG However the noise in the obtainedputative CPs is random
542 Characteristics and Limitations of the CF Model andEG Model In the two models there are no restrictionson the depth range of the imaged scene However theEG is a global describing model and the CF is a localdescribing model In the EG model any pair of putativeCPs can influence the overall model estimation whereas inthe CF model a pair of putative CPs only influence themodel estimation within a local area in their vicinity Theshortcoming of the global model EG is that the overallmodel estimation can be degraded or ruined by PCs withconsiderable noise or mismatches in any area of the imagesThe shortcoming of the local model CF is that a pair of goodCPs cannot improve the estimation of CF outside of a localarea around itself Thus the uneven distribution of putativecorrespondences with different quality results in the fact thatthe estimated CF may be of high quality in some areas butof low quality in others for example in the experiment ofFigure 10 the qualities of the CF estimations over the second-nearest-to-the-cameras crossbeam and the third-nearest-to-the-cameras crossbeam are better than estimation over thefirst-nearest-to-the-cameras crossbeam The ldquolocal areardquo isdetermined by the CF estimation method and its chosenparameters for example in the CF estimation method SP(SVM) [2] the local area is determined by the chosen kerneland the scale parameter The above limitations of EG and CFcan be alleviated by investigating robust estimation methodsfor example the RANSAC M-estimators [8] LMedS (LeastMedian of Squares) [9] and MLESAC [10 12] for EGIn future work we will investigate further the estimationmethod for CF to improve its robustness
6 Conclusion
The CF is a recently introduced nonparametric model forrejecting mismatchesoutliers in image point matching In
Mathematical Problems in Engineering 11
(a) RMSD = 1386 (b) RMSD = 22707 (c) RMSD = 384 (d) RMSD = 23626Figure 11 Quantitative evaluation on two popular benchmark sets bikes and leuven The ground-truth CPs are presented by red circles thedetected interest points by black plus signs The ground-truth correspondences are displayed as blue dashed lines and the computed putativecorrespondences are shown by a solid green solid line (a) and (c) present some EG PMs that are accepted by CF as matches (b) and (d)present some CF PMs that are accepted by EG as matches 120572
119868= 10597 120572
119877= 21435 times 10
minus5 in (b) and 120572119877
= 58 times 10minus5 in (d)
(a) RMSD = 1631 (b) RMSD = 40098 (c) RMSD = 358 (d) RMSD = 32288Figure 12 Quantitative evaluation on two popular benchmark sets wall and bark ((a) and (c)) The EG PMs that are accepted by CF asmatches ((b) and (d)) the CF PMs that are accepted by EG as matches 120572
119868= 10597 120572
119877= 2029 times 10
minus4 in (b) and 120572119877
= 105 times 10minus4 in (d)
(a) RMSD = 1017 (b) RMSD = 42639 (c) RMSD = 329 (d) RMSD = 17646Figure 13 Quantitative evaluation on two popular benchmark sets graf and boat The ground-truth CPs are presented by red circles and thedetected interest points by black plus signs The ground-truth correspondences are displayed as blue dashed lines and the computed putativecorrespondences are shown by a solid green solid line (a) and (c) present some EG PMs that are accepted by CF as matches (b) and (d)present some CF PMs that are accepted by EG as matches 120572
Figure 14 Quantitative evaluation on two popular benchmark sets trees and ubc The ground-truth correspondences are displayed as bluedashed lines and the computed putative correspondences are shown by a yellow solid line (a) and (c) present some EG PMs that are acceptedbyCF asmatches (b) and (d) present someCFPMs that are accepted by EG asmatches120572
119868= 10597120572
119877= 59times10
minus5 in (b) and120572119877
= 185times10minus4
in (d)
(a) 119878EG119888cap 119878
CF119888
(b) 119878EG119888cap 119878
CF119888
(c) 119878EG119888cap 119878
CF119888
(d) 119878EG119888cap 119878
CF119888
Figure 15 Evaluation on Middlebury Stereo Datasets art and books ((a) and (c)) Some potential EG mismatches that are accepted by CF asmatches ((b) and (d)) some potential CF mismatches that are accepted by EG as matches 120572
119877= 0013 in (a) and (b) 120572
119877= 000018 in (c) and
(d) and 120572119868
= 10597
this study we investigated the principles of the CF andstudied its characteristics by comparing it with the widelyused parametric model epipolar geometry (EG) constraint
It is shown that the CF describes the mapping relation-ships between two CPs and should be able to estimate theposition of the corresponding point Therefore in additionto mismatch rejection a potential application of the CF is to
guide the point matching process by incorporating it into thecorrespondence propagation [39]
In practice putative correspondences are usually cor-rupted by noise and outliers and the number of putativecorrespondences is limited Therefore only an estimationof the CF with a certain accuracy can be estimated andthe actual results will violate the ldquonecessary and sufficient
Mathematical Problems in Engineering 13
(a) 119878EG119888cap SCF119888
(b) 119878EG119888cap 119878
CF119888
(c) 119878EG119888cap 119878
CF119888
(d) 119878EG119888cap 119878
CF119888
Figure 16 Evaluation on Middlebury Stereo Datasets computer and dolls ((a) and (c)) Some potential EG mismatches that are accepted byCF as matches ((b) and (d)) some potential CF mismatches that are accepted by EG as matches 120572
119877= 000018 in (a) and (b) 120572
119877= 000048 in
(c) and (d) and 120572119868
= 10597
(a) 119878EG119888cap 119878
CF119888
(b) 119878EG119888cap 119878
CF119888
(c) 119878EG119888cap 119878
CF119888
(d) 119878EG119888cap 119878
CF119888
Figure 17 Evaluation onMiddlebury Stereo Datasets drumsticks and dwarves ((a) and (c)) Some potential EGmismatches that are acceptedby CF as matches ((b) and (d)) some potential CF mismatches that are accepted by EG as matches 120572
119877= 000078 in (a) and (b) 120572
119877= 00035
in (c) and (d) and 120572119868
= 10597
conditionrdquoThe characteristic is that theCFmodel can reducethe searching space of CPs into an elliptical area and themismatches of the CF are usually near the correct CPs
Conflict of Interests
The authors declare that there is no conflict of interestsregarding the publication of this paper
Acknowledgments
The authors are grateful for valuable advice from DrDibyendu Mukherjee and the anonymous reviewers andwould like to thank the National Natural Science Foundationof China (Grant nos 61075033 61005033 and 61273248)the Natural Science Foundation of Guangdong Province(2014A030313425 and S2011010003348) and theOpenProject
14 Mathematical Problems in Engineering
(a) 119878EG119888cap 119878
CF119888
(b) 119878EG119888cap 119878
CF119888
(c) 119878EG119888cap 119878
CF119888
(d) 119878EG119888cap 119878
CF119888
Figure 18 Evaluation onMiddlebury Stereo Datasets laundry andmoebius ((a) and (c)) Some potential EGmismatches that are accepted byCF asmatches ((b) and (d)) some potential CFmismatches that are accepted by EG asmatches 120572
119877= 000000275 in (a) and (b) 120572
119877= 000035
in (c) and (d) and 120572119868
= 10597
(a) 119878EG119888cap 119878
CF119888
(b) 119878EG119888cap 119878
CF119888
Figure 19 Evaluation on Middlebury Stereo Dataset reindeer (a)Two potential EG mismatches that are accepted by CF as matches(b) two potential CFmismatches that are accepted by EG asmatches120572119877
= 0005 and 120572119868
= 30
Program of the National Laboratory of Pattern Recognition(NLPR) (201001060) for their support
References
[1] J P Monaco A C Bovik and L K Cormack ldquoActive foveateduncalibrated stereovisionrdquo International Journal of ComputerVision vol 85 no 2 pp 192ndash207 2009
[2] X Li and Z Hu ldquoRejecting mismatches by correspondencefunctionrdquo International Journal of Computer Vision vol 89 no1 pp 1ndash17 2010
[3] R Hartley and A Zisserman Multiple View Geometry inComputer Vision Cambridge University Press 2nd edition2003
[4] M Sonka V Hlavac and R Boyle Image Processing Analysisand Machine Vision Thomson-Engineering 3rd edition 2007
[5] G Carneiro and A D Jepson ldquoFlexible spatial configuration oflocal image featuresrdquo IEEE Transactions on Pattern Analysis andMachine Intelligence vol 29 no 12 pp 2089ndash2104 2007
[6] W Aguilar Y Frauel F Escolano M E Martinez-Perez AEspinosa-Romero andM A Lozano ldquoA robust graph transfor-mation matching for non-rigid registrationrdquo Image and VisionComputing vol 27 no 7 pp 897ndash910 2009
[7] L Wang F Duan and K Lv ldquoFisheye-lens-based visual suncompass for perception of spatial orientationrdquo MathematicalProblems in Engineering vol 2012 Article ID 460430 p 15 2012
[8] P Huber Robust Statistics Wiley New York NY USA 1981[9] P J Rousseeuw ldquoLeast median of squares regressionrdquo Journal
of the American Statistical Association vol 79 no 388 pp 871ndash880 1984
[10] P H S Torr and A Zisserman ldquoMLESAC a new robustestimator with application to estimating image geometryrdquoComputer Vision and Image Understanding vol 78 no 1 pp138ndash156 2000
[11] M A Fischler and R C Bolles ldquoRandom sample consensus aparadigm for model fitting with applications to image analysisand automated cartographyrdquo Communications of the ACM vol24 no 6 pp 381ndash395 1981
[12] B J Tordoff and D W Murray ldquoGuided-MLESAC fasterimage transform estimation by using matching priorsrdquo IEEETransactions on Pattern Analysis and Machine Intelligence vol27 no 10 pp 1523ndash1535 2005
Mathematical Problems in Engineering 15
[13] D Sidibe P Montesinos and S Janaqi ldquoMatching localinvariant features with contextual information an experimentalevaluationrdquo Electronic Letters on Computer Vision and ImageAnalysis vol 7 no 1 pp 26ndash39 2008
[14] Q-H Tran T-J Chin G Carneiro M S Brown and D SuterldquoIn defence of RANSAC for outlier rejection in deformableregistrationrdquo Lecture Notes in Computer Science vol 7575 no4 pp 274ndash287 2012
[15] A Goshtasby ldquoTransformation functions for image registra-tionrdquo in Proceedings of the Computer Vision and Pattern Recog-nition (CVPR rsquo11) June 2011
[16] X Li Y Wang J Zhou and X Li ldquoComparative research oncorrespondence functionrdquo in Proceedings of the InternationalConference on Intelligent System Design and Engineering Appli-cation (ISDEA rsquo10) pp 444ndash447 October 2010
[17] X-R Li X-M Li H-L Li and M-Y Cao ldquoRejecting outliersbased on correspondence manifoldrdquo Acta Automatica Sinicavol 35 no 1 pp 17ndash22 2009
[18] D Pizarro and A Bartoli ldquoFeature-based deformable surfacedetection with self-occlusion reasoningrdquo International Journalof Computer Vision vol 97 no 1 pp 54ndash70 2012
[19] J Yu A Eriksson T-J Chin and D Suter ldquoAn adversarialoptimization approach to efficient outlier removalrdquo Journal ofMathematical Imaging and Vision vol 48 no 3 pp 451ndash4662014
[20] A Goshtasby ldquoRegistration of images with geometric distor-tionsrdquo IEEETransactions onGeoscience andRemote Sensing vol26 no 1 pp 60ndash64 1988
[21] F L Bookstein ldquoPrincipal warps thin-plate splines and thedecomposition of deformationsrdquo IEEE Transactions on PatternAnalysis and Machine Intelligence vol 11 no 6 pp 567ndash5851992
[22] B Scholkopf F Steinke and V Blanz ldquoObject correspondenceas a machine learning problemrdquo in Proceedings of the 22ndInternational Conference on Machine Learning (ICML rsquo05) pp776ndash783 2005
[23] L Zagorchev and A Goshtasby ldquoA comparative study oftransformation functions for nonrigid image registrationrdquo IEEETransactions on Image Processing vol 15 no 3 pp 529ndash5382006
[24] J Zhao JMa J Tian J Ma andD Zhang ldquoA robustmethod forvector field learning with application to mismatch removingrdquoin Proceedings of the IEEE Conference on Computer Vision andPattern Recognition (CVPR rsquo11) Providence RI USA June 2011
[25] JMa J Zhao J Tian X Bai and Z Tu ldquoRegularized vector fieldlearning with sparse approximation for mismatch removalrdquoPattern Recognition vol 46 no 12 pp 3519ndash3532 2013
[26] J Ma J Zhao J Tian Z Tu and A L Yuille ldquoRobust estimationof nonrigid transformation for point set registrationrdquo in Pro-ceedings of the 26th IEEE Conference on Computer Vision andPattern Recognition (CVPR rsquo13) pp 2147ndash2154 June 2013
[27] J MaW Qiu J Zhao Y Ma A L Yuille and Z Tu ldquoRobust l2e
estimation of transformation for non-rigid registrationrdquo IEEETransactions on Signal Processing vol 63 no 5 pp 1115ndash11292015
[28] J Ma J Zhao J Tian A L Yuille and Z Tu ldquoRobust pointmatching via vector field consensusrdquo IEEE Transactions onImage Processing vol 23 no 4 pp 1706ndash1721 2014
[29] L Goshen and I Shimshoni ldquoBalanced exploration andexploitation model search for efficient epipolar geometry esti-mationrdquo IEEE Transactions on Pattern Analysis and MachineIntelligence vol 30 no 7 pp 1230ndash1242 2008
[30] O Chum and J Matas ldquoMatching with prosacmdashprogressivesample consensusrdquo in Proceedings of the IEEE Computer SocietyConference on Computer Vision and Pattern Recognition (CVPRrsquo05) vol 1 pp 220ndash226 IEEE June 2005
[31] K Mikolajczyk and C Schmid ldquoA performance evaluation oflocal descriptorsrdquo IEEE Transactions on Pattern Analysis andMachine Intelligence vol 27 no 10 pp 1615ndash1630 2005
[32] D G Lowe ldquoDistinctive image features from scale-invariantkeypointsrdquo International Journal of Computer Vision vol 60 no2 pp 91ndash110 2004
[33] S Gauglitz T Hollerer and M Turk ldquoEvaluation of interestpoint detectors and feature descriptors for visual trackingrdquoInternational Journal of Computer Vision vol 94 no 3 pp 335ndash360 2011
[34] Y Yu K Huang W Chen and T Tan ldquoA novel algorithmfor view and illumination invariant image matchingrdquo IEEETransactions on Image Processing vol 21 no 1 pp 229ndash2402012
[35] K Mikolajczyk T Tuytelaars C Schmid et al ldquoA comparisonof affine region detectorsrdquo International Journal of ComputerVision vol 65 no 1-2 pp 43ndash72 2005
[36] H Hirschmuller and D Scharstein ldquoEvaluation of cost func-tions for stereo matchingrdquo in Proceedings of the IEEE ComputerSociety Conference on Computer Vision and Pattern Recognition(CVPR rsquo07) June 2007
[37] D Scharstein and C Pal ldquoLearning conditional random fieldsfor stereordquo in Proceedings of the IEEE Computer Society Confer-ence on Computer Vision and Pattern Recognition (CVPR rsquo07)IEEE Minneapolis Minn USA June 2007
[38] D Scharstein andR Szeliski ldquoHigh-accuracy stereo depthmapsusing structured lightrdquo in Proceedings of the IEEE ComputerSociety Conference on Computer Vision and Pattern Recognition(CVPR rsquo03) vol 1 pp I-195ndashI-202 IEEE June 2003
[39] H Wang S Yan J Liu X Tang and T S Huang ldquoCorre-spondence propagationwithweak priorsrdquo IEEETransactions onImage Processing vol 18 no 1 pp 140ndash150 2009
iteratively by gradually reducing the undue effects fromoutliers The objective of this work is to investigate thefundamental principles and characteristics of CF Thereforewe do not compare the various implementation of CF anddo experiments by IECF The CF 119891
1015840 can be estimatedsimilarly
Theoretically the correctness of a pair of putative corre-spondences (p p1015840) can be determined by checking whetherit is consistent with one of the estimated correspondencefunctions 119891 or 119891
1015840 However due to the influence of imagenoise the observed corresponding points usually are notstrictly consistent with the estimated CFs 119891 and 119891
1015840 and atolerance parameter 120572
119868is needed to determine whether they
are consistent It is shown that the consistency of (p p1015840) with119891 can be determined by
119888119891
(p p1015840) = (119891 (p) minus p1015840) Σminus1
(119891 (p) minus p1015840)119879
(5)
where the superscript119879 represents a transpose of a vector andΣ is the covariance matrix of random variable 119891(p) minus p1015840 andcan be estimated using IECF algorithm [2]The consistency of(p p1015840)with119891
1015840 ismeasured similarly and denoted by 1198881198911015840(p p1015840)
Then some mismatches can be rejected by the constraints119888119891(p p1015840) gt 120572
In this section we experimentally investigate the characteris-tics of theCF by comparing it with thewidely used parametricmodel epipolar geometry (EG) ((1) Figure 3) in the contextof rejecting mismatches
51 Experimental Configuration The mismatch-rejectingmethods based onCF and EG are implemented by algorithmsICF (identifying point correspondences by correspondencefunction) [2] and RANSAC (Random Sample Consensus)[11 29 30] respectively in this study Since the coordinatesof the observed putative CPs are usually corrupted by noisethey are not strictly consistent with the CF and EG equationsTherefore two tolerance parameters are needed for thecomputation we name the two parameters 120572
119877in EG +
RANSAC and 120572119868in CF + ICF
In evaluating the performance of local descriptors andpoints matching models there are several schemes forexample recall precision [31] and ROC curve [14] Howeverthe objective of this workrsquos experiments is to show thecharacteristic of fail-detected mismatches of CF Thereforeto make a fair comparison we adjusted the values of 120572
119877and
120572119868such that a same number of putative correspondences are
identified as possible correct matches in every experiment
and the RANSAC are implemented without a limit oniteration number
Suppose 119868 and 1198681015840 are two images of a scene to be matched
and suppose 119878 is a set of putative correspondences betweenthem A mismatch-rejecting method will partition 119878 intotwo subsets 119878
119888and 119878
119898 as possible correct correspondences
and possible mismatches For readability we denote the twosubsets by 119878
EG119888
and 119878EG119898
in the method based on EG constraintand 119878
CF119888
and 119878CF119898
in the method based on CF Thus set 119878 canbe divided into four subsets 119878
EG119888
cap 119878CF119888 119878
EG119888
cap 119878CF119888 119878
EG119888
cap 119878CF119888
and 119878EG119888
cap 119878CF119888 where 119878
EG119888
= 119878 minus 119878EG119888
= 119878EG119898
and 119878CF119888
=
119878 minus 119878CF119888
= 119878CF119898
The meanings of the above four subsets are asfollows
(1) 119878EG119888
cap 119878CF119888
= 119878EG119898
cap 119878CF119898 the putative correspondences
that are regarded as possible correct matches by themethods based on EG and CF
(2) 119878EG119888
cap 119878CF119888
= 119878EG119898
cap 119878CF119898 the putative correspondences
that are regarded as possible mismatches by EGmethod and possible correct matches by CF
(3) 119878EG119888
cap 119878CF119888
= 119878EG119898
cap 119878CF119898 the putative correspondences
that are regarded as possible correct matches by theEG method and possible mismatches by CF
(4) 119878EG119888
cap 119878CF119888
= 119878EG119898
cap 119878CF119898 the putative correspondences
that are regarded as possible mismatches by themethods based on EG and CF
On subsets 119878EG119888
cap119878CF119888
and 119878EG119888
cap119878CF119888 the twomethods based on
CF and EG constraint are consistentTherefore it is sufficientto focus on 119878
EG119888
cap 119878CFc and 119878
EG119888
cap 119878CF119888
to compare the CFmodeland the EG constraint
For convenience in comparing the results of EG and CFwe manually removed some of the correct and near-correctputative correspondences from 119878
EG119888
cap 119878CF119888
and 119878EG119888
cap 119878CF119888
in some experiments and the two sets composed of theremaining putative correspondences are denoted by 119879(119878
EG119888
cap
119878CF119888
) and 119879(119878EG119888
cap 119878CF119888
) respectively
(i) 119879(119878EG119888
cap 119878CF119888
) the typical putative correspondencesthat are classified as possible correct matches by EGand as possible mismatches by CF
(ii) 119879(119878EG119888
cap 119878CF119888
) the typical putative correspondencesthat are classified as possible mismatches by EG andas possible correct correspondences by CF
52 Experiments
521 Main Results One of the used image pairs is pre-sented in Figure 5 There are 1278 and 1207 SIFT featurepoints extracted from two images respectively (the featurepoints are detected by the multiscale DoG (Difference ofGaussian) scheme and described by the orientation his-togram technique [32]) and 448 putative correspondences
Mathematical Problems in Engineering 7
(a) 119878EG119888cap 119878
CF119888
(b) 119878EG119888cap 119878
CF119888
(c) 119879(119878EG119888cap 119878
CF119888) (d) 119879(119878EG
119888cap 119878
CF119888)
Figure 5 An image pair of a toy bear (a) 26 EG potential mismatches (PMs) that are accepted by CF as matches (b) 26 CF PMs that areaccepted by EG as matches (c) some typical EG PMs that are accepted by CF (d) some typical CF PMs that are accepted by EG 120572
119868= 10597
and 120572119877
= 000019
(a) 119878EG119888cap 119878
CF119888
(b) 119878EG119888cap 119878
CF119888
(c) 119878EG119888cap 119878
CF119888
(d) 119878EG119888cap 119878
CF119888
Figure 6 An image pair of a corridor (a) 25 EG PMs that are accepted by CF as matches (b) 25 CF PMs that are accepted by EG as matches(c) 31 EG PMs that are accepted by CF as matches (d) 31 CF PMs that are accepted by EG as matches In (a)-(b) 120572
119868= 5991 and 120572
119877= 000048
and in (c)-(d) 120572119868
= 110 and 120572119877
= 005
computed from the feature points by the NNDR (Near-est Neighbor and Distance Ratio) method [32] Due tothe ambiguity of local information some of the putativecorrespondences are incorrect and need to be rejected Inthis experiment the two methods based on EG and CFare inconsistent on 52 putative correspondences (Figures5(a) and 5(b)) For example for every putative correspon-dence (p p1015840) in Figure 5(a) the CF method identified it as
a possible correct match but the EG method identified it asa possible mismatch Therefore based on the definition of119878EG119888
cap 119878CF119888
and 119878EG119888
cap 119878CF119888 we can regard 119878
EG119888
cap 119878CF119888
as a setof potential mismatches that the EG constraint fails to detectand 119878
EG119888
cap 119878CF119888
as a set of potential mismatches that the CFmodel fails to reject More results are presented in Figures 67 and 8
8 Mathematical Problems in Engineering
(a) 119878EG119888cap 119878
CF119888
(b) 119878EG119888cap 119878
CF119888
(c) 119879(119878EG119888cap 119878
CF119888) (d) 119879(119878EG
119888cap 119878
CF119888)
Figure 7 An image pair of Baxia (a) Some EG PMs that are accepted by CF asmatches (b) some CF PMs that are accepted by EG asmatches(c) some typical EG PMs that are accepted by CF (d) some typical CF PMs that are accepted by EG 120572
119868= 10597 and 120572
119877= 005
(a) 119878EG119888cap 119878
CF119888
(b) 119878EG119888cap 119878
CF119888
(c) 119879(119878EGc cap 119878CF119888) (d) 119879(119878EG
119888cap 119878
CF119888)
Figure 8 An image pair of temple Fayu (a) 204 EG PMs that are accepted by CF as matches (b) 204 CF PMs that are accepted by EG asmatches (c) some typical EG PMs that are accepted by CF (d) some typical CF PMs that are accepted by EG 120572
119868= 31 and 120572
119877= 00001515
(for visibility only 100 randomly chosen PMs are presented in (a) and (b))
522 Comparing the CF and EG Constraint by Analyzing theUndetected Mismatches For ease of explanation we intro-duce the following two definitions type-I mismatches andtype-II mismatches Suppose (p p1015840) is a mismatch If thereexists a correct corresponding point pair (pp1015840) satisfying thatthe points p and p1015840 are in the neighborhood of p and p1015840respectively then (p p1015840) is defined as a mismatch of type-IOtherwise we call (p p1015840) a type-II mismatch For example
the mismatches in Figure 7(d) and (p p10158403) in Figure 3(b)
belong to type-II and (p p10158402) in Figure 3(b) belong to type-
I By the EG constraint we can only determine whether aputative correspondence is in a band around the epipolar line(Figures 3(b) 9(b) and 9(d)) but not whether it is close to thecorrect one Therefore the undetected mismatches usuallybelong to type-II in the methods based on the EG constraint(Figures 3 5(b) 5(d) 6(b) 6(d) 7(b) 7(d) 8(b) and 8(d))
Mathematical Problems in Engineering 9
l1
l2
p1
l3p3 p2
l4p4
l9984001 p9984001
l9984002 p9984002
l9984003 p9984003
l9984004p9984004
(a)
l1
l3
p1
p3
l2 p2
l9984001p9984001
l9984002p9984002
l9984003p9984003
(b)
l2p2
l1 p1l3p3
l9984001p9984001
l9984002p9984002
l9984003 p9984003
(c)
l1
l6
p1
p6
l3
l4 p4
p3
l2
l5
p2
p5
l9984001p9984001
l9984002 p9984002
l9984003
l9984006
p9984003
p9984006
l9984004 p9984004
l9984005p9984005
(d)
Figure 9 Some PMs and their corresponding epipolar lines The EG model cannot detect the mismatches that are near the epipolar line butfar from the correct CPs The undetected mismatches by CF are usually near both the correct CPs and the epipolar line (a) and (c) presentthe EG PMs that are accepted by CF as matches in the experiments of Figures 7(c) and 8(c) respectively (b) and (d) present the CF PMs thatare accepted by EG as matches in the experiments of Figures 7(d) and 8(d) respectively In these experiments (p
119894 p1015840119894) is a pair of putative
CPs and l119894is the epipolar line corresponding to p1015840
119894and l1015840119894is the epipolar line corresponding to p
119894
The CF constraint determines the mapping relationshipsbetween two CPs and directly estimates the position of thecorresponding point Therefore by the mismatch-rejectingmethods based on CF we can restrict the error bound ofthe putative correspondences by a tolerance parameter 120572
119868
the undetected mismatches by CF usually belong to type-I To show the difference between the CF model and EGconstraints a typical result is presented in Figure 6 in whichwe relax the tolerance parameters 120572
119868and 120572119877to ensure the two
sets 119878EG119888
cap 119878CF119888
and 119878EG119888
cap 119878CF119888
change at equal speedHowever themismatches of type-I and type-II are usually
dramatically different from each other in usefulness andundue influence From the mismatches of type-I we cansearch for the correct CPs by including more informationOn the contrary other than ruining the traditional estimationmethods and distorting the final application results (egimage mosaicing and three-dimensional reconstruction) themismatches of the type-II provide less useful information
53 Further Evaluation on Some Popular Benchmark DataThe characteristics of the CF are also investigated on twoseries of popular benchmark data sets widely used to compareand evaluate related algorithms [33 34]
To evaluate the two schemes quantitatively this workconducted experiments on a series of image sets with ground-truth correspondences [31 35] The images were capturedunder special consideration to enable the encoding of theground-truth correspondences with homographies between
the reference image and other images The ground-truthhomographies are computed based on the following stepsfirst a set of putative correspondences are manually selectedand an initial estimation of the homography is computedbased on the selected putative correspondences second theimages are approximately aligned with the rough estimationof homography third reliable interest points are detectedand matched automatically and an accurate homography iscomputed from them
The experimental results are presented in Figures 11 12 13and 14 In every experiment the upper image and the lowerimage are called the reference image and the target imagerespectively In addition to the computed putative correspon-dences we also present the ground-truth CPs and ground-truth correspondence relationships in these experimentsTheground-truth CPs in the target images are computed for all ofthe marked interest points in reference images based on theground-truth homographies and denoted by red circles Thedetected interest points are denoted by black plus signs Theground-truth correspondence relationships are displayed asblue dashed lines and the computed putative correspondencerelationships are shown by a solid line with red or green forvisibility
Suppose that 119878 = (p1 p10158401) (p
119899 p1015840119899) are a set of
putative correspondences and 119878 = (p1 p10158401) (p
119899 p1015840119899) are
the ground-truth correspondences of 119878 We quantitativelyevaluate the quality of 119878 by the root-mean-square deviation(RMSD) RMSD(119878) = radicsum
Figure 10The possible correct correspondences identified by CF in the experiment of Figure 8 are presented based on depth (a) 34 putativecorrespondences (PCs) over the first crossbeam (b) 324 PCs over the second crossbeam (c) 214 PCs over the third crossbeam (d) 20 PCsaround the fourth crossbeam (e) 2 PCs on the wall (for clarity only 50 randomly chosen PCs are presented in (b) and (c))
The second series of evaluation data are the Middle-bury Stereo Datasets [36 37] (httpvisionmiddleburyedustereodata) These data sets were created using an auto-mated version of structured-lighting technique [38] Eachdataset consists of seven rectified views This work used thefirst and the fifth views for evaluation The experimentalresults are presented in Figures 15 16 17 18 and 19
54 CF and EG in Practice
541 Necessity and Sufficiency As analyzed the EG con-straint is a necessary but insufficient condition in the corre-spondence problem and theCFmodel describes themappingbetween CPs directly However in applications some correctCPsmay be inconsistent with the estimated EGmodel andorthe estimated CF model there may be some mismatchesthat are ldquoconsistentrdquo with the estimated CF model there alsomay be some of the putative correspondences which areldquoconsistentrdquo with the estimated CF but ldquoinconsistentrdquo withthe estimated EG for example the experimental results inFigures 5(a) 5(b) 7(a) 7(b) 8(a) and 8(b) The reasons forthe above results are as follows
(i) The coordinates of the obtained putative CPs areusually corrupted by noise
(ii) There are usually some errors in the estimation of theCF and EG
(iii) As pointed out in the first paragraph of Section 5a preset tolerance parameter is needed to define theldquoconsistentrdquo and ldquoinconsistentrdquo in the applicationsof CF and EG However the noise in the obtainedputative CPs is random
542 Characteristics and Limitations of the CF Model andEG Model In the two models there are no restrictionson the depth range of the imaged scene However theEG is a global describing model and the CF is a localdescribing model In the EG model any pair of putativeCPs can influence the overall model estimation whereas inthe CF model a pair of putative CPs only influence themodel estimation within a local area in their vicinity Theshortcoming of the global model EG is that the overallmodel estimation can be degraded or ruined by PCs withconsiderable noise or mismatches in any area of the imagesThe shortcoming of the local model CF is that a pair of goodCPs cannot improve the estimation of CF outside of a localarea around itself Thus the uneven distribution of putativecorrespondences with different quality results in the fact thatthe estimated CF may be of high quality in some areas butof low quality in others for example in the experiment ofFigure 10 the qualities of the CF estimations over the second-nearest-to-the-cameras crossbeam and the third-nearest-to-the-cameras crossbeam are better than estimation over thefirst-nearest-to-the-cameras crossbeam The ldquolocal areardquo isdetermined by the CF estimation method and its chosenparameters for example in the CF estimation method SP(SVM) [2] the local area is determined by the chosen kerneland the scale parameter The above limitations of EG and CFcan be alleviated by investigating robust estimation methodsfor example the RANSAC M-estimators [8] LMedS (LeastMedian of Squares) [9] and MLESAC [10 12] for EGIn future work we will investigate further the estimationmethod for CF to improve its robustness
6 Conclusion
The CF is a recently introduced nonparametric model forrejecting mismatchesoutliers in image point matching In
Mathematical Problems in Engineering 11
(a) RMSD = 1386 (b) RMSD = 22707 (c) RMSD = 384 (d) RMSD = 23626Figure 11 Quantitative evaluation on two popular benchmark sets bikes and leuven The ground-truth CPs are presented by red circles thedetected interest points by black plus signs The ground-truth correspondences are displayed as blue dashed lines and the computed putativecorrespondences are shown by a solid green solid line (a) and (c) present some EG PMs that are accepted by CF as matches (b) and (d)present some CF PMs that are accepted by EG as matches 120572
119868= 10597 120572
119877= 21435 times 10
minus5 in (b) and 120572119877
= 58 times 10minus5 in (d)
(a) RMSD = 1631 (b) RMSD = 40098 (c) RMSD = 358 (d) RMSD = 32288Figure 12 Quantitative evaluation on two popular benchmark sets wall and bark ((a) and (c)) The EG PMs that are accepted by CF asmatches ((b) and (d)) the CF PMs that are accepted by EG as matches 120572
119868= 10597 120572
119877= 2029 times 10
minus4 in (b) and 120572119877
= 105 times 10minus4 in (d)
(a) RMSD = 1017 (b) RMSD = 42639 (c) RMSD = 329 (d) RMSD = 17646Figure 13 Quantitative evaluation on two popular benchmark sets graf and boat The ground-truth CPs are presented by red circles and thedetected interest points by black plus signs The ground-truth correspondences are displayed as blue dashed lines and the computed putativecorrespondences are shown by a solid green solid line (a) and (c) present some EG PMs that are accepted by CF as matches (b) and (d)present some CF PMs that are accepted by EG as matches 120572
Figure 14 Quantitative evaluation on two popular benchmark sets trees and ubc The ground-truth correspondences are displayed as bluedashed lines and the computed putative correspondences are shown by a yellow solid line (a) and (c) present some EG PMs that are acceptedbyCF asmatches (b) and (d) present someCFPMs that are accepted by EG asmatches120572
119868= 10597120572
119877= 59times10
minus5 in (b) and120572119877
= 185times10minus4
in (d)
(a) 119878EG119888cap 119878
CF119888
(b) 119878EG119888cap 119878
CF119888
(c) 119878EG119888cap 119878
CF119888
(d) 119878EG119888cap 119878
CF119888
Figure 15 Evaluation on Middlebury Stereo Datasets art and books ((a) and (c)) Some potential EG mismatches that are accepted by CF asmatches ((b) and (d)) some potential CF mismatches that are accepted by EG as matches 120572
119877= 0013 in (a) and (b) 120572
119877= 000018 in (c) and
(d) and 120572119868
= 10597
this study we investigated the principles of the CF andstudied its characteristics by comparing it with the widelyused parametric model epipolar geometry (EG) constraint
It is shown that the CF describes the mapping relation-ships between two CPs and should be able to estimate theposition of the corresponding point Therefore in additionto mismatch rejection a potential application of the CF is to
guide the point matching process by incorporating it into thecorrespondence propagation [39]
In practice putative correspondences are usually cor-rupted by noise and outliers and the number of putativecorrespondences is limited Therefore only an estimationof the CF with a certain accuracy can be estimated andthe actual results will violate the ldquonecessary and sufficient
Mathematical Problems in Engineering 13
(a) 119878EG119888cap SCF119888
(b) 119878EG119888cap 119878
CF119888
(c) 119878EG119888cap 119878
CF119888
(d) 119878EG119888cap 119878
CF119888
Figure 16 Evaluation on Middlebury Stereo Datasets computer and dolls ((a) and (c)) Some potential EG mismatches that are accepted byCF as matches ((b) and (d)) some potential CF mismatches that are accepted by EG as matches 120572
119877= 000018 in (a) and (b) 120572
119877= 000048 in
(c) and (d) and 120572119868
= 10597
(a) 119878EG119888cap 119878
CF119888
(b) 119878EG119888cap 119878
CF119888
(c) 119878EG119888cap 119878
CF119888
(d) 119878EG119888cap 119878
CF119888
Figure 17 Evaluation onMiddlebury Stereo Datasets drumsticks and dwarves ((a) and (c)) Some potential EGmismatches that are acceptedby CF as matches ((b) and (d)) some potential CF mismatches that are accepted by EG as matches 120572
119877= 000078 in (a) and (b) 120572
119877= 00035
in (c) and (d) and 120572119868
= 10597
conditionrdquoThe characteristic is that theCFmodel can reducethe searching space of CPs into an elliptical area and themismatches of the CF are usually near the correct CPs
Conflict of Interests
The authors declare that there is no conflict of interestsregarding the publication of this paper
Acknowledgments
The authors are grateful for valuable advice from DrDibyendu Mukherjee and the anonymous reviewers andwould like to thank the National Natural Science Foundationof China (Grant nos 61075033 61005033 and 61273248)the Natural Science Foundation of Guangdong Province(2014A030313425 and S2011010003348) and theOpenProject
14 Mathematical Problems in Engineering
(a) 119878EG119888cap 119878
CF119888
(b) 119878EG119888cap 119878
CF119888
(c) 119878EG119888cap 119878
CF119888
(d) 119878EG119888cap 119878
CF119888
Figure 18 Evaluation onMiddlebury Stereo Datasets laundry andmoebius ((a) and (c)) Some potential EGmismatches that are accepted byCF asmatches ((b) and (d)) some potential CFmismatches that are accepted by EG asmatches 120572
119877= 000000275 in (a) and (b) 120572
119877= 000035
in (c) and (d) and 120572119868
= 10597
(a) 119878EG119888cap 119878
CF119888
(b) 119878EG119888cap 119878
CF119888
Figure 19 Evaluation on Middlebury Stereo Dataset reindeer (a)Two potential EG mismatches that are accepted by CF as matches(b) two potential CFmismatches that are accepted by EG asmatches120572119877
= 0005 and 120572119868
= 30
Program of the National Laboratory of Pattern Recognition(NLPR) (201001060) for their support
References
[1] J P Monaco A C Bovik and L K Cormack ldquoActive foveateduncalibrated stereovisionrdquo International Journal of ComputerVision vol 85 no 2 pp 192ndash207 2009
[2] X Li and Z Hu ldquoRejecting mismatches by correspondencefunctionrdquo International Journal of Computer Vision vol 89 no1 pp 1ndash17 2010
[3] R Hartley and A Zisserman Multiple View Geometry inComputer Vision Cambridge University Press 2nd edition2003
[4] M Sonka V Hlavac and R Boyle Image Processing Analysisand Machine Vision Thomson-Engineering 3rd edition 2007
[5] G Carneiro and A D Jepson ldquoFlexible spatial configuration oflocal image featuresrdquo IEEE Transactions on Pattern Analysis andMachine Intelligence vol 29 no 12 pp 2089ndash2104 2007
[6] W Aguilar Y Frauel F Escolano M E Martinez-Perez AEspinosa-Romero andM A Lozano ldquoA robust graph transfor-mation matching for non-rigid registrationrdquo Image and VisionComputing vol 27 no 7 pp 897ndash910 2009
[7] L Wang F Duan and K Lv ldquoFisheye-lens-based visual suncompass for perception of spatial orientationrdquo MathematicalProblems in Engineering vol 2012 Article ID 460430 p 15 2012
[8] P Huber Robust Statistics Wiley New York NY USA 1981[9] P J Rousseeuw ldquoLeast median of squares regressionrdquo Journal
of the American Statistical Association vol 79 no 388 pp 871ndash880 1984
[10] P H S Torr and A Zisserman ldquoMLESAC a new robustestimator with application to estimating image geometryrdquoComputer Vision and Image Understanding vol 78 no 1 pp138ndash156 2000
[11] M A Fischler and R C Bolles ldquoRandom sample consensus aparadigm for model fitting with applications to image analysisand automated cartographyrdquo Communications of the ACM vol24 no 6 pp 381ndash395 1981
[12] B J Tordoff and D W Murray ldquoGuided-MLESAC fasterimage transform estimation by using matching priorsrdquo IEEETransactions on Pattern Analysis and Machine Intelligence vol27 no 10 pp 1523ndash1535 2005
Mathematical Problems in Engineering 15
[13] D Sidibe P Montesinos and S Janaqi ldquoMatching localinvariant features with contextual information an experimentalevaluationrdquo Electronic Letters on Computer Vision and ImageAnalysis vol 7 no 1 pp 26ndash39 2008
[14] Q-H Tran T-J Chin G Carneiro M S Brown and D SuterldquoIn defence of RANSAC for outlier rejection in deformableregistrationrdquo Lecture Notes in Computer Science vol 7575 no4 pp 274ndash287 2012
[15] A Goshtasby ldquoTransformation functions for image registra-tionrdquo in Proceedings of the Computer Vision and Pattern Recog-nition (CVPR rsquo11) June 2011
[16] X Li Y Wang J Zhou and X Li ldquoComparative research oncorrespondence functionrdquo in Proceedings of the InternationalConference on Intelligent System Design and Engineering Appli-cation (ISDEA rsquo10) pp 444ndash447 October 2010
[17] X-R Li X-M Li H-L Li and M-Y Cao ldquoRejecting outliersbased on correspondence manifoldrdquo Acta Automatica Sinicavol 35 no 1 pp 17ndash22 2009
[18] D Pizarro and A Bartoli ldquoFeature-based deformable surfacedetection with self-occlusion reasoningrdquo International Journalof Computer Vision vol 97 no 1 pp 54ndash70 2012
[19] J Yu A Eriksson T-J Chin and D Suter ldquoAn adversarialoptimization approach to efficient outlier removalrdquo Journal ofMathematical Imaging and Vision vol 48 no 3 pp 451ndash4662014
[20] A Goshtasby ldquoRegistration of images with geometric distor-tionsrdquo IEEETransactions onGeoscience andRemote Sensing vol26 no 1 pp 60ndash64 1988
[21] F L Bookstein ldquoPrincipal warps thin-plate splines and thedecomposition of deformationsrdquo IEEE Transactions on PatternAnalysis and Machine Intelligence vol 11 no 6 pp 567ndash5851992
[22] B Scholkopf F Steinke and V Blanz ldquoObject correspondenceas a machine learning problemrdquo in Proceedings of the 22ndInternational Conference on Machine Learning (ICML rsquo05) pp776ndash783 2005
[23] L Zagorchev and A Goshtasby ldquoA comparative study oftransformation functions for nonrigid image registrationrdquo IEEETransactions on Image Processing vol 15 no 3 pp 529ndash5382006
[24] J Zhao JMa J Tian J Ma andD Zhang ldquoA robustmethod forvector field learning with application to mismatch removingrdquoin Proceedings of the IEEE Conference on Computer Vision andPattern Recognition (CVPR rsquo11) Providence RI USA June 2011
[25] JMa J Zhao J Tian X Bai and Z Tu ldquoRegularized vector fieldlearning with sparse approximation for mismatch removalrdquoPattern Recognition vol 46 no 12 pp 3519ndash3532 2013
[26] J Ma J Zhao J Tian Z Tu and A L Yuille ldquoRobust estimationof nonrigid transformation for point set registrationrdquo in Pro-ceedings of the 26th IEEE Conference on Computer Vision andPattern Recognition (CVPR rsquo13) pp 2147ndash2154 June 2013
[27] J MaW Qiu J Zhao Y Ma A L Yuille and Z Tu ldquoRobust l2e
estimation of transformation for non-rigid registrationrdquo IEEETransactions on Signal Processing vol 63 no 5 pp 1115ndash11292015
[28] J Ma J Zhao J Tian A L Yuille and Z Tu ldquoRobust pointmatching via vector field consensusrdquo IEEE Transactions onImage Processing vol 23 no 4 pp 1706ndash1721 2014
[29] L Goshen and I Shimshoni ldquoBalanced exploration andexploitation model search for efficient epipolar geometry esti-mationrdquo IEEE Transactions on Pattern Analysis and MachineIntelligence vol 30 no 7 pp 1230ndash1242 2008
[30] O Chum and J Matas ldquoMatching with prosacmdashprogressivesample consensusrdquo in Proceedings of the IEEE Computer SocietyConference on Computer Vision and Pattern Recognition (CVPRrsquo05) vol 1 pp 220ndash226 IEEE June 2005
[31] K Mikolajczyk and C Schmid ldquoA performance evaluation oflocal descriptorsrdquo IEEE Transactions on Pattern Analysis andMachine Intelligence vol 27 no 10 pp 1615ndash1630 2005
[32] D G Lowe ldquoDistinctive image features from scale-invariantkeypointsrdquo International Journal of Computer Vision vol 60 no2 pp 91ndash110 2004
[33] S Gauglitz T Hollerer and M Turk ldquoEvaluation of interestpoint detectors and feature descriptors for visual trackingrdquoInternational Journal of Computer Vision vol 94 no 3 pp 335ndash360 2011
[34] Y Yu K Huang W Chen and T Tan ldquoA novel algorithmfor view and illumination invariant image matchingrdquo IEEETransactions on Image Processing vol 21 no 1 pp 229ndash2402012
[35] K Mikolajczyk T Tuytelaars C Schmid et al ldquoA comparisonof affine region detectorsrdquo International Journal of ComputerVision vol 65 no 1-2 pp 43ndash72 2005
[36] H Hirschmuller and D Scharstein ldquoEvaluation of cost func-tions for stereo matchingrdquo in Proceedings of the IEEE ComputerSociety Conference on Computer Vision and Pattern Recognition(CVPR rsquo07) June 2007
[37] D Scharstein and C Pal ldquoLearning conditional random fieldsfor stereordquo in Proceedings of the IEEE Computer Society Confer-ence on Computer Vision and Pattern Recognition (CVPR rsquo07)IEEE Minneapolis Minn USA June 2007
[38] D Scharstein andR Szeliski ldquoHigh-accuracy stereo depthmapsusing structured lightrdquo in Proceedings of the IEEE ComputerSociety Conference on Computer Vision and Pattern Recognition(CVPR rsquo03) vol 1 pp I-195ndashI-202 IEEE June 2003
[39] H Wang S Yan J Liu X Tang and T S Huang ldquoCorre-spondence propagationwithweak priorsrdquo IEEETransactions onImage Processing vol 18 no 1 pp 140ndash150 2009
Figure 5 An image pair of a toy bear (a) 26 EG potential mismatches (PMs) that are accepted by CF as matches (b) 26 CF PMs that areaccepted by EG as matches (c) some typical EG PMs that are accepted by CF (d) some typical CF PMs that are accepted by EG 120572
119868= 10597
and 120572119877
= 000019
(a) 119878EG119888cap 119878
CF119888
(b) 119878EG119888cap 119878
CF119888
(c) 119878EG119888cap 119878
CF119888
(d) 119878EG119888cap 119878
CF119888
Figure 6 An image pair of a corridor (a) 25 EG PMs that are accepted by CF as matches (b) 25 CF PMs that are accepted by EG as matches(c) 31 EG PMs that are accepted by CF as matches (d) 31 CF PMs that are accepted by EG as matches In (a)-(b) 120572
119868= 5991 and 120572
119877= 000048
and in (c)-(d) 120572119868
= 110 and 120572119877
= 005
computed from the feature points by the NNDR (Near-est Neighbor and Distance Ratio) method [32] Due tothe ambiguity of local information some of the putativecorrespondences are incorrect and need to be rejected Inthis experiment the two methods based on EG and CFare inconsistent on 52 putative correspondences (Figures5(a) and 5(b)) For example for every putative correspon-dence (p p1015840) in Figure 5(a) the CF method identified it as
a possible correct match but the EG method identified it asa possible mismatch Therefore based on the definition of119878EG119888
cap 119878CF119888
and 119878EG119888
cap 119878CF119888 we can regard 119878
EG119888
cap 119878CF119888
as a setof potential mismatches that the EG constraint fails to detectand 119878
EG119888
cap 119878CF119888
as a set of potential mismatches that the CFmodel fails to reject More results are presented in Figures 67 and 8
8 Mathematical Problems in Engineering
(a) 119878EG119888cap 119878
CF119888
(b) 119878EG119888cap 119878
CF119888
(c) 119879(119878EG119888cap 119878
CF119888) (d) 119879(119878EG
119888cap 119878
CF119888)
Figure 7 An image pair of Baxia (a) Some EG PMs that are accepted by CF asmatches (b) some CF PMs that are accepted by EG asmatches(c) some typical EG PMs that are accepted by CF (d) some typical CF PMs that are accepted by EG 120572
119868= 10597 and 120572
119877= 005
(a) 119878EG119888cap 119878
CF119888
(b) 119878EG119888cap 119878
CF119888
(c) 119879(119878EGc cap 119878CF119888) (d) 119879(119878EG
119888cap 119878
CF119888)
Figure 8 An image pair of temple Fayu (a) 204 EG PMs that are accepted by CF as matches (b) 204 CF PMs that are accepted by EG asmatches (c) some typical EG PMs that are accepted by CF (d) some typical CF PMs that are accepted by EG 120572
119868= 31 and 120572
119877= 00001515
(for visibility only 100 randomly chosen PMs are presented in (a) and (b))
522 Comparing the CF and EG Constraint by Analyzing theUndetected Mismatches For ease of explanation we intro-duce the following two definitions type-I mismatches andtype-II mismatches Suppose (p p1015840) is a mismatch If thereexists a correct corresponding point pair (pp1015840) satisfying thatthe points p and p1015840 are in the neighborhood of p and p1015840respectively then (p p1015840) is defined as a mismatch of type-IOtherwise we call (p p1015840) a type-II mismatch For example
the mismatches in Figure 7(d) and (p p10158403) in Figure 3(b)
belong to type-II and (p p10158402) in Figure 3(b) belong to type-
I By the EG constraint we can only determine whether aputative correspondence is in a band around the epipolar line(Figures 3(b) 9(b) and 9(d)) but not whether it is close to thecorrect one Therefore the undetected mismatches usuallybelong to type-II in the methods based on the EG constraint(Figures 3 5(b) 5(d) 6(b) 6(d) 7(b) 7(d) 8(b) and 8(d))
Mathematical Problems in Engineering 9
l1
l2
p1
l3p3 p2
l4p4
l9984001 p9984001
l9984002 p9984002
l9984003 p9984003
l9984004p9984004
(a)
l1
l3
p1
p3
l2 p2
l9984001p9984001
l9984002p9984002
l9984003p9984003
(b)
l2p2
l1 p1l3p3
l9984001p9984001
l9984002p9984002
l9984003 p9984003
(c)
l1
l6
p1
p6
l3
l4 p4
p3
l2
l5
p2
p5
l9984001p9984001
l9984002 p9984002
l9984003
l9984006
p9984003
p9984006
l9984004 p9984004
l9984005p9984005
(d)
Figure 9 Some PMs and their corresponding epipolar lines The EG model cannot detect the mismatches that are near the epipolar line butfar from the correct CPs The undetected mismatches by CF are usually near both the correct CPs and the epipolar line (a) and (c) presentthe EG PMs that are accepted by CF as matches in the experiments of Figures 7(c) and 8(c) respectively (b) and (d) present the CF PMs thatare accepted by EG as matches in the experiments of Figures 7(d) and 8(d) respectively In these experiments (p
119894 p1015840119894) is a pair of putative
CPs and l119894is the epipolar line corresponding to p1015840
119894and l1015840119894is the epipolar line corresponding to p
119894
The CF constraint determines the mapping relationshipsbetween two CPs and directly estimates the position of thecorresponding point Therefore by the mismatch-rejectingmethods based on CF we can restrict the error bound ofthe putative correspondences by a tolerance parameter 120572
119868
the undetected mismatches by CF usually belong to type-I To show the difference between the CF model and EGconstraints a typical result is presented in Figure 6 in whichwe relax the tolerance parameters 120572
119868and 120572119877to ensure the two
sets 119878EG119888
cap 119878CF119888
and 119878EG119888
cap 119878CF119888
change at equal speedHowever themismatches of type-I and type-II are usually
dramatically different from each other in usefulness andundue influence From the mismatches of type-I we cansearch for the correct CPs by including more informationOn the contrary other than ruining the traditional estimationmethods and distorting the final application results (egimage mosaicing and three-dimensional reconstruction) themismatches of the type-II provide less useful information
53 Further Evaluation on Some Popular Benchmark DataThe characteristics of the CF are also investigated on twoseries of popular benchmark data sets widely used to compareand evaluate related algorithms [33 34]
To evaluate the two schemes quantitatively this workconducted experiments on a series of image sets with ground-truth correspondences [31 35] The images were capturedunder special consideration to enable the encoding of theground-truth correspondences with homographies between
the reference image and other images The ground-truthhomographies are computed based on the following stepsfirst a set of putative correspondences are manually selectedand an initial estimation of the homography is computedbased on the selected putative correspondences second theimages are approximately aligned with the rough estimationof homography third reliable interest points are detectedand matched automatically and an accurate homography iscomputed from them
The experimental results are presented in Figures 11 12 13and 14 In every experiment the upper image and the lowerimage are called the reference image and the target imagerespectively In addition to the computed putative correspon-dences we also present the ground-truth CPs and ground-truth correspondence relationships in these experimentsTheground-truth CPs in the target images are computed for all ofthe marked interest points in reference images based on theground-truth homographies and denoted by red circles Thedetected interest points are denoted by black plus signs Theground-truth correspondence relationships are displayed asblue dashed lines and the computed putative correspondencerelationships are shown by a solid line with red or green forvisibility
Suppose that 119878 = (p1 p10158401) (p
119899 p1015840119899) are a set of
putative correspondences and 119878 = (p1 p10158401) (p
119899 p1015840119899) are
the ground-truth correspondences of 119878 We quantitativelyevaluate the quality of 119878 by the root-mean-square deviation(RMSD) RMSD(119878) = radicsum
Figure 10The possible correct correspondences identified by CF in the experiment of Figure 8 are presented based on depth (a) 34 putativecorrespondences (PCs) over the first crossbeam (b) 324 PCs over the second crossbeam (c) 214 PCs over the third crossbeam (d) 20 PCsaround the fourth crossbeam (e) 2 PCs on the wall (for clarity only 50 randomly chosen PCs are presented in (b) and (c))
The second series of evaluation data are the Middle-bury Stereo Datasets [36 37] (httpvisionmiddleburyedustereodata) These data sets were created using an auto-mated version of structured-lighting technique [38] Eachdataset consists of seven rectified views This work used thefirst and the fifth views for evaluation The experimentalresults are presented in Figures 15 16 17 18 and 19
54 CF and EG in Practice
541 Necessity and Sufficiency As analyzed the EG con-straint is a necessary but insufficient condition in the corre-spondence problem and theCFmodel describes themappingbetween CPs directly However in applications some correctCPsmay be inconsistent with the estimated EGmodel andorthe estimated CF model there may be some mismatchesthat are ldquoconsistentrdquo with the estimated CF model there alsomay be some of the putative correspondences which areldquoconsistentrdquo with the estimated CF but ldquoinconsistentrdquo withthe estimated EG for example the experimental results inFigures 5(a) 5(b) 7(a) 7(b) 8(a) and 8(b) The reasons forthe above results are as follows
(i) The coordinates of the obtained putative CPs areusually corrupted by noise
(ii) There are usually some errors in the estimation of theCF and EG
(iii) As pointed out in the first paragraph of Section 5a preset tolerance parameter is needed to define theldquoconsistentrdquo and ldquoinconsistentrdquo in the applicationsof CF and EG However the noise in the obtainedputative CPs is random
542 Characteristics and Limitations of the CF Model andEG Model In the two models there are no restrictionson the depth range of the imaged scene However theEG is a global describing model and the CF is a localdescribing model In the EG model any pair of putativeCPs can influence the overall model estimation whereas inthe CF model a pair of putative CPs only influence themodel estimation within a local area in their vicinity Theshortcoming of the global model EG is that the overallmodel estimation can be degraded or ruined by PCs withconsiderable noise or mismatches in any area of the imagesThe shortcoming of the local model CF is that a pair of goodCPs cannot improve the estimation of CF outside of a localarea around itself Thus the uneven distribution of putativecorrespondences with different quality results in the fact thatthe estimated CF may be of high quality in some areas butof low quality in others for example in the experiment ofFigure 10 the qualities of the CF estimations over the second-nearest-to-the-cameras crossbeam and the third-nearest-to-the-cameras crossbeam are better than estimation over thefirst-nearest-to-the-cameras crossbeam The ldquolocal areardquo isdetermined by the CF estimation method and its chosenparameters for example in the CF estimation method SP(SVM) [2] the local area is determined by the chosen kerneland the scale parameter The above limitations of EG and CFcan be alleviated by investigating robust estimation methodsfor example the RANSAC M-estimators [8] LMedS (LeastMedian of Squares) [9] and MLESAC [10 12] for EGIn future work we will investigate further the estimationmethod for CF to improve its robustness
6 Conclusion
The CF is a recently introduced nonparametric model forrejecting mismatchesoutliers in image point matching In
Mathematical Problems in Engineering 11
(a) RMSD = 1386 (b) RMSD = 22707 (c) RMSD = 384 (d) RMSD = 23626Figure 11 Quantitative evaluation on two popular benchmark sets bikes and leuven The ground-truth CPs are presented by red circles thedetected interest points by black plus signs The ground-truth correspondences are displayed as blue dashed lines and the computed putativecorrespondences are shown by a solid green solid line (a) and (c) present some EG PMs that are accepted by CF as matches (b) and (d)present some CF PMs that are accepted by EG as matches 120572
119868= 10597 120572
119877= 21435 times 10
minus5 in (b) and 120572119877
= 58 times 10minus5 in (d)
(a) RMSD = 1631 (b) RMSD = 40098 (c) RMSD = 358 (d) RMSD = 32288Figure 12 Quantitative evaluation on two popular benchmark sets wall and bark ((a) and (c)) The EG PMs that are accepted by CF asmatches ((b) and (d)) the CF PMs that are accepted by EG as matches 120572
119868= 10597 120572
119877= 2029 times 10
minus4 in (b) and 120572119877
= 105 times 10minus4 in (d)
(a) RMSD = 1017 (b) RMSD = 42639 (c) RMSD = 329 (d) RMSD = 17646Figure 13 Quantitative evaluation on two popular benchmark sets graf and boat The ground-truth CPs are presented by red circles and thedetected interest points by black plus signs The ground-truth correspondences are displayed as blue dashed lines and the computed putativecorrespondences are shown by a solid green solid line (a) and (c) present some EG PMs that are accepted by CF as matches (b) and (d)present some CF PMs that are accepted by EG as matches 120572
Figure 14 Quantitative evaluation on two popular benchmark sets trees and ubc The ground-truth correspondences are displayed as bluedashed lines and the computed putative correspondences are shown by a yellow solid line (a) and (c) present some EG PMs that are acceptedbyCF asmatches (b) and (d) present someCFPMs that are accepted by EG asmatches120572
119868= 10597120572
119877= 59times10
minus5 in (b) and120572119877
= 185times10minus4
in (d)
(a) 119878EG119888cap 119878
CF119888
(b) 119878EG119888cap 119878
CF119888
(c) 119878EG119888cap 119878
CF119888
(d) 119878EG119888cap 119878
CF119888
Figure 15 Evaluation on Middlebury Stereo Datasets art and books ((a) and (c)) Some potential EG mismatches that are accepted by CF asmatches ((b) and (d)) some potential CF mismatches that are accepted by EG as matches 120572
119877= 0013 in (a) and (b) 120572
119877= 000018 in (c) and
(d) and 120572119868
= 10597
this study we investigated the principles of the CF andstudied its characteristics by comparing it with the widelyused parametric model epipolar geometry (EG) constraint
It is shown that the CF describes the mapping relation-ships between two CPs and should be able to estimate theposition of the corresponding point Therefore in additionto mismatch rejection a potential application of the CF is to
guide the point matching process by incorporating it into thecorrespondence propagation [39]
In practice putative correspondences are usually cor-rupted by noise and outliers and the number of putativecorrespondences is limited Therefore only an estimationof the CF with a certain accuracy can be estimated andthe actual results will violate the ldquonecessary and sufficient
Mathematical Problems in Engineering 13
(a) 119878EG119888cap SCF119888
(b) 119878EG119888cap 119878
CF119888
(c) 119878EG119888cap 119878
CF119888
(d) 119878EG119888cap 119878
CF119888
Figure 16 Evaluation on Middlebury Stereo Datasets computer and dolls ((a) and (c)) Some potential EG mismatches that are accepted byCF as matches ((b) and (d)) some potential CF mismatches that are accepted by EG as matches 120572
119877= 000018 in (a) and (b) 120572
119877= 000048 in
(c) and (d) and 120572119868
= 10597
(a) 119878EG119888cap 119878
CF119888
(b) 119878EG119888cap 119878
CF119888
(c) 119878EG119888cap 119878
CF119888
(d) 119878EG119888cap 119878
CF119888
Figure 17 Evaluation onMiddlebury Stereo Datasets drumsticks and dwarves ((a) and (c)) Some potential EGmismatches that are acceptedby CF as matches ((b) and (d)) some potential CF mismatches that are accepted by EG as matches 120572
119877= 000078 in (a) and (b) 120572
119877= 00035
in (c) and (d) and 120572119868
= 10597
conditionrdquoThe characteristic is that theCFmodel can reducethe searching space of CPs into an elliptical area and themismatches of the CF are usually near the correct CPs
Conflict of Interests
The authors declare that there is no conflict of interestsregarding the publication of this paper
Acknowledgments
The authors are grateful for valuable advice from DrDibyendu Mukherjee and the anonymous reviewers andwould like to thank the National Natural Science Foundationof China (Grant nos 61075033 61005033 and 61273248)the Natural Science Foundation of Guangdong Province(2014A030313425 and S2011010003348) and theOpenProject
14 Mathematical Problems in Engineering
(a) 119878EG119888cap 119878
CF119888
(b) 119878EG119888cap 119878
CF119888
(c) 119878EG119888cap 119878
CF119888
(d) 119878EG119888cap 119878
CF119888
Figure 18 Evaluation onMiddlebury Stereo Datasets laundry andmoebius ((a) and (c)) Some potential EGmismatches that are accepted byCF asmatches ((b) and (d)) some potential CFmismatches that are accepted by EG asmatches 120572
119877= 000000275 in (a) and (b) 120572
119877= 000035
in (c) and (d) and 120572119868
= 10597
(a) 119878EG119888cap 119878
CF119888
(b) 119878EG119888cap 119878
CF119888
Figure 19 Evaluation on Middlebury Stereo Dataset reindeer (a)Two potential EG mismatches that are accepted by CF as matches(b) two potential CFmismatches that are accepted by EG asmatches120572119877
= 0005 and 120572119868
= 30
Program of the National Laboratory of Pattern Recognition(NLPR) (201001060) for their support
References
[1] J P Monaco A C Bovik and L K Cormack ldquoActive foveateduncalibrated stereovisionrdquo International Journal of ComputerVision vol 85 no 2 pp 192ndash207 2009
[2] X Li and Z Hu ldquoRejecting mismatches by correspondencefunctionrdquo International Journal of Computer Vision vol 89 no1 pp 1ndash17 2010
[3] R Hartley and A Zisserman Multiple View Geometry inComputer Vision Cambridge University Press 2nd edition2003
[4] M Sonka V Hlavac and R Boyle Image Processing Analysisand Machine Vision Thomson-Engineering 3rd edition 2007
[5] G Carneiro and A D Jepson ldquoFlexible spatial configuration oflocal image featuresrdquo IEEE Transactions on Pattern Analysis andMachine Intelligence vol 29 no 12 pp 2089ndash2104 2007
[6] W Aguilar Y Frauel F Escolano M E Martinez-Perez AEspinosa-Romero andM A Lozano ldquoA robust graph transfor-mation matching for non-rigid registrationrdquo Image and VisionComputing vol 27 no 7 pp 897ndash910 2009
[7] L Wang F Duan and K Lv ldquoFisheye-lens-based visual suncompass for perception of spatial orientationrdquo MathematicalProblems in Engineering vol 2012 Article ID 460430 p 15 2012
[8] P Huber Robust Statistics Wiley New York NY USA 1981[9] P J Rousseeuw ldquoLeast median of squares regressionrdquo Journal
of the American Statistical Association vol 79 no 388 pp 871ndash880 1984
[10] P H S Torr and A Zisserman ldquoMLESAC a new robustestimator with application to estimating image geometryrdquoComputer Vision and Image Understanding vol 78 no 1 pp138ndash156 2000
[11] M A Fischler and R C Bolles ldquoRandom sample consensus aparadigm for model fitting with applications to image analysisand automated cartographyrdquo Communications of the ACM vol24 no 6 pp 381ndash395 1981
[12] B J Tordoff and D W Murray ldquoGuided-MLESAC fasterimage transform estimation by using matching priorsrdquo IEEETransactions on Pattern Analysis and Machine Intelligence vol27 no 10 pp 1523ndash1535 2005
Mathematical Problems in Engineering 15
[13] D Sidibe P Montesinos and S Janaqi ldquoMatching localinvariant features with contextual information an experimentalevaluationrdquo Electronic Letters on Computer Vision and ImageAnalysis vol 7 no 1 pp 26ndash39 2008
[14] Q-H Tran T-J Chin G Carneiro M S Brown and D SuterldquoIn defence of RANSAC for outlier rejection in deformableregistrationrdquo Lecture Notes in Computer Science vol 7575 no4 pp 274ndash287 2012
[15] A Goshtasby ldquoTransformation functions for image registra-tionrdquo in Proceedings of the Computer Vision and Pattern Recog-nition (CVPR rsquo11) June 2011
[16] X Li Y Wang J Zhou and X Li ldquoComparative research oncorrespondence functionrdquo in Proceedings of the InternationalConference on Intelligent System Design and Engineering Appli-cation (ISDEA rsquo10) pp 444ndash447 October 2010
[17] X-R Li X-M Li H-L Li and M-Y Cao ldquoRejecting outliersbased on correspondence manifoldrdquo Acta Automatica Sinicavol 35 no 1 pp 17ndash22 2009
[18] D Pizarro and A Bartoli ldquoFeature-based deformable surfacedetection with self-occlusion reasoningrdquo International Journalof Computer Vision vol 97 no 1 pp 54ndash70 2012
[19] J Yu A Eriksson T-J Chin and D Suter ldquoAn adversarialoptimization approach to efficient outlier removalrdquo Journal ofMathematical Imaging and Vision vol 48 no 3 pp 451ndash4662014
[20] A Goshtasby ldquoRegistration of images with geometric distor-tionsrdquo IEEETransactions onGeoscience andRemote Sensing vol26 no 1 pp 60ndash64 1988
[21] F L Bookstein ldquoPrincipal warps thin-plate splines and thedecomposition of deformationsrdquo IEEE Transactions on PatternAnalysis and Machine Intelligence vol 11 no 6 pp 567ndash5851992
[22] B Scholkopf F Steinke and V Blanz ldquoObject correspondenceas a machine learning problemrdquo in Proceedings of the 22ndInternational Conference on Machine Learning (ICML rsquo05) pp776ndash783 2005
[23] L Zagorchev and A Goshtasby ldquoA comparative study oftransformation functions for nonrigid image registrationrdquo IEEETransactions on Image Processing vol 15 no 3 pp 529ndash5382006
[24] J Zhao JMa J Tian J Ma andD Zhang ldquoA robustmethod forvector field learning with application to mismatch removingrdquoin Proceedings of the IEEE Conference on Computer Vision andPattern Recognition (CVPR rsquo11) Providence RI USA June 2011
[25] JMa J Zhao J Tian X Bai and Z Tu ldquoRegularized vector fieldlearning with sparse approximation for mismatch removalrdquoPattern Recognition vol 46 no 12 pp 3519ndash3532 2013
[26] J Ma J Zhao J Tian Z Tu and A L Yuille ldquoRobust estimationof nonrigid transformation for point set registrationrdquo in Pro-ceedings of the 26th IEEE Conference on Computer Vision andPattern Recognition (CVPR rsquo13) pp 2147ndash2154 June 2013
[27] J MaW Qiu J Zhao Y Ma A L Yuille and Z Tu ldquoRobust l2e
estimation of transformation for non-rigid registrationrdquo IEEETransactions on Signal Processing vol 63 no 5 pp 1115ndash11292015
[28] J Ma J Zhao J Tian A L Yuille and Z Tu ldquoRobust pointmatching via vector field consensusrdquo IEEE Transactions onImage Processing vol 23 no 4 pp 1706ndash1721 2014
[29] L Goshen and I Shimshoni ldquoBalanced exploration andexploitation model search for efficient epipolar geometry esti-mationrdquo IEEE Transactions on Pattern Analysis and MachineIntelligence vol 30 no 7 pp 1230ndash1242 2008
[30] O Chum and J Matas ldquoMatching with prosacmdashprogressivesample consensusrdquo in Proceedings of the IEEE Computer SocietyConference on Computer Vision and Pattern Recognition (CVPRrsquo05) vol 1 pp 220ndash226 IEEE June 2005
[31] K Mikolajczyk and C Schmid ldquoA performance evaluation oflocal descriptorsrdquo IEEE Transactions on Pattern Analysis andMachine Intelligence vol 27 no 10 pp 1615ndash1630 2005
[32] D G Lowe ldquoDistinctive image features from scale-invariantkeypointsrdquo International Journal of Computer Vision vol 60 no2 pp 91ndash110 2004
[33] S Gauglitz T Hollerer and M Turk ldquoEvaluation of interestpoint detectors and feature descriptors for visual trackingrdquoInternational Journal of Computer Vision vol 94 no 3 pp 335ndash360 2011
[34] Y Yu K Huang W Chen and T Tan ldquoA novel algorithmfor view and illumination invariant image matchingrdquo IEEETransactions on Image Processing vol 21 no 1 pp 229ndash2402012
[35] K Mikolajczyk T Tuytelaars C Schmid et al ldquoA comparisonof affine region detectorsrdquo International Journal of ComputerVision vol 65 no 1-2 pp 43ndash72 2005
[36] H Hirschmuller and D Scharstein ldquoEvaluation of cost func-tions for stereo matchingrdquo in Proceedings of the IEEE ComputerSociety Conference on Computer Vision and Pattern Recognition(CVPR rsquo07) June 2007
[37] D Scharstein and C Pal ldquoLearning conditional random fieldsfor stereordquo in Proceedings of the IEEE Computer Society Confer-ence on Computer Vision and Pattern Recognition (CVPR rsquo07)IEEE Minneapolis Minn USA June 2007
[38] D Scharstein andR Szeliski ldquoHigh-accuracy stereo depthmapsusing structured lightrdquo in Proceedings of the IEEE ComputerSociety Conference on Computer Vision and Pattern Recognition(CVPR rsquo03) vol 1 pp I-195ndashI-202 IEEE June 2003
[39] H Wang S Yan J Liu X Tang and T S Huang ldquoCorre-spondence propagationwithweak priorsrdquo IEEETransactions onImage Processing vol 18 no 1 pp 140ndash150 2009
Figure 7 An image pair of Baxia (a) Some EG PMs that are accepted by CF asmatches (b) some CF PMs that are accepted by EG asmatches(c) some typical EG PMs that are accepted by CF (d) some typical CF PMs that are accepted by EG 120572
119868= 10597 and 120572
119877= 005
(a) 119878EG119888cap 119878
CF119888
(b) 119878EG119888cap 119878
CF119888
(c) 119879(119878EGc cap 119878CF119888) (d) 119879(119878EG
119888cap 119878
CF119888)
Figure 8 An image pair of temple Fayu (a) 204 EG PMs that are accepted by CF as matches (b) 204 CF PMs that are accepted by EG asmatches (c) some typical EG PMs that are accepted by CF (d) some typical CF PMs that are accepted by EG 120572
119868= 31 and 120572
119877= 00001515
(for visibility only 100 randomly chosen PMs are presented in (a) and (b))
522 Comparing the CF and EG Constraint by Analyzing theUndetected Mismatches For ease of explanation we intro-duce the following two definitions type-I mismatches andtype-II mismatches Suppose (p p1015840) is a mismatch If thereexists a correct corresponding point pair (pp1015840) satisfying thatthe points p and p1015840 are in the neighborhood of p and p1015840respectively then (p p1015840) is defined as a mismatch of type-IOtherwise we call (p p1015840) a type-II mismatch For example
the mismatches in Figure 7(d) and (p p10158403) in Figure 3(b)
belong to type-II and (p p10158402) in Figure 3(b) belong to type-
I By the EG constraint we can only determine whether aputative correspondence is in a band around the epipolar line(Figures 3(b) 9(b) and 9(d)) but not whether it is close to thecorrect one Therefore the undetected mismatches usuallybelong to type-II in the methods based on the EG constraint(Figures 3 5(b) 5(d) 6(b) 6(d) 7(b) 7(d) 8(b) and 8(d))
Mathematical Problems in Engineering 9
l1
l2
p1
l3p3 p2
l4p4
l9984001 p9984001
l9984002 p9984002
l9984003 p9984003
l9984004p9984004
(a)
l1
l3
p1
p3
l2 p2
l9984001p9984001
l9984002p9984002
l9984003p9984003
(b)
l2p2
l1 p1l3p3
l9984001p9984001
l9984002p9984002
l9984003 p9984003
(c)
l1
l6
p1
p6
l3
l4 p4
p3
l2
l5
p2
p5
l9984001p9984001
l9984002 p9984002
l9984003
l9984006
p9984003
p9984006
l9984004 p9984004
l9984005p9984005
(d)
Figure 9 Some PMs and their corresponding epipolar lines The EG model cannot detect the mismatches that are near the epipolar line butfar from the correct CPs The undetected mismatches by CF are usually near both the correct CPs and the epipolar line (a) and (c) presentthe EG PMs that are accepted by CF as matches in the experiments of Figures 7(c) and 8(c) respectively (b) and (d) present the CF PMs thatare accepted by EG as matches in the experiments of Figures 7(d) and 8(d) respectively In these experiments (p
119894 p1015840119894) is a pair of putative
CPs and l119894is the epipolar line corresponding to p1015840
119894and l1015840119894is the epipolar line corresponding to p
119894
The CF constraint determines the mapping relationshipsbetween two CPs and directly estimates the position of thecorresponding point Therefore by the mismatch-rejectingmethods based on CF we can restrict the error bound ofthe putative correspondences by a tolerance parameter 120572
119868
the undetected mismatches by CF usually belong to type-I To show the difference between the CF model and EGconstraints a typical result is presented in Figure 6 in whichwe relax the tolerance parameters 120572
119868and 120572119877to ensure the two
sets 119878EG119888
cap 119878CF119888
and 119878EG119888
cap 119878CF119888
change at equal speedHowever themismatches of type-I and type-II are usually
dramatically different from each other in usefulness andundue influence From the mismatches of type-I we cansearch for the correct CPs by including more informationOn the contrary other than ruining the traditional estimationmethods and distorting the final application results (egimage mosaicing and three-dimensional reconstruction) themismatches of the type-II provide less useful information
53 Further Evaluation on Some Popular Benchmark DataThe characteristics of the CF are also investigated on twoseries of popular benchmark data sets widely used to compareand evaluate related algorithms [33 34]
To evaluate the two schemes quantitatively this workconducted experiments on a series of image sets with ground-truth correspondences [31 35] The images were capturedunder special consideration to enable the encoding of theground-truth correspondences with homographies between
the reference image and other images The ground-truthhomographies are computed based on the following stepsfirst a set of putative correspondences are manually selectedand an initial estimation of the homography is computedbased on the selected putative correspondences second theimages are approximately aligned with the rough estimationof homography third reliable interest points are detectedand matched automatically and an accurate homography iscomputed from them
The experimental results are presented in Figures 11 12 13and 14 In every experiment the upper image and the lowerimage are called the reference image and the target imagerespectively In addition to the computed putative correspon-dences we also present the ground-truth CPs and ground-truth correspondence relationships in these experimentsTheground-truth CPs in the target images are computed for all ofthe marked interest points in reference images based on theground-truth homographies and denoted by red circles Thedetected interest points are denoted by black plus signs Theground-truth correspondence relationships are displayed asblue dashed lines and the computed putative correspondencerelationships are shown by a solid line with red or green forvisibility
Suppose that 119878 = (p1 p10158401) (p
119899 p1015840119899) are a set of
putative correspondences and 119878 = (p1 p10158401) (p
119899 p1015840119899) are
the ground-truth correspondences of 119878 We quantitativelyevaluate the quality of 119878 by the root-mean-square deviation(RMSD) RMSD(119878) = radicsum
Figure 10The possible correct correspondences identified by CF in the experiment of Figure 8 are presented based on depth (a) 34 putativecorrespondences (PCs) over the first crossbeam (b) 324 PCs over the second crossbeam (c) 214 PCs over the third crossbeam (d) 20 PCsaround the fourth crossbeam (e) 2 PCs on the wall (for clarity only 50 randomly chosen PCs are presented in (b) and (c))
The second series of evaluation data are the Middle-bury Stereo Datasets [36 37] (httpvisionmiddleburyedustereodata) These data sets were created using an auto-mated version of structured-lighting technique [38] Eachdataset consists of seven rectified views This work used thefirst and the fifth views for evaluation The experimentalresults are presented in Figures 15 16 17 18 and 19
54 CF and EG in Practice
541 Necessity and Sufficiency As analyzed the EG con-straint is a necessary but insufficient condition in the corre-spondence problem and theCFmodel describes themappingbetween CPs directly However in applications some correctCPsmay be inconsistent with the estimated EGmodel andorthe estimated CF model there may be some mismatchesthat are ldquoconsistentrdquo with the estimated CF model there alsomay be some of the putative correspondences which areldquoconsistentrdquo with the estimated CF but ldquoinconsistentrdquo withthe estimated EG for example the experimental results inFigures 5(a) 5(b) 7(a) 7(b) 8(a) and 8(b) The reasons forthe above results are as follows
(i) The coordinates of the obtained putative CPs areusually corrupted by noise
(ii) There are usually some errors in the estimation of theCF and EG
(iii) As pointed out in the first paragraph of Section 5a preset tolerance parameter is needed to define theldquoconsistentrdquo and ldquoinconsistentrdquo in the applicationsof CF and EG However the noise in the obtainedputative CPs is random
542 Characteristics and Limitations of the CF Model andEG Model In the two models there are no restrictionson the depth range of the imaged scene However theEG is a global describing model and the CF is a localdescribing model In the EG model any pair of putativeCPs can influence the overall model estimation whereas inthe CF model a pair of putative CPs only influence themodel estimation within a local area in their vicinity Theshortcoming of the global model EG is that the overallmodel estimation can be degraded or ruined by PCs withconsiderable noise or mismatches in any area of the imagesThe shortcoming of the local model CF is that a pair of goodCPs cannot improve the estimation of CF outside of a localarea around itself Thus the uneven distribution of putativecorrespondences with different quality results in the fact thatthe estimated CF may be of high quality in some areas butof low quality in others for example in the experiment ofFigure 10 the qualities of the CF estimations over the second-nearest-to-the-cameras crossbeam and the third-nearest-to-the-cameras crossbeam are better than estimation over thefirst-nearest-to-the-cameras crossbeam The ldquolocal areardquo isdetermined by the CF estimation method and its chosenparameters for example in the CF estimation method SP(SVM) [2] the local area is determined by the chosen kerneland the scale parameter The above limitations of EG and CFcan be alleviated by investigating robust estimation methodsfor example the RANSAC M-estimators [8] LMedS (LeastMedian of Squares) [9] and MLESAC [10 12] for EGIn future work we will investigate further the estimationmethod for CF to improve its robustness
6 Conclusion
The CF is a recently introduced nonparametric model forrejecting mismatchesoutliers in image point matching In
Mathematical Problems in Engineering 11
(a) RMSD = 1386 (b) RMSD = 22707 (c) RMSD = 384 (d) RMSD = 23626Figure 11 Quantitative evaluation on two popular benchmark sets bikes and leuven The ground-truth CPs are presented by red circles thedetected interest points by black plus signs The ground-truth correspondences are displayed as blue dashed lines and the computed putativecorrespondences are shown by a solid green solid line (a) and (c) present some EG PMs that are accepted by CF as matches (b) and (d)present some CF PMs that are accepted by EG as matches 120572
119868= 10597 120572
119877= 21435 times 10
minus5 in (b) and 120572119877
= 58 times 10minus5 in (d)
(a) RMSD = 1631 (b) RMSD = 40098 (c) RMSD = 358 (d) RMSD = 32288Figure 12 Quantitative evaluation on two popular benchmark sets wall and bark ((a) and (c)) The EG PMs that are accepted by CF asmatches ((b) and (d)) the CF PMs that are accepted by EG as matches 120572
119868= 10597 120572
119877= 2029 times 10
minus4 in (b) and 120572119877
= 105 times 10minus4 in (d)
(a) RMSD = 1017 (b) RMSD = 42639 (c) RMSD = 329 (d) RMSD = 17646Figure 13 Quantitative evaluation on two popular benchmark sets graf and boat The ground-truth CPs are presented by red circles and thedetected interest points by black plus signs The ground-truth correspondences are displayed as blue dashed lines and the computed putativecorrespondences are shown by a solid green solid line (a) and (c) present some EG PMs that are accepted by CF as matches (b) and (d)present some CF PMs that are accepted by EG as matches 120572
Figure 14 Quantitative evaluation on two popular benchmark sets trees and ubc The ground-truth correspondences are displayed as bluedashed lines and the computed putative correspondences are shown by a yellow solid line (a) and (c) present some EG PMs that are acceptedbyCF asmatches (b) and (d) present someCFPMs that are accepted by EG asmatches120572
119868= 10597120572
119877= 59times10
minus5 in (b) and120572119877
= 185times10minus4
in (d)
(a) 119878EG119888cap 119878
CF119888
(b) 119878EG119888cap 119878
CF119888
(c) 119878EG119888cap 119878
CF119888
(d) 119878EG119888cap 119878
CF119888
Figure 15 Evaluation on Middlebury Stereo Datasets art and books ((a) and (c)) Some potential EG mismatches that are accepted by CF asmatches ((b) and (d)) some potential CF mismatches that are accepted by EG as matches 120572
119877= 0013 in (a) and (b) 120572
119877= 000018 in (c) and
(d) and 120572119868
= 10597
this study we investigated the principles of the CF andstudied its characteristics by comparing it with the widelyused parametric model epipolar geometry (EG) constraint
It is shown that the CF describes the mapping relation-ships between two CPs and should be able to estimate theposition of the corresponding point Therefore in additionto mismatch rejection a potential application of the CF is to
guide the point matching process by incorporating it into thecorrespondence propagation [39]
In practice putative correspondences are usually cor-rupted by noise and outliers and the number of putativecorrespondences is limited Therefore only an estimationof the CF with a certain accuracy can be estimated andthe actual results will violate the ldquonecessary and sufficient
Mathematical Problems in Engineering 13
(a) 119878EG119888cap SCF119888
(b) 119878EG119888cap 119878
CF119888
(c) 119878EG119888cap 119878
CF119888
(d) 119878EG119888cap 119878
CF119888
Figure 16 Evaluation on Middlebury Stereo Datasets computer and dolls ((a) and (c)) Some potential EG mismatches that are accepted byCF as matches ((b) and (d)) some potential CF mismatches that are accepted by EG as matches 120572
119877= 000018 in (a) and (b) 120572
119877= 000048 in
(c) and (d) and 120572119868
= 10597
(a) 119878EG119888cap 119878
CF119888
(b) 119878EG119888cap 119878
CF119888
(c) 119878EG119888cap 119878
CF119888
(d) 119878EG119888cap 119878
CF119888
Figure 17 Evaluation onMiddlebury Stereo Datasets drumsticks and dwarves ((a) and (c)) Some potential EGmismatches that are acceptedby CF as matches ((b) and (d)) some potential CF mismatches that are accepted by EG as matches 120572
119877= 000078 in (a) and (b) 120572
119877= 00035
in (c) and (d) and 120572119868
= 10597
conditionrdquoThe characteristic is that theCFmodel can reducethe searching space of CPs into an elliptical area and themismatches of the CF are usually near the correct CPs
Conflict of Interests
The authors declare that there is no conflict of interestsregarding the publication of this paper
Acknowledgments
The authors are grateful for valuable advice from DrDibyendu Mukherjee and the anonymous reviewers andwould like to thank the National Natural Science Foundationof China (Grant nos 61075033 61005033 and 61273248)the Natural Science Foundation of Guangdong Province(2014A030313425 and S2011010003348) and theOpenProject
14 Mathematical Problems in Engineering
(a) 119878EG119888cap 119878
CF119888
(b) 119878EG119888cap 119878
CF119888
(c) 119878EG119888cap 119878
CF119888
(d) 119878EG119888cap 119878
CF119888
Figure 18 Evaluation onMiddlebury Stereo Datasets laundry andmoebius ((a) and (c)) Some potential EGmismatches that are accepted byCF asmatches ((b) and (d)) some potential CFmismatches that are accepted by EG asmatches 120572
119877= 000000275 in (a) and (b) 120572
119877= 000035
in (c) and (d) and 120572119868
= 10597
(a) 119878EG119888cap 119878
CF119888
(b) 119878EG119888cap 119878
CF119888
Figure 19 Evaluation on Middlebury Stereo Dataset reindeer (a)Two potential EG mismatches that are accepted by CF as matches(b) two potential CFmismatches that are accepted by EG asmatches120572119877
= 0005 and 120572119868
= 30
Program of the National Laboratory of Pattern Recognition(NLPR) (201001060) for their support
References
[1] J P Monaco A C Bovik and L K Cormack ldquoActive foveateduncalibrated stereovisionrdquo International Journal of ComputerVision vol 85 no 2 pp 192ndash207 2009
[2] X Li and Z Hu ldquoRejecting mismatches by correspondencefunctionrdquo International Journal of Computer Vision vol 89 no1 pp 1ndash17 2010
[3] R Hartley and A Zisserman Multiple View Geometry inComputer Vision Cambridge University Press 2nd edition2003
[4] M Sonka V Hlavac and R Boyle Image Processing Analysisand Machine Vision Thomson-Engineering 3rd edition 2007
[5] G Carneiro and A D Jepson ldquoFlexible spatial configuration oflocal image featuresrdquo IEEE Transactions on Pattern Analysis andMachine Intelligence vol 29 no 12 pp 2089ndash2104 2007
[6] W Aguilar Y Frauel F Escolano M E Martinez-Perez AEspinosa-Romero andM A Lozano ldquoA robust graph transfor-mation matching for non-rigid registrationrdquo Image and VisionComputing vol 27 no 7 pp 897ndash910 2009
[7] L Wang F Duan and K Lv ldquoFisheye-lens-based visual suncompass for perception of spatial orientationrdquo MathematicalProblems in Engineering vol 2012 Article ID 460430 p 15 2012
[8] P Huber Robust Statistics Wiley New York NY USA 1981[9] P J Rousseeuw ldquoLeast median of squares regressionrdquo Journal
of the American Statistical Association vol 79 no 388 pp 871ndash880 1984
[10] P H S Torr and A Zisserman ldquoMLESAC a new robustestimator with application to estimating image geometryrdquoComputer Vision and Image Understanding vol 78 no 1 pp138ndash156 2000
[11] M A Fischler and R C Bolles ldquoRandom sample consensus aparadigm for model fitting with applications to image analysisand automated cartographyrdquo Communications of the ACM vol24 no 6 pp 381ndash395 1981
[12] B J Tordoff and D W Murray ldquoGuided-MLESAC fasterimage transform estimation by using matching priorsrdquo IEEETransactions on Pattern Analysis and Machine Intelligence vol27 no 10 pp 1523ndash1535 2005
Mathematical Problems in Engineering 15
[13] D Sidibe P Montesinos and S Janaqi ldquoMatching localinvariant features with contextual information an experimentalevaluationrdquo Electronic Letters on Computer Vision and ImageAnalysis vol 7 no 1 pp 26ndash39 2008
[14] Q-H Tran T-J Chin G Carneiro M S Brown and D SuterldquoIn defence of RANSAC for outlier rejection in deformableregistrationrdquo Lecture Notes in Computer Science vol 7575 no4 pp 274ndash287 2012
[15] A Goshtasby ldquoTransformation functions for image registra-tionrdquo in Proceedings of the Computer Vision and Pattern Recog-nition (CVPR rsquo11) June 2011
[16] X Li Y Wang J Zhou and X Li ldquoComparative research oncorrespondence functionrdquo in Proceedings of the InternationalConference on Intelligent System Design and Engineering Appli-cation (ISDEA rsquo10) pp 444ndash447 October 2010
[17] X-R Li X-M Li H-L Li and M-Y Cao ldquoRejecting outliersbased on correspondence manifoldrdquo Acta Automatica Sinicavol 35 no 1 pp 17ndash22 2009
[18] D Pizarro and A Bartoli ldquoFeature-based deformable surfacedetection with self-occlusion reasoningrdquo International Journalof Computer Vision vol 97 no 1 pp 54ndash70 2012
[19] J Yu A Eriksson T-J Chin and D Suter ldquoAn adversarialoptimization approach to efficient outlier removalrdquo Journal ofMathematical Imaging and Vision vol 48 no 3 pp 451ndash4662014
[20] A Goshtasby ldquoRegistration of images with geometric distor-tionsrdquo IEEETransactions onGeoscience andRemote Sensing vol26 no 1 pp 60ndash64 1988
[21] F L Bookstein ldquoPrincipal warps thin-plate splines and thedecomposition of deformationsrdquo IEEE Transactions on PatternAnalysis and Machine Intelligence vol 11 no 6 pp 567ndash5851992
[22] B Scholkopf F Steinke and V Blanz ldquoObject correspondenceas a machine learning problemrdquo in Proceedings of the 22ndInternational Conference on Machine Learning (ICML rsquo05) pp776ndash783 2005
[23] L Zagorchev and A Goshtasby ldquoA comparative study oftransformation functions for nonrigid image registrationrdquo IEEETransactions on Image Processing vol 15 no 3 pp 529ndash5382006
[24] J Zhao JMa J Tian J Ma andD Zhang ldquoA robustmethod forvector field learning with application to mismatch removingrdquoin Proceedings of the IEEE Conference on Computer Vision andPattern Recognition (CVPR rsquo11) Providence RI USA June 2011
[25] JMa J Zhao J Tian X Bai and Z Tu ldquoRegularized vector fieldlearning with sparse approximation for mismatch removalrdquoPattern Recognition vol 46 no 12 pp 3519ndash3532 2013
[26] J Ma J Zhao J Tian Z Tu and A L Yuille ldquoRobust estimationof nonrigid transformation for point set registrationrdquo in Pro-ceedings of the 26th IEEE Conference on Computer Vision andPattern Recognition (CVPR rsquo13) pp 2147ndash2154 June 2013
[27] J MaW Qiu J Zhao Y Ma A L Yuille and Z Tu ldquoRobust l2e
estimation of transformation for non-rigid registrationrdquo IEEETransactions on Signal Processing vol 63 no 5 pp 1115ndash11292015
[28] J Ma J Zhao J Tian A L Yuille and Z Tu ldquoRobust pointmatching via vector field consensusrdquo IEEE Transactions onImage Processing vol 23 no 4 pp 1706ndash1721 2014
[29] L Goshen and I Shimshoni ldquoBalanced exploration andexploitation model search for efficient epipolar geometry esti-mationrdquo IEEE Transactions on Pattern Analysis and MachineIntelligence vol 30 no 7 pp 1230ndash1242 2008
[30] O Chum and J Matas ldquoMatching with prosacmdashprogressivesample consensusrdquo in Proceedings of the IEEE Computer SocietyConference on Computer Vision and Pattern Recognition (CVPRrsquo05) vol 1 pp 220ndash226 IEEE June 2005
[31] K Mikolajczyk and C Schmid ldquoA performance evaluation oflocal descriptorsrdquo IEEE Transactions on Pattern Analysis andMachine Intelligence vol 27 no 10 pp 1615ndash1630 2005
[32] D G Lowe ldquoDistinctive image features from scale-invariantkeypointsrdquo International Journal of Computer Vision vol 60 no2 pp 91ndash110 2004
[33] S Gauglitz T Hollerer and M Turk ldquoEvaluation of interestpoint detectors and feature descriptors for visual trackingrdquoInternational Journal of Computer Vision vol 94 no 3 pp 335ndash360 2011
[34] Y Yu K Huang W Chen and T Tan ldquoA novel algorithmfor view and illumination invariant image matchingrdquo IEEETransactions on Image Processing vol 21 no 1 pp 229ndash2402012
[35] K Mikolajczyk T Tuytelaars C Schmid et al ldquoA comparisonof affine region detectorsrdquo International Journal of ComputerVision vol 65 no 1-2 pp 43ndash72 2005
[36] H Hirschmuller and D Scharstein ldquoEvaluation of cost func-tions for stereo matchingrdquo in Proceedings of the IEEE ComputerSociety Conference on Computer Vision and Pattern Recognition(CVPR rsquo07) June 2007
[37] D Scharstein and C Pal ldquoLearning conditional random fieldsfor stereordquo in Proceedings of the IEEE Computer Society Confer-ence on Computer Vision and Pattern Recognition (CVPR rsquo07)IEEE Minneapolis Minn USA June 2007
[38] D Scharstein andR Szeliski ldquoHigh-accuracy stereo depthmapsusing structured lightrdquo in Proceedings of the IEEE ComputerSociety Conference on Computer Vision and Pattern Recognition(CVPR rsquo03) vol 1 pp I-195ndashI-202 IEEE June 2003
[39] H Wang S Yan J Liu X Tang and T S Huang ldquoCorre-spondence propagationwithweak priorsrdquo IEEETransactions onImage Processing vol 18 no 1 pp 140ndash150 2009
Figure 9 Some PMs and their corresponding epipolar lines The EG model cannot detect the mismatches that are near the epipolar line butfar from the correct CPs The undetected mismatches by CF are usually near both the correct CPs and the epipolar line (a) and (c) presentthe EG PMs that are accepted by CF as matches in the experiments of Figures 7(c) and 8(c) respectively (b) and (d) present the CF PMs thatare accepted by EG as matches in the experiments of Figures 7(d) and 8(d) respectively In these experiments (p
119894 p1015840119894) is a pair of putative
CPs and l119894is the epipolar line corresponding to p1015840
119894and l1015840119894is the epipolar line corresponding to p
119894
The CF constraint determines the mapping relationshipsbetween two CPs and directly estimates the position of thecorresponding point Therefore by the mismatch-rejectingmethods based on CF we can restrict the error bound ofthe putative correspondences by a tolerance parameter 120572
119868
the undetected mismatches by CF usually belong to type-I To show the difference between the CF model and EGconstraints a typical result is presented in Figure 6 in whichwe relax the tolerance parameters 120572
119868and 120572119877to ensure the two
sets 119878EG119888
cap 119878CF119888
and 119878EG119888
cap 119878CF119888
change at equal speedHowever themismatches of type-I and type-II are usually
dramatically different from each other in usefulness andundue influence From the mismatches of type-I we cansearch for the correct CPs by including more informationOn the contrary other than ruining the traditional estimationmethods and distorting the final application results (egimage mosaicing and three-dimensional reconstruction) themismatches of the type-II provide less useful information
53 Further Evaluation on Some Popular Benchmark DataThe characteristics of the CF are also investigated on twoseries of popular benchmark data sets widely used to compareand evaluate related algorithms [33 34]
To evaluate the two schemes quantitatively this workconducted experiments on a series of image sets with ground-truth correspondences [31 35] The images were capturedunder special consideration to enable the encoding of theground-truth correspondences with homographies between
the reference image and other images The ground-truthhomographies are computed based on the following stepsfirst a set of putative correspondences are manually selectedand an initial estimation of the homography is computedbased on the selected putative correspondences second theimages are approximately aligned with the rough estimationof homography third reliable interest points are detectedand matched automatically and an accurate homography iscomputed from them
The experimental results are presented in Figures 11 12 13and 14 In every experiment the upper image and the lowerimage are called the reference image and the target imagerespectively In addition to the computed putative correspon-dences we also present the ground-truth CPs and ground-truth correspondence relationships in these experimentsTheground-truth CPs in the target images are computed for all ofthe marked interest points in reference images based on theground-truth homographies and denoted by red circles Thedetected interest points are denoted by black plus signs Theground-truth correspondence relationships are displayed asblue dashed lines and the computed putative correspondencerelationships are shown by a solid line with red or green forvisibility
Suppose that 119878 = (p1 p10158401) (p
119899 p1015840119899) are a set of
putative correspondences and 119878 = (p1 p10158401) (p
119899 p1015840119899) are
the ground-truth correspondences of 119878 We quantitativelyevaluate the quality of 119878 by the root-mean-square deviation(RMSD) RMSD(119878) = radicsum
Figure 10The possible correct correspondences identified by CF in the experiment of Figure 8 are presented based on depth (a) 34 putativecorrespondences (PCs) over the first crossbeam (b) 324 PCs over the second crossbeam (c) 214 PCs over the third crossbeam (d) 20 PCsaround the fourth crossbeam (e) 2 PCs on the wall (for clarity only 50 randomly chosen PCs are presented in (b) and (c))
The second series of evaluation data are the Middle-bury Stereo Datasets [36 37] (httpvisionmiddleburyedustereodata) These data sets were created using an auto-mated version of structured-lighting technique [38] Eachdataset consists of seven rectified views This work used thefirst and the fifth views for evaluation The experimentalresults are presented in Figures 15 16 17 18 and 19
54 CF and EG in Practice
541 Necessity and Sufficiency As analyzed the EG con-straint is a necessary but insufficient condition in the corre-spondence problem and theCFmodel describes themappingbetween CPs directly However in applications some correctCPsmay be inconsistent with the estimated EGmodel andorthe estimated CF model there may be some mismatchesthat are ldquoconsistentrdquo with the estimated CF model there alsomay be some of the putative correspondences which areldquoconsistentrdquo with the estimated CF but ldquoinconsistentrdquo withthe estimated EG for example the experimental results inFigures 5(a) 5(b) 7(a) 7(b) 8(a) and 8(b) The reasons forthe above results are as follows
(i) The coordinates of the obtained putative CPs areusually corrupted by noise
(ii) There are usually some errors in the estimation of theCF and EG
(iii) As pointed out in the first paragraph of Section 5a preset tolerance parameter is needed to define theldquoconsistentrdquo and ldquoinconsistentrdquo in the applicationsof CF and EG However the noise in the obtainedputative CPs is random
542 Characteristics and Limitations of the CF Model andEG Model In the two models there are no restrictionson the depth range of the imaged scene However theEG is a global describing model and the CF is a localdescribing model In the EG model any pair of putativeCPs can influence the overall model estimation whereas inthe CF model a pair of putative CPs only influence themodel estimation within a local area in their vicinity Theshortcoming of the global model EG is that the overallmodel estimation can be degraded or ruined by PCs withconsiderable noise or mismatches in any area of the imagesThe shortcoming of the local model CF is that a pair of goodCPs cannot improve the estimation of CF outside of a localarea around itself Thus the uneven distribution of putativecorrespondences with different quality results in the fact thatthe estimated CF may be of high quality in some areas butof low quality in others for example in the experiment ofFigure 10 the qualities of the CF estimations over the second-nearest-to-the-cameras crossbeam and the third-nearest-to-the-cameras crossbeam are better than estimation over thefirst-nearest-to-the-cameras crossbeam The ldquolocal areardquo isdetermined by the CF estimation method and its chosenparameters for example in the CF estimation method SP(SVM) [2] the local area is determined by the chosen kerneland the scale parameter The above limitations of EG and CFcan be alleviated by investigating robust estimation methodsfor example the RANSAC M-estimators [8] LMedS (LeastMedian of Squares) [9] and MLESAC [10 12] for EGIn future work we will investigate further the estimationmethod for CF to improve its robustness
6 Conclusion
The CF is a recently introduced nonparametric model forrejecting mismatchesoutliers in image point matching In
Mathematical Problems in Engineering 11
(a) RMSD = 1386 (b) RMSD = 22707 (c) RMSD = 384 (d) RMSD = 23626Figure 11 Quantitative evaluation on two popular benchmark sets bikes and leuven The ground-truth CPs are presented by red circles thedetected interest points by black plus signs The ground-truth correspondences are displayed as blue dashed lines and the computed putativecorrespondences are shown by a solid green solid line (a) and (c) present some EG PMs that are accepted by CF as matches (b) and (d)present some CF PMs that are accepted by EG as matches 120572
119868= 10597 120572
119877= 21435 times 10
minus5 in (b) and 120572119877
= 58 times 10minus5 in (d)
(a) RMSD = 1631 (b) RMSD = 40098 (c) RMSD = 358 (d) RMSD = 32288Figure 12 Quantitative evaluation on two popular benchmark sets wall and bark ((a) and (c)) The EG PMs that are accepted by CF asmatches ((b) and (d)) the CF PMs that are accepted by EG as matches 120572
119868= 10597 120572
119877= 2029 times 10
minus4 in (b) and 120572119877
= 105 times 10minus4 in (d)
(a) RMSD = 1017 (b) RMSD = 42639 (c) RMSD = 329 (d) RMSD = 17646Figure 13 Quantitative evaluation on two popular benchmark sets graf and boat The ground-truth CPs are presented by red circles and thedetected interest points by black plus signs The ground-truth correspondences are displayed as blue dashed lines and the computed putativecorrespondences are shown by a solid green solid line (a) and (c) present some EG PMs that are accepted by CF as matches (b) and (d)present some CF PMs that are accepted by EG as matches 120572
Figure 14 Quantitative evaluation on two popular benchmark sets trees and ubc The ground-truth correspondences are displayed as bluedashed lines and the computed putative correspondences are shown by a yellow solid line (a) and (c) present some EG PMs that are acceptedbyCF asmatches (b) and (d) present someCFPMs that are accepted by EG asmatches120572
119868= 10597120572
119877= 59times10
minus5 in (b) and120572119877
= 185times10minus4
in (d)
(a) 119878EG119888cap 119878
CF119888
(b) 119878EG119888cap 119878
CF119888
(c) 119878EG119888cap 119878
CF119888
(d) 119878EG119888cap 119878
CF119888
Figure 15 Evaluation on Middlebury Stereo Datasets art and books ((a) and (c)) Some potential EG mismatches that are accepted by CF asmatches ((b) and (d)) some potential CF mismatches that are accepted by EG as matches 120572
119877= 0013 in (a) and (b) 120572
119877= 000018 in (c) and
(d) and 120572119868
= 10597
this study we investigated the principles of the CF andstudied its characteristics by comparing it with the widelyused parametric model epipolar geometry (EG) constraint
It is shown that the CF describes the mapping relation-ships between two CPs and should be able to estimate theposition of the corresponding point Therefore in additionto mismatch rejection a potential application of the CF is to
guide the point matching process by incorporating it into thecorrespondence propagation [39]
In practice putative correspondences are usually cor-rupted by noise and outliers and the number of putativecorrespondences is limited Therefore only an estimationof the CF with a certain accuracy can be estimated andthe actual results will violate the ldquonecessary and sufficient
Mathematical Problems in Engineering 13
(a) 119878EG119888cap SCF119888
(b) 119878EG119888cap 119878
CF119888
(c) 119878EG119888cap 119878
CF119888
(d) 119878EG119888cap 119878
CF119888
Figure 16 Evaluation on Middlebury Stereo Datasets computer and dolls ((a) and (c)) Some potential EG mismatches that are accepted byCF as matches ((b) and (d)) some potential CF mismatches that are accepted by EG as matches 120572
119877= 000018 in (a) and (b) 120572
119877= 000048 in
(c) and (d) and 120572119868
= 10597
(a) 119878EG119888cap 119878
CF119888
(b) 119878EG119888cap 119878
CF119888
(c) 119878EG119888cap 119878
CF119888
(d) 119878EG119888cap 119878
CF119888
Figure 17 Evaluation onMiddlebury Stereo Datasets drumsticks and dwarves ((a) and (c)) Some potential EGmismatches that are acceptedby CF as matches ((b) and (d)) some potential CF mismatches that are accepted by EG as matches 120572
119877= 000078 in (a) and (b) 120572
119877= 00035
in (c) and (d) and 120572119868
= 10597
conditionrdquoThe characteristic is that theCFmodel can reducethe searching space of CPs into an elliptical area and themismatches of the CF are usually near the correct CPs
Conflict of Interests
The authors declare that there is no conflict of interestsregarding the publication of this paper
Acknowledgments
The authors are grateful for valuable advice from DrDibyendu Mukherjee and the anonymous reviewers andwould like to thank the National Natural Science Foundationof China (Grant nos 61075033 61005033 and 61273248)the Natural Science Foundation of Guangdong Province(2014A030313425 and S2011010003348) and theOpenProject
14 Mathematical Problems in Engineering
(a) 119878EG119888cap 119878
CF119888
(b) 119878EG119888cap 119878
CF119888
(c) 119878EG119888cap 119878
CF119888
(d) 119878EG119888cap 119878
CF119888
Figure 18 Evaluation onMiddlebury Stereo Datasets laundry andmoebius ((a) and (c)) Some potential EGmismatches that are accepted byCF asmatches ((b) and (d)) some potential CFmismatches that are accepted by EG asmatches 120572
119877= 000000275 in (a) and (b) 120572
119877= 000035
in (c) and (d) and 120572119868
= 10597
(a) 119878EG119888cap 119878
CF119888
(b) 119878EG119888cap 119878
CF119888
Figure 19 Evaluation on Middlebury Stereo Dataset reindeer (a)Two potential EG mismatches that are accepted by CF as matches(b) two potential CFmismatches that are accepted by EG asmatches120572119877
= 0005 and 120572119868
= 30
Program of the National Laboratory of Pattern Recognition(NLPR) (201001060) for their support
References
[1] J P Monaco A C Bovik and L K Cormack ldquoActive foveateduncalibrated stereovisionrdquo International Journal of ComputerVision vol 85 no 2 pp 192ndash207 2009
[2] X Li and Z Hu ldquoRejecting mismatches by correspondencefunctionrdquo International Journal of Computer Vision vol 89 no1 pp 1ndash17 2010
[3] R Hartley and A Zisserman Multiple View Geometry inComputer Vision Cambridge University Press 2nd edition2003
[4] M Sonka V Hlavac and R Boyle Image Processing Analysisand Machine Vision Thomson-Engineering 3rd edition 2007
[5] G Carneiro and A D Jepson ldquoFlexible spatial configuration oflocal image featuresrdquo IEEE Transactions on Pattern Analysis andMachine Intelligence vol 29 no 12 pp 2089ndash2104 2007
[6] W Aguilar Y Frauel F Escolano M E Martinez-Perez AEspinosa-Romero andM A Lozano ldquoA robust graph transfor-mation matching for non-rigid registrationrdquo Image and VisionComputing vol 27 no 7 pp 897ndash910 2009
[7] L Wang F Duan and K Lv ldquoFisheye-lens-based visual suncompass for perception of spatial orientationrdquo MathematicalProblems in Engineering vol 2012 Article ID 460430 p 15 2012
[8] P Huber Robust Statistics Wiley New York NY USA 1981[9] P J Rousseeuw ldquoLeast median of squares regressionrdquo Journal
of the American Statistical Association vol 79 no 388 pp 871ndash880 1984
[10] P H S Torr and A Zisserman ldquoMLESAC a new robustestimator with application to estimating image geometryrdquoComputer Vision and Image Understanding vol 78 no 1 pp138ndash156 2000
[11] M A Fischler and R C Bolles ldquoRandom sample consensus aparadigm for model fitting with applications to image analysisand automated cartographyrdquo Communications of the ACM vol24 no 6 pp 381ndash395 1981
[12] B J Tordoff and D W Murray ldquoGuided-MLESAC fasterimage transform estimation by using matching priorsrdquo IEEETransactions on Pattern Analysis and Machine Intelligence vol27 no 10 pp 1523ndash1535 2005
Mathematical Problems in Engineering 15
[13] D Sidibe P Montesinos and S Janaqi ldquoMatching localinvariant features with contextual information an experimentalevaluationrdquo Electronic Letters on Computer Vision and ImageAnalysis vol 7 no 1 pp 26ndash39 2008
[14] Q-H Tran T-J Chin G Carneiro M S Brown and D SuterldquoIn defence of RANSAC for outlier rejection in deformableregistrationrdquo Lecture Notes in Computer Science vol 7575 no4 pp 274ndash287 2012
[15] A Goshtasby ldquoTransformation functions for image registra-tionrdquo in Proceedings of the Computer Vision and Pattern Recog-nition (CVPR rsquo11) June 2011
[16] X Li Y Wang J Zhou and X Li ldquoComparative research oncorrespondence functionrdquo in Proceedings of the InternationalConference on Intelligent System Design and Engineering Appli-cation (ISDEA rsquo10) pp 444ndash447 October 2010
[17] X-R Li X-M Li H-L Li and M-Y Cao ldquoRejecting outliersbased on correspondence manifoldrdquo Acta Automatica Sinicavol 35 no 1 pp 17ndash22 2009
[18] D Pizarro and A Bartoli ldquoFeature-based deformable surfacedetection with self-occlusion reasoningrdquo International Journalof Computer Vision vol 97 no 1 pp 54ndash70 2012
[19] J Yu A Eriksson T-J Chin and D Suter ldquoAn adversarialoptimization approach to efficient outlier removalrdquo Journal ofMathematical Imaging and Vision vol 48 no 3 pp 451ndash4662014
[20] A Goshtasby ldquoRegistration of images with geometric distor-tionsrdquo IEEETransactions onGeoscience andRemote Sensing vol26 no 1 pp 60ndash64 1988
[21] F L Bookstein ldquoPrincipal warps thin-plate splines and thedecomposition of deformationsrdquo IEEE Transactions on PatternAnalysis and Machine Intelligence vol 11 no 6 pp 567ndash5851992
[22] B Scholkopf F Steinke and V Blanz ldquoObject correspondenceas a machine learning problemrdquo in Proceedings of the 22ndInternational Conference on Machine Learning (ICML rsquo05) pp776ndash783 2005
[23] L Zagorchev and A Goshtasby ldquoA comparative study oftransformation functions for nonrigid image registrationrdquo IEEETransactions on Image Processing vol 15 no 3 pp 529ndash5382006
[24] J Zhao JMa J Tian J Ma andD Zhang ldquoA robustmethod forvector field learning with application to mismatch removingrdquoin Proceedings of the IEEE Conference on Computer Vision andPattern Recognition (CVPR rsquo11) Providence RI USA June 2011
[25] JMa J Zhao J Tian X Bai and Z Tu ldquoRegularized vector fieldlearning with sparse approximation for mismatch removalrdquoPattern Recognition vol 46 no 12 pp 3519ndash3532 2013
[26] J Ma J Zhao J Tian Z Tu and A L Yuille ldquoRobust estimationof nonrigid transformation for point set registrationrdquo in Pro-ceedings of the 26th IEEE Conference on Computer Vision andPattern Recognition (CVPR rsquo13) pp 2147ndash2154 June 2013
[27] J MaW Qiu J Zhao Y Ma A L Yuille and Z Tu ldquoRobust l2e
estimation of transformation for non-rigid registrationrdquo IEEETransactions on Signal Processing vol 63 no 5 pp 1115ndash11292015
[28] J Ma J Zhao J Tian A L Yuille and Z Tu ldquoRobust pointmatching via vector field consensusrdquo IEEE Transactions onImage Processing vol 23 no 4 pp 1706ndash1721 2014
[29] L Goshen and I Shimshoni ldquoBalanced exploration andexploitation model search for efficient epipolar geometry esti-mationrdquo IEEE Transactions on Pattern Analysis and MachineIntelligence vol 30 no 7 pp 1230ndash1242 2008
[30] O Chum and J Matas ldquoMatching with prosacmdashprogressivesample consensusrdquo in Proceedings of the IEEE Computer SocietyConference on Computer Vision and Pattern Recognition (CVPRrsquo05) vol 1 pp 220ndash226 IEEE June 2005
[31] K Mikolajczyk and C Schmid ldquoA performance evaluation oflocal descriptorsrdquo IEEE Transactions on Pattern Analysis andMachine Intelligence vol 27 no 10 pp 1615ndash1630 2005
[32] D G Lowe ldquoDistinctive image features from scale-invariantkeypointsrdquo International Journal of Computer Vision vol 60 no2 pp 91ndash110 2004
[33] S Gauglitz T Hollerer and M Turk ldquoEvaluation of interestpoint detectors and feature descriptors for visual trackingrdquoInternational Journal of Computer Vision vol 94 no 3 pp 335ndash360 2011
[34] Y Yu K Huang W Chen and T Tan ldquoA novel algorithmfor view and illumination invariant image matchingrdquo IEEETransactions on Image Processing vol 21 no 1 pp 229ndash2402012
[35] K Mikolajczyk T Tuytelaars C Schmid et al ldquoA comparisonof affine region detectorsrdquo International Journal of ComputerVision vol 65 no 1-2 pp 43ndash72 2005
[36] H Hirschmuller and D Scharstein ldquoEvaluation of cost func-tions for stereo matchingrdquo in Proceedings of the IEEE ComputerSociety Conference on Computer Vision and Pattern Recognition(CVPR rsquo07) June 2007
[37] D Scharstein and C Pal ldquoLearning conditional random fieldsfor stereordquo in Proceedings of the IEEE Computer Society Confer-ence on Computer Vision and Pattern Recognition (CVPR rsquo07)IEEE Minneapolis Minn USA June 2007
[38] D Scharstein andR Szeliski ldquoHigh-accuracy stereo depthmapsusing structured lightrdquo in Proceedings of the IEEE ComputerSociety Conference on Computer Vision and Pattern Recognition(CVPR rsquo03) vol 1 pp I-195ndashI-202 IEEE June 2003
[39] H Wang S Yan J Liu X Tang and T S Huang ldquoCorre-spondence propagationwithweak priorsrdquo IEEETransactions onImage Processing vol 18 no 1 pp 140ndash150 2009
Figure 10The possible correct correspondences identified by CF in the experiment of Figure 8 are presented based on depth (a) 34 putativecorrespondences (PCs) over the first crossbeam (b) 324 PCs over the second crossbeam (c) 214 PCs over the third crossbeam (d) 20 PCsaround the fourth crossbeam (e) 2 PCs on the wall (for clarity only 50 randomly chosen PCs are presented in (b) and (c))
The second series of evaluation data are the Middle-bury Stereo Datasets [36 37] (httpvisionmiddleburyedustereodata) These data sets were created using an auto-mated version of structured-lighting technique [38] Eachdataset consists of seven rectified views This work used thefirst and the fifth views for evaluation The experimentalresults are presented in Figures 15 16 17 18 and 19
54 CF and EG in Practice
541 Necessity and Sufficiency As analyzed the EG con-straint is a necessary but insufficient condition in the corre-spondence problem and theCFmodel describes themappingbetween CPs directly However in applications some correctCPsmay be inconsistent with the estimated EGmodel andorthe estimated CF model there may be some mismatchesthat are ldquoconsistentrdquo with the estimated CF model there alsomay be some of the putative correspondences which areldquoconsistentrdquo with the estimated CF but ldquoinconsistentrdquo withthe estimated EG for example the experimental results inFigures 5(a) 5(b) 7(a) 7(b) 8(a) and 8(b) The reasons forthe above results are as follows
(i) The coordinates of the obtained putative CPs areusually corrupted by noise
(ii) There are usually some errors in the estimation of theCF and EG
(iii) As pointed out in the first paragraph of Section 5a preset tolerance parameter is needed to define theldquoconsistentrdquo and ldquoinconsistentrdquo in the applicationsof CF and EG However the noise in the obtainedputative CPs is random
542 Characteristics and Limitations of the CF Model andEG Model In the two models there are no restrictionson the depth range of the imaged scene However theEG is a global describing model and the CF is a localdescribing model In the EG model any pair of putativeCPs can influence the overall model estimation whereas inthe CF model a pair of putative CPs only influence themodel estimation within a local area in their vicinity Theshortcoming of the global model EG is that the overallmodel estimation can be degraded or ruined by PCs withconsiderable noise or mismatches in any area of the imagesThe shortcoming of the local model CF is that a pair of goodCPs cannot improve the estimation of CF outside of a localarea around itself Thus the uneven distribution of putativecorrespondences with different quality results in the fact thatthe estimated CF may be of high quality in some areas butof low quality in others for example in the experiment ofFigure 10 the qualities of the CF estimations over the second-nearest-to-the-cameras crossbeam and the third-nearest-to-the-cameras crossbeam are better than estimation over thefirst-nearest-to-the-cameras crossbeam The ldquolocal areardquo isdetermined by the CF estimation method and its chosenparameters for example in the CF estimation method SP(SVM) [2] the local area is determined by the chosen kerneland the scale parameter The above limitations of EG and CFcan be alleviated by investigating robust estimation methodsfor example the RANSAC M-estimators [8] LMedS (LeastMedian of Squares) [9] and MLESAC [10 12] for EGIn future work we will investigate further the estimationmethod for CF to improve its robustness
6 Conclusion
The CF is a recently introduced nonparametric model forrejecting mismatchesoutliers in image point matching In
Mathematical Problems in Engineering 11
(a) RMSD = 1386 (b) RMSD = 22707 (c) RMSD = 384 (d) RMSD = 23626Figure 11 Quantitative evaluation on two popular benchmark sets bikes and leuven The ground-truth CPs are presented by red circles thedetected interest points by black plus signs The ground-truth correspondences are displayed as blue dashed lines and the computed putativecorrespondences are shown by a solid green solid line (a) and (c) present some EG PMs that are accepted by CF as matches (b) and (d)present some CF PMs that are accepted by EG as matches 120572
119868= 10597 120572
119877= 21435 times 10
minus5 in (b) and 120572119877
= 58 times 10minus5 in (d)
(a) RMSD = 1631 (b) RMSD = 40098 (c) RMSD = 358 (d) RMSD = 32288Figure 12 Quantitative evaluation on two popular benchmark sets wall and bark ((a) and (c)) The EG PMs that are accepted by CF asmatches ((b) and (d)) the CF PMs that are accepted by EG as matches 120572
119868= 10597 120572
119877= 2029 times 10
minus4 in (b) and 120572119877
= 105 times 10minus4 in (d)
(a) RMSD = 1017 (b) RMSD = 42639 (c) RMSD = 329 (d) RMSD = 17646Figure 13 Quantitative evaluation on two popular benchmark sets graf and boat The ground-truth CPs are presented by red circles and thedetected interest points by black plus signs The ground-truth correspondences are displayed as blue dashed lines and the computed putativecorrespondences are shown by a solid green solid line (a) and (c) present some EG PMs that are accepted by CF as matches (b) and (d)present some CF PMs that are accepted by EG as matches 120572
Figure 14 Quantitative evaluation on two popular benchmark sets trees and ubc The ground-truth correspondences are displayed as bluedashed lines and the computed putative correspondences are shown by a yellow solid line (a) and (c) present some EG PMs that are acceptedbyCF asmatches (b) and (d) present someCFPMs that are accepted by EG asmatches120572
119868= 10597120572
119877= 59times10
minus5 in (b) and120572119877
= 185times10minus4
in (d)
(a) 119878EG119888cap 119878
CF119888
(b) 119878EG119888cap 119878
CF119888
(c) 119878EG119888cap 119878
CF119888
(d) 119878EG119888cap 119878
CF119888
Figure 15 Evaluation on Middlebury Stereo Datasets art and books ((a) and (c)) Some potential EG mismatches that are accepted by CF asmatches ((b) and (d)) some potential CF mismatches that are accepted by EG as matches 120572
119877= 0013 in (a) and (b) 120572
119877= 000018 in (c) and
(d) and 120572119868
= 10597
this study we investigated the principles of the CF andstudied its characteristics by comparing it with the widelyused parametric model epipolar geometry (EG) constraint
It is shown that the CF describes the mapping relation-ships between two CPs and should be able to estimate theposition of the corresponding point Therefore in additionto mismatch rejection a potential application of the CF is to
guide the point matching process by incorporating it into thecorrespondence propagation [39]
In practice putative correspondences are usually cor-rupted by noise and outliers and the number of putativecorrespondences is limited Therefore only an estimationof the CF with a certain accuracy can be estimated andthe actual results will violate the ldquonecessary and sufficient
Mathematical Problems in Engineering 13
(a) 119878EG119888cap SCF119888
(b) 119878EG119888cap 119878
CF119888
(c) 119878EG119888cap 119878
CF119888
(d) 119878EG119888cap 119878
CF119888
Figure 16 Evaluation on Middlebury Stereo Datasets computer and dolls ((a) and (c)) Some potential EG mismatches that are accepted byCF as matches ((b) and (d)) some potential CF mismatches that are accepted by EG as matches 120572
119877= 000018 in (a) and (b) 120572
119877= 000048 in
(c) and (d) and 120572119868
= 10597
(a) 119878EG119888cap 119878
CF119888
(b) 119878EG119888cap 119878
CF119888
(c) 119878EG119888cap 119878
CF119888
(d) 119878EG119888cap 119878
CF119888
Figure 17 Evaluation onMiddlebury Stereo Datasets drumsticks and dwarves ((a) and (c)) Some potential EGmismatches that are acceptedby CF as matches ((b) and (d)) some potential CF mismatches that are accepted by EG as matches 120572
119877= 000078 in (a) and (b) 120572
119877= 00035
in (c) and (d) and 120572119868
= 10597
conditionrdquoThe characteristic is that theCFmodel can reducethe searching space of CPs into an elliptical area and themismatches of the CF are usually near the correct CPs
Conflict of Interests
The authors declare that there is no conflict of interestsregarding the publication of this paper
Acknowledgments
The authors are grateful for valuable advice from DrDibyendu Mukherjee and the anonymous reviewers andwould like to thank the National Natural Science Foundationof China (Grant nos 61075033 61005033 and 61273248)the Natural Science Foundation of Guangdong Province(2014A030313425 and S2011010003348) and theOpenProject
14 Mathematical Problems in Engineering
(a) 119878EG119888cap 119878
CF119888
(b) 119878EG119888cap 119878
CF119888
(c) 119878EG119888cap 119878
CF119888
(d) 119878EG119888cap 119878
CF119888
Figure 18 Evaluation onMiddlebury Stereo Datasets laundry andmoebius ((a) and (c)) Some potential EGmismatches that are accepted byCF asmatches ((b) and (d)) some potential CFmismatches that are accepted by EG asmatches 120572
119877= 000000275 in (a) and (b) 120572
119877= 000035
in (c) and (d) and 120572119868
= 10597
(a) 119878EG119888cap 119878
CF119888
(b) 119878EG119888cap 119878
CF119888
Figure 19 Evaluation on Middlebury Stereo Dataset reindeer (a)Two potential EG mismatches that are accepted by CF as matches(b) two potential CFmismatches that are accepted by EG asmatches120572119877
= 0005 and 120572119868
= 30
Program of the National Laboratory of Pattern Recognition(NLPR) (201001060) for their support
References
[1] J P Monaco A C Bovik and L K Cormack ldquoActive foveateduncalibrated stereovisionrdquo International Journal of ComputerVision vol 85 no 2 pp 192ndash207 2009
[2] X Li and Z Hu ldquoRejecting mismatches by correspondencefunctionrdquo International Journal of Computer Vision vol 89 no1 pp 1ndash17 2010
[3] R Hartley and A Zisserman Multiple View Geometry inComputer Vision Cambridge University Press 2nd edition2003
[4] M Sonka V Hlavac and R Boyle Image Processing Analysisand Machine Vision Thomson-Engineering 3rd edition 2007
[5] G Carneiro and A D Jepson ldquoFlexible spatial configuration oflocal image featuresrdquo IEEE Transactions on Pattern Analysis andMachine Intelligence vol 29 no 12 pp 2089ndash2104 2007
[6] W Aguilar Y Frauel F Escolano M E Martinez-Perez AEspinosa-Romero andM A Lozano ldquoA robust graph transfor-mation matching for non-rigid registrationrdquo Image and VisionComputing vol 27 no 7 pp 897ndash910 2009
[7] L Wang F Duan and K Lv ldquoFisheye-lens-based visual suncompass for perception of spatial orientationrdquo MathematicalProblems in Engineering vol 2012 Article ID 460430 p 15 2012
[8] P Huber Robust Statistics Wiley New York NY USA 1981[9] P J Rousseeuw ldquoLeast median of squares regressionrdquo Journal
of the American Statistical Association vol 79 no 388 pp 871ndash880 1984
[10] P H S Torr and A Zisserman ldquoMLESAC a new robustestimator with application to estimating image geometryrdquoComputer Vision and Image Understanding vol 78 no 1 pp138ndash156 2000
[11] M A Fischler and R C Bolles ldquoRandom sample consensus aparadigm for model fitting with applications to image analysisand automated cartographyrdquo Communications of the ACM vol24 no 6 pp 381ndash395 1981
[12] B J Tordoff and D W Murray ldquoGuided-MLESAC fasterimage transform estimation by using matching priorsrdquo IEEETransactions on Pattern Analysis and Machine Intelligence vol27 no 10 pp 1523ndash1535 2005
Mathematical Problems in Engineering 15
[13] D Sidibe P Montesinos and S Janaqi ldquoMatching localinvariant features with contextual information an experimentalevaluationrdquo Electronic Letters on Computer Vision and ImageAnalysis vol 7 no 1 pp 26ndash39 2008
[14] Q-H Tran T-J Chin G Carneiro M S Brown and D SuterldquoIn defence of RANSAC for outlier rejection in deformableregistrationrdquo Lecture Notes in Computer Science vol 7575 no4 pp 274ndash287 2012
[15] A Goshtasby ldquoTransformation functions for image registra-tionrdquo in Proceedings of the Computer Vision and Pattern Recog-nition (CVPR rsquo11) June 2011
[16] X Li Y Wang J Zhou and X Li ldquoComparative research oncorrespondence functionrdquo in Proceedings of the InternationalConference on Intelligent System Design and Engineering Appli-cation (ISDEA rsquo10) pp 444ndash447 October 2010
[17] X-R Li X-M Li H-L Li and M-Y Cao ldquoRejecting outliersbased on correspondence manifoldrdquo Acta Automatica Sinicavol 35 no 1 pp 17ndash22 2009
[18] D Pizarro and A Bartoli ldquoFeature-based deformable surfacedetection with self-occlusion reasoningrdquo International Journalof Computer Vision vol 97 no 1 pp 54ndash70 2012
[19] J Yu A Eriksson T-J Chin and D Suter ldquoAn adversarialoptimization approach to efficient outlier removalrdquo Journal ofMathematical Imaging and Vision vol 48 no 3 pp 451ndash4662014
[20] A Goshtasby ldquoRegistration of images with geometric distor-tionsrdquo IEEETransactions onGeoscience andRemote Sensing vol26 no 1 pp 60ndash64 1988
[21] F L Bookstein ldquoPrincipal warps thin-plate splines and thedecomposition of deformationsrdquo IEEE Transactions on PatternAnalysis and Machine Intelligence vol 11 no 6 pp 567ndash5851992
[22] B Scholkopf F Steinke and V Blanz ldquoObject correspondenceas a machine learning problemrdquo in Proceedings of the 22ndInternational Conference on Machine Learning (ICML rsquo05) pp776ndash783 2005
[23] L Zagorchev and A Goshtasby ldquoA comparative study oftransformation functions for nonrigid image registrationrdquo IEEETransactions on Image Processing vol 15 no 3 pp 529ndash5382006
[24] J Zhao JMa J Tian J Ma andD Zhang ldquoA robustmethod forvector field learning with application to mismatch removingrdquoin Proceedings of the IEEE Conference on Computer Vision andPattern Recognition (CVPR rsquo11) Providence RI USA June 2011
[25] JMa J Zhao J Tian X Bai and Z Tu ldquoRegularized vector fieldlearning with sparse approximation for mismatch removalrdquoPattern Recognition vol 46 no 12 pp 3519ndash3532 2013
[26] J Ma J Zhao J Tian Z Tu and A L Yuille ldquoRobust estimationof nonrigid transformation for point set registrationrdquo in Pro-ceedings of the 26th IEEE Conference on Computer Vision andPattern Recognition (CVPR rsquo13) pp 2147ndash2154 June 2013
[27] J MaW Qiu J Zhao Y Ma A L Yuille and Z Tu ldquoRobust l2e
estimation of transformation for non-rigid registrationrdquo IEEETransactions on Signal Processing vol 63 no 5 pp 1115ndash11292015
[28] J Ma J Zhao J Tian A L Yuille and Z Tu ldquoRobust pointmatching via vector field consensusrdquo IEEE Transactions onImage Processing vol 23 no 4 pp 1706ndash1721 2014
[29] L Goshen and I Shimshoni ldquoBalanced exploration andexploitation model search for efficient epipolar geometry esti-mationrdquo IEEE Transactions on Pattern Analysis and MachineIntelligence vol 30 no 7 pp 1230ndash1242 2008
[30] O Chum and J Matas ldquoMatching with prosacmdashprogressivesample consensusrdquo in Proceedings of the IEEE Computer SocietyConference on Computer Vision and Pattern Recognition (CVPRrsquo05) vol 1 pp 220ndash226 IEEE June 2005
[31] K Mikolajczyk and C Schmid ldquoA performance evaluation oflocal descriptorsrdquo IEEE Transactions on Pattern Analysis andMachine Intelligence vol 27 no 10 pp 1615ndash1630 2005
[32] D G Lowe ldquoDistinctive image features from scale-invariantkeypointsrdquo International Journal of Computer Vision vol 60 no2 pp 91ndash110 2004
[33] S Gauglitz T Hollerer and M Turk ldquoEvaluation of interestpoint detectors and feature descriptors for visual trackingrdquoInternational Journal of Computer Vision vol 94 no 3 pp 335ndash360 2011
[34] Y Yu K Huang W Chen and T Tan ldquoA novel algorithmfor view and illumination invariant image matchingrdquo IEEETransactions on Image Processing vol 21 no 1 pp 229ndash2402012
[35] K Mikolajczyk T Tuytelaars C Schmid et al ldquoA comparisonof affine region detectorsrdquo International Journal of ComputerVision vol 65 no 1-2 pp 43ndash72 2005
[36] H Hirschmuller and D Scharstein ldquoEvaluation of cost func-tions for stereo matchingrdquo in Proceedings of the IEEE ComputerSociety Conference on Computer Vision and Pattern Recognition(CVPR rsquo07) June 2007
[37] D Scharstein and C Pal ldquoLearning conditional random fieldsfor stereordquo in Proceedings of the IEEE Computer Society Confer-ence on Computer Vision and Pattern Recognition (CVPR rsquo07)IEEE Minneapolis Minn USA June 2007
[38] D Scharstein andR Szeliski ldquoHigh-accuracy stereo depthmapsusing structured lightrdquo in Proceedings of the IEEE ComputerSociety Conference on Computer Vision and Pattern Recognition(CVPR rsquo03) vol 1 pp I-195ndashI-202 IEEE June 2003
[39] H Wang S Yan J Liu X Tang and T S Huang ldquoCorre-spondence propagationwithweak priorsrdquo IEEETransactions onImage Processing vol 18 no 1 pp 140ndash150 2009
(a) RMSD = 1386 (b) RMSD = 22707 (c) RMSD = 384 (d) RMSD = 23626Figure 11 Quantitative evaluation on two popular benchmark sets bikes and leuven The ground-truth CPs are presented by red circles thedetected interest points by black plus signs The ground-truth correspondences are displayed as blue dashed lines and the computed putativecorrespondences are shown by a solid green solid line (a) and (c) present some EG PMs that are accepted by CF as matches (b) and (d)present some CF PMs that are accepted by EG as matches 120572
119868= 10597 120572
119877= 21435 times 10
minus5 in (b) and 120572119877
= 58 times 10minus5 in (d)
(a) RMSD = 1631 (b) RMSD = 40098 (c) RMSD = 358 (d) RMSD = 32288Figure 12 Quantitative evaluation on two popular benchmark sets wall and bark ((a) and (c)) The EG PMs that are accepted by CF asmatches ((b) and (d)) the CF PMs that are accepted by EG as matches 120572
119868= 10597 120572
119877= 2029 times 10
minus4 in (b) and 120572119877
= 105 times 10minus4 in (d)
(a) RMSD = 1017 (b) RMSD = 42639 (c) RMSD = 329 (d) RMSD = 17646Figure 13 Quantitative evaluation on two popular benchmark sets graf and boat The ground-truth CPs are presented by red circles and thedetected interest points by black plus signs The ground-truth correspondences are displayed as blue dashed lines and the computed putativecorrespondences are shown by a solid green solid line (a) and (c) present some EG PMs that are accepted by CF as matches (b) and (d)present some CF PMs that are accepted by EG as matches 120572
Figure 14 Quantitative evaluation on two popular benchmark sets trees and ubc The ground-truth correspondences are displayed as bluedashed lines and the computed putative correspondences are shown by a yellow solid line (a) and (c) present some EG PMs that are acceptedbyCF asmatches (b) and (d) present someCFPMs that are accepted by EG asmatches120572
119868= 10597120572
119877= 59times10
minus5 in (b) and120572119877
= 185times10minus4
in (d)
(a) 119878EG119888cap 119878
CF119888
(b) 119878EG119888cap 119878
CF119888
(c) 119878EG119888cap 119878
CF119888
(d) 119878EG119888cap 119878
CF119888
Figure 15 Evaluation on Middlebury Stereo Datasets art and books ((a) and (c)) Some potential EG mismatches that are accepted by CF asmatches ((b) and (d)) some potential CF mismatches that are accepted by EG as matches 120572
119877= 0013 in (a) and (b) 120572
119877= 000018 in (c) and
(d) and 120572119868
= 10597
this study we investigated the principles of the CF andstudied its characteristics by comparing it with the widelyused parametric model epipolar geometry (EG) constraint
It is shown that the CF describes the mapping relation-ships between two CPs and should be able to estimate theposition of the corresponding point Therefore in additionto mismatch rejection a potential application of the CF is to
guide the point matching process by incorporating it into thecorrespondence propagation [39]
In practice putative correspondences are usually cor-rupted by noise and outliers and the number of putativecorrespondences is limited Therefore only an estimationof the CF with a certain accuracy can be estimated andthe actual results will violate the ldquonecessary and sufficient
Mathematical Problems in Engineering 13
(a) 119878EG119888cap SCF119888
(b) 119878EG119888cap 119878
CF119888
(c) 119878EG119888cap 119878
CF119888
(d) 119878EG119888cap 119878
CF119888
Figure 16 Evaluation on Middlebury Stereo Datasets computer and dolls ((a) and (c)) Some potential EG mismatches that are accepted byCF as matches ((b) and (d)) some potential CF mismatches that are accepted by EG as matches 120572
119877= 000018 in (a) and (b) 120572
119877= 000048 in
(c) and (d) and 120572119868
= 10597
(a) 119878EG119888cap 119878
CF119888
(b) 119878EG119888cap 119878
CF119888
(c) 119878EG119888cap 119878
CF119888
(d) 119878EG119888cap 119878
CF119888
Figure 17 Evaluation onMiddlebury Stereo Datasets drumsticks and dwarves ((a) and (c)) Some potential EGmismatches that are acceptedby CF as matches ((b) and (d)) some potential CF mismatches that are accepted by EG as matches 120572
119877= 000078 in (a) and (b) 120572
119877= 00035
in (c) and (d) and 120572119868
= 10597
conditionrdquoThe characteristic is that theCFmodel can reducethe searching space of CPs into an elliptical area and themismatches of the CF are usually near the correct CPs
Conflict of Interests
The authors declare that there is no conflict of interestsregarding the publication of this paper
Acknowledgments
The authors are grateful for valuable advice from DrDibyendu Mukherjee and the anonymous reviewers andwould like to thank the National Natural Science Foundationof China (Grant nos 61075033 61005033 and 61273248)the Natural Science Foundation of Guangdong Province(2014A030313425 and S2011010003348) and theOpenProject
14 Mathematical Problems in Engineering
(a) 119878EG119888cap 119878
CF119888
(b) 119878EG119888cap 119878
CF119888
(c) 119878EG119888cap 119878
CF119888
(d) 119878EG119888cap 119878
CF119888
Figure 18 Evaluation onMiddlebury Stereo Datasets laundry andmoebius ((a) and (c)) Some potential EGmismatches that are accepted byCF asmatches ((b) and (d)) some potential CFmismatches that are accepted by EG asmatches 120572
119877= 000000275 in (a) and (b) 120572
119877= 000035
in (c) and (d) and 120572119868
= 10597
(a) 119878EG119888cap 119878
CF119888
(b) 119878EG119888cap 119878
CF119888
Figure 19 Evaluation on Middlebury Stereo Dataset reindeer (a)Two potential EG mismatches that are accepted by CF as matches(b) two potential CFmismatches that are accepted by EG asmatches120572119877
= 0005 and 120572119868
= 30
Program of the National Laboratory of Pattern Recognition(NLPR) (201001060) for their support
References
[1] J P Monaco A C Bovik and L K Cormack ldquoActive foveateduncalibrated stereovisionrdquo International Journal of ComputerVision vol 85 no 2 pp 192ndash207 2009
[2] X Li and Z Hu ldquoRejecting mismatches by correspondencefunctionrdquo International Journal of Computer Vision vol 89 no1 pp 1ndash17 2010
[3] R Hartley and A Zisserman Multiple View Geometry inComputer Vision Cambridge University Press 2nd edition2003
[4] M Sonka V Hlavac and R Boyle Image Processing Analysisand Machine Vision Thomson-Engineering 3rd edition 2007
[5] G Carneiro and A D Jepson ldquoFlexible spatial configuration oflocal image featuresrdquo IEEE Transactions on Pattern Analysis andMachine Intelligence vol 29 no 12 pp 2089ndash2104 2007
[6] W Aguilar Y Frauel F Escolano M E Martinez-Perez AEspinosa-Romero andM A Lozano ldquoA robust graph transfor-mation matching for non-rigid registrationrdquo Image and VisionComputing vol 27 no 7 pp 897ndash910 2009
[7] L Wang F Duan and K Lv ldquoFisheye-lens-based visual suncompass for perception of spatial orientationrdquo MathematicalProblems in Engineering vol 2012 Article ID 460430 p 15 2012
[8] P Huber Robust Statistics Wiley New York NY USA 1981[9] P J Rousseeuw ldquoLeast median of squares regressionrdquo Journal
of the American Statistical Association vol 79 no 388 pp 871ndash880 1984
[10] P H S Torr and A Zisserman ldquoMLESAC a new robustestimator with application to estimating image geometryrdquoComputer Vision and Image Understanding vol 78 no 1 pp138ndash156 2000
[11] M A Fischler and R C Bolles ldquoRandom sample consensus aparadigm for model fitting with applications to image analysisand automated cartographyrdquo Communications of the ACM vol24 no 6 pp 381ndash395 1981
[12] B J Tordoff and D W Murray ldquoGuided-MLESAC fasterimage transform estimation by using matching priorsrdquo IEEETransactions on Pattern Analysis and Machine Intelligence vol27 no 10 pp 1523ndash1535 2005
Mathematical Problems in Engineering 15
[13] D Sidibe P Montesinos and S Janaqi ldquoMatching localinvariant features with contextual information an experimentalevaluationrdquo Electronic Letters on Computer Vision and ImageAnalysis vol 7 no 1 pp 26ndash39 2008
[14] Q-H Tran T-J Chin G Carneiro M S Brown and D SuterldquoIn defence of RANSAC for outlier rejection in deformableregistrationrdquo Lecture Notes in Computer Science vol 7575 no4 pp 274ndash287 2012
[15] A Goshtasby ldquoTransformation functions for image registra-tionrdquo in Proceedings of the Computer Vision and Pattern Recog-nition (CVPR rsquo11) June 2011
[16] X Li Y Wang J Zhou and X Li ldquoComparative research oncorrespondence functionrdquo in Proceedings of the InternationalConference on Intelligent System Design and Engineering Appli-cation (ISDEA rsquo10) pp 444ndash447 October 2010
[17] X-R Li X-M Li H-L Li and M-Y Cao ldquoRejecting outliersbased on correspondence manifoldrdquo Acta Automatica Sinicavol 35 no 1 pp 17ndash22 2009
[18] D Pizarro and A Bartoli ldquoFeature-based deformable surfacedetection with self-occlusion reasoningrdquo International Journalof Computer Vision vol 97 no 1 pp 54ndash70 2012
[19] J Yu A Eriksson T-J Chin and D Suter ldquoAn adversarialoptimization approach to efficient outlier removalrdquo Journal ofMathematical Imaging and Vision vol 48 no 3 pp 451ndash4662014
[20] A Goshtasby ldquoRegistration of images with geometric distor-tionsrdquo IEEETransactions onGeoscience andRemote Sensing vol26 no 1 pp 60ndash64 1988
[21] F L Bookstein ldquoPrincipal warps thin-plate splines and thedecomposition of deformationsrdquo IEEE Transactions on PatternAnalysis and Machine Intelligence vol 11 no 6 pp 567ndash5851992
[22] B Scholkopf F Steinke and V Blanz ldquoObject correspondenceas a machine learning problemrdquo in Proceedings of the 22ndInternational Conference on Machine Learning (ICML rsquo05) pp776ndash783 2005
[23] L Zagorchev and A Goshtasby ldquoA comparative study oftransformation functions for nonrigid image registrationrdquo IEEETransactions on Image Processing vol 15 no 3 pp 529ndash5382006
[24] J Zhao JMa J Tian J Ma andD Zhang ldquoA robustmethod forvector field learning with application to mismatch removingrdquoin Proceedings of the IEEE Conference on Computer Vision andPattern Recognition (CVPR rsquo11) Providence RI USA June 2011
[25] JMa J Zhao J Tian X Bai and Z Tu ldquoRegularized vector fieldlearning with sparse approximation for mismatch removalrdquoPattern Recognition vol 46 no 12 pp 3519ndash3532 2013
[26] J Ma J Zhao J Tian Z Tu and A L Yuille ldquoRobust estimationof nonrigid transformation for point set registrationrdquo in Pro-ceedings of the 26th IEEE Conference on Computer Vision andPattern Recognition (CVPR rsquo13) pp 2147ndash2154 June 2013
[27] J MaW Qiu J Zhao Y Ma A L Yuille and Z Tu ldquoRobust l2e
estimation of transformation for non-rigid registrationrdquo IEEETransactions on Signal Processing vol 63 no 5 pp 1115ndash11292015
[28] J Ma J Zhao J Tian A L Yuille and Z Tu ldquoRobust pointmatching via vector field consensusrdquo IEEE Transactions onImage Processing vol 23 no 4 pp 1706ndash1721 2014
[29] L Goshen and I Shimshoni ldquoBalanced exploration andexploitation model search for efficient epipolar geometry esti-mationrdquo IEEE Transactions on Pattern Analysis and MachineIntelligence vol 30 no 7 pp 1230ndash1242 2008
[30] O Chum and J Matas ldquoMatching with prosacmdashprogressivesample consensusrdquo in Proceedings of the IEEE Computer SocietyConference on Computer Vision and Pattern Recognition (CVPRrsquo05) vol 1 pp 220ndash226 IEEE June 2005
[31] K Mikolajczyk and C Schmid ldquoA performance evaluation oflocal descriptorsrdquo IEEE Transactions on Pattern Analysis andMachine Intelligence vol 27 no 10 pp 1615ndash1630 2005
[32] D G Lowe ldquoDistinctive image features from scale-invariantkeypointsrdquo International Journal of Computer Vision vol 60 no2 pp 91ndash110 2004
[33] S Gauglitz T Hollerer and M Turk ldquoEvaluation of interestpoint detectors and feature descriptors for visual trackingrdquoInternational Journal of Computer Vision vol 94 no 3 pp 335ndash360 2011
[34] Y Yu K Huang W Chen and T Tan ldquoA novel algorithmfor view and illumination invariant image matchingrdquo IEEETransactions on Image Processing vol 21 no 1 pp 229ndash2402012
[35] K Mikolajczyk T Tuytelaars C Schmid et al ldquoA comparisonof affine region detectorsrdquo International Journal of ComputerVision vol 65 no 1-2 pp 43ndash72 2005
[36] H Hirschmuller and D Scharstein ldquoEvaluation of cost func-tions for stereo matchingrdquo in Proceedings of the IEEE ComputerSociety Conference on Computer Vision and Pattern Recognition(CVPR rsquo07) June 2007
[37] D Scharstein and C Pal ldquoLearning conditional random fieldsfor stereordquo in Proceedings of the IEEE Computer Society Confer-ence on Computer Vision and Pattern Recognition (CVPR rsquo07)IEEE Minneapolis Minn USA June 2007
[38] D Scharstein andR Szeliski ldquoHigh-accuracy stereo depthmapsusing structured lightrdquo in Proceedings of the IEEE ComputerSociety Conference on Computer Vision and Pattern Recognition(CVPR rsquo03) vol 1 pp I-195ndashI-202 IEEE June 2003
[39] H Wang S Yan J Liu X Tang and T S Huang ldquoCorre-spondence propagationwithweak priorsrdquo IEEETransactions onImage Processing vol 18 no 1 pp 140ndash150 2009
Figure 14 Quantitative evaluation on two popular benchmark sets trees and ubc The ground-truth correspondences are displayed as bluedashed lines and the computed putative correspondences are shown by a yellow solid line (a) and (c) present some EG PMs that are acceptedbyCF asmatches (b) and (d) present someCFPMs that are accepted by EG asmatches120572
119868= 10597120572
119877= 59times10
minus5 in (b) and120572119877
= 185times10minus4
in (d)
(a) 119878EG119888cap 119878
CF119888
(b) 119878EG119888cap 119878
CF119888
(c) 119878EG119888cap 119878
CF119888
(d) 119878EG119888cap 119878
CF119888
Figure 15 Evaluation on Middlebury Stereo Datasets art and books ((a) and (c)) Some potential EG mismatches that are accepted by CF asmatches ((b) and (d)) some potential CF mismatches that are accepted by EG as matches 120572
119877= 0013 in (a) and (b) 120572
119877= 000018 in (c) and
(d) and 120572119868
= 10597
this study we investigated the principles of the CF andstudied its characteristics by comparing it with the widelyused parametric model epipolar geometry (EG) constraint
It is shown that the CF describes the mapping relation-ships between two CPs and should be able to estimate theposition of the corresponding point Therefore in additionto mismatch rejection a potential application of the CF is to
guide the point matching process by incorporating it into thecorrespondence propagation [39]
In practice putative correspondences are usually cor-rupted by noise and outliers and the number of putativecorrespondences is limited Therefore only an estimationof the CF with a certain accuracy can be estimated andthe actual results will violate the ldquonecessary and sufficient
Mathematical Problems in Engineering 13
(a) 119878EG119888cap SCF119888
(b) 119878EG119888cap 119878
CF119888
(c) 119878EG119888cap 119878
CF119888
(d) 119878EG119888cap 119878
CF119888
Figure 16 Evaluation on Middlebury Stereo Datasets computer and dolls ((a) and (c)) Some potential EG mismatches that are accepted byCF as matches ((b) and (d)) some potential CF mismatches that are accepted by EG as matches 120572
119877= 000018 in (a) and (b) 120572
119877= 000048 in
(c) and (d) and 120572119868
= 10597
(a) 119878EG119888cap 119878
CF119888
(b) 119878EG119888cap 119878
CF119888
(c) 119878EG119888cap 119878
CF119888
(d) 119878EG119888cap 119878
CF119888
Figure 17 Evaluation onMiddlebury Stereo Datasets drumsticks and dwarves ((a) and (c)) Some potential EGmismatches that are acceptedby CF as matches ((b) and (d)) some potential CF mismatches that are accepted by EG as matches 120572
119877= 000078 in (a) and (b) 120572
119877= 00035
in (c) and (d) and 120572119868
= 10597
conditionrdquoThe characteristic is that theCFmodel can reducethe searching space of CPs into an elliptical area and themismatches of the CF are usually near the correct CPs
Conflict of Interests
The authors declare that there is no conflict of interestsregarding the publication of this paper
Acknowledgments
The authors are grateful for valuable advice from DrDibyendu Mukherjee and the anonymous reviewers andwould like to thank the National Natural Science Foundationof China (Grant nos 61075033 61005033 and 61273248)the Natural Science Foundation of Guangdong Province(2014A030313425 and S2011010003348) and theOpenProject
14 Mathematical Problems in Engineering
(a) 119878EG119888cap 119878
CF119888
(b) 119878EG119888cap 119878
CF119888
(c) 119878EG119888cap 119878
CF119888
(d) 119878EG119888cap 119878
CF119888
Figure 18 Evaluation onMiddlebury Stereo Datasets laundry andmoebius ((a) and (c)) Some potential EGmismatches that are accepted byCF asmatches ((b) and (d)) some potential CFmismatches that are accepted by EG asmatches 120572
119877= 000000275 in (a) and (b) 120572
119877= 000035
in (c) and (d) and 120572119868
= 10597
(a) 119878EG119888cap 119878
CF119888
(b) 119878EG119888cap 119878
CF119888
Figure 19 Evaluation on Middlebury Stereo Dataset reindeer (a)Two potential EG mismatches that are accepted by CF as matches(b) two potential CFmismatches that are accepted by EG asmatches120572119877
= 0005 and 120572119868
= 30
Program of the National Laboratory of Pattern Recognition(NLPR) (201001060) for their support
References
[1] J P Monaco A C Bovik and L K Cormack ldquoActive foveateduncalibrated stereovisionrdquo International Journal of ComputerVision vol 85 no 2 pp 192ndash207 2009
[2] X Li and Z Hu ldquoRejecting mismatches by correspondencefunctionrdquo International Journal of Computer Vision vol 89 no1 pp 1ndash17 2010
[3] R Hartley and A Zisserman Multiple View Geometry inComputer Vision Cambridge University Press 2nd edition2003
[4] M Sonka V Hlavac and R Boyle Image Processing Analysisand Machine Vision Thomson-Engineering 3rd edition 2007
[5] G Carneiro and A D Jepson ldquoFlexible spatial configuration oflocal image featuresrdquo IEEE Transactions on Pattern Analysis andMachine Intelligence vol 29 no 12 pp 2089ndash2104 2007
[6] W Aguilar Y Frauel F Escolano M E Martinez-Perez AEspinosa-Romero andM A Lozano ldquoA robust graph transfor-mation matching for non-rigid registrationrdquo Image and VisionComputing vol 27 no 7 pp 897ndash910 2009
[7] L Wang F Duan and K Lv ldquoFisheye-lens-based visual suncompass for perception of spatial orientationrdquo MathematicalProblems in Engineering vol 2012 Article ID 460430 p 15 2012
[8] P Huber Robust Statistics Wiley New York NY USA 1981[9] P J Rousseeuw ldquoLeast median of squares regressionrdquo Journal
of the American Statistical Association vol 79 no 388 pp 871ndash880 1984
[10] P H S Torr and A Zisserman ldquoMLESAC a new robustestimator with application to estimating image geometryrdquoComputer Vision and Image Understanding vol 78 no 1 pp138ndash156 2000
[11] M A Fischler and R C Bolles ldquoRandom sample consensus aparadigm for model fitting with applications to image analysisand automated cartographyrdquo Communications of the ACM vol24 no 6 pp 381ndash395 1981
[12] B J Tordoff and D W Murray ldquoGuided-MLESAC fasterimage transform estimation by using matching priorsrdquo IEEETransactions on Pattern Analysis and Machine Intelligence vol27 no 10 pp 1523ndash1535 2005
Mathematical Problems in Engineering 15
[13] D Sidibe P Montesinos and S Janaqi ldquoMatching localinvariant features with contextual information an experimentalevaluationrdquo Electronic Letters on Computer Vision and ImageAnalysis vol 7 no 1 pp 26ndash39 2008
[14] Q-H Tran T-J Chin G Carneiro M S Brown and D SuterldquoIn defence of RANSAC for outlier rejection in deformableregistrationrdquo Lecture Notes in Computer Science vol 7575 no4 pp 274ndash287 2012
[15] A Goshtasby ldquoTransformation functions for image registra-tionrdquo in Proceedings of the Computer Vision and Pattern Recog-nition (CVPR rsquo11) June 2011
[16] X Li Y Wang J Zhou and X Li ldquoComparative research oncorrespondence functionrdquo in Proceedings of the InternationalConference on Intelligent System Design and Engineering Appli-cation (ISDEA rsquo10) pp 444ndash447 October 2010
[17] X-R Li X-M Li H-L Li and M-Y Cao ldquoRejecting outliersbased on correspondence manifoldrdquo Acta Automatica Sinicavol 35 no 1 pp 17ndash22 2009
[18] D Pizarro and A Bartoli ldquoFeature-based deformable surfacedetection with self-occlusion reasoningrdquo International Journalof Computer Vision vol 97 no 1 pp 54ndash70 2012
[19] J Yu A Eriksson T-J Chin and D Suter ldquoAn adversarialoptimization approach to efficient outlier removalrdquo Journal ofMathematical Imaging and Vision vol 48 no 3 pp 451ndash4662014
[20] A Goshtasby ldquoRegistration of images with geometric distor-tionsrdquo IEEETransactions onGeoscience andRemote Sensing vol26 no 1 pp 60ndash64 1988
[21] F L Bookstein ldquoPrincipal warps thin-plate splines and thedecomposition of deformationsrdquo IEEE Transactions on PatternAnalysis and Machine Intelligence vol 11 no 6 pp 567ndash5851992
[22] B Scholkopf F Steinke and V Blanz ldquoObject correspondenceas a machine learning problemrdquo in Proceedings of the 22ndInternational Conference on Machine Learning (ICML rsquo05) pp776ndash783 2005
[23] L Zagorchev and A Goshtasby ldquoA comparative study oftransformation functions for nonrigid image registrationrdquo IEEETransactions on Image Processing vol 15 no 3 pp 529ndash5382006
[24] J Zhao JMa J Tian J Ma andD Zhang ldquoA robustmethod forvector field learning with application to mismatch removingrdquoin Proceedings of the IEEE Conference on Computer Vision andPattern Recognition (CVPR rsquo11) Providence RI USA June 2011
[25] JMa J Zhao J Tian X Bai and Z Tu ldquoRegularized vector fieldlearning with sparse approximation for mismatch removalrdquoPattern Recognition vol 46 no 12 pp 3519ndash3532 2013
[26] J Ma J Zhao J Tian Z Tu and A L Yuille ldquoRobust estimationof nonrigid transformation for point set registrationrdquo in Pro-ceedings of the 26th IEEE Conference on Computer Vision andPattern Recognition (CVPR rsquo13) pp 2147ndash2154 June 2013
[27] J MaW Qiu J Zhao Y Ma A L Yuille and Z Tu ldquoRobust l2e
estimation of transformation for non-rigid registrationrdquo IEEETransactions on Signal Processing vol 63 no 5 pp 1115ndash11292015
[28] J Ma J Zhao J Tian A L Yuille and Z Tu ldquoRobust pointmatching via vector field consensusrdquo IEEE Transactions onImage Processing vol 23 no 4 pp 1706ndash1721 2014
[29] L Goshen and I Shimshoni ldquoBalanced exploration andexploitation model search for efficient epipolar geometry esti-mationrdquo IEEE Transactions on Pattern Analysis and MachineIntelligence vol 30 no 7 pp 1230ndash1242 2008
[30] O Chum and J Matas ldquoMatching with prosacmdashprogressivesample consensusrdquo in Proceedings of the IEEE Computer SocietyConference on Computer Vision and Pattern Recognition (CVPRrsquo05) vol 1 pp 220ndash226 IEEE June 2005
[31] K Mikolajczyk and C Schmid ldquoA performance evaluation oflocal descriptorsrdquo IEEE Transactions on Pattern Analysis andMachine Intelligence vol 27 no 10 pp 1615ndash1630 2005
[32] D G Lowe ldquoDistinctive image features from scale-invariantkeypointsrdquo International Journal of Computer Vision vol 60 no2 pp 91ndash110 2004
[33] S Gauglitz T Hollerer and M Turk ldquoEvaluation of interestpoint detectors and feature descriptors for visual trackingrdquoInternational Journal of Computer Vision vol 94 no 3 pp 335ndash360 2011
[34] Y Yu K Huang W Chen and T Tan ldquoA novel algorithmfor view and illumination invariant image matchingrdquo IEEETransactions on Image Processing vol 21 no 1 pp 229ndash2402012
[35] K Mikolajczyk T Tuytelaars C Schmid et al ldquoA comparisonof affine region detectorsrdquo International Journal of ComputerVision vol 65 no 1-2 pp 43ndash72 2005
[36] H Hirschmuller and D Scharstein ldquoEvaluation of cost func-tions for stereo matchingrdquo in Proceedings of the IEEE ComputerSociety Conference on Computer Vision and Pattern Recognition(CVPR rsquo07) June 2007
[37] D Scharstein and C Pal ldquoLearning conditional random fieldsfor stereordquo in Proceedings of the IEEE Computer Society Confer-ence on Computer Vision and Pattern Recognition (CVPR rsquo07)IEEE Minneapolis Minn USA June 2007
[38] D Scharstein andR Szeliski ldquoHigh-accuracy stereo depthmapsusing structured lightrdquo in Proceedings of the IEEE ComputerSociety Conference on Computer Vision and Pattern Recognition(CVPR rsquo03) vol 1 pp I-195ndashI-202 IEEE June 2003
[39] H Wang S Yan J Liu X Tang and T S Huang ldquoCorre-spondence propagationwithweak priorsrdquo IEEETransactions onImage Processing vol 18 no 1 pp 140ndash150 2009
Figure 16 Evaluation on Middlebury Stereo Datasets computer and dolls ((a) and (c)) Some potential EG mismatches that are accepted byCF as matches ((b) and (d)) some potential CF mismatches that are accepted by EG as matches 120572
119877= 000018 in (a) and (b) 120572
119877= 000048 in
(c) and (d) and 120572119868
= 10597
(a) 119878EG119888cap 119878
CF119888
(b) 119878EG119888cap 119878
CF119888
(c) 119878EG119888cap 119878
CF119888
(d) 119878EG119888cap 119878
CF119888
Figure 17 Evaluation onMiddlebury Stereo Datasets drumsticks and dwarves ((a) and (c)) Some potential EGmismatches that are acceptedby CF as matches ((b) and (d)) some potential CF mismatches that are accepted by EG as matches 120572
119877= 000078 in (a) and (b) 120572
119877= 00035
in (c) and (d) and 120572119868
= 10597
conditionrdquoThe characteristic is that theCFmodel can reducethe searching space of CPs into an elliptical area and themismatches of the CF are usually near the correct CPs
Conflict of Interests
The authors declare that there is no conflict of interestsregarding the publication of this paper
Acknowledgments
The authors are grateful for valuable advice from DrDibyendu Mukherjee and the anonymous reviewers andwould like to thank the National Natural Science Foundationof China (Grant nos 61075033 61005033 and 61273248)the Natural Science Foundation of Guangdong Province(2014A030313425 and S2011010003348) and theOpenProject
14 Mathematical Problems in Engineering
(a) 119878EG119888cap 119878
CF119888
(b) 119878EG119888cap 119878
CF119888
(c) 119878EG119888cap 119878
CF119888
(d) 119878EG119888cap 119878
CF119888
Figure 18 Evaluation onMiddlebury Stereo Datasets laundry andmoebius ((a) and (c)) Some potential EGmismatches that are accepted byCF asmatches ((b) and (d)) some potential CFmismatches that are accepted by EG asmatches 120572
119877= 000000275 in (a) and (b) 120572
119877= 000035
in (c) and (d) and 120572119868
= 10597
(a) 119878EG119888cap 119878
CF119888
(b) 119878EG119888cap 119878
CF119888
Figure 19 Evaluation on Middlebury Stereo Dataset reindeer (a)Two potential EG mismatches that are accepted by CF as matches(b) two potential CFmismatches that are accepted by EG asmatches120572119877
= 0005 and 120572119868
= 30
Program of the National Laboratory of Pattern Recognition(NLPR) (201001060) for their support
References
[1] J P Monaco A C Bovik and L K Cormack ldquoActive foveateduncalibrated stereovisionrdquo International Journal of ComputerVision vol 85 no 2 pp 192ndash207 2009
[2] X Li and Z Hu ldquoRejecting mismatches by correspondencefunctionrdquo International Journal of Computer Vision vol 89 no1 pp 1ndash17 2010
[3] R Hartley and A Zisserman Multiple View Geometry inComputer Vision Cambridge University Press 2nd edition2003
[4] M Sonka V Hlavac and R Boyle Image Processing Analysisand Machine Vision Thomson-Engineering 3rd edition 2007
[5] G Carneiro and A D Jepson ldquoFlexible spatial configuration oflocal image featuresrdquo IEEE Transactions on Pattern Analysis andMachine Intelligence vol 29 no 12 pp 2089ndash2104 2007
[6] W Aguilar Y Frauel F Escolano M E Martinez-Perez AEspinosa-Romero andM A Lozano ldquoA robust graph transfor-mation matching for non-rigid registrationrdquo Image and VisionComputing vol 27 no 7 pp 897ndash910 2009
[7] L Wang F Duan and K Lv ldquoFisheye-lens-based visual suncompass for perception of spatial orientationrdquo MathematicalProblems in Engineering vol 2012 Article ID 460430 p 15 2012
[8] P Huber Robust Statistics Wiley New York NY USA 1981[9] P J Rousseeuw ldquoLeast median of squares regressionrdquo Journal
of the American Statistical Association vol 79 no 388 pp 871ndash880 1984
[10] P H S Torr and A Zisserman ldquoMLESAC a new robustestimator with application to estimating image geometryrdquoComputer Vision and Image Understanding vol 78 no 1 pp138ndash156 2000
[11] M A Fischler and R C Bolles ldquoRandom sample consensus aparadigm for model fitting with applications to image analysisand automated cartographyrdquo Communications of the ACM vol24 no 6 pp 381ndash395 1981
[12] B J Tordoff and D W Murray ldquoGuided-MLESAC fasterimage transform estimation by using matching priorsrdquo IEEETransactions on Pattern Analysis and Machine Intelligence vol27 no 10 pp 1523ndash1535 2005
Mathematical Problems in Engineering 15
[13] D Sidibe P Montesinos and S Janaqi ldquoMatching localinvariant features with contextual information an experimentalevaluationrdquo Electronic Letters on Computer Vision and ImageAnalysis vol 7 no 1 pp 26ndash39 2008
[14] Q-H Tran T-J Chin G Carneiro M S Brown and D SuterldquoIn defence of RANSAC for outlier rejection in deformableregistrationrdquo Lecture Notes in Computer Science vol 7575 no4 pp 274ndash287 2012
[15] A Goshtasby ldquoTransformation functions for image registra-tionrdquo in Proceedings of the Computer Vision and Pattern Recog-nition (CVPR rsquo11) June 2011
[16] X Li Y Wang J Zhou and X Li ldquoComparative research oncorrespondence functionrdquo in Proceedings of the InternationalConference on Intelligent System Design and Engineering Appli-cation (ISDEA rsquo10) pp 444ndash447 October 2010
[17] X-R Li X-M Li H-L Li and M-Y Cao ldquoRejecting outliersbased on correspondence manifoldrdquo Acta Automatica Sinicavol 35 no 1 pp 17ndash22 2009
[18] D Pizarro and A Bartoli ldquoFeature-based deformable surfacedetection with self-occlusion reasoningrdquo International Journalof Computer Vision vol 97 no 1 pp 54ndash70 2012
[19] J Yu A Eriksson T-J Chin and D Suter ldquoAn adversarialoptimization approach to efficient outlier removalrdquo Journal ofMathematical Imaging and Vision vol 48 no 3 pp 451ndash4662014
[20] A Goshtasby ldquoRegistration of images with geometric distor-tionsrdquo IEEETransactions onGeoscience andRemote Sensing vol26 no 1 pp 60ndash64 1988
[21] F L Bookstein ldquoPrincipal warps thin-plate splines and thedecomposition of deformationsrdquo IEEE Transactions on PatternAnalysis and Machine Intelligence vol 11 no 6 pp 567ndash5851992
[22] B Scholkopf F Steinke and V Blanz ldquoObject correspondenceas a machine learning problemrdquo in Proceedings of the 22ndInternational Conference on Machine Learning (ICML rsquo05) pp776ndash783 2005
[23] L Zagorchev and A Goshtasby ldquoA comparative study oftransformation functions for nonrigid image registrationrdquo IEEETransactions on Image Processing vol 15 no 3 pp 529ndash5382006
[24] J Zhao JMa J Tian J Ma andD Zhang ldquoA robustmethod forvector field learning with application to mismatch removingrdquoin Proceedings of the IEEE Conference on Computer Vision andPattern Recognition (CVPR rsquo11) Providence RI USA June 2011
[25] JMa J Zhao J Tian X Bai and Z Tu ldquoRegularized vector fieldlearning with sparse approximation for mismatch removalrdquoPattern Recognition vol 46 no 12 pp 3519ndash3532 2013
[26] J Ma J Zhao J Tian Z Tu and A L Yuille ldquoRobust estimationof nonrigid transformation for point set registrationrdquo in Pro-ceedings of the 26th IEEE Conference on Computer Vision andPattern Recognition (CVPR rsquo13) pp 2147ndash2154 June 2013
[27] J MaW Qiu J Zhao Y Ma A L Yuille and Z Tu ldquoRobust l2e
estimation of transformation for non-rigid registrationrdquo IEEETransactions on Signal Processing vol 63 no 5 pp 1115ndash11292015
[28] J Ma J Zhao J Tian A L Yuille and Z Tu ldquoRobust pointmatching via vector field consensusrdquo IEEE Transactions onImage Processing vol 23 no 4 pp 1706ndash1721 2014
[29] L Goshen and I Shimshoni ldquoBalanced exploration andexploitation model search for efficient epipolar geometry esti-mationrdquo IEEE Transactions on Pattern Analysis and MachineIntelligence vol 30 no 7 pp 1230ndash1242 2008
[30] O Chum and J Matas ldquoMatching with prosacmdashprogressivesample consensusrdquo in Proceedings of the IEEE Computer SocietyConference on Computer Vision and Pattern Recognition (CVPRrsquo05) vol 1 pp 220ndash226 IEEE June 2005
[31] K Mikolajczyk and C Schmid ldquoA performance evaluation oflocal descriptorsrdquo IEEE Transactions on Pattern Analysis andMachine Intelligence vol 27 no 10 pp 1615ndash1630 2005
[32] D G Lowe ldquoDistinctive image features from scale-invariantkeypointsrdquo International Journal of Computer Vision vol 60 no2 pp 91ndash110 2004
[33] S Gauglitz T Hollerer and M Turk ldquoEvaluation of interestpoint detectors and feature descriptors for visual trackingrdquoInternational Journal of Computer Vision vol 94 no 3 pp 335ndash360 2011
[34] Y Yu K Huang W Chen and T Tan ldquoA novel algorithmfor view and illumination invariant image matchingrdquo IEEETransactions on Image Processing vol 21 no 1 pp 229ndash2402012
[35] K Mikolajczyk T Tuytelaars C Schmid et al ldquoA comparisonof affine region detectorsrdquo International Journal of ComputerVision vol 65 no 1-2 pp 43ndash72 2005
[36] H Hirschmuller and D Scharstein ldquoEvaluation of cost func-tions for stereo matchingrdquo in Proceedings of the IEEE ComputerSociety Conference on Computer Vision and Pattern Recognition(CVPR rsquo07) June 2007
[37] D Scharstein and C Pal ldquoLearning conditional random fieldsfor stereordquo in Proceedings of the IEEE Computer Society Confer-ence on Computer Vision and Pattern Recognition (CVPR rsquo07)IEEE Minneapolis Minn USA June 2007
[38] D Scharstein andR Szeliski ldquoHigh-accuracy stereo depthmapsusing structured lightrdquo in Proceedings of the IEEE ComputerSociety Conference on Computer Vision and Pattern Recognition(CVPR rsquo03) vol 1 pp I-195ndashI-202 IEEE June 2003
[39] H Wang S Yan J Liu X Tang and T S Huang ldquoCorre-spondence propagationwithweak priorsrdquo IEEETransactions onImage Processing vol 18 no 1 pp 140ndash150 2009
Figure 18 Evaluation onMiddlebury Stereo Datasets laundry andmoebius ((a) and (c)) Some potential EGmismatches that are accepted byCF asmatches ((b) and (d)) some potential CFmismatches that are accepted by EG asmatches 120572
119877= 000000275 in (a) and (b) 120572
119877= 000035
in (c) and (d) and 120572119868
= 10597
(a) 119878EG119888cap 119878
CF119888
(b) 119878EG119888cap 119878
CF119888
Figure 19 Evaluation on Middlebury Stereo Dataset reindeer (a)Two potential EG mismatches that are accepted by CF as matches(b) two potential CFmismatches that are accepted by EG asmatches120572119877
= 0005 and 120572119868
= 30
Program of the National Laboratory of Pattern Recognition(NLPR) (201001060) for their support
References
[1] J P Monaco A C Bovik and L K Cormack ldquoActive foveateduncalibrated stereovisionrdquo International Journal of ComputerVision vol 85 no 2 pp 192ndash207 2009
[2] X Li and Z Hu ldquoRejecting mismatches by correspondencefunctionrdquo International Journal of Computer Vision vol 89 no1 pp 1ndash17 2010
[3] R Hartley and A Zisserman Multiple View Geometry inComputer Vision Cambridge University Press 2nd edition2003
[4] M Sonka V Hlavac and R Boyle Image Processing Analysisand Machine Vision Thomson-Engineering 3rd edition 2007
[5] G Carneiro and A D Jepson ldquoFlexible spatial configuration oflocal image featuresrdquo IEEE Transactions on Pattern Analysis andMachine Intelligence vol 29 no 12 pp 2089ndash2104 2007
[6] W Aguilar Y Frauel F Escolano M E Martinez-Perez AEspinosa-Romero andM A Lozano ldquoA robust graph transfor-mation matching for non-rigid registrationrdquo Image and VisionComputing vol 27 no 7 pp 897ndash910 2009
[7] L Wang F Duan and K Lv ldquoFisheye-lens-based visual suncompass for perception of spatial orientationrdquo MathematicalProblems in Engineering vol 2012 Article ID 460430 p 15 2012
[8] P Huber Robust Statistics Wiley New York NY USA 1981[9] P J Rousseeuw ldquoLeast median of squares regressionrdquo Journal
of the American Statistical Association vol 79 no 388 pp 871ndash880 1984
[10] P H S Torr and A Zisserman ldquoMLESAC a new robustestimator with application to estimating image geometryrdquoComputer Vision and Image Understanding vol 78 no 1 pp138ndash156 2000
[11] M A Fischler and R C Bolles ldquoRandom sample consensus aparadigm for model fitting with applications to image analysisand automated cartographyrdquo Communications of the ACM vol24 no 6 pp 381ndash395 1981
[12] B J Tordoff and D W Murray ldquoGuided-MLESAC fasterimage transform estimation by using matching priorsrdquo IEEETransactions on Pattern Analysis and Machine Intelligence vol27 no 10 pp 1523ndash1535 2005
Mathematical Problems in Engineering 15
[13] D Sidibe P Montesinos and S Janaqi ldquoMatching localinvariant features with contextual information an experimentalevaluationrdquo Electronic Letters on Computer Vision and ImageAnalysis vol 7 no 1 pp 26ndash39 2008
[14] Q-H Tran T-J Chin G Carneiro M S Brown and D SuterldquoIn defence of RANSAC for outlier rejection in deformableregistrationrdquo Lecture Notes in Computer Science vol 7575 no4 pp 274ndash287 2012
[15] A Goshtasby ldquoTransformation functions for image registra-tionrdquo in Proceedings of the Computer Vision and Pattern Recog-nition (CVPR rsquo11) June 2011
[16] X Li Y Wang J Zhou and X Li ldquoComparative research oncorrespondence functionrdquo in Proceedings of the InternationalConference on Intelligent System Design and Engineering Appli-cation (ISDEA rsquo10) pp 444ndash447 October 2010
[17] X-R Li X-M Li H-L Li and M-Y Cao ldquoRejecting outliersbased on correspondence manifoldrdquo Acta Automatica Sinicavol 35 no 1 pp 17ndash22 2009
[18] D Pizarro and A Bartoli ldquoFeature-based deformable surfacedetection with self-occlusion reasoningrdquo International Journalof Computer Vision vol 97 no 1 pp 54ndash70 2012
[19] J Yu A Eriksson T-J Chin and D Suter ldquoAn adversarialoptimization approach to efficient outlier removalrdquo Journal ofMathematical Imaging and Vision vol 48 no 3 pp 451ndash4662014
[20] A Goshtasby ldquoRegistration of images with geometric distor-tionsrdquo IEEETransactions onGeoscience andRemote Sensing vol26 no 1 pp 60ndash64 1988
[21] F L Bookstein ldquoPrincipal warps thin-plate splines and thedecomposition of deformationsrdquo IEEE Transactions on PatternAnalysis and Machine Intelligence vol 11 no 6 pp 567ndash5851992
[22] B Scholkopf F Steinke and V Blanz ldquoObject correspondenceas a machine learning problemrdquo in Proceedings of the 22ndInternational Conference on Machine Learning (ICML rsquo05) pp776ndash783 2005
[23] L Zagorchev and A Goshtasby ldquoA comparative study oftransformation functions for nonrigid image registrationrdquo IEEETransactions on Image Processing vol 15 no 3 pp 529ndash5382006
[24] J Zhao JMa J Tian J Ma andD Zhang ldquoA robustmethod forvector field learning with application to mismatch removingrdquoin Proceedings of the IEEE Conference on Computer Vision andPattern Recognition (CVPR rsquo11) Providence RI USA June 2011
[25] JMa J Zhao J Tian X Bai and Z Tu ldquoRegularized vector fieldlearning with sparse approximation for mismatch removalrdquoPattern Recognition vol 46 no 12 pp 3519ndash3532 2013
[26] J Ma J Zhao J Tian Z Tu and A L Yuille ldquoRobust estimationof nonrigid transformation for point set registrationrdquo in Pro-ceedings of the 26th IEEE Conference on Computer Vision andPattern Recognition (CVPR rsquo13) pp 2147ndash2154 June 2013
[27] J MaW Qiu J Zhao Y Ma A L Yuille and Z Tu ldquoRobust l2e
estimation of transformation for non-rigid registrationrdquo IEEETransactions on Signal Processing vol 63 no 5 pp 1115ndash11292015
[28] J Ma J Zhao J Tian A L Yuille and Z Tu ldquoRobust pointmatching via vector field consensusrdquo IEEE Transactions onImage Processing vol 23 no 4 pp 1706ndash1721 2014
[29] L Goshen and I Shimshoni ldquoBalanced exploration andexploitation model search for efficient epipolar geometry esti-mationrdquo IEEE Transactions on Pattern Analysis and MachineIntelligence vol 30 no 7 pp 1230ndash1242 2008
[30] O Chum and J Matas ldquoMatching with prosacmdashprogressivesample consensusrdquo in Proceedings of the IEEE Computer SocietyConference on Computer Vision and Pattern Recognition (CVPRrsquo05) vol 1 pp 220ndash226 IEEE June 2005
[31] K Mikolajczyk and C Schmid ldquoA performance evaluation oflocal descriptorsrdquo IEEE Transactions on Pattern Analysis andMachine Intelligence vol 27 no 10 pp 1615ndash1630 2005
[32] D G Lowe ldquoDistinctive image features from scale-invariantkeypointsrdquo International Journal of Computer Vision vol 60 no2 pp 91ndash110 2004
[33] S Gauglitz T Hollerer and M Turk ldquoEvaluation of interestpoint detectors and feature descriptors for visual trackingrdquoInternational Journal of Computer Vision vol 94 no 3 pp 335ndash360 2011
[34] Y Yu K Huang W Chen and T Tan ldquoA novel algorithmfor view and illumination invariant image matchingrdquo IEEETransactions on Image Processing vol 21 no 1 pp 229ndash2402012
[35] K Mikolajczyk T Tuytelaars C Schmid et al ldquoA comparisonof affine region detectorsrdquo International Journal of ComputerVision vol 65 no 1-2 pp 43ndash72 2005
[36] H Hirschmuller and D Scharstein ldquoEvaluation of cost func-tions for stereo matchingrdquo in Proceedings of the IEEE ComputerSociety Conference on Computer Vision and Pattern Recognition(CVPR rsquo07) June 2007
[37] D Scharstein and C Pal ldquoLearning conditional random fieldsfor stereordquo in Proceedings of the IEEE Computer Society Confer-ence on Computer Vision and Pattern Recognition (CVPR rsquo07)IEEE Minneapolis Minn USA June 2007
[38] D Scharstein andR Szeliski ldquoHigh-accuracy stereo depthmapsusing structured lightrdquo in Proceedings of the IEEE ComputerSociety Conference on Computer Vision and Pattern Recognition(CVPR rsquo03) vol 1 pp I-195ndashI-202 IEEE June 2003
[39] H Wang S Yan J Liu X Tang and T S Huang ldquoCorre-spondence propagationwithweak priorsrdquo IEEETransactions onImage Processing vol 18 no 1 pp 140ndash150 2009
[13] D Sidibe P Montesinos and S Janaqi ldquoMatching localinvariant features with contextual information an experimentalevaluationrdquo Electronic Letters on Computer Vision and ImageAnalysis vol 7 no 1 pp 26ndash39 2008
[14] Q-H Tran T-J Chin G Carneiro M S Brown and D SuterldquoIn defence of RANSAC for outlier rejection in deformableregistrationrdquo Lecture Notes in Computer Science vol 7575 no4 pp 274ndash287 2012
[15] A Goshtasby ldquoTransformation functions for image registra-tionrdquo in Proceedings of the Computer Vision and Pattern Recog-nition (CVPR rsquo11) June 2011
[16] X Li Y Wang J Zhou and X Li ldquoComparative research oncorrespondence functionrdquo in Proceedings of the InternationalConference on Intelligent System Design and Engineering Appli-cation (ISDEA rsquo10) pp 444ndash447 October 2010
[17] X-R Li X-M Li H-L Li and M-Y Cao ldquoRejecting outliersbased on correspondence manifoldrdquo Acta Automatica Sinicavol 35 no 1 pp 17ndash22 2009
[18] D Pizarro and A Bartoli ldquoFeature-based deformable surfacedetection with self-occlusion reasoningrdquo International Journalof Computer Vision vol 97 no 1 pp 54ndash70 2012
[19] J Yu A Eriksson T-J Chin and D Suter ldquoAn adversarialoptimization approach to efficient outlier removalrdquo Journal ofMathematical Imaging and Vision vol 48 no 3 pp 451ndash4662014
[20] A Goshtasby ldquoRegistration of images with geometric distor-tionsrdquo IEEETransactions onGeoscience andRemote Sensing vol26 no 1 pp 60ndash64 1988
[21] F L Bookstein ldquoPrincipal warps thin-plate splines and thedecomposition of deformationsrdquo IEEE Transactions on PatternAnalysis and Machine Intelligence vol 11 no 6 pp 567ndash5851992
[22] B Scholkopf F Steinke and V Blanz ldquoObject correspondenceas a machine learning problemrdquo in Proceedings of the 22ndInternational Conference on Machine Learning (ICML rsquo05) pp776ndash783 2005
[23] L Zagorchev and A Goshtasby ldquoA comparative study oftransformation functions for nonrigid image registrationrdquo IEEETransactions on Image Processing vol 15 no 3 pp 529ndash5382006
[24] J Zhao JMa J Tian J Ma andD Zhang ldquoA robustmethod forvector field learning with application to mismatch removingrdquoin Proceedings of the IEEE Conference on Computer Vision andPattern Recognition (CVPR rsquo11) Providence RI USA June 2011
[25] JMa J Zhao J Tian X Bai and Z Tu ldquoRegularized vector fieldlearning with sparse approximation for mismatch removalrdquoPattern Recognition vol 46 no 12 pp 3519ndash3532 2013
[26] J Ma J Zhao J Tian Z Tu and A L Yuille ldquoRobust estimationof nonrigid transformation for point set registrationrdquo in Pro-ceedings of the 26th IEEE Conference on Computer Vision andPattern Recognition (CVPR rsquo13) pp 2147ndash2154 June 2013
[27] J MaW Qiu J Zhao Y Ma A L Yuille and Z Tu ldquoRobust l2e
estimation of transformation for non-rigid registrationrdquo IEEETransactions on Signal Processing vol 63 no 5 pp 1115ndash11292015
[28] J Ma J Zhao J Tian A L Yuille and Z Tu ldquoRobust pointmatching via vector field consensusrdquo IEEE Transactions onImage Processing vol 23 no 4 pp 1706ndash1721 2014
[29] L Goshen and I Shimshoni ldquoBalanced exploration andexploitation model search for efficient epipolar geometry esti-mationrdquo IEEE Transactions on Pattern Analysis and MachineIntelligence vol 30 no 7 pp 1230ndash1242 2008
[30] O Chum and J Matas ldquoMatching with prosacmdashprogressivesample consensusrdquo in Proceedings of the IEEE Computer SocietyConference on Computer Vision and Pattern Recognition (CVPRrsquo05) vol 1 pp 220ndash226 IEEE June 2005
[31] K Mikolajczyk and C Schmid ldquoA performance evaluation oflocal descriptorsrdquo IEEE Transactions on Pattern Analysis andMachine Intelligence vol 27 no 10 pp 1615ndash1630 2005
[32] D G Lowe ldquoDistinctive image features from scale-invariantkeypointsrdquo International Journal of Computer Vision vol 60 no2 pp 91ndash110 2004
[33] S Gauglitz T Hollerer and M Turk ldquoEvaluation of interestpoint detectors and feature descriptors for visual trackingrdquoInternational Journal of Computer Vision vol 94 no 3 pp 335ndash360 2011
[34] Y Yu K Huang W Chen and T Tan ldquoA novel algorithmfor view and illumination invariant image matchingrdquo IEEETransactions on Image Processing vol 21 no 1 pp 229ndash2402012
[35] K Mikolajczyk T Tuytelaars C Schmid et al ldquoA comparisonof affine region detectorsrdquo International Journal of ComputerVision vol 65 no 1-2 pp 43ndash72 2005
[36] H Hirschmuller and D Scharstein ldquoEvaluation of cost func-tions for stereo matchingrdquo in Proceedings of the IEEE ComputerSociety Conference on Computer Vision and Pattern Recognition(CVPR rsquo07) June 2007
[37] D Scharstein and C Pal ldquoLearning conditional random fieldsfor stereordquo in Proceedings of the IEEE Computer Society Confer-ence on Computer Vision and Pattern Recognition (CVPR rsquo07)IEEE Minneapolis Minn USA June 2007
[38] D Scharstein andR Szeliski ldquoHigh-accuracy stereo depthmapsusing structured lightrdquo in Proceedings of the IEEE ComputerSociety Conference on Computer Vision and Pattern Recognition(CVPR rsquo03) vol 1 pp I-195ndashI-202 IEEE June 2003
[39] H Wang S Yan J Liu X Tang and T S Huang ldquoCorre-spondence propagationwithweak priorsrdquo IEEETransactions onImage Processing vol 18 no 1 pp 140ndash150 2009