-
Hindawi Publishing CorporationInternational Journal of
Biomedical ImagingVolume 2011, Article ID 592924, 20
pagesdoi:10.1155/2011/592924
Research Article
Contour Detection and Completion forInpainting and Segmentation
Based onTopological Gradient and Fast Marching Algorithms
Didier Auroux,1 Laurent D. Cohen,2 and Mohamed Masmoudi3
1 Laboratoire J. A. Dieudonné, Université de Nice Sophia
Antipolis, Parc Valrose, 06108 Nice Cedex 2, France2 CEREMADE, UMR
CNRS 7534, Université Paris Dauphine, Place du Marchal De Lattre
De Tassigny, 75775 Paris Cedex 16, France3 Institut de
Mathématiques de Toulouse, Université Paul Sabatier, 31062
Toulouse Cedex 9, France
Correspondence should be addressed to Didier Auroux,
[email protected]
Received 30 May 2011; Revised 12 September 2011; Accepted 12
September 2011
Academic Editor: Shan Zhao
Copyright © 2011 Didier Auroux et al. This is an open access
article distributed under the Creative Commons Attribution
License,which permits unrestricted use, distribution, and
reproduction in any medium, provided the original work is properly
cited.
We combine in this paper the topological gradient, which is a
powerful method for edge detection in image processing, and
avariant of the minimal path method in order to find connected
contours. The topological gradient provides a more global
analysisof the image than the standard gradient and identifies the
main edges of an image. Several image processing problems
(e.g.,inpainting and segmentation) require continuous contours. For
this purpose, we consider the fast marching algorithm in order
tofind minimal paths in the topological gradient image. This
coupled algorithm quickly provides accurate and connected
contours.We present then two numerical applications, to image
inpainting and segmentation, of this hybrid algorithm.
1. Introduction
Contour detection is a major issue in image processing.
Forinstance, in classification and segmentation, the goal is
tosplit the image into several parts. This problem is strongly
re-lated to the detection of the connected contours separatingthese
parts. It is quite easy to detect edges using local imageanalysis
techniques, but the detection of continuous contoursis more
complicated and needs a global analysis of the image.
Several image processing problems like image inpaint-ing and
denoising (or enhancement) are classically solvedwithout detecting
edges and contours. The goal of imageenhancement is to denoise the
image without blurring it. Aclassical idea is to identify the edges
in order to preserve themand to smooth the image outside them. In
this particularcase, contour completion is not prerequisite, as the
qualityof the result is not too much related to the completeness
ofthe identified edges, but missing edges may lead to
blurredboundaries. For most of the other image processing
problems(segmentation, inpainting, classification), the detection
ofconnected contours can drastically simplify the resolutionand
improve the quality of the results. For instance, the
imagesegmentation problem is a very good example, as the goal
is
to split the image into its characteristic parts. In other
words,one has to find connected contours, which define
differentsubsets of the image.
For solving all these problems, various approaches havebeen
considered in the literature. We can cite here the mostcommonly
used models: the structural approach by regiongrowing [1], the
stochastic approaches [2–4], and the vari-ational approaches, which
are based on various strategies likelevel set formulations,
minimizing the total variation of aquantity or the Mumford-Shah
functional, active contoursand geodesic active contours methods,
snakes, wavelet trans-forms, or shape gradient [5–19, 19–24].
Another approach is based on the topological asymptoticanalysis
and consists of defining edges as cracks [25, 26].The goal of
topological optimization is to look for an opti-mal design (i.e., a
subset) and its complementary. Findingthe optimal subdomain is
equivalent to identifying its char-acteristic function. At first
sight, this problem is not differ-entiable. But the topological
asymptotic expansion gives thevariation of a cost function j(Ω)
(see Section 2 for examples)when one switches the characteristic
function from one tozero (or from zero to one) in a small region
[27].
-
2 International Journal of Biomedical Imaging
More precisely, we consider the perturbation of themain domain Ω
by the insertion of a small crack (or hole)σρ : Ωρ = Ω \ σρ, ρ
being the size of the crack. Thetopological sensitivity theory
provides then an asymptoticexpansion of the considered cost
function when the size ofthe crack tends to zero. It takes the
general form: j(Ωρ) −j(Ω) = f (ρ)g(x) + o( f (ρ)), where f (ρ) is
an explicit positivefunction going to zero with ρ, and g(x) is the
topologicalgradient at point x. Then, in order to minimize the
criterion(or at least its first order expansion), one has to
insertsmall cracks at points where the topological gradient is
themost negative. Using this gradient type information, it
ispossible to build fast algorithms. In most applications,
asatisfying approximation of the optimal solution is reached atthe
first iteration of the optimization process. A
topologicalsensitivity framework allowing to obtain such an
expansionfor general cost functions has been proposed in [27].
An efficient edge detection technique, based on the topo-logical
gradient, has been introduced in [28, 29]. It is alsoshown that
edge detection can make all these image proc-essing problems
straightforward to solve [25, 26, 30, 31].But the identified edges
are usually not connected, and theresults can be degraded. Our goal
is to improve these resultsby replacing dashed discontinuous edges
by connectedcontours.
In the inpainting problem, we assume that there is ahidden part
of the image, and our goal is to recover thispart from the known
part of the image. We assume thatthe missing part is a quite large
part of the image, we donot consider the case of random sets or
narrow lines. Thisproblem has been widely studied and the most
commonapproaches are: learning approches (neural networks,
radialbasis functions, . . .) [32, 33], minimization of an energy
costfunction based on a total variation norm [34, 35],
morpho-logical component analysis methods separating texture
andcartoon [36]. We also refer to [6, 8] for the description
ofseveral inpainting algorithms.
We now consider the crack detection technique, withinthe
framework of the identification of the image edges, eitherin the
hidden part of the image for the inpainting application,or in the
whole image for the segmentation application [26].The topological
asymptotic analysis provides very quickly thelocation of the edges,
as they are precisely defined by themost negative points of the
topological gradient. The greatadvantage of the topological
gradient in comparison withlevel line completion and TV-based
inpainting methods (seee.g., [6, 8, 12, 13]) is that the identified
edges in the unknownpart of the image correspond to a regular
extrapolation ofthe known edges, and as we will see on a numerical
example,the topological gradient preserves the continuity of the
edgecurvature. Thus, the proposed approach is much more thansimple
edge detection.
The main issue of the approach based on the topologicalgradient
is the need for connected complete contours. Thiscan be easily
understood since the hidden part of the imageis filled in using the
Laplace operator in each subdomainof the missing zone, and a
discontinuous contour wouldlead to some blurred reconstruction. Up
to now, one had tothreshold the topological gradient with a not too
small value,
in order to identify connected contours, but this leads tothick
identified edges, and also to consider more noisy pointsas
potential edges. In order to overcome this limitation, weconsider a
minimal path technique in order to connect theedges identified by
the topological gradient.
Minimal paths have been first introduced for findingthe global
minimum of active contour models, using thefast marching technique
[37, 38]. They have then beenused to find contours or tubular
structures and also forperceptual grouping using a path or a set of
paths minimizinga functional [38–43]. In our case, the energy to be
minimizedwill be an increasing function of the topological
gradient. Asthe topological gradient takes its minimal (negative)
valueson the edges of the image, the idea is indeed to find
contoursfor contour completion from the various minima and
smallvalues of the topological gradient.
The energy to be minimized can be seen as a distancefunction.
The idea is then to compute this distance functionbetween a given
starting point and all other points. For thispurpose, a front
propagation equation is considered. Usingthe fast marching
propagation, the definition of the distancefunction is
straightforward: the distance between a point xand the starting
point is exactly the time at which the frontreached x. Then,
minimal paths between these points can beidentified using a
gradient descent. For perceptual grouping,a set of keypoints is
considered as starting points and a setof minimal paths connecting
some pairs of these keypointsis considered as a contour completion.
This approach isextremely satisfactory in 2D problems, with quite
few keypoints. It is also extremely fast. In 3D images, minimal
pathsfind tubular structures, but in order to identify
minimalsurfaces, this approach is much more difficult to
consider.It was dealt in the case of a surface connecting two
curves in[44]. We only consider here the 2D case.
The application of the minimal path technique to thetopological
gradient allows us to obtain an automaticidentification of the main
(missing or not) edges of theimage. These edges will be continuous,
by construction, andwill allow us to simply apply the Laplace
operator to fill in theimage for inpainting applications, or will
directly provide thesegmented image, with very good results.
Another advantageof this technique is to be very fast, as it does
not degradethe O(n · log(n)) complexity of the topological
gradientbased algorithm introduced in [26]. We refer to [26, 45]
forthe inpainting and segmentation algorithms by
topologicalasymptotic expansion, and for a detailed presentation of
thetopological gradient.
The paper is organized as follows. In Section 2, wepresent the
edge detection method using the topological gra-dient, and the
corresponding segmentation and inpaintingalgorithms. In Section 3,
we propose an algorithm based onthe minimal path and fast marching
techniques in order toidentify the valley lines of the topological
gradient, whichcorrespond to the main edges of the image. Then, we
reportthe results of several numerical experiments in Section 4.
Wealso compare this hybrid scheme with the fast marching algo-rithm
applied to the standard gradient. Two particular imageprocessing
problems are considered: segmentation andinpainting. Finally, some
conclusions are given in Section 5.
-
International Journal of Biomedical Imaging 3
σ(n)
n
Ωρ
x0σρ
σ Γ
n
Figure 1: Example of domain and inserted crack (with its
orienta-tion).
2. Edge Detection by TopologicalAsymptotic Analysis and Its
Application toInpainting and Segmentation
2.1. Topological Asymptotic Analysis. Let Ω be an openbounded
domain ofR2 (note that it can easily be extended toRn). We consider
a partial differential equation (PDE) prob-lem defined in Ω, and we
denote by uΩ its solution (we willfurther see under which
assumptions it can be considered).We finally consider a cost
function J(Ω,uΩ) to be minimized,where uΩ is the solution to the
PDE in Ω. The idea oftopological asymptotic analysis is to measure
the impact ofa perturbation of the domain Ω on the cost
function.
For a small ρ ≥ 0, let Ωρ = Ω \ σρ be the perturbeddomain by the
insertion of a crack σρ = x0 + ρσ(n), wherex0 ∈ Ω. We denote by σ a
fixed bounded straight crackcontaining the origin, n is a unit
vector, and σ(n) is the resultof the rotation of σ so that n is the
normal to σ(n). Thefixed crack σ is rotated (normal n), stretched
(size ρ), andtranslated (center x0) in order to get σρ (see Figure
1). Thetopological gradient theory can also be applied in the case
ofarbitrary shaped holes [46–49], but we will only consider thecase
of crack perturbations in our applications. The small pa-rameter ρ
will represent the size of the inserted crack. Finally,we denote by
V a Hilbert space on Ω, usually H1(Ω) in ourapplications.
We now consider the variational formulation of the PDEproblem on
Ω
Find u ∈ V such thata(u,w) = l(w), ∀w ∈ V,
(1)
and the corresponding variational formulation of the PDEproblem
on the perturbed domain
Find uρ ∈ Vρ such that
aρ(uρ,w
)= lρ(w), ∀w ∈ Vρ.
(2)
One should notice that for ρ = 0, the perturbed PDEproblem
becomes the original PDE problem.
We assume in the following that aρ is a bilinear continu-ous and
coercive form defined on Vρ, a Hilbert space on Ωρ,and that lρ is a
linear continuous form on Vρ.
We can rewrite the cost function J as a function of ρ
byconsidering the following map:
j : ρ �−→ Ωρ �−→ uρ, solution of Equation (2) �−→ j(ρ)
:= J(Ωρ,uρ
).
(3)
In order to apply the topological asymptotic theory, aρ,lρ, and
J have to satisfy the hypotheses of the following result[50,
51].
If there exist a linear form Lρ defined on Vρ, a functionf : R+
→ R+, and four real numbers δJ1, δJ2, δa, and δlsuch that
(1) limρ→ 0 f (ρ) = 0,(2) J(Ωρ,uρ)−J(Ωρ,u0) = Lρ(uρ−u0)+ f
(ρ)δJ1+o( f (ρ)),(3) J(Ωρ,u0)− J(Ω,u0) = f (ρ)δJ2 + o( f (ρ)),(4)
(aρ − a0)(u0, pρ) = f (ρ)δa + o( f (ρ)),(5) (lρ − l0)(pρ) = f (ρ)δl
+ o( f (ρ)),
where the adjoint state pρ is solution of the adjoint
equation
aρ(w, pρ
)= −Lρ(w) ∀w ∈ Vρ, (4)
and uρ is solution of the direct (2), then the cost function
hasthe following asymptotic expansion:
j(ρ)− j(0) = f (ρ)g(x) + o( f (ρ)), (5)
where g(x) is the topological gradient, given by
g(x) = δJ1 + δJ2 + δa− δl. (6)Indeed, from second and third
items, j(ρ) − j(0) =
J(Ωρ,uρ)−J(Ω,u0) = Lρ(uρ−u0)+ f (ρ)(δJ1 +δJ2)+o( f (ρ)).From the
definition of the adjoint state and the directequation, Lρ(uρ − u0)
= −aρ(uρ, pρ) + aρ(u0, pρ). Fromfourth item and direction (2),
−aρ(uρ, pρ) + aρ(u0, pρ) =−lρ(pρ) + a0(u0, pρ) + f (ρ)δa+ o( f (ρ))
= −lρ(pρ) + l0(pρ) +f (ρ)δa + o( f (ρ)). Finally, from fifth item,
this term is equalto f (ρ)(δa− δl) + o( f (ρ)).
Then, from an asymptotic point of view, as f (ρ) ≥ 0,the idea is
to create cracks in the domain Ω, where thetopological gradient g
is the most negative, because
J(Ωρ,uρ
)= J(Ω,u) + f (ρ)g(x) + o( f (ρ)), (7)
and the cost function corresponding to the perturbedproblem will
be smaller than the original one. The mainadvantage of this method
is that it only requires theresolution of the direct (2) and
adjoint (4) problems.
2.2. Application to Edge Detection. LetΩ be an open
boundeddomain of R2, representing the image domain. For agiven
function v in L2(Ω) (in our application, v representsthe input
image), the initial problem is defined on theunperturbed domain and
reads as follows: find u ∈ H1(Ω)such that
−div(c∇u) + u = v in Ω,∂nu = 0 on ∂Ω,
(8)
-
4 International Journal of Biomedical Imaging
where n denotes the outward unit normal to ∂Ω and c is agiven
function. Note that this problem is equivalent to lineardiffusion
restoration.
For a given x0 ∈ Ω and a small ρ ≥ 0, let us now considerΩρ =
Ω\σρ the perturbed domain by the insertion of a crackσρ = x0 +
ρσ(n), where x0 ∈ Ω, σ(n) is a straight crack, andn a unit vector
normal to the crack. Then, the new solutionuρ ∈ H1(Ωρ)
satisfies
−div(c∇uρ
)+ uρ = v in Ωρ,
∂nuρ = 0 on ∂Ωρ.(9)
Edge detection is equivalent to looking for a subdomainof Ω in
which the energy is small. Indeed, we consider theimage gradient
energy function, and the edges correspond tohigh variations of the
image intensity, and then to high valuesof the gradient. So, our
goal is to find the most energeticparts of the image (in order to
identify the edges), and wereformulate this problem as the
minimization of the energynorm outside the edges
j(ρ) = J
(Ωρ,uρ
)=∫
Ωρ
∥∥∥∇uρ∥∥∥2. (10)
Then, the cost function j has the following asymptoticexpansion
(see, e.g., [52] for more details):
j(ρ)− j(0) = ρ2G(x0,n) + o
(ρ2), (11)
with
G(x0,n) = −πc(∇u0(x0) · n)(∇p0(x0) · n
)
− π|∇u0(x0) · n|2,(12)
and where p0 is the solution to the adjoint problem
−div(c∇p0)
+ p0 = −∂uJ(Ω,u0) in Ω,∂np0 = 0 on ∂Ω.
(13)
The topological gradient could be written as
G(x,n) = (M(x)n) · n, (14)
where M(x) is the 2 × 2 symmetric matrix defined by
M(x) = −πc∇u0(x)∇p0(x)T +∇p0(x)∇u0(x)T
2
− π∇u0(x)∇u0(x)T .(15)
For a given x, G(x,n) takes its minimal value when n isthe
eigenvector associated to the lowest eigenvalue λmin ofM. This
value will be considered as the topological gradientassociated to
the optimal orientation of the crack σρ(n).
Then, we can define the identified edge set
σ = {x ∈ Ω; λmin(x) < δ < 0}, (16)
where δ is a negative threshold.
We first illustrate this technique on a synthetic twodimensional
image, in grey level, defined by a sigmoidfunction in x-coordinate
(cumulative distribution functionof a Gaussian). The image is
represented in Figure 2(a).Then, the L2 norm of its standard
gradient ‖∇u(x)‖ and itstopological gradient λmin(M(x)) are
represented in Figures2(b) and 2(c), respectively.
One can see that the topological gradient is less sensitiveto a
smooth variation of the image intensity than thestandard gradient.
The support of the topological gradient isindeed much smaller.
Thanks to the homogeneous Neumanncondition on the crack, the
solution of the perturbedproblem is discontinuous along the crack,
and the solutionhas a much smaller energy if one inserts a crack in
the imagenear the middle of the x-axis.
We now apply this edge detection technique to the
imagerepresented in Figure 3(a). The opposite of the L2 norm ofits
standard gradient is represented in Figure 3(b). Note thatwe
represent its opposite in order to have comparable imageswith the
topological gradient, which has negative values.
The topological gradient is represented on Figure 3(c). Asit
quantifies in a global way whether a pixel is part of an edgeor
not, it is much less sensitive to noise and small variationsof the
image than the standard gradient. For instance, thetopological
gradient takes much larger absolute values onthe edges than
outside, contrary to the standard gradient.Note also that the time
required for the computation ofthe topological gradient is not much
higher than for thestandard gradient, thanks to the O(n · logn)
complexity ofthe topological gradient algorithm.
However, for segmentation (or simply edge detection),the next
step of topological gradient algorithms usuallyconsists of
thresholding the topological gradient in orderto define the edge
set. Such a threshold is represented inFigure 3(d). One can see
that in order to obtain at least themain connected edge, the
threshold coefficient has been setto a large value, leading to add
many unwanted points to theedge set, but also to thick edges. And
even in this case, themain contour is not totally continuous. This
is why we needto hybridize this method with the fast marching
algorithm(see Section 3.4) in order to obtain continuous edges for
thesegmentation and to remove the isolated unwanted pixels.
We will also see below that the fast marching algorithmneeds a
potential function highly related to the edges ofthe image, much
more than the standard gradient of theimage. Then, we will see that
the topological gradient alsoimproves the fast marching method
within the segmentationframework, as the quality of the
segmentation is directlyrelated to the choice of the potential
function.
2.3. Inpainting Algorithm by Topological Asymptotic Analysis.We
also consider the inpainting application. We present herethe
topological gradient-based algorithm. Let ω ⊂ Ω be themissing part
of the image and γ its boundary. We still de-note by v the input
image (assumed to be known in Ω \ ω,and unknown in ω). The
algorithm is based on the factthat two measurements are available
on the boundary of thehidden part of the image: the value of the
image (Dirichletcondition) and its normal derivative (Neumann
condition).
-
International Journal of Biomedical Imaging 5
(a) (b)
(c)
Figure 2: (a) Original image; (b) L2 norm of the (standard)
gradient of (a); (c) Topological gradient of (a).
From these two measurements, by considering the standardcrack
localization problem (see, e.g., [50]), it is possible tosolve a
Dirichlet problem and a Neumann problem for agiven crack σ
ΔuD = 0 in ω \ σ ,uD = v on γ,∂nuD = 0 on σ ,uD = v in Ω \
ω,
(17)
where uD ∈ H1(Ω \ σ), andΔuN = 0 in ω \ σ ,∂nuN = ∂nv on γ,∂nuN
= 0 on σ ,uN = v in Ω \ ω,
(18)
where uN is in H1(Ω \ σ).
Then, in order to identify the missing edges, one has tominimize
the following cost function:
J(σ) = 12‖uD − uN‖2L2(Ω). (19)
For the actual cracks (hidden edges), the solutions uD anduN
should be equal, as the actual solution satisfies bothNeumann and
Dirichlet conditions. By minimizing this costfunction, one tries to
find a solution that is consistent withboth conditions on the
boundary.
The topological gradient corresponding to this costfunction is
given by
G(x,n) = − [(∇uD(x) · n)(∇pD(x) · n
)
+(∇uN (x) · n)(∇pN (x) · n
)],
(20)
where pN and pD are the two corresponding adjoint states[25,
50]. As previously, the topological gradient can berewritten as
G(x,n) = nTM(x)n, where M(x) is a symmetricmatrix, and G takes its
minimal value when n is theeigenvector associated to the lowest
eigenvalue of M.
-
6 International Journal of Biomedical Imaging
50
100
150
200
50 100 150 200
(a)
50
100
150
200
50 100 150 200
−35
−30
−25
−20
−15
−10
−5
(b)
50
100
150
200
50 100 150 200
−180−160−140−120−100−80−60−40−20
(c)
50
100
150
200
50 100 150 200
(d)
Figure 3: (a) Original image; (b) L2 norm of the (standard)
gradient of (a); (c) Topological gradient of (a); (d) Identified
edges bythresholding the topological gradient.
The inpainting algorithm is then the following:
(i) calculation of uD and uN ,
(ii) calculation of pD and pN ,
(iii) computation of matrix M(x) and its lowest eigen-value λmin
at each point of the missing domain ω,
(iv) definition of the set of cracks: {x ∈ ω; λmin(x) < δ
<0}, where δ is a negative threshold,
(v) dalculation of u solution to the Neumann problemtaking into
account the cracks location.
This algorithm has a complexity of O(n · log(n)), wheren is the
size of the image (i.e., number of pixels). We refer to[25] for
more details about this algorithm.
We now illustrate this algorithm on two syntheticexamples. We
first want to restore a black square, partiallyhidden by a red
square. The degraded image is representedin Figure 4(a).
If no edge is inserted in the hidden zone, then theresolution of
a Poisson problem gives a blurred image, as theLaplace operator
provides a smooth reconstruction between
the black square and the white background, as shown inFigure
4(c). The restored image by the inpainting algorithmis represented
in Figure 4(e). Using the edges identified bythe topological
gradient, the reconstruction by the Laplacianis much better, as
there is now an insulating crack betweenthe black and white
zones.
The second synthetic example is the reconstruction of ablack
circle, partially hidden by a red square. The degradedimage is
represented in Figure 4(b), the restored imageby the Laplacian
without any inserted edge is shown inFigure 4(d), and the restored
image the Laplacian using theedges identified by the topological
gradient is represented inFigure 4(f). As one can see on these two
synthetic examples,the curvature of the reconstructed edges is
continuous inthe neighborhood of the boundary of the occlusion. It
isnot common that an inpainted image has C1 edges, and forinstance,
TV-based methods would connect the boundarypoints with a straight
line.
We now explain why we also decided to hybridize thetopological
gradient and minimal paths methods on a morerealistic case.
-
International Journal of Biomedical Imaging 7
5
10
15
20
25
30
35
40
45
505 10 15 20 25 30 35 40 45 50
(a)
5
10
15
20
25
30
35
40
45
505 10 15 20 25 30 35 40 45 50
(b)
5
10
15
20
25
30
35
40
45
505 10 15 20 25 30 35 40 45 50
(c)
5
10
15
20
25
30
35
40
45
505 10 15 20 25 30 35 40 45 50
(d)
5
10
15
20
25
30
35
40
45
505 10 15 20 25 30 35 40 45 50
(e)
5
10
15
20
25
30
35
40
45
505 10 15 20 25 30 35 40 45 50
(f)
Figure 4: (a) Occluded image, defined by a black square on a
white background, the occlusion being represented by a red square;
(b)Occluded image, defined by a black circle on a white background,
the occlusion being represented by a red square; (c) Inpainted
image bydiffusion (see (a) for the original degraded image),
without any inserted edge in the occlusion; (d) Same as (c) in the
circle case; (e) Inpaintedimage using the missing edges identified
by the topological gradient, and then diffusion to fill in the
image (see (a) for the original degradedimage); (f) Same as (e) in
the circle case.
-
8 International Journal of Biomedical Imaging
Figures 5(a) and 5(b) show an example of image, inwhich we added
a mask on a quite large part of the image(800 pixels). The goal of
inpainting is to reconstruct asprecisely as possible the original
image from the occludedimage. We also want the inpainted image to
have sharp(unblurred) edges.
Figure 5(c) shows the corresponding topological gradi-ent,
provided by the inpainting algorithm. In this case, thetopological
gradient gives some information about the mostprobable location of
the missing edges. In the inpaintingalgorithm presented in [25],
the idea is then to thresholdthe topological gradient and to define
the edge set of theoccluded zone as being the set of points below
the threshold.The main issue is that the identified missing edges
mustbe connected in order to avoid blurry effects (due to
theLaplacian) in the reconstruction. Then, the threshold
issometimes set manually in order to have connected contours.In our
example, the identified edge set is represented by whitepoints in
Figure 5(d).
Figures 5(e) and 5(f) show the corresponding inpaintedimage. One
can see that the reconstruction is not very good,particularly in
the top part. This is mainly due to the fact thatthe missing edges
identified by the topological gradient areeither connected but
thick with a lot of wrong identifications(if the threshold is too
small) or discontinuous (otherwise).
The idea is then to apply the fast marching algorithmon the
topological gradient obtained during the inpaintingprocess in order
to identify connected contours in the hiddenpart of the image.
3. A 2D Algorithm Based on theMinimal Paths and Fast Marching
Methods
3.1. Minimal Paths. In this section, we describe the
standardminimal path technique, adapted to our needs. We referto
[37, 38, 41] for more details about the minimal pathsmethod.
In the following, let Ω be the considered image domain.We assume
that Ω is a regular subset of R2. In order tocompute some minimal
paths, we need to define a potentialfunction, measuring in some
sense for any point of Ω the costfor a path to contain this point.
As we want to identify pathsin the topological gradient image, and
considering that thispotential function must be positive, we will
define a potentialfunction as follows:
P(x) = g(x)−miny∈Ω
{g(y)}
, ∀x ∈ Ω, (21)
where g is the topological gradient, defined in all the domainΩ.
We simply shift the topological gradient from its minimalvalue, in
order to obtain a positive function P. We can seethat the points
where the topological gradient g reachesits minimal values are
quite costless. This is a way to saythat these points must be on
the minimal paths. On thecontrary, if the topological gradient
takes high values, thenthe corresponding potential values lead to
very expensivepaths.
Once each point has a cost (defined by the potentialfunction),
we need to define the corresponding cost of a path.
We denote by C(s) a path, or curve, drawn in the imagedomain,
where s represents the arc length. We can now definea functional,
measuring the cost of such a path
J(C) =∫
C(P(C(s)) + α)ds, (22)
where α is a positive real coefficient that represents
regu-larization. The first part of the cost function measures
thecost itself of the path C(s) simply by summing the value ofthe
potential function on this path, and the second part isa
regularization term that measures the length of this path.In our
applications, α is usually very small, as the goal is toconnect the
most negative parts of the topological gradient,whatever the
Euclidean distance is. Note also that we do notconsider any
regularization terms on the curvature of thecontour, as the
topological gradient already provides suchregularity on the
curvature, contrary to TV-based methods.Typically, α = 0 would be a
good choice, as we really wantthe minimal path along the
topological gradient values, butas the minimum of P is 0 (at the
minimum of the topologicalgradient), one has to set α to a very
small value in order toavoid numerical instability (see (24)).
We now consider a key point x0 ∈ Ω of the image, andx will
represent any point of the image. The energy J(C)of a given path C
can be seen as a distance between thetwo endings of C, weighted by
the potential function (andthe regularization). The goal is to find
the minimal energyintegrated along the path C. We can now define
the weighteddistance between key point x0 and point x by
D(x; x0) = infC∈A(x,x0)
J(C) = infC∈A(x,x0)
∫
C(P(C(s)) + α)ds, (23)
where A(x, x0) is the set of all paths going from point x0
topoint x in the image. The idea is that finding the minimalpath
between points x and x0 is now equivalent to computingthe weighted
distance function between these two points.If x and x0 are on the
same contour of the image, thenthe minimal path between these two
points is obviously acontinuous contour of the image, connecting
these points.The minimal path has indeed the lowest cost, that is,
thepoints on this path have low topological gradient values.
Thegoal is now to compute the distance function given by (23).
3.2. Fast Marching. An efficient way to compute this
distancefunction is to solve a front propagation equation:
∂F (s, t)∂t
= 1P(F (s, t)) + α
nF (s, t), (24)
where nF (s, t) is the outer normal unit vector to the frontF .
We initialize the propagation with F (s, 0), an infinitelysmall
circle centered at key point x0. This front evolvesthen with a
propagation speed inversely proportional to thepotential function.
If for example a point in the outer partof the front has a large
potential (i.e., a large cost), then thepropagation speed will be
nearly equal to zero, and the frontwill not expand much at this
point. On the other hand, ifthe potential is small (i.e., this
point is nearly costless), then
-
International Journal of Biomedical Imaging 9
50 100 150 200 250 300 350
20
40
60
80
100
120
140
160
180
200
(a)
10 20 30 40 50 60
5
10
15
20
25
30
35
40
45
50
(b)
10 20 30 40 50 60
5
10
15
20
25
30
35
40
45
50−220−200−180−160−140−120−100−80−60−40−20
(c)
10 20 30 40 50 60
5
10
15
20
25
30
35
40
45
50
(d)
50 100 150 200 250 300 350
20
40
60
80
100
120
140
160
180
200
(e)
5 10 15 20 25 30 35 40 45 50
5
10
15
20
25
30
35
40
(f)
Figure 5: (a) Occluded image (by a white rectangle); (b) Zoom of
the occluded zone (see (a)); (c) Topological gradient of (b); (d)
Identifiededges in the occluded zone by thresholding the
topological gradient; (e) Inpainted image using the topological
gradient; (f) Zoom of theoccluded zone (see (e)).
the propagation speed is large, and the front will
quicklypropagate in this direction.
The distance D(x; x0) introduced in (23), between keypoint x0
and point x, is then simply the instant t at whichthe front,
initialized at key point x0, reaches point x. The
algorithm to compute the distance function is called thefast
marching technique and is justified by the fact that thedistance
satisfies the following Eikonal equation:
‖∇xD(x; x0)‖ = P(x) + α, (25)
-
10 International Journal of Biomedical Imaging
with the initialization D(x0) = 0. We refer to [37, 38, 44,53,
54] for more details about the fast marching techniqueand the
justification of (25). If n is the size of the image,the complexity
of this fast marching method is bounded byO(n · log(n)), which is
also the complexity of the topologicalgradient algorithm.
3.3. Multiple Minimal Paths. The main issue is now to extendthis
minimal path technique to more than one keypoint inorder to connect
several points. This is exactly what we needin order to connect the
identified edges by the topologicalgradient, as we have many
identified keypoints (e.g., allnegative local minima of the
topological gradient) that wewant to connect. As explained in [41],
the first point ofa multiple minimal path algorithm is to reduce
the set ofkeypoints for computational reasons. Moreover, the
selectedkeypoints should not be too close to each other. One
usuallychooses a total number N of keypoints and the first (or
main)one. Then, the N − 1 other keypoints can be chosen forexample
as described in [41].
The next step consists of connecting these N points.One has to
compute the distance function from each ofthese key points, and the
common minimal paths algorithmsprovide then the Voronoı̈ diagram of
the distance andthe corresponding saddle points (minimal distance
alongthe edges of the diagram and maximal distance from
thekeypoints). The Voronoı̈ diagram defines a partition of theimage
in as many subsets as the number of keypoints. Eachsubset is
defined by the set of points that are closer to thecorresponding
keypoint than to all others. The saddle pointsminimize the distance
function on the edges of the diagram:minimal distance on the edge
and maximal distance to thekeypoints [38]. It is useful to compute
these saddle pointsto save computation time, since it reduces the
domain ofthe image where the fast marching computes or updates
theweighted distance map.
Finally, the idea is to consider the saddle points as
initialconditions for minimizing the distance function. For
eachsaddle point as an initial point, a minimization is
performedtowards each of the two corresponding keypoints
(recallthat the saddle points are located at the interface
betweentwo subsets of the Voronoı̈ diagram). Each
minimizationproduces a path between the saddle point (initial
condition)and a keypoint (local minimum of the distance
function).This step is usually called back propagation, as it
consists ofa gradient descent from the saddle point, back to the
linkedkeypoints. The back-propagation step is straightforward,
asthere is no local minimum of the distance function, exceptthe
keypoints. The union of all these paths gives a continuouspath,
connecting the keypoints together.
The interesting part of the approach introduced in [41]is that
each keypoint should not be connected to all theothers, but only to
at most two others, as we are looking for aset of closed connected
paths. Thus, the keypoints have tobe ordered in a way such that
they are only connected tothe other keypoints that are closest to
them in the energysense [41]. For this reason, we sort all the
saddle pointsfrom smaller to larger distance, and we first try to
connectthe pairs of keypoints corresponding to the saddle
points
of smallest distance. These keypoints are indeed more likelyto
be connected than distant keypoints, corresponding tosaddle points
of large potential. Once the close keypointsare connected, we
repeat the process with the new closestpairs of keypoints, provided
each point remains connectedto at most two other ones. At the end
of the process, allthe keypoints are connected to at most two other
keypoints,and the union of all minimal paths between the
keypointsrepresents one (or several) continuous contour of the
image.An interesting feature of this method is that the key
pointsare by construction widely distributed around.
If all the selected keypoints are on the same contour ofthe
image, we are almost sure that at the end, they will all
beconnected together, and we will retrieve the
correspondingcontour, as the potential function (related to the
topologicalgradient) is very low on this contour. If, on the
contrary,one keypoint is not part of the contour, the large values
ofthe topological gradient, and hence of the potential
function,will isolate this keypoint from the other ones, and it
will notdisturb the contour completion process.
3.4. Algorithm. The hybrid algorithm we propose is then
thefollowing.
Fast Marching Algorithm Applied to the Topological Gradient
(i) Compute the topological gradient of the image.
(ii) Set N the number of keypoints and choose the Nkeypoints:
the main one will be for example theglobal minimum of the
topological gradient, theother ones being the most negative local
minima ofthe topological gradient.
(iii) Compute the distance function (23) with all
thesekeypoints, and the corresponding Voronoı̈ diagram.
(iv) Compute the set of saddle points: on each edge ofthe
Voronoı̈ diagram, determine the point of minimaldistance.
(v) Sort all these points of minimal distance, fromsmaller to
larger distance.
(vi) For each of these saddle points, from smaller to
largerdistance, check if it will not be used to connect
twokeypoints, one of which is already connected to twoother
keypoints.
(vii) If this is not the case, perform the back propagationfrom
this point: use this saddle point as an initializa-tion for a
descent type algorithm in order to connectthe two corresponding
keypoints.
It is straightforward to see that this algorithm convergesand
that at convergence, all the keypoints are connected toat most two
other keypoints. This provides one or severalcontinuous contours
containing the keypoints. As the firstkeypoint is usually the
global minimum of the topologicalgradient, it is on one of the main
edges of the image.Consequently, using this algorithm, we can
identify this edge.Then, it is possible to restart the algorithm,
using otherkeypoints that are not on this identified edge, by
initializing,for instance, the first keypoint as the minimum of
thetopological gradient outside the neighborhood of this edge.
-
International Journal of Biomedical Imaging 11
Note that for inpainting applications, the number ofkeypoints
can be set automatically, as the topological gra-dient takes its
minimal values on the edges located on theboundary of the hidden
zone, and all these minima (close tothe global minimal value of the
topological gradient) can bechosen as keypoints.
4. Numerical Experiments
4.1. Numerical Results for 2D Segmentation. We consideragain the
grey level image represented in Figure 3(a) for thesegmentation
application, and we now present the resultscorresponding to the
hybrid method.
Using an automatic thresholding for identifying the mostnegative
values of the topological gradient, Figure 6(a) showsthe set of
points (or admissible keypoints, in blue), in whichwe will choose
the keypoints for the minimal path algorithm.The first keypoint is
set to the minimum of the topologicalgradient. Then, we have set
the number of keypoints toN = 3. From the first keypoint, we start
the minimal pathalgorithm, and we choose the second keypoint as
being thepoint (in the admissible set) maximizing the distance to
thefirst keypoint. Then, we start again the minimal path algo-rithm
from these two points, and we set the third keypointin a similar
way. These three keypoints are represented byblack points in Figure
6(a). Note that the keypoints can alsobe (manually) provided by the
user, for instance, with theaim of identifying a specific edge of
the image.
From these keypoints, we run the minimal path algo-rithm in
order to compute the distance map. Figure 6(b)shows this distance
function. One can clearly see that the dis-tance does not
correspond to the Euclidean metric in theplane, as the distance
remains very small on the commonedge of the 3 keypoints, whereas it
takes much larger valuesoutside.
The corresponding Voronoı̈ diagram is represented inFigure 6(c).
The three keypoints are still represented by blackpoints. Each
color represents the subset Ωi of points thatare closer to keypoint
i than to the others. For instance,all the points in the green zone
are closer to the rightkeypoint than to any of the two others. This
diagram isautomatically provided during the distance computation
bythe fast marching algorithm.
For any i /= j, we consider the interface Γi j = Ωi ∩ Ω jbetween
two subsets of the Voronoı̈ diagram. Γi j representsthen the set of
points equidistant from keypoints i and j. Asaddle point minimizes
the distance function on Γi j : samedistance to keypoints i and j,
minimal distance on Γi j . Thesesaddle points are represented by
blue points on Figure 6(c).These saddle points can be found during
the fast marchingpropagation as the first meeting points of the
fronts startingfrom each of the keypoints.
From these saddle points, the idea is finally to performa
descent-type algorithm in order to minimize the distancefunction
from the saddle points to the keypoints. Weconsider a saddle point
on an edge Γi j as an initial conditionfor two minimizations of the
distance function, one towardseach of the corresponding keypoints
(i and j). Each of thesetwo minimizations provides a continuous
path from the
saddle point to one of the two keypoints. The union of thesetwo
paths connects the two keypoints. This process is donefor all pairs
of keypoints.
The final set of paths is represented in green on thedistance
function in Figure 6(d). The three keypoints arealso represented
(in white). These paths correspond to thecontour of the original
image that contains the 3 keypoints.
The minimal path is also represented on the originalimage in
Figure 6(e). It also confirms that the identified pathperfectly
matches the edge we were looking at.
By applying again this algorithm, with other keypoints(selected
outside the first identified contour), it is possibleto detect
other contours of the image. Figure 6(f) shows,for instance, the
first main contour in green and a secondone in red. Contrary to the
first one, we can see that thiscontour is not perfectly detected,
as the algorithm missedsome parts of the contour in the bottom left
and top parts ofthe red zone. One should probably consider more
keypoints,and maybe a different regularization coefficient, in
orderto avoid this phenomenon. But for the application of
thetopological gradient to image segmentation, the main issuewas
the discontinuity of the identified contours (see, e.g.,[26]). With
this approach, we ensure the continuity of thecontours, and hence,
assuming the edges are well identified,we can obtain a perfectly
segmented image.
Finally, we illustrate the fact that the topological
gradientprovides better information about the edges of the
imagethan the standard gradient, as previously observed (see
Fig-ures 3(b) and 3(c)). We have manually selected 3 keypointson an
edge of the image. These keypoints are representedin blue on Figure
7(a). From these keypoints, we have runthe fast marching algorithm
(see Section 3.4) applied to boththe standard gradient and the
topological gradient (hybridscheme). The identified paths are
represented in Figures 7(b)and 7(c), respectively.
The topological gradient clearly provides the best
iden-tification of the edge. This can easily be explained by thebad
shape of the standard gradient in this region (seeFigure 3(b)). On
the contrary, the topological gradient is lesssensitive to small
local variations, and it is more likely todefine a potential
function than the standard gradient.
4.2. Numerical Results for 2D Inpainting. We now consideranother
application of this hybrid scheme to image inpaint-ing. We recall
that the idea of the topological gradientalgorithm is to identify
the missing edges in the occludedpart of the image, and then to
reconstruct the image fromthe solution of a Poisson problem with
Neumann boundaryconditions [25]. In this application also, it is
crucial to haveconnected contours; otherwise, the reconstruction
with theLaplacian will not be satisfactory.
We first present a comparison between the standard top-ological
gradient approach, a TV-based inpainting method,and the new hybrid
scheme. The original image is a blackrectangle, and we consider
various perturbations of thisimage. Figure 8(a) shows a first
perturbation of the image, inwhich the missing region is
represented by the red rectangle.The length of the hidden zone is
20 pixels. As previouslyshown, the missing zone is quite large, and
as the identified
-
12 International Journal of Biomedical Imaging
50
100
150
200
50 100 150 200
−180−160−140−120−100−80−60−40−20
Admissible pointsKeypoints automatically selected
(a)
50 100 150 200
50
100
150
200
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
(b)
50
100
150
200
50 100 150 200
Saddle pointsKeypoints
(c)
Minimal path
50 100 150 200
50
100
150
200
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
(d)
50
100
150
200
50 100 150 200
(e)
50
100
150
200
50 100 150 200
(f)
Figure 6: (a) Admissible set of points (i.e., most negative
values of the topological gradient) in blue, and 3 keypoints
automatically selectedin black; (b) Distance function computed from
these 3 keypoints with the fast marching algorithm; (c)
Corresponding Voronoı̈ diagram,with the 3 keypoints and saddle
points; (d) Identified minimal path between the keypoints
represented on the distance function; (e) Minimalpath between the
keypoints represented on the original image; (f) Another identified
continuous contour from other keypoints.
-
International Journal of Biomedical Imaging 13
50
100
150
200
50 100 150 200
Key-points
(a)
50
100
150
200
50 100 150 200
Minimal path
(b)
50
100
150
200
50 100 150 200
Minimal path
(c)
Figure 7: (a) Three selected keypoints on the original image;
(b) Contours identified by the fast marching algorithm applied to
the standardgradient with the three selected keypoints (see (a));
(c) Contours identified by the fast marching algorithm applied to
the topological gradientwith the three selected keypoints (see
(a)).
edges have to be connected in order to avoid blurry effectsin
the reconstruction, the threshold is set manually to aquite small
negative value. And then, the identified edgesare then quite thick
with a lot of wrong identifications. Thereconstructed image by the
topological gradient is shownin Figure 8(b). The reconstruction is
not very good, asmany wrong edges are considered in order to
connect thecontours. Figure 8(c) shows the identified minimal
pathbetween keypoints (that have been automatically selected,as
being the main edges on the boundary of the missingzone) in green,
represented on a zoom of the perturbedimage. Figure 8(d) shows the
corresponding inpainted imageby the hybrid scheme: the image is
reconstructed using thetopological gradient method, with the edges
identified bythe minimal path technique. In this case, the
reconstructionis perfectly done, and the inpainted image is
identical tothe original image. A TV-based inpainting method gives
the
same result (see Figure 8(e)), as the missing zone is not
toowide (20 pixels, which is also the size of the black
rectangle).
Figure 9 is similar to Figure 8 in the case of a
largerperturbation. The missing zone corresponds now to 40pixels,
twice the size of the black rectangle. In this case, thetopological
gradient is much less negative near the middleof the hidden zone,
and the threshold has to be increasedto a smaller negative value in
order to have closed contours.The corresponding inpainted image is
not good at all. Butthe minimal path technique still identifies
correct edges, andthe inpainted image by the hybrid scheme is
almost perfect,whereas a TV-based inpainting method does not
connectanymore the two regions of the rectangle.
Figure 10 is similar to Figures 8 and 9, in the case of alarger
perturbation. The missing zone now corresponds to80 pixels, which
is much larger than the size of the blackrectangle. In this case,
the topological gradient still gives
-
14 International Journal of Biomedical Imaging
20
40
60
80
100
120
140
160
180
20020 40 60 80 100 120 140 160 180 200
(a)
20
40
60
80
100
120
140
160
180
20020 40 60 80 100 120 140 160 180 200
(b)
5 10 15 20 25 30 35 40
10
20
30
40
50
60
(c)
20
40
60
80
100
120
140
160
180
20020 40 60 80 100 120 140 160 180 200
(d)
20
40
60
80
100
120
140
160
180
20020 40 60 80 100 120 140 160 180 200
(e)
Figure 8: (a) Occluded image (black rectangle) by a red
rectangle; (b) Inpainted image using the standard topological
gradient; (c) Minimalpath between the keypoints represented on the
topological gradient; (d) Inpainted image using the hybrid scheme
(fast marching algorithmfor closing the contours identified by the
topological gradient); (e) Inpainted image using a TV-based
method.
unsatisfactory results, due to badly connected edges. Even ifthe
topological gradient has strongly negative values alongthe missing
edges close to the boundary of the perturbation,the missing zone is
too wide, and the minimal path technique
now connects wrong keypoints, and the inpainted imageby the
hybrid scheme is no more connected. As before, theTV-based method
does not connect the two parts of therectangle.
-
International Journal of Biomedical Imaging 15
20 40 60 80 100 120 140 160 180 200
20
40
60
80
100
120
140
160
180
200
(a)
20 40 60 80 100 120 140 160 180 200
20
40
60
80
100
120
140
160
180
200
(b)
10
20
30
40
50
60
10 20 30 40 50 60
(c)
20 40 60 80 100 120 140 160 180 200
20
40
60
80
100
120
140
160
180
200
(d)
20 40 60 80 100 120 140 160 180 200
20
40
60
80
100
120
140
160
180
200
(e)
Figure 9: (a) Occluded image (black rectangle) by a red
rectangle; (b) Inpainted image using the standard topological
gradient; (c) Minimalpath between the keypoints represented on the
topological gradient; (d) Inpainted image using the hybrid scheme
(fast marching algorithmfor closing the contours identified by the
topological gradient); (e) Inpainted image using a TV-based
method.
We now consider again the occluded image given inFigure
5(a).
After thresholding the topological gradient, severalpoints
(identified by blue circles) have been identified
and define the admissible set of keypoints represented inFigure
11(a). We choose then the most negative point ofthe topological
gradient as the first keypoint and then thefurther admissible point
as the second one. The keypoints
-
16 International Journal of Biomedical Imaging
20 40 60 80 100 120 140 160 180 200
20
40
60
80
100
120
140
160
180
200
(a)
20 40 60 80 100 120 140 160 180 200
20
40
60
80
100
120
140
160
180
200
(b)
10
10
20
30
30
40
50
50
60
20 40 60 80 10070 90
(c)
20 40 60 80 100 120 140 160 180 200
20
40
60
80
100
120
140
160
180
200
(d)
20 40 60 80 100 120 140 160 180 200
20
40
60
80
100
120
140
160
180
200
(e)
Figure 10: (a) Occluded image (black rectangle) by a red
rectangle; (b) Inpainted image using the standard topological
gradient; (c) Minimalpath between the keypoints represented on the
topological gradient; (d) Inpainted image using the hybrid scheme
(fast marching algorithmfor closing the contours identified by the
topological gradient); (e) Inpainted image using a TV-based
method.
are represented by a large black point on the same image.They
are located on the edge of the domain, as the inpaintingtopological
gradient always takes its minimal values there.
Then, the minimal path algorithm is run, and it providesa path
between the keypoints, represented in green in
Figure 11(b). We can see that the path follows very well
thevalley line of the topological gradient, from one side to
theother. By choosing 3 keypoints instead of 2, there will
beanother keypoint on the bottom edge, near the first one, andit
will simply add a small contour located all along on the
-
International Journal of Biomedical Imaging 17
10 20 30 40 50 60
5
10
15
20
25
30
35
40
45
50−220−200−180−160−140−120−100−80−60−40−20
Admissible pointsKeypoints automatically selected
(a)
10 20 30 40 50 60
5
10
15
20
25
30
35
40
45
50
Minimal path between keypoints
(b)
10 20 30 40 50 60
5
10
15
20
25
30
35
40
45
50
(c)
10 20 30 40 50 60
5
10
15
20
25
30
35
40
45
50
(d)
50 100 150 200 250 300 350
20
40
60
80
100
120
140
160
180
200
(e)
10 20 30 40 50 60
5
10
15
20
25
30
35
40
45
50
(f)
Figure 11: (a) Admissible set of keypoints and selected
keypoints on the topological gradient; (b) Minimal path between the
keypointsrepresented on the topological gradient; (c) Minimal path
between the keypoints represented on the occluded image; (d)
Correspondingidentified missing edge in white; (e) Inpainted image
using the fast marching algorithm for closing the contours
identified by the topologicalgradient in the hidden part of the
image; (f) Zoom of (e).
-
18 International Journal of Biomedical Imaging
edge of the domain, and consequently, there is absolutely
noimpact on the reconstruction of the hidden part of the image.
Figure 11(c) shows the same identified path representedon the
occluded image. This allows one to see that the pathclearly gives a
good approximation of the missing edge andalso that the topological
gradient is very powerful for thisidentification problem. The
corresponding identified edgeset is represented in Figure 11(d).
This image should becompared with the thresholded edge set of
Figure 5(d). Fromthese two images, we can conclude that the minimal
pathalgorithm is an excellent tool for extracting the valley
linesof the topological gradient.
Finally, using this minimal path as the set of missingedges in
the occluded zone, the inpainting topological gradi-ent algorithm
produces a much better reconstructed image,shown in Figures 11(e)
and 11(f). The quality of the image isvery good, as the missing
edges used for the reconstructionare connected, and the Laplace
operator will not produceany blurring effect due to a discontinuous
contour. Notethat there are some small discontinuities on the top
leftboundary due to the fact that we used the Neumann solutionof
the perturbed problem. The construction of a Dirichletsolution
would be better, but it is also much more difficultto solve the
Dirichlet problem in this case, as it is ill posed.This example
confirms that the quality of all topologicalgradient applications
in image processing can be improvedby replacing a simple
thresholding technique by a minimalpath algorithm.
As already shown in [25], the topological gradientextrapolates
the edges and their curvature in the missing partof the image (see
also Figure 11(d), in which the identifiededge is not a straight
line), contrary to total variation-basedmethods. Thus, provided the
identified missing edges areconnected (this point is now ensured by
the application ofthe fast marching algorithm to the topological
gradient), theinpainted image has edges with continuous curvature,
whichis not the case with many other inpainting schemes.
5. Conclusions and Perspectives
We have introduced a hybrid scheme, based on one side onthe
topological gradient for edge detection, and on the otherside on
the fast marching and minimal paths methods forcontour completion.
These approaches allow us to extractconnected contours in 2D images
and to solve the mainissue of all topological gradient-based
algorithms for imageprocessing problems (discontinuity of the
edges). Moreover,the minimal path algorithm does not degrade the
complexityof the topological asymptotic analysis.
We have considered two specific applications in imageprocessing:
segmentation and inpainting. In the first one(segmentation), we
showed that the topological gradient ismore efficient than the
standard gradient for edge detectionand the hybrid scheme provides
better results than the fastmarching method applied to the standard
gradient of theimage. In the second application (inpainting), we
showedthat the hybrid scheme particularly improves the quality
ofthe inpainted image, as the contour completion ensures a
nonblurred inpainted image and as it also helps removingthe
manual thresholding of the topological gradient.
The hybrid scheme is very efficient and quite automatic,as there
is no more thresholding process. The topologicalgradient algorithm
has been shown in previous inpaintingarticles to propagate the main
edges inside the hidden zone,with some continuity of their
curvature, and the use ofa minimal path technique helps detect the
valley lines ofthe topological gradient. The main drawback of the
hybridscheme is the same as for the standard topological
gradientalgorithm: the image is filled in with the Laplacian,
andthis part has to be improved in order to also recovertexture
information. Some preliminary results show that itis possible with
the same kind of approach, thanks to higherorder operators.
An interesting and natural perspective is to apply thishybrid
scheme to 3D images and movies. The topologicalgradient can very
easily be extended to 3D images. Theminimal path technique has also
been adapted to the iden-tification of tubular structures in 3D
[37]. Another perspec-tive consists of dealing with the changes of
topology of theedges in order to automatically detect bifurcations
and T-junctions.
References
[1] T. Pavlidis and Y. T. Liow, “Integrating region growing
andedge detection,” IEEE Transactions on Pattern Analysis
andMachine Intelligence, vol. 12, no. 3, pp. 225–233, 1990.
[2] M. Berthod, Z. Kato, S. Yu, and J. Zerubia, “Bayesian
imageclassification using Markov random fields,” Image and
VisionComputing, vol. 14, no. 4, pp. 285–295, 1996.
[3] C. A. Bouman and M. Shapiro, “Multiscale random fieldmodel
for Bayesian image segmentation,” IEEE Transactions onImage
Processing, vol. 3, no. 2, pp. 162–177, 1994.
[4] S. C. Zhu, “Region competition: unifying snakes, region
grow-ing, and bayes/mdl for multiband image segmentation,”
IEEETransactions on Pattern Analysis and Machine Intelligence,
vol.18, no. 9, pp. 884–900, 1996.
[5] G. Aubert and J.-F. Aujol, “Optimal partitions, regularized
so-lutions, and application to image classification,”
ApplicableAnalysis, vol. 84, no. 1, pp. 15–35, 2005.
[6] G. Aubert and P. Kornprobst, “Mathematical Problems inImage
Processing,” in Applied Mathematical Sciences, vol. 147,Springer,
New York, NY, USA, 2001.
[7] J. F. Aujol, G. Aubert, and L. Blanc-Féraud,
“Wavelet-basedlevel set evolution for classification of textured
images,” IEEETransactions on Image Processing, vol. 12, no. 12, pp.
1634–1641, 2003.
[8] T. F. Chan and J. Shen, Image Processing and Analysis:
Varia-tional, PDE, Wavelets, and Stochastic Methods, SIAM
Publisher,Philadelphia, Pa, USA, 2005.
[9] L. D. Cohen, “Avoiding local minima for deformable curvesin
image analysis,” in Curves and Surfaces with Applications inCAGD,
A. Le Méhauté, C. Rabut, and L. L. Schumaker, Eds.,pp. 77–84,
1997.
[10] S. Jehan-Besson, A. Herbulot, M. Barlaud, and G.
Aubert,“Shape gradient for image and video segmentation,” in
Mathe-matical Models in Computer Vision: The Handbook, Springer,New
York, NY, USA, 2005.
-
International Journal of Biomedical Imaging 19
[11] M. Kass, A. Witkin, and D. Terzopoulos, “Snakes:
activecontour models,” International Journal of Computer
Vision,vol. 1, no. 4, pp. 321–331, 1988.
[12] S. Masnou, “Disocclusion: a variational approach using
levellines,” IEEE Transactions on Image Processing, vol. 11, no.
2,pp. 68–76, 2002.
[13] S. Masnou and J. M. Morel, “Level lines based
disocclusion,”in Proceedings of the IEEE International Conference
on ImageProcessing, vol. 3, pp. 259–263, 1998, 1998.
[14] T. McInerney and D. Terzopoulos, “T-snakes: topology
adap-tive snakes,” Medical Image Analysis, vol. 4, no. 2, pp.
73–91,2000.
[15] D. Mumford and J. Shah, “Optimal approximations by
piece-wise smooth functions and associated variational
problems,”Communications on Pure and Applied Mathematics, vol. 42,
no.5, pp. 577–685, 1989.
[16] S. Osher, Geometric Level Set Methods in Imaging, Vision
andGraphics, Springer, New York, NY, USA, 2003.
[17] N. Paragios, N. Ayache, and J. Duncan, Biomedical
ImageAnalysis: Methodologies and Applications, Springer, New
York,NY, USA, 2008.
[18] N. Paragios and R. Deriche, “Geodesic active regions and
levelset methods for motion estimation and tracking,”
ComputerVision and Image Understanding, vol. 97, no. 3, pp.
259–282,2005.
[19] C. Samson, L. Blanc-Féraud, G. Aubert, and J. Zerubia,
“Levelset model for image classification,” International Journal
ofComputer Vision, vol. 40, no. 3, pp. 187–197, 2000.
[20] C. Samson, L. Blanc-Féraud, G. Aubert, and J. Zerubia,
“Avariational model for image classification and restoration,”IEEE
Transactions on Pattern Analysis and Machine Intelligence,vol. 22,
no. 5, pp. 460–472, 2000.
[21] A. Tsai, A. Yezzi, and A. S. Willsky, “Curve
evolutionimplementation of the Mumford-Shah functional for
imagesegmentation, denoising, interpolation, and
magnification,”IEEE Transactions on Image Processing, vol. 10, no.
8, pp. 1169–1186, 2001.
[22] L. A. Vese and T. F. Chan, “A multiphase level set
frameworkfor image segmentation using the Mumford and Shah
model,”International Journal of Computer Vision, vol. 50, no. 3,
pp.271–293, 2002.
[23] J. Weickert, “Efficient image segmentation using partial
dif-ferential equations and morphology,” Pattern Recognition,
vol.34, no. 9, pp. 1813–1824, 2001.
[24] A. Yezzi, A. Tsai, and A. Willsky, “A fully global approach
toimage segmentation via coupled curve evolution equations,”Journal
of Visual Communication and Image Representation,vol. 13, no. 1-2,
pp. 195–216, 2002.
[25] D. Auroux and M. Masmoudi, “A one-shot inpaintingalgorithm
based on the topological asymptotic analysis,”Computational and
Applied Mathematics, vol. 25, no. 2-3, pp.251–267, 2006.
[26] D. Auroux and M. Masmoudi, “Image processing by
topolog-ical asymptotic expansion,” Journal of Mathematical
Imagingand Vision, vol. 33, no. 2, pp. 122–134, 2009.
[27] M. Masmoudi, “The topological asymptotic,” in
Computa-tional Methods for Control Applications, R. Glowinski,
H.Karawada, and J. Périaux, Eds., vol. 16, pp. 53–72,
GAKUTOInternational Series. Mathematical Sciences and
Applications,Tokyo, Japan,, 2001.
[28] D. Auroux, M. Masmoudi, and L. Jaafar Belaid, “Image
res-toration and classification by topological asymptotic
expan-sion,” in Variational Formulations in Mechanics: Theory
and
Applications, E. Taroco, E. A. de Souza Neto, and A. A.
Nov-otny, Eds., CIMNE, Barcelona, Spain, 2006.
[29] L. J. Belaid, M. Jaoua, M. Masmoudi, and L. Siala,
“Imagerestoration and edge detection by topological
asymptoticexpansion,” Comptes Rendus Mathematique, vol. 342, no. 5,
pp.313–318, 2006.
[30] L. Jaafar Belaid, “An overview of the topological
gradientapproach in image processing: advantages and
inconve-niences,” Journal of Applied Mathematics, vol. 2010,
Article ID761783, 19 pages, 2010.
[31] H. G. Senel, “Topological gradient operators for edge
detec-tion,” in Proceedings of the IEEE International Conference
onImage Processing (ICIP ’07), vol. 3, pp. 61–64, 2004.
[32] P. Wen, X. Wu, and C. Wu, “An interactive image
inpaintingmethod based on rbf networks,” in Proceedings of the
3rdInternational Symposium on Neural Networks, pp.
629–637,2006.
[33] T. Zhou, F. Tang, J. Wang, Z. Peng, and Q. Wang, “Digital
im-age inpainting with radial basis functions,” Journal of Imageand
Graphics, vol. 9, no. 10, pp. 1190–1196, 2004.
[34] T. Chan and J. Shen, “Mathematical models for local
deter-ministic inpaintings,” Tech. Rep. 00-11, CAM
Reports—UCLAMathematics, 2000.
[35] T. Chan and J. Shen, “Non-texture inpainting by
curvature-driven diffusions (CDD),” Tech. Rep. 00-35, CAM
Reports—UCLA Mathematics, 2000.
[36] M. Elad, J. L. Starck, P. Querre, and D. L. Donoho,
“Simultane-ous cartoon and texture image inpainting using
morphologi-cal component analysis (MCA),” Applied and
ComputationalHarmonic Analysis, vol. 19, no. 3, pp. 340–358,
2005.
[37] L. D. Cohen, “Minimal paths and fast marching methods
forimage analysis,” in Mathematical Models in Computer Vision:The
Handbook, N. Paragios, Y. Chen, and O. Faugeras, Eds.,Springer, New
York, NY, USA, 2005.
[38] L. D. Cohen and R. Kimmel, “Global minimum for active
con-tour models: a minimal path approach,” International Journalof
Computer Vision, vol. 24, no. 1, pp. 57–78, 1997.
[39] F. Benmansour and L. D. Cohen, “Tubular structure
seg-mentation based on minimal path method and
anisotropicenhancement,” International Journal of Computer Vision,
vol.92, no. 2, pp. 192–210, 2011.
[40] F. Benmansour and L. D. Cohen, “Fast object segmentationby
growing minimal paths from a single point on 2D or 3Dimages,”
Journal of Mathematical Imaging and Vision, vol. 33,no. 2, pp.
209–221, 2009.
[41] L. D. Cohen, “Multiple contour finding and perceptual
group-ing using minimal paths,” Journal of Mathematical Imagingand
Vision, vol. 14, no. 3, pp. 225–236, 2001.
[42] T. Deschamps and L. D. Cohen, “Fast extraction of
minimalpaths in 3D images and applications to virtual
endoscopy,”Medical Image Analysis, vol. 5, no. 4, pp. 281–299,
2001.
[43] H. Li, A. Yezzi, and L. Cohen, “3D multi-branch
tubularsurface and centerline extraction with 4D iterative key
points,”in Proceedings of the 12th International Conference on
MedicalImage Computing and Computer Assisted Intervention (MIC-CAI
’09), Imperial College, London, UK, 2009.
[44] R. Ardon, L. D. Cohen, and A. Yezzi, “A new implicit
methodfor surface segmentation by minimal paths in 3D
images,”Applied Mathematics and Optimization, vol. 55, no. 2, pp.
127–144, 2007.
[45] D. Auroux, “From restoration by topological gradient to
medi-cal image segmentation via an asymptotic expansion,”
Mathe-matical and Computer Modelling, vol. 49, no. 11-12, pp.
2191–2205, 2009.
-
20 International Journal of Biomedical Imaging
[46] H. Ammari, M. S. Vogelius, and D. Volkov, “Asymptotic
for-mulas for perturbations in the electromagnetic fields due tothe
presence of inhomogeneities of small diameter II. Thefull Maxwell
equations,” Journal des Mathematiques Pures etAppliquees, vol. 80,
no. 8, pp. 769–814, 2001.
[47] S. Garreau, P. Guillaume, and M. Masmoudi, “The
topologicalasymptotic for PDE systems: the elasticity case,” SIAM
Journalon Control and Optimization, vol. 39, no. 6, pp.
1756–1778,2001.
[48] P. Guillaume and K. Sididris, “The topological asymptotic
ex-pansion for the dirichlet problem,” SIAM Journal on Controland
Optimization, vol. 41, no. 4, pp. 1042–1072, 2002.
[49] P. Guillaume and K. Sididris, “The topological sensitivity
andshape optimization for the stokes equations,” SIAM Journal
onControl and Optimization, vol. 43, no. 1, pp. 1–31, 2004.
[50] S. Amstutz, I. Horchani, and M. Masmoudi, “Crack
detectionby the topological gradient method,” Control and
Cybernetics,vol. 34, no. 1, pp. 81–101, 2005.
[51] B. Samet, S. Amstutz, and M. Masmoudi, “The
topologicalasymptotic for the Helmholtz equation,” SIAM Journal
onControl and Optimization, vol. 42, no. 5, pp. 1523–1544,
2004.
[52] L. Jaafar Belaid, M. Jaoua, M. Masmoudi, and L. Siala,
“Appli-cation of the topological gradient to image restoration
andedge detection,” Engineering Analysis with Boundary
Elements,vol. 32, no. 11, pp. 891–899, 2008.
[53] J. Dicker, Fast marching methods and level set methods: an
im-plementation, Ph.D. thesis, University of British
Columbia,2006.
[54] J. A. Sethian, Level set methods and fast marching
methods,Cambridge University Press, 1999.
-
International Journal of
AerospaceEngineeringHindawi Publishing
Corporationhttp://www.hindawi.com Volume 2010
RoboticsJournal of
Hindawi Publishing Corporationhttp://www.hindawi.com Volume
2014
Hindawi Publishing Corporationhttp://www.hindawi.com Volume
2014
Active and Passive Electronic Components
Control Scienceand Engineering
Journal of
Hindawi Publishing Corporationhttp://www.hindawi.com Volume
2014
International Journal of
RotatingMachinery
Hindawi Publishing Corporationhttp://www.hindawi.com Volume
2014
Hindawi Publishing Corporation http://www.hindawi.com
Journal ofEngineeringVolume 2014
Submit your manuscripts athttp://www.hindawi.com
VLSI Design
Hindawi Publishing Corporationhttp://www.hindawi.com Volume
2014
Hindawi Publishing Corporationhttp://www.hindawi.com Volume
2014
Shock and Vibration
Hindawi Publishing Corporationhttp://www.hindawi.com Volume
2014
Civil EngineeringAdvances in
Acoustics and VibrationAdvances in
Hindawi Publishing Corporationhttp://www.hindawi.com Volume
2014
Hindawi Publishing Corporationhttp://www.hindawi.com Volume
2014
Electrical and Computer Engineering
Journal of
Advances inOptoElectronics
Hindawi Publishing Corporation http://www.hindawi.com
Volume 2014
The Scientific World JournalHindawi Publishing Corporation
http://www.hindawi.com Volume 2014
SensorsJournal of
Hindawi Publishing Corporationhttp://www.hindawi.com Volume
2014
Modelling & Simulation in EngineeringHindawi Publishing
Corporation http://www.hindawi.com Volume 2014
Hindawi Publishing Corporationhttp://www.hindawi.com Volume
2014
Chemical EngineeringInternational Journal of Antennas and
Propagation
International Journal of
Hindawi Publishing Corporationhttp://www.hindawi.com Volume
2014
Hindawi Publishing Corporationhttp://www.hindawi.com Volume
2014
Navigation and Observation
International Journal of
Hindawi Publishing Corporationhttp://www.hindawi.com Volume
2014
DistributedSensor Networks
International Journal of