Top Banner
Segmentation of Partially Overlapping Convex Objects Using Branch and Bound Algorithm Sahar Zafari 1 , Tuomas Eerola 1 , Jouni Sampo 2 , Heikki K¨ alvi¨ ainen 1 , and Heikki Haario 2 1 Machine Vision and Pattern Recognition Laboratory,School of Engineering Science, Lappeenranta University of Technology, Lappeenranta Finland 2 Mathematics Laboratory, School of Engineering Science, Lappeenranta University of Technology,, Lappeenranta Finland [email protected] Abstract. This paper presents a novel method for the segmentation of partially overlapping convex shape objects in silhouette images. The proposed method involves two main steps: contour evidence extraction and contour estimation. Contour evidence extraction starts by recover- ing contour segments from a binarized image using concave contour point detection. The contour segments which belong to the same objects are grouped by utilizing a criterion defining the convexity, symmetry and ellipticity of the resulting object. The grouping is formulated as a com- binatorial optimization problem and solved using the well-known branch and bound algorithm. Finally, the contour estimation is implemented through a non-linear ellipse fitting problem in which partially observed objects are modeled in the form of ellipse-shape objects. The experiments on a dataset of consisting of nanoparticles demonstrate that the proposed method outperforms four current state-of-art approaches in overlapping convex objects segmentation. The method relies only on edge informa- tion and can be applied to any segmentation problems where the objects are partially overlapping and have an approximately convex shape. 1 Introduction Segmentation of overlapping objects aims to address the issue of representation of multiple objects with partial views. Overlapping or occluded objects occur in various applications, such as morphology analysis of molecular or cellular objects in biomedical and industrial imagery where quantitative analysis of individual objects by their size and shape is desired [1–3]. In many such applications, the objects can often be assumed to have approximately elliptical shape. For exam- ple, the most commonly measured properties of nanoparticles are their length and width, which can correspond to the major and minor axis of an ellipse fitted over the particle contour [4]. Even with rather strong shape priors, segmentation of overlapping objects remains a challenging task. Deficient information from the objects with occluded
15

Segmentation of Partially Overlapping Convex Objects Using ...

Jan 29, 2022

Download

Documents

dariahiddleston
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: Segmentation of Partially Overlapping Convex Objects Using ...

Segmentation of Partially Overlapping ConvexObjects Using Branch and Bound Algorithm

Sahar Zafari1, Tuomas Eerola1, Jouni Sampo2, Heikki Kalviainen1, and HeikkiHaario2

1 Machine Vision and Pattern Recognition Laboratory,School of Engineering Science,Lappeenranta University of Technology, Lappeenranta Finland

2 Mathematics Laboratory, School of Engineering Science, Lappeenranta Universityof Technology,, Lappeenranta Finland

[email protected]

Abstract. This paper presents a novel method for the segmentationof partially overlapping convex shape objects in silhouette images. Theproposed method involves two main steps: contour evidence extractionand contour estimation. Contour evidence extraction starts by recover-ing contour segments from a binarized image using concave contour pointdetection. The contour segments which belong to the same objects aregrouped by utilizing a criterion defining the convexity, symmetry andellipticity of the resulting object. The grouping is formulated as a com-binatorial optimization problem and solved using the well-known branchand bound algorithm. Finally, the contour estimation is implementedthrough a non-linear ellipse fitting problem in which partially observedobjects are modeled in the form of ellipse-shape objects. The experimentson a dataset of consisting of nanoparticles demonstrate that the proposedmethod outperforms four current state-of-art approaches in overlappingconvex objects segmentation. The method relies only on edge informa-tion and can be applied to any segmentation problems where the objectsare partially overlapping and have an approximately convex shape.

1 Introduction

Segmentation of overlapping objects aims to address the issue of representationof multiple objects with partial views. Overlapping or occluded objects occur invarious applications, such as morphology analysis of molecular or cellular objectsin biomedical and industrial imagery where quantitative analysis of individualobjects by their size and shape is desired [1–3]. In many such applications, theobjects can often be assumed to have approximately elliptical shape. For exam-ple, the most commonly measured properties of nanoparticles are their lengthand width, which can correspond to the major and minor axis of an ellipse fittedover the particle contour [4].

Even with rather strong shape priors, segmentation of overlapping objectsremains a challenging task. Deficient information from the objects with occluded

Page 2: Segmentation of Partially Overlapping Convex Objects Using ...

2 Sahar Zafari et al.

or overlapping parts introduces considerable complexity into the segmentationprocess. For example, in the context of contour estimation, the contours of ob-jects intersecting with each other do not usually contain enough visible geomet-rical evidence, which can make contour estimation problematic and challenging.Frequently, the segmentation method has to rely purely on edges between thebackground and the foreground, which makes the processed image essentially asilhouette image (see Fig. 1). Furthermore, the task involves simultaneous seg-mentation of multiple objects. A large number of objects in the image causes alarge number of variations in pose, size and shape of the objects, and leads to amore complex segmentation problem.

Fig. 1: Overlapping nanoparticles.

Several approaches have been proposed for the segmentation of overlappingobjects in various applications. The watershed transform is one of the commonlyused approaches in overlapping cell segmentation [5–7]. However, methods basedon the watershed transform suffer from a poor or inadequate initialization andmay experience difficulties with segmentation of highly overlapped objects inwhich a strong gradient is not present.

Other commonly used approach is to resolve the segmentation of overlap-ping objects using active contours [8, 9]. The efficiency of the active contourbased methods depends highly on the accuracy of the model initialization. Ac-tive contour based methods are also computationally heavy when the amount ofobjects is large.

Morphological operations have been also used for overlapping object seg-mentation. In [1], an automated morphology analysis coupled with a statisticalmodel to inference and segment partially overlapping nanoparticles. The methodis prone to under-segmentation with highly overlapped objects.

In [2], the problem of overlapping objects segmentation was approached usingconcave points extraction and ellipse fitting. Although this approach is efficientfor regular shaped objects, such as bubbles, objects with a shape that deviatesfrom elliptical shape, such as nanoparticles, cause problems for the method.In [10] and [11], the problem of overlapping objects with non-elliptical shapes

Page 3: Segmentation of Partially Overlapping Convex Objects Using ...

Segmentation of Partially Overlapping Convex Objects 3

were addressed by proposing a method combining various heuristics with theconcave point extraction.

In [12], a method for segmentation of overlapping objects with close to el-liptical shape was proposed. The method consists of three steps: seed pointextraction using radial symmetry, contour evidence extraction via edge to seedpoint matching, and contour estimation through ellipse fitting.

In this paper, a novel method for the segmentation of partially overlappingconvex objects is proposed. The proposed method utilizes both generic and spe-cific object shape properties such as convexity, symmetry and ellipticity. Theproposed method follows two sequential steps of contour evidence extractionand contour estimation. The contour evidence extraction step consists of twosub-steps of contour segmentation and segment grouping. In the contour seg-mentation, contours are divided into separate contour segments. In the segmentgrouping, contour evidences are built by joining the contour segments that belongto the same object. Once the contour evidence is obtained, contour estimationis performed using numerically stable direct ellipse fitting.

The key contribution of this work is to introduce a novel framework for seg-ment grouping based on the branch and bound (BB) optimization algorithm [13–16]. This is done by optimizing a cost function that combines different objectproperties including, but not limited to, convexity, symmetry and ellipticity. An-other contribution is the integration of the proposed segment grouping methodinto the segmentation of overlapping objects, enabling improvements comparedto four existing methods with higher detection rate and segmentation accuracy.

2 Overlapping Object Segmentation

Fig. 2 summarizes the proposed method. Given a gray-scale image as an input,the segmentation process starts with pre-processing to build an image silhouetteand the corresponding edge map. The binarization of the image is obtained bybackground suppression based on the Otsu’s method [17] along with morpholog-ical opening to smooth the object boundaries. For computational reasons, theconnected component are extracted and further analysis is performed for eachconnected component separately. The edge map is constructed using the Cannyedge detector [18]. In the contour evidence extraction steps, edge points thatbelonged to each object are grouped using concave points and proposed branchand bound algorithm. Once the contour evidence has been obtained, contourestimation is carried out to infer the missing parts of the overlapping objects.

2.1 Contour Segmentation

The first step of the proposed method is to extract the contour evidence contain-ing the visible parts of the objects boundaries that can be used to inference theoccluded parts of overlapped objects. The contour evidence extraction involvestwo separate tasks: contour segmentation and segment grouping.

Page 4: Segmentation of Partially Overlapping Convex Objects Using ...

4 Sahar Zafari et al.

Contour Evidence Extraction

Pre-Processing

ellipse fitting

contour segmentation

segment grouping

Contour Estimation

image binarization

Fig. 2: Proposed method.

For contour segmentation, first the image edge are extracted by the Cannyedge detector, and then the concave points are obtained through the detectionof corner points followed by the concavity test. The corner points are detectedusing the modified curvature scale space (CSS) method based on curvature anal-ysis [19]. The output of the corner detector includes the points with the max-imum curvature lying on both concave and convex regions of object contours.Since being only interested in the concave points joining the contours of over-lapping objects, the detected corner points are examined if they lie on concaveregions.

Let pi be a detected corner point and pi−k and pi+k its two kth adjacentcontour points. The corner point pi is qualified as concave if the line connectingpi−k to pi+k does not reside inside the object. The obtained concave points areused to split the contours into contour segments. Fig. 3 shows an example ofconcave point extraction and contour segmentation.

A partial overlap between two or more convex shape objects leads to a con-cave shape with concave edge points that correspond to the intersections of theobject boundaries. It is a common practice to utilize these concave points tosegment the contour of overlapping objects. Different methods such as polygo-nal approximation [2, 10], curvature [20], and angle [21, 10] have been applied todetermine the location of concave points in the image.

2.2 Segment Grouping

Due to the overlap between the objects and the irregularities in the object shapes,a single object may produce multiple contour segments. Segment grouping isneeded to merge all the contour segments belonging to the same object. In thissection, we lay out the mathematical definition of the contour segment groupingproblem in terms of a grouping criterion. We formulate the grouping task as acombinatorial optimization problem and solve it using the branch and boundalgorithm. The efficiency of this algorithm depends upon the definition of thegrouping criterion. We will later study the grouping criterion in more details.

Page 5: Segmentation of Partially Overlapping Convex Objects Using ...

Segmentation of Partially Overlapping Convex Objects 5

(a) (b) (c) (d)

Fig. 3: Contour segmentation: (a) Edge map; (b) Corner detection by [19]; (c)Concavity test to extract concave corners (green circle) and removed convexcorners (pink square); (d) Contour segmentation by concave points (the colorsare used only for illustrative purpose to visualize the segmented contour byconcave points).

Problem Formulation: Let S = S1, S2, ..., SN be an ordered set of N con-tour segments in a connected component of an image. The aim is to group thecontour segments into M subsets such that the contour segments that belong toindividual objects are grouped together and M ≤ N . Let ωi be the group mem-bership indicator giving the group index to which the contour segment Si belongsto. Denote Ω be the ordered set of all membership indicators: ω1, ω2, ...., ωN.

The grouping criterion is given by a scalar function J(·) of Ω which mapsa possible grouping of the given contour segments onto the set of real numbersR. That is, J is the cost of grouping that ideally measures how the groupingΩ resembles the true contour segments of the objects. The grouping problemfor the given set of S is to find the optimal membership set Ω∗ such that thegrouping criterion (cost of grouping) is minimized:

Ω∗ = argminΩ

J(Ω;S) (1)

Branch and Bound Algorithm: The contour segment grouping problem canbe considered as a combinatorial optimization problem. As the number of groups

(objects) is not known, there are at most 1 + 2N

2! + . . . + MN

M ! . . . + NN

N ! distinctcandidate solutions. The optimal solution can be obtained by evaluating thegrouping criterion J(·) for every candidate grouping and by selecting the solutionwith minimum grouping criterion. However, this optimization problem is NPhard where even with modest values of M and N , the explicit enumeration isimpossible due to the exponentially increasing number of candidate solutions.The BB algorithm is known as an efficient approach for NP-hard combinatorialoptimization problems [13–15]. BB is efficient for our optimization problem since

Page 6: Segmentation of Partially Overlapping Convex Objects Using ...

6 Sahar Zafari et al.

it avoids exhaustive enumeration using the value of the current optimal solutionand defining bounds for the function to be optimized.

The BB algorithm evaluates the partial solutions for the grouping problemas Ωk = ω1, ω2, . . . ωk, 1 < k < N . Let us denote b1, b2, . . . , bN as a sequenceof lower bounds satisfying

bk = bk(ω1, ω2, . . . , ωk) ≤ J(Ω;S). (2)

Let Ωi and Ωj be two sets of partial solutions for i < j. If the lower bound onΩi is greater than the lower bound on Ωj , i.e. bi > bj , bj is replaced with bi.This implies that:

b1 ≤ b2 ≤ . . . ≤ bN = J(Ω;S). (3)

Let us denote B the upper bound on the optimal J , i.e.,

J(Ω∗;S) ≤ B. (4)

If the lower bound bk of a given partial solution Ωk is greater than the upperbound of optimal solution B, bk(Ωk) ≥ B, then

1. Ωk is suboptimal and explicitly rejected, since J(Ω;S) ≥ bk > B > J(Ω∗;S),and

2. all grouping of the form ω1, ω2...., ωk, ωk+1, . . . , ωN are considered to besuboptimal and implicitly rejected since bk is unchanged.

Hence, given a single candidate solution to the optimization problem and a se-quence of lower and upper bounds as defined in Eqs. 2 and 4, one can eliminatethe candidate solutions to the grouping problem that are evaluated as subopti-mal. To be more illustrative, we demonstrate the branch and bound algorithm inthe contour segment grouping using an example shown in Fig. 4. Given the setof extracted contour segments S = S1, S2, S3, S4, the BB algorithm systemat-ically searches all the candidate groupings and attempts to find the optimum.It is convenient to represent the whole grouping process of the BB algorithmthrough a search tree as shown in Fig 5. The root of such search tree repre-sents the initial state where each contour segment is considered as an individualgroup, that is S1, S2, S3, and S4 as four distinct groups. The descenders of theroot represent all the other candidate grouping obtained by merging of S. Bythis means, every node at the level m of the search tree represents a distinctgrouping of m segments.

The nodes in search tree are explored by the depth-first search such that thereare at most four optimal groupings. With a naive strategy to expand the searchtree, a large number of redundant nodes is generated. To increase the efficiencyof the search, this should be avoided. For example, the grouping S2, S1 is notcreated as it has been already generated in the previous branch as S1, S2.

The expansion of the search tree is controlled by the grouping criterion. Theoptimal grouping is the node with the minimum cost. The upper bound forthe optimal grouping is set as soon as the first node is generated (see Fig. 6).

Page 7: Segmentation of Partially Overlapping Convex Objects Using ...

Segmentation of Partially Overlapping Convex Objects 7

Provided that the node already takes part in an optimal grouping, the upperbound is set by the cost of the optimal grouping. Otherwise, the upper boundis set by the cost of the node itself. The initial upper bound in S1, S2, S4 are setto the their costs as the nodes have not been appeared in any optimal groupingso far. The initial upper bound in S3 is set to J(S1S3) as it is already part ofthe optimal solution S1, S3.

The BB terminates search earlier if the current solution is recognized assuboptimal, i.e., the cost of current solution is greater than the upper bound foroptimal solution J(Ω;S) > B. The upper bound B is updated when a solutionis found which cost is less than the current value of B.

S1 S2

S3

S4

Fig. 4: An example of the contour segmentation result.

Intial State S1 S2 S3 S4

First level S1S2 S1S3 S1S4 S2S3 S2S4 S3S4

Second level S1S2S3 S1S2S4 S1S3S4 S2S3S4

Third level S1S2S3S4

Fig. 5: The corresponding non-redundant search tree for the coutour segmentspresented in Fig 4.

Grouping Criterion: The BB algorithm can be applied with any groupingcriterion J . However, the selection of the grouping criterion has significant effect

Page 8: Segmentation of Partially Overlapping Convex Objects Using ...

8 Sahar Zafari et al.

B = J(S1) S1 B = J(S2) S2 B = J(S1S3) S3 B = J(S4) S4

J(S1S2) > B

S1S2

J(S1S3) < B,B = J(S1S3)

S1S3J(S1S4) > B

S1S4J(S2S3) > B

S2S3J(S2S4) > B

S2S4J(S3S4) > B

S3S4

J(S1S3S4) > B S1S3S4

Fig. 6: An example of the termination criterion in the search tree (red colorrepresents the suboptimal solutions and the green color represents the optimalsolutions).

on the overall method performance. Our proposal for the grouping criterionis a hybrid cost function consisting of two parts: 1) generic part (Jconcavity)that encapsulate the general convexity properties of the objects and 2) specificpart that encapsulates the properties of objects that are exclusive to a certainapplication, e.g., symmetry (Jsymmetry) and ellipticity (Jellipticity). The proposedcost function is defined as follows:

J = αJconcavity︸ ︷︷ ︸Generic

+βJellipticity + γJsymmetry︸ ︷︷ ︸Specific

(5)

where α, β, and γ are the weights for each term respectively.

The generic part encourages the convexity assumption of the objects in orderto penalize the grouping of the contour segments belonging to different objects.This is achieved by incorporating a quantitative concavity measure. Given twocontour segments Si and Sj the generic part of the cost function is defined asfollows:

Jconcavity =

1, there is concave point between Sj and Sj

(1− ASi∪Sj

Ach,Si∪Sj) otherwise,

(6)

where ASi∪Sj is the area of a region bounded by Si, Sj , and the lines connect-ing the endpoint of Sj to the starting point of Si and the end point of Si tothe starting point of Sj . Ach,Si∪Sj

is the corresponding area of the convex hullbounded by Si and Sj .

The specific part is adapted to consider the application criteria and certainobject properties. Considering the object under examination, several functionscan be utilized. Here, the specific term is formed by the ellipticity and symmetrycost functions.

Page 9: Segmentation of Partially Overlapping Convex Objects Using ...

Segmentation of Partially Overlapping Convex Objects 9

The ellipticity term is defined by measuring the discrepancy between the fit-ted ellipse and the contour segments [2]. Given the contour segment Si consistingof n points, Si = pk(xk, yk)nk=1, and the corresponding fitted ellipse points,Sf,i = pf,k(xf,k, yf,k)nk=1, the ellipticity term is defined as follows:

Jellipticity =1

n

n∑k=1

√(xk − xf,k)2 + (yk − yf,k)2. (7)

The symmetry term penalize for the non-symmetry of the resulting object.Let Oi and Oj be the center of symmetry of the contour segments Si and Sjobtained by aggregating the normal vector of the contour segments. The proce-dure is similar to fast radial symmetry transform [22], but the gradient vectorsare replaced by contour segments normal vector. We call this transform as thenormal symmetry transform (NST).

In NST every contour segment point gives a vote for the plausible radialsymmetry at some specific distance from that point. Given the distance valuer of the predefined range [Rmin Rmax], for every contour segment point (x, y),NST determines the negatively affected pixels p−e and sequentially constructsthe orientation projection image Or:

p−e(x, y) = (x, y)− round( n(x, y)

‖n(x, y)‖× r), (8)

Or(p−e(x, y)) = Or(p−e(x, y))− 1, (9)

In order to construct the orientation image, the radial symmetry contributionSr for the radius m ∈ [Rmin, Rmax] is formulated as

Sr(x, y) = ( |Or(x, y)|kr

, (10)

where α and kr are the radial strictness and the scaling factor that normalizesOr across different radii, respectively. Or is defined as

Or(x, y) =

Or(x, y), if Or(x, y) < kr.

kr, otherwise.(11)

The full NST transform S by which the interest symmetric regions are definedis given by the average of the symmetry contributions over all the radii r ∈[Rmin, Rmax] considered:

S =1

|N |∑

r∈[Rmin,Rmax]

Sr. (12)

The center of symmetry Oi and Oj of the contour segments are estimated as theaverage locations of the detected symmetric regions in S. The symmetry termJsymmetry is defined as the Euclidean distance between Oi and Oj . The distanceis normalized to [0...1] according to maximum diameter of the object.

Page 10: Segmentation of Partially Overlapping Convex Objects Using ...

10 Sahar Zafari et al.

2.3 Contour Estimation

The last step of proposed method is the contour estimation, where, by meansof the visual information produced from the previous step, the missing parts ofthe object contours are estimated. Ellipse fitting is a very common approach inoverlapping object segmentation, especially in the medical and industrial appli-cations.

The most efficient recent ellipse fitting methods based on shape boundarypoints are generally addressed through the classic least square fitting problem.In this work, the contour estimation is addressed through a stable direct leastsquare fitting method [23] where the partially observed objects are modeled inthe form of ellipse-shape objects. Fig. 7 shows an example of contour estimationapplied to contour evidences.

(a) (b) (c) (d)

Fig. 7: Contour estimation: (a) Original image; (b) Contour segmentation; (c)Segment grouping (the thin gray lines are added to illustrate the grouping ofnon-adjacent segments); (d) Contour estimation.

3 Experiments

3.1 Data

The experiments were carried out using a dataset consisting of nanoparticles im-ages captured by transmission electron microscopy. In total, the dataset contains11 images of 4008× 2672 pixels. Around 200 particles were marked manually ineach image by an expert. The annotations consist of manually drawn contoursof the objects. Since not all the objects are marked, a pre-processing step wasapplied to eliminate the unmarked objects from the images. It should be notedthat the images consist of dark objects on a white background and, therefore,pixels outside the marked objects could be colored white without making theimages considerably easier to analyze.

Page 11: Segmentation of Partially Overlapping Convex Objects Using ...

Segmentation of Partially Overlapping Convex Objects 11

3.2 Results

To evaluate the method performance and to compare the methods, two specificperformance measures, True Positive Rate (TPR) and Positive Predictive Value(PPV), were used:

TPR =TP

TP + FN(13)

PPV =TP

TP + FP(14)

where True Positive (TP) is the number of correctly segmented objects, FalsePositive (FP) is the number of incorrectly segmented objects, and False Negative(FN) is the number of missed objects.

To decide whether the segmentation result was correct or incorrect, JaccardSimilarity coefficient (JSC) [24] was used. Given a binary map of the segmentedobject Os and the ground truth particle Og, JSC is computed as

JSC =Os ∩OgOs ∪Og

. (15)

The threshold value for the ratio of overlap (JSC threshold ) in the JSC was setto 0.5. The average JSC (AJSC) value was used as the third measure to evaluatethe segmentation performance.

Considering the smallest and largest object axes, the method parametersRmin and Rmax were set to 10 and 20 respectively. The parameter k was exper-imentally set to 10. The weighting parameters α, β, and γ were set to 0.4, 0.1and 0.5 based on the prior knowledge about the shape of objects in the image.Fig. 8 shows an example of segmentation result.

The performance of the proposed segmentation method was compared tofour existing state-of-the-art methods, Concave point detection and Contourevidence extraction (CC) [11], a method based on Seed point extraction, Con-tour evidence extraction and Contour estimation (SCC) [12], Nanoparticles Seg-mentation (NPA) [1], and Concave-point Extraction and Contour Segmentation(CECS) [2]. These methods are particularly chosen as previously applied forsegmentation of overlapping convex and elliptical shape objects. The implemen-tations [25] made by the corresponding authors were used for NPA, CC, andSCC. CECS was implemented by ourselves based on [2]. The source code ofproposed method written in Matlab can be downloaded from [25].

Examples of typical segmentation results are presented in Fig. 9. SCC andNPA suffers from under-segmentation while CC and CECS tend to over-segmentthe objects. The proposed method neither under- nor over-segments the objects.

The corresponding performance statistics of the competing methods appliedto the dataset are shown in Table 1. As it can be seen, the proposed methodoutperforms the other four with respect to the TPR and JSC and achieves acomparable performance with NPA in terms of PPV. The high JSC value of the

Page 12: Segmentation of Partially Overlapping Convex Objects Using ...

12 Sahar Zafari et al.

(a) (b)

Fig. 8: An example of the proposed method segmentation result on nanoparticlesdataset: (a) Ground truth; (b) Proposed method.

proposed method indicates its superiority with respect to the resolved overlapratio.

Table 1: Comparison of the performance of the proposed method for the nanopar-ticles dataset.

MethodsTPR PPV AJSC[%] [%] [%]

Proposed 86 88 77CC 85 84 73SCC 79 89 72NPA 62 90 58CECS 66 73 53

Figs. 10 shows the effect of the Jaccard similarity threshold on the TPR,PPV, and AJSC scores with the proposed and competing segmentation methods.As expected, the segmentation performance of all methods degrades when theJSC threshold is increased. However, the threshold value has only minor effecton the ranking order of the methods and the proposed segmentation methodoutperforms the other methods with all JSC threshold values below 0.9. Withthe threshold value of 0.9 NPA is slightly better due to the fact that the objectsare not perfect ellipses and the ellipse fitting cannot estimate the contours witha such accuracy.

Page 13: Segmentation of Partially Overlapping Convex Objects Using ...

Segmentation of Partially Overlapping Convex Objects 13

(a) (b) (c)

(d) (e) (f)

Fig. 9: Examples of segmentation results on the slice of nanoparticles dataset:(a) Ground truth; (b) Proposed method; (c) CC; (d) SCC ; (e) CECS; (f) NPA.

3.3 Computation Time

The proposed method was implemented in MATLAB, using a PC with a 3.20GHz CPU and 8 GB of RAM. With the selected combination of parametersthe computational time was 170 seconds per image, while CC demanded 21 sec-onds, CECS 77 seconds, SCC 150 seconds, NPA demanded 200 seconds. Thecomputational time breakdown was as follows: contour segmentation 2%, seg-ment grouping 96%, and ellipse fitting 2%. However, it should be noted thatthe method performance was not optimized and the computation time could beimproved.

4 Conclusions

This paper presented a method to segment multiple partially overlapping convexshaped objects in silhouette images using concave points and branch and boundalgorithm. The proposed method consisted of two main steps: contour evidenceextraction to detected the visible part of each object and contour estimation to

Page 14: Segmentation of Partially Overlapping Convex Objects Using ...

14 Sahar Zafari et al.

JSC threshold 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Tru

e P

ositiv

e R

ate

[ %

]

0

10

20

30

40

50

60

70

80

90

100

TPR vs JSC threshold

Proposed

CC

SCC

NPA

CECS

JSC threshold0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1P

ositiv

e P

red

ictive

Va

lue

[ %

]

0

10

20

30

40

50

60

70

80

90

100

PPV vs JSC threshold

Proposed

CC

SCC

NPA

CECS

JSC threshold 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1A

ve

rag

e J

acca

rd S

imila

rity

[ %

]

0

10

20

30

40

50

60

70

80

90

100

AJSC vs JSC threshold

Proposed

CC

SCC

NPA

CECS

(a) (b) (c)

Fig. 10: Performance of the segmentation methods with different values of theJSC threshold in the nanoparticle dataset: (a) TPR; (b) PPV; (c) AJSC.

estimate the final objects contours. The experiments showed that the proposedmethod achieved high detection and segmentation accuracies and outperformedfour competing methods on a dataset of nanoparticles images. The proposedmethod relies only on edge information and can be applied also to other seg-mentation problems where the objects are partially overlapping and have anapproximately convex shape.

References

1. Park, C., Huang, J.Z., Ji, J.X., Ding, Y.: Segmentation, inference and classificationof partially overlapping nanoparticles. IEEE Transactions on Pattern Analysis andMachine Intelligence 35 (2013) 669–681

2. Zhang, W.H., Jiang, X., Liu, Y.M.: A method for recognizing overlapping ellipticalbubbles in bubble image. Pattern Recognition Letters 33 (2012) 1543 – 1548

3. Kothari, S., Chaudry, Q., Wang, M.: Automated cell counting and cluster segmen-tation using concavity detection and ellipse fitting techniques. In: IEEE Interna-tional Symposium on Biomedical Imaging. (2009) 795–798

4. Fisker, R., Carstensen, J., Hansen, M., Bødker, F., Mørup, S.: Estimation ofnanoparticle size distributions by image analysis. Journal of Nanoparticle Research2 (2000) 267–277

5. Shu, J., Fu, H., Qiu, G., Kaye, P., Ilyas, M.: Segmenting overlapping cell nuclei indigital histopathology images. In: 35th International Conference on Medicine andBiology Society (EMBC). (2013) 5445–5448

6. Cheng, J., Rajapakse, J.: Segmentation of clustered nuclei with shape markersand marking function. IEEE Transactions on Biomedical Engineering, 56 (2009)741–748

7. Jung, C., Kim, C.: Segmenting clustered nuclei using h-minima transform-basedmarker extraction and contour parameterization. IEEE Transactions on BiomedicalEngineering 57 (2010) 2600–2604

8. Zhang, Q., Pless, R.: Segmenting multiple familiar objects under mutual occlusion.In: IEEE International Conference on Image Processing (ICIP). (2006) 197–200

Page 15: Segmentation of Partially Overlapping Convex Objects Using ...

Segmentation of Partially Overlapping Convex Objects 15

9. Ali, S., Madabhushi, A.: An integrated region-, boundary-, shape-based activecontour for multiple object overlap resolution in histological imagery. IEEE Trans-actions on Medical Imaging 31 (2012) 1448–1460

10. Bai, X., Sun, C., Zhou, F.: Splitting touching cells based on concave points andellipse fitting. Pattern Recognition 42 (2009) 2434 – 2446

11. Zafari, S., Eerola, T., Sampo, J., Kalviainen, H., Haario, H.: Segmentation ofpartially overlapping nanoparticles using concave points. In: 11th InternationalSymposium on Visual Computing (ISVC 2015). (2015) 187–197

12. Zafari, S., Eerola, T., Sampo, J., Kalviainen, H., Haario, H.: Segmentation ofoverlapping elliptical objects in silhouette images. IEEE Transactions on ImageProcessing 24 (2015) 5942–5952

13. Doig, A.G., Land, B.H., Doig, A.G.: An automatic method for solving discreteprogramming problems. Econometrica (1960) 497–520

14. Principles, Examples, Clausen, J.: Branch and bound algorithms (2003)15. Koontz, W.L.G., Narendra, P.M., Fukunaga, K.: A branch and bound clustering

algorithm. IEEE Transactions on Computers (1975) 908–91516. Lempitsky, V., Blake, A., Rother, C. In: Image Segmentation by Branch-and-

Mincut. Springer Berlin Heidelberg, Berlin, Heidelberg (2008) 15–2917. Otsu, N.: A threshold selection method from gray-level histograms. Automatica

11 (1975) 23–2718. Canny, J.: A computational approach to edge detection. IEEE Transactions on

Pattern Analysis and Machine Intelligence 8 (1986) 679–69819. He, X., Yung, N.: Curvature scale space corner detector with adaptive threshold

and dynamic region of support. In: Proceedings of the 17th International Confer-ence on Pattern Recognition. (2004) 791–794

20. Wu, X., Kemeny, J.: A segmentation method for multi-connected particle delin-eation. In: IEEE Workshop on Applications of Computer Vision. (1992) 240–247

21. Wang, W.: Binary image segmentation of aggregates based on polygonal approx-imation and classification of concavities. Pattern Recognition 31 (1998) 1503 –1524

22. Loy, G., Zelinsky, A.: Fast radial symmetry for detecting points of interest. IEEETransactions on Pattern Analysis and Machine Intelligence 25 (2003) 959–973

23. Fitzgibbon, A., Pilu, M., Fisher, R.B.: Direct least square fitting of ellipses. IEEETransactions on Pattern Analysis and Machine Intelligence 21 (1999) 476–480

24. Choi, S.S., Cha, S.H., Tappert, C.C.: A survey of binary similarity and distancemeasures. Journal of Systemics, Cybernetics and Informatics 8 (2010) 43–48

25. Zafari, S., Eerola, T., Sampo, J., Kalviainen, H., Haario, H.: Segmentation of over-lapping objects. http://www2.it.lut.fi/project/comphi1/index.shtml (2016)[Online; Accessed August -2016].