Fuzzy segmentation for object‐based image classification

This article was downloaded by: [Pontificia Universidad Javeria]On: 23 January 2015, At: 07:33Publisher: Taylor & FrancisInforma Ltd Registered in England and Wales Registered Number: 1072954 Registeredoffice: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK

International Journal of RemoteSensingPublication details, including instructions for authors andsubscription information:http://www.tandfonline.com/loi/tres20

Fuzzy segmentation for object‐basedimage classificationI. Lizarazo a & P. Elsner ba Cadastral Engineering and Geodesy Department , UniversidadDistrital Francisco Jose de Caldas , Bogota, Colombiab Birkbeck College , University of London , London, UnitedKingdomPublished online: 22 Apr 2009.

To cite this article: I. Lizarazo & P. Elsner (2009) Fuzzy segmentation for object‐basedimage classification, International Journal of Remote Sensing, 30:6, 1643-1649, DOI:10.1080/01431160802460062

To link to this article: http://dx.doi.org/10.1080/01431160802460062

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all the information (the“Content”) contained in the publications on our platform. However, Taylor & Francis,our agents, and our licensors make no representations or warranties whatsoever as tothe accuracy, completeness, or suitability for any purpose of the Content. Any opinionsand views expressed in this publication are the opinions and views of the authors,and are not the views of or endorsed by Taylor & Francis. The accuracy of the Contentshould not be relied upon and should be independently verified with primary sourcesof information. Taylor and Francis shall not be liable for any losses, actions, claims,proceedings, demands, costs, expenses, damages, and other liabilities whatsoever orhowsoever caused arising directly or indirectly in connection with, in relation to or arisingout of the use of the Content.

This article may be used for research, teaching, and private study purposes. Anysubstantial or systematic reproduction, redistribution, reselling, loan, sub-licensing,systematic supply, or distribution in any form to anyone is expressly forbidden. Terms &Conditions of access and use can be found at http://www.tandfonline.com/page/terms-and-conditions

Letter

Fuzzy segmentation for object-based image classification

I. LIZARAZO*{ and P. ELSNER{{Cadastral Engineering and Geodesy Department, Universidad Distrital Francisco Jose

de Caldas, Bogota, Colombia

{Birkbeck College, University of London, London, United Kingdom

(Received 30 April 2008; in final form 8 September 2008 )

This Letter proposes an object-based image classification procedure which is

based on fuzzy image-regions instead of crisp image-objects. The approach has

three stages: (a) fuzzification in which fuzzy image-regions are developed,

resulting in a set of images whose digital values express the degree of membership

of each pixel to target land-cover classes; (b) feature analysis in which contextual

properties of fuzzy image-regions are quantified; and (c) defuzzification in which

fuzzy image-regions are allocated to target land-cover classes. The proposed

procedure is implemented using automated statistical techniques that require very

little user interaction. The results indicate that fuzzy segmentation-based methods

produce acceptable thematic accuracy and could represent a viable alternative to

current crisp image segmentation approaches.

1. Introduction

How to produce accurate thematic maps from remotely sensed data sets remains an

important challenge for the remote sensing community. This task depends both on a

robust classifier and an appropriate classification method. The effective use of

multiple features of remotely sensed data is particularly significant for increasing

thematic accuracy (Lu and Weng 2007). An emerging classification method which is

becoming popular is segmentation-based classification, also known as object-based

image analysis (Jensen 2005).

Image segmentation aims to divide an image into parts that have high

correlation with geographic objects represented in the image. A complete

segmentation of an image R is a finite set of regions R1,…,RS, which fill

completely the image space without overlapping (Sonka et al. 2008). This ‘hard’ or

‘crisp’ segmentation produces image-objects that are delimited by clearly defined

boundaries. Measurement of spectral, textural and geometric properties of image-

objects at one or several spatial scales is conducted in order to enhance the vector

of features used in the subsequent classification stage (Blaschke et al. 2006). It has

been reported that this method is often more accurate than the traditional spectral

pixel-wise classification (Blaschke et al. 2006; Carleer et al. 2005). However, the

parameterization of crisp segmentation models commonly requires significant

user-interaction, making it difficult to employ such methods for the automated

processing of large data sets. This is particularly valid for urban landscapes. These

environments are heterogeneous and complex zones which result in images with

*Corresponding author. Email: [email protected]

International Journal of Remote Sensing

Vol. 30, No. 6, 20 March 2009, 1643–1649

International Journal of Remote SensingISSN 0143-1161 print/ISSN 1366-5901 online # 2009 Taylor & Francis

http://www.tandf.co.uk/journalsDOI: 10.1080/01431160802460062

Dow

nloa

ded

by [

Pont

ific

ia U

nive

rsid

ad J

aver

ia]

at 0

7:33

23

Janu

ary

2015

mixed signals at every scale, particularly at fine spatial resolution (Herold et al.

2003; Schiewe et al. 2001). This Letter proposes an alternative fuzzy segmentation-

based classification technique that requires very little user interaction. It is centred

on building fuzzy image-regions instead of producing clearly defined crisp image-

objects (Bezdeck and Pal 1992).

Pixel-based fuzzy classification is well established in environmental remote

sensing. An example is the analysis of spatial structures of ecological systems

without reducing them to arbitrary pure classes (Andrefout and Roux 1998; Foody

1996). Fuzzy concepts have also been combined with crisp image segmentation.

Aplin and Atkinson (2001), for example, proposed a method of subpixel mapping for

per-field classification based on soft-classification and vector data. Shackelford and

Davis (2003) presented an approach for urban land-cover classification that

produced a fuzzy classification and integrated properties of hard image-objects to

increase thematic accuracy. Benz et al. (2004) used a fuzzy classifier to allocate hard

image-objects to intended land-cover classes.

Fuzzy image segmentation for geographic object-based image analysis, however, is

a less explored territory. Recent research reported the use of unsupervised fuzzy

clustering algorithms for producing vague objects for change analysis and detection

(Fisher and Arnot 2007). This Letter now focuses on exploring the suitability of a

supervised fuzzy image segmentation approach.

Fuzzy segmentation produces fuzzy image-regions which are a set of images whose

digital values express degrees of membership of each pixel to target land-cover classes.

In opposition to the single layer space of crisp image-objects, a fuzzy image

segmentation produces an n-dimensional space in which there are as many layers of

fuzzy image-regions as target classes exist. Membership degree to regions is expressed

by any value in the interval [0, 1], expressing null, partial or full membership

possibility. The higher the membership degree is for a pixel, the more a pixel belongs to

the respective target class. Once fuzzy image-regions have been established, a number

of contextual properties may then be measured to enhance the set of attributes to be

used for the subsequent image classification (Lizarazo and Elsner 2008).

A fuzzy segmentation-based classification may be conducted by splitting the

classification process into three different stages:

(i) Fuzzification: the input image is converted into fuzzy image-regions which hold

degrees of membership of each pixel to a given number of target land-cover

classes. Note that fuzzy memberships must sum to 1 for a given pixel;

(ii) Feature analysis: contextual properties of fuzzy image-regions are measured

using contextual indices representing overlap between pairs of classes which are

spectrally similar; and

(iii) Defuzzification: fuzzy image-regions are allocated to target land-cover classes

according to their membership values and contextual properties. Final output is

a single classified image.

2. Data and methods

2.1 Data

The proposed method was evaluated by classifying the University image, a

hyperspectral data set of the University of Pavia (Italy) that was collected by the

Hysens project on 8th July 2002 (Gamba 2004). The University data set size is

1644 I. Lizarazo and P. Elsner

Dow

nloa

ded

by [

Pont

ific

ia U

nive

rsid

ad J

aver

ia]

at 0

7:33

23

Janu

ary

2015

6106339 pixels with 1.2 m spatial resolution and 112 hyperspectral channels ranging

from 400 to 1260 nm (Gamba 2004). This data set was chosen because it was used in

a previous study focusing on object-based image analysis (Aksoy 2006) which makes

it possible to evaluate directly the performance of the proposed fuzzy segmentation

approach. Seven spectral bands which roughly correspond to the middle of the

Landsat Thematic Mapper (TM) multi-spectral channel range were selected as input

data for the image classification experiment. Figure 1(a) shows a false colour

composition using the composition RGB753. The University data set includes

training and testing samples for nine land cover classes: asphalt, meadows, gravel,

trees, (painted) metal sheets, bare soil, bitumen, self-blocking bricks, and shadow.

The training sample comprises 3921 pixels and the testing sample 42 762 pixels.

Figure 1(b) shows the testing sample used.

2.2 Methods

Figure 2 depicts the workflow of the proposed approach. A number of robust

algorithms were potentially available for implementing the classification process.

Non-parametric classifiers such as support vector machines (SVM), neural networks,

and decision trees have increasingly been used for remote sensing applications (Lu

and Weng 2007). Among these novel classifiers, SVM stands out because of its

generalization capabilities and use of small training samples. The generalized

additive model (GAM) is a novel regression method. SVM and GAM approaches

were selected for this research. SVM transforms the input data set into a higher-

dimensional space using special functions called kernels. SVM uses the structural

risk minimization (SRM) principle to find a separating hyperplane which minimizes

the margin between classes (Duda et al. 2001).

Figure 1. (a) False colour composition of the input data set. (b) Testing sample for thematicaccuracy evaluation. (c) Classified image using GAM-SVM method. In (b) and (c) Asphaltclass is shown in grey, Meadows in light green, Gravel in cyan, Trees in dark green, Metal inmagenta, Soil in brown, Bitumen in purple, Brick in red, and Shadow in yellow.

Remote Sensing Letters 1645

Dow

nloa

ded

by [

Pont

ific

ia U

nive

rsid

ad J

aver

ia]

at 0

7:33

23

Janu

ary

2015

Linear models are statistical models in which a univariate response is modelled as

the sum of a ‘linear predictor’. Generalized linear models (GLMs) relax the linearity

assumption by allowing the expected value of the response to depend on a smooth

monotonic function of the linear predictor. A GAM is a GLM in which part of the

linear predictor is specified in terms of a sum of smooth functions of predictor

variables. The exact parametric form of these functions is unknown as is the degree

of smoothness appropriate for each of them (Wood 2006).

Two alternative means of conducting fuzzification and defuzzification were tested

in this research project: (i) use of SVM for both fuzzification and defuzzification

(Pure SVM), and (ii) use of GAM for fuzzification and SVM for defuzzification

(Combined GAM-SVM).

2.2.1 Fuzzification. Nine fuzzy image-regions (i.e. one image-region per intended

land-cover class) were obtained by using a model with seven hyperspectral channels

as predictor variables. A one-class-against-all technique was applied to each

statistical method used as follows:

(i) Pure SVM: a SVM regression with the Gaussian radial basis function (RBF) was

used to produce fuzzy image-regions. The SVM model was tuned automatically,

by finding the optima values of parameters by five cross-fold validation in the

search space: gamma52(23 : 3) and cost52(1 : 3).

(ii) Combined GAM-SVM: A GAM regression was used to output a quantitative

prediction of every class using a binomial distribution of the seven input bands. Fuzzy

image-regions were modelled as the sum of smoothed functions of the input image.

2.2.2 Feature analysis. Contextual relationships between fuzzy regions were

obtained to resolve the inherent ambiguity of fuzzy image-regions. In this study,

Figure 2. Diagram depicting the three-stage method for land-cover classification based onfuzzy image-segmentation.

1646 I. Lizarazo and P. Elsner

Dow

nloa

ded

by [

Pont

ific

ia U

nive

rsid

ad J

aver

ia]

at 0

7:33

23

Janu

ary

2015

the absolute normalized difference index (ANDI) was calculated as follows:

ANDI~ miA{miBj j ð1Þ

where miA and miB are the membership values of the i-th individual to the classes A

and B, evaluated as the average of membership values on a 363 pixel window. The

ANDI index measures the overlap existing between two specific classes. ANDI

indices were calculated for the following pairs of classes as they were visually

identified as a potential source of spectral confusion: asphalt and gravel, asphalt and

bitumen, asphalt and bricks, meadows and trees, and meadows and bare soil.

2.2.3 Defuzzification. The eventual target land-cover classes were obtained by

SVM-based classification. Input to the defuzzification process were nine fuzzy

image-regions plus five ANDI indices described above. Output of the process is the

land-cover classification.

The proposed procedure was implemented using R, a free software environment

for statistical computing and analysis. In addition to the R base package which

provides basic statistical capabilities, the additional packages rgdal, sp, maptools,

e1071, and gam were used. They provide, respectively, functions for reading and

writing images, creating and manipulating spatial classes, and applying SVM and

GAM algorithms.

3. Results and discussion

The predictive power of the proposed fuzzy segmentation-based image analysis was

evaluated by using an earlier classification of the University data set (Aksoy 2006) as

benchmark. In that work, a three stage object-based image classification procedure

was applied. First, a segmentation step in which a total of 24 bands were used as

input for producing crisp image-objects from 8 Linear discriminant analysis (LDA)

bands, 10 principal components analysis (PCA) bands and 16 Gabor bands. This

was followed by a feature analysis step, in which a feature vector of four values was

measured for each image-object by clustering statistics from different features (i.e.

the LDA bands, the PCA bands, the Gabor bands, and 10 shape attributes). In the

final step, the allocation of pixels to land-cover classes was conducted using Bayesian

Table 1. Error matrix of the Combined GAM-SVM classification.

Map Asphalt Meadows Gravel Trees Metal Soil Bitumen Brick Shadow

Asphalt 5884 1 5 6 4 11 168 7 3Meadows 0 13313 0 57 0 590 0 20 0Gravel 115 2 1491 0 0 1 17 118 5Trees 34 2525 2 2974 0 59 0 11 0Metal 1 0 0 7 1339 33 0 0 0Soil 0 2767 10 20 0 4319 0 3 0Bitumen 261 0 1 0 0 0 1136 0 0Brick 320 41 590 0 0 16 9 3523 2Shadow 2 0 0 0 2 0 0 0 937

Total 6617 18649 2099 3064 1345 5029 1330 3682 947

PRODUCER’SACC.

89 71 71 97 100 86 85 96 99

USER’S ACC. 97 95 85 53 97 61 81 78 100

Remote Sensing Letters 1647

Dow

nloa

ded

by [

Pont

ific

ia U

nive

rsid

ad J

aver

ia]

at 0

7:33

23

Janu

ary

2015

classifiers. The overall accuracy reported by Aksoy (2006) was a percentage of

correct classification (PCC) of 84%.

In the research project presented, two classification experiments were conducted.

The Pure SVM method used SVM for both fuzzification and defuzzification and

reached a global accuracy (PCC) of 79%. The Combined GAM-SVM method used

GAM for fuzzification and SVM for defuzzification and achieved a global accuracy

(PCC) of 82%. Figure 1(c) shows the image classified using the Combined GAM-

SVM method. Table 1 shows the corresponding error matrix. It can be seen that

major misclassification rates corresponded to confusion between the classes

Meadows and Trees and Meadows and Soil, indicating the significant spectraloverlap between these classes.

Table 2 lists the results of the accuracy assessment, including PCC and KHAT

statistics (K ) as quality indicators (Congalton 1991). It can be seen that the

difference in thematic accuracy between all methods tested here is significant at the

95% level of confidence. Table 2 also shows that the work of Aksoy (2006) is slightly

more accurate than the Combined GAM-SVM method presented here.

4. Conclusions

This Letter proposed a novel fuzzy segmentation approach for object-based image

classification. The results demonstrate that the predictive power of the method is

comparable to other object-based image analysis procedures such as those of Aksoy

(2006). The procedure implemented here utilized only seven bands out of a muchlarger hyperspectral data set. This indicates that fuzzy segmentation-based image

analysis appears to have the potential to work with data sets that have much lower

spectral resolution, such as images collected by the Landsat Thematic Mapper.

Fuzzy segmentation also requires very little user interaction. This allows the

automation of the segmentation stage so that iterative time-consuming procedures

for finding the optimum scale can be avoided, a concern which remains with object-based image analysis based on crisp image-objects (Schiewe et al. 2001).

Overall, the results obtained in this study suggest that fuzzy image segmentation

could represent a viable alternative to current crisp image segmentation approaches.

Acknowledgments

The authors are grateful to Dr. Paolo Gamba from the University of Pavia, Italy, for

providing the University data set. The authors thank the referees for their valuable

comments and suggestions. The work reported in this Letter is part of PhD researchpartially funded by a Birkbeck International Research Studentship.

ReferencesAKSOY, S., 2006, Spatial techniques for image classification. In Signal and Image Processing

for Remote Sensing, C.H. Chen, (Ed), pp. 491–514 (Boca Raton: CRC Press).

Table 2. Accuracy of the proposed fuzzy segmentation approaches.

MethodPCC(%) K Variance of K Z 0.95 interval

Pure SVM 79 0.742 0.000014 195.93 [0.737, 0.756]Combined GAM-SVM 82 0.769 0.000015 201.48 [0.764, 0.784]Aksoy (2006) 84 0.796 0.000021 172.30 [0.788, 0.814]

1648 I. Lizarazo and P. Elsner

Dow

nloa

ded

by [

Pont

ific

ia U

nive

rsid

ad J

aver

ia]

at 0

7:33

23

Janu

ary

2015

ANDREFOUT, S. and ROUX, L., 1998, Characterization of ecotones using membership degrees

computed with a fuzzy classifier. International Journal of Remote Sensing, 19, pp.

3205–3211.

APLIN, P. and ATKINSON, P.M., 2001, Sub-pixel land cover mapping for per-field

classification. International Journal of Remote Sensing, 22, pp. 2853–2858.

BENZ, U.C., HOFMANN, P., WILLHAUCK, G., LINGENFELDER, I. and HEYNEN, M., 2004, Multi-

resolution, object-oriented fuzzy analysis of remote sensing data for GIS-ready

information. ISPRS Journal of Photogrammetry and Remote Sensing, 58, pp. 239–258.

BEZDECK, J.C. and PAL, S.K., 1992, Fuzzy Models for Pattern Recognition (New York: IEEE

Press).

BLASCHKE, T., BURNETT, CH. and PEKKARINEN, A., 2006, Image segmentation methods for

object-based analysis and classification. In Remote Sensing Image Analysis: Including

the Spatial Domain, S.M. de Jong and F.D. van der Meer (Eds), pp. 211–236 (Berlin:

Springer).

CARLEER, A.P., DEBEIR, O. and WOLFF, E., 2005, Assessment of very high spatial resolution

satellite image segmentations. Photogrammetric Engineering and Remote Sensing, 71,

pp. 1285–1294.

CONGALTON, R.G., 1991, A review of assessing the accuracy of classifications of remotely

sensed data. Remote Sensing of Environment, 37, pp. 35–46.

DUDA, R., HART, P.E. and STORK, D.G., 2001, Pattern Classification (New York: Wiley

Interscience).

FISHER, P. and ARNOT, Ch., 2007, Mapping type 2 change in fuzzy land cover. In Geographic

Uncertainty in Environmental Security, A. Morris and S. Kokhan (Eds), pp. 167–186

(Berlin: Springer).

FOODY, G.M., 1996, Approaches for the production and evaluation of fuzzy land cover

classifications from remotely sensed data. International Journal of Remote Sensing, 17,

pp. 1317–1340.

GAMBA, P., 2004, A collection of data for urban area characterization. In Proceedings of

Geoscience and Remote Sensing Symposium, 20-24 September 2004, pp. 69–72.

HEROLD, M., GARDNER, M.E. and ROBERTS, D.A., 2003, Spectral resolution requirements for

mapping urban areas. IEEE Transactions on Geoscience and Remote Sensing, 41, pp.

1907–1919.

JENSEN, J., 2005, Introductory Image Processing: a Remote Sensing perspective. (Upper Saddle

River, NJ: Prentice Hall).

LIZARAZO, I. and ELSNER, P., 2008, Fuzzy regions for handling uncertainty in remote sensing

image segmentation. In International Conference on Computational Science and

Applications, (ICCSA 2008), O. Gervasi, B. Murgante, A. Lagana, D. Taniar, Y. Mun,

and M.L. Gavrilova (Eds), pp. 724–739 (Berlin: Springer).

LU, D. and WENG, Q., 2007, A survey of image classification methods and techniques for

improving classification performance. International Journal of Remote Sensing, 28, pp.

823–870.

SCHIEWE, J., TUFTE, L. and EHLERS, M., 2001, Potential and problems of multi-scale

segmentation methods in remote sensing. GIS: Geo-Informations-Systeme, 6, pp.

34–39.

SHACKELFORD, A.K. and DAVIS, C.H., 2003, A combined fuzzy pixel-based and object-based

approach for classification of high-resolution multispectral data over urban areas.

IEEE Transactions on Geoscience and Remote Sensing, 41, pp. 2354–2363.

SONKA, M., HLAVAC, V. and BOYLE, R., 2008, Image Processing, Analysis, and Machine

Vision. (Toronto: Thomson).

WOOD, S., 2006, Generalized Additive Models: An Introduction with R (London: Chapman and

Hall).

Remote Sensing Letters 1649

Dow

nloa

ded

by [

Pont

ific

ia U

nive

rsid

ad J

aver

ia]

at 0

7:33

23

Janu

ary

2015