Author's personal copy - Université Lavalyvanbedard.scg.ulaval.ca/wp-content/documents/publications/500.pdf · to populate MRDB with a photogrammetric approach using MRAP is introduced

This article appeared in a journal published by Elsevier. The attachedcopy is furnished to the author for internal non-commercial researchand education use, including for instruction at the authors institution

and sharing with colleagues.

Other uses, including reproduction and distribution, or selling orlicensing copies, or posting to personal, institutional or third party

websites are prohibited.

In most cases authors are permitted to post their version of thearticle (e.g. in Word or Tex form) to their personal website orinstitutional repository. Authors requiring further information

regarding Elsevier’s archiving and manuscript policies areencouraged to visit:

http://www.elsevier.com/copyright

http://www.elsevier.com/copyright

Author's personal copy

ISPRS Journal of Photogrammetry & Remote Sensing 63 (2008) 441–460www.elsevier.com/locate/isprsjprs

Populating a building Multi Representation Data Base withphotogrammetric tools: Recent progress

Benoit Fredericquea,∗, Sylvie Daniela, Yvan Bedarda, Nicolas Paparoditisb

a Universite Laval, SCG, Canadab IGN, Laboratoire MATIS, France

Received 29 June 2007; received in revised form 18 January 2008; accepted 21 January 2008Available online 23 May 2008

Abstract

More and more frequently, the populating of 3D MRDB’s (Multi Representations Data Base) is required to support advancedcartographic applications and advanced geospatial decision analysis. We previously proposed a manual photogrammetric processbased on the Multi Representation Acquisition Pattern (MRAP) concept to tackle simultaneously Fine Level Geometries (FLG)extraction and Coarse Level Geometries (CLG) extraction. This paper presents our progress, from an automation standpoint,regarding our approach to populate MRDBs containing building geometries through a photogrammetric multiple representationsacquisition process. Two new algorithms dedicated to Multi Representation (MR) acquisition are introduced and constitute twocontributions. These combine passive imagery and Digital Surface Model analysis in order to address automation issues. The firstalgorithm allows for the automatic determination of the MRAP parameters from a single click initialization. The second algorithmaims at supporting part of the MR acquisition when MRAP are not relevant and allows the automatic building footprints extraction.This paper describes the project motivation and its actual progress. It is divided into four parts. The first part concerns the MR dataacquisition specifications. A description of the manual prototype is provided and the MRAP concept is described. In the secondpart, the first algorithm allowing the automatic determination of the MRAP parameters is introduced. The third part is dedicatedto the review of the automation performances through the study of three test sites. Finally, our second algorithm, allowing theautomatic building footprints extraction is introduced with preliminary results.c© 2008 International Society for Photogrammetry and Remote Sensing, Inc. (ISPRS). Published by Elsevier B.V. All rights reserved.

Keywords: Building extraction; Multi representation Data Base; Multi representation acquisition pattern; Graph matching; A priori knowledge

1. Introduction

Emerging web-mapping applications and SOLAP(Spatial On-Line Analytical Processing) applications

∗ Corresponding address: Departement des sciences geomatiques,Pavillon Louis-Jacques-Casault, Universite Laval, Quebec (Quebec),G1K 7P4, Canada. Tel.: +1 4186562131.

E-mail addresses: [email protected](B. Fredericque), [email protected] (S. Daniel),[email protected] (Y. Bedard), [email protected](N. Paparoditis).

(Bedard et al., 2007) have become increasinglydemanding from a map production standpoint. On-demand map production with highly customizablecapabilities is needed. In fact, these web-mappingand SOLAP applications need to manage the mapcontents at the instance level (Bernier et al., 2005).Instead of having the abstraction levels provided inthe maps stored by dataset layers, each geographicalobject should manage its own abstraction levels. Themap user interface should also support navigation

0924-2716/$ - see front matter c© 2008 International Society for Photogrammetry and Remote Sensing, Inc. (ISPRS). Published by Elsevier B.V.All rights reserved.doi:10.1016/j.isprsjprs.2008.01.002


442 B. Fredericque et al. / ISPRS Journal of Photogrammetry & Remote Sensing 63 (2008) 441–460

operations through the different abstraction levels, likedrilling-up, drilling-down or drilling across. Producingthis kind of drillable map is a complex task. Thishas motivated a lot of research work, particularlyin the field of cartographic generalization (Cecconiet al., 2002; Mackaness et al., 2007). In fact, usinga unique detailed data source (preliminarily extracted)to generate on-the-fly (using generalization) simplifiedgeometric representations of geographic objects wouldfulfill these needs. Despite a lot of research work,generalization is still time consuming and involves alarge amount of human intervention. An alternative toon-the-fly generalization is using Multi RepresentationsData Bases (MRDB), that store an explicit link betweenvarious geometries of geographical objects. Storingand linking several geometric representations of eachgeographical object in a geospatial database allowsfulfilling the needs described previously. It is worthnoticing that CityGML1 defines and manages themost relevant topographic objects in cities at severalabstraction levels. In this information model, the linksbetween the geometries corresponding to the differentabstraction levels, the Levels Of Details (LODs), arealso explicit. The use of MRDB to generate maps-on-demand has motivated a lot of research (Weibeland Dutton, 1999; Cecconi et al., 2002; Bedard, 2004;Mackaness et al., 2007; Bernier and Bedard, 2007).

When populating a MRDB, three approaches can beused to extract and link fine and coarse geometries:(1) Fine Level Geometry (FLG) extraction andgeneralization (the Coarse Level Geometries (CLG) arededuced from FLG using the generalization process)(Cecconi et al., 2002; Mackaness et al., 2007),(2) geometric and semantic matching of differentsources at different scales (FLG and CLG arealready available, the link is built through a matchingprocess) (Bernier and Bedard, 2007; Olteanu et al.,2006), (3) multi representations (MR) data acquisition(simultaneous acquisition and link of CLG andFLG). Obtaining multiple geometric representationsto populate MRDBs implies several difficulties fromthe automation standpoint and is therefore a costissue. Despite important research work to automatethem, a large amount of human interventions are stillneeded for the first two approaches. More detailedinformation on the above topics and reviews of relatedwork can be found in Fredericque et al. (2005)and Fredericque et al. (2008). The third approach,

1 CityGML is an OGC adopted best practice paper providing acommon semantic information model for the representation of 3Durban objects, http://www.citygml.org/.

which consists in defining several abstraction levelsduring the acquisition step (Fine and Coarse), is veryrecent. We are, to our knowledge, the only researchgroup to work on such an approach. Although FLGand CLG extractions are both extremely complextasks that, today, cannot be entirely automated, wethink these two major stages of populating MRDBspresent some similarities. Tackling them simultaneouslyusing a photogrammetric data acquisition approachcould be advantageous. In Fredericque et al. (2005),Fredericque et al. (2008), we proposed a multirepresentation data acquisition framework and a manualphotogrammetric tool to perform MR acquisition. Theproposed framework was designed based on our reviewof existing works addressing (1) MRDB populationand (2) buildings extraction through photogrammetricprocess. We proposed a semi-automatic strategyinvolving human interventions at the beginning ofthe acquisition process. The purpose of the humaninterventions is to introduce a priori knowledgeuseful for both FLG and CLG extraction. TheMRAP concept, formalizing a priori knowledgeabout building geometries, was introduced to supportMR acquisition. This links two existing concepts,parametric models and geometric patterns, introducedrespectively in the photogrammetric and generalizationcommunities. The MRAP concept was implemented inour manual photogrammetric tool. Since MRAP relieson parametric models, this approach is not relevant in allconfigurations. In other words, all the buildings cannotbe described a priori with MRAP. However, we thinkthat they are relevant in most cases, particularly in aNorth American context, and that they can consequentlyplay a major role in a MR acquisition system. Moreflexible approaches have to be considered to completea MR acquisition system.

This paper describes our recent progress from anautomation standpoint. We are addressing contextswhere MRAP is relevant and not. Two new algorithms,using aerial imagery and Digital Surface Models(computed from aerial imagery), are introduced. Theyrepresent two innovative contributions to the automaticbuilding extraction field of work. The first can beapplied when the MRAP concept is relevant andallows for the automatic determination of the MRAPparameters from a single click initialization. The secondalgorithm aims at supporting part of the MR acquisitionwhen MRAP are not relevant to tackle FLG and CLGextraction. This second algorithm allows for automaticbuilding footprint extraction and representation as acombination of rectangles.


B. Fredericque et al. / ISPRS Journal of Photogrammetry & Remote Sensing 63 (2008) 441–460 443

This paper is divided in four sections. The firstsection concerns the MR data acquisition specificationsand our precedent works. This section is proposedin order to better understand the approach so far,as some results have only been published in French(i.e. the manual MR acquisition prototype). In thissection, the manual prototype and the MRAP conceptare described. In the second part, the first algorithmallowing enhancement of the level of automation ofthe MR acquisition using MRAP is proposed. A briefreview of related works is also provided. The thirdpart is dedicated to the assessment of the automationperformance using MRAP and three sites have beenused to test the algorithm. Finally, our second algorithmas well as preliminary results are introduced. Thissecond algorithm aims at supporting part of the MRacquisition when MRAP are not relevant through theautomatic building footprint extraction.

2. Multi representation data acquisition: existingtools

2.1. Goal

Given a territory and a range of abstraction levels,the photogrammetric MR data acquisition process aimsat extracting and linking geometric representations ofgeographic objects. These geometric representationscan be described by traditional CAD structures, suchas line, polyline, point, etc. or by more expressivetopological GIS structures, such as surfaces, multi-surfaces, and so on (a more detailed description of thisstructure can be found in the ISO-TC19125-1 standard).As suggested by our research group (Sabo, 2007), it canalso be described using geometric patterns wheneverpossible.

The geometric pattern concept as well as its useto populate MRDB with a photogrammetric approachusing MRAP is introduced below. More detailedinformation on these concepts can be found in Saboet al. (2005a,b), Cardenas (2004) and Fredericque et al.(2008).

2.2. Geometric pattern concept

2.2.1. DefinitionA geometric pattern is defined as “a geometric object

with basic geometric characteristics that are typicaland representative of a large number of occurrencesof a mapping feature-type or of a geometric primitiveand that is able to adapt itself to the geometry ofthese occurrences of object at different scales and that

can be reused several times” (Bedard, 2004; Cardenas,2004). A geometric pattern consists of primitives thatcorrespond to the indivisible atomic elements it is madeof.

2.2.2. PurposeGeometric patterns can be used to describe the

geometries of an object stored in a database (DB). Theuse of geometric patterns enhances the integrity of aDB by reducing information redundancy. Moreover, theuse of geometric patterns provides several advantagesfrom the generalization standpoint. In Sabo (2007),the author suggests the combination of geometricpatterns with simple generalization algorithms andgeneralization constraints to obtain Self-GeneralizingObjects (SGO). SGO can be abstracted as softwareagents that perform generalization operations throughthe manipulation of the geometric pattern implantationparameters. For example, an exaggeration operation,applied on a rectangular building that is defined witha geometric pattern, is simply performed by increasingthe building width and length parameters. A SGO canproduce geometric representations at several arbitraryabstraction levels (which is impossible when usingonly MR stored in DB). It can also take into accountthe relations between different geographic objects. Thereader should refer to Sabo (2007) for more detailsabout SGO, their use and the quality of the resultingcartographic representations.

2.2.3. Links with the parametric model conceptThe literature review we conducted reveals proximity

between the 2D2 geometric pattern concept usedto define CLG and the parametric models conceptdefined to capture FLG. Geometric patterns have beenintroduced by cartographers and database specialistsfor generalization purposes while the parametric modelhas been introduced to support photogrammetric datacapture. Despite these two different origins, theseconcepts display strong similarities. In fact, bothconcepts use a library of a priori defined shapesto describe the geometries of geographical objects.The geometry description is carried out by definingspecific implementation settings for these shapes. Likegeometric patterns, parametric models can be used toreduce redundancy in a geospatial database and toensure MRDB integrity. Several differences still remainbetween these two concepts. The differences can be

2 We consider a geometric object as 3D as soon as its geometrydefines a volume and as 2D when its geometry is only defined in aplane.



Fig. 1. Basic volumetric primitive types: (a) flat roof, (b) pent roof,(c) gable roof, (d) hip roof.

categorized according to the three following aspects: (1)their dimensions (2D for the geometric patterns and 3Dfor the parametric models), (2) the addressed levels ofabstraction (geometric patterns address CLG definitionwhile parametric models address FLG), (3) the prioritygiven to the geometric precision (the geometricpatterns defined by Sabo (2007) voluntarily favor thegeneralization speed and data volume reduction whilethe parametric models favor the geometric fidelity).

2.3. MRAP concept

As mentioned before, our goal consists in populat-ing MRDB using, when possible, geometric patternsand parametric models. Following such an approach,populating MRDBs requires, for each level of abstrac-tion, to identify, implement, and link geometric patternsand parametric models corresponding to geographicalobjects. Eventually, generalization algorithms and gen-eralization constraints can be defined to manage re-lations between spatial objects. This can be long andmonotonous if done manually. To facilitate this process,we introduced the concepts of Multi Representation Ac-quisition Pattern (MRAP).

Table 1Source images characteristics

Charlesbourg Beauport Montreal

Scale 1/5000 1/8000 1/4000Camera type Digital Film FilmFocal length 120 mm 152 mm 306 mmPixel size 12 microns 21 microns 14 microns

The principle of MRAP is to define a library ofobjects consisting of several 3D parametric modelsand 2D geometric patterns, and eventually consistingof generalization algorithms and constraints. Eachparametric model and geometric pattern of the MRAPcorresponds to a single predefined abstraction level. Allthe parametric models and geometric patterns includedin a MRAP are linked to each other. Knowing theparameters of the more detailed parametric model ofthe MRAP allows for inferring the implementationparameters of the other parametric models andgeometric patterns included in the MRAP. Whencreating a new MRAP, the computation method of thecoarse level parameters using the fine level parametersmust be specified by a cartographer during an a prioristep. Fig. 2 represents some MRAPs identified in ourtest sites (defined in Table 1, Section 4.1).

Four types of basic volumetric primitives can be usedto create a 3D parametric model. They are illustrated inFig. 1. When defining a geometry with such volumetricprimitives (combined in a parametric model), we needto define its global parameters (i.e. implementationpoint and orientation in the horizontal plane) and itsshape parameters. Shape parameters of the volumetricprimitive number two are also represented (w: width, l:length, α: roof slope, h: building height).

Fig. 2. MRAPs examples.



Fig. 3. Examples of results obtained with the photogrammetric multi-representations acquisition system manual prototype. (a) image sources (b)(c) and (d) different levels of abstraction.

MRAP is an extension of the Multi-Scale Patternconcept, proposed in Cardenas (2004). The latter isrestricted to 2D and tackles only CLG geometryextraction. Moreover, it involves neither constraints norgeneralization algorithms. Adding those constraints andalgorithms to the MRAP allows for supporting the SGOconcept (currently under development in our researchgroup) (Sabo, 2007). Linking the multi-scale patternconcept with the parametric model makes the MRAPusable in the FLG photogrammetric acquisition context.

Data acquisition with MRAP consists in implement-ing (i.e. setting the following parameters: anchor point,rotation, height, . . . ) the most detailed 3D parametricmodel of the MRAP. It is also possible to define groupsof geographic objects that must fulfill the same general-ization constraints (e.g.: the buildings belonging to thegroup must all be lined up during the generalization) orthe same generalization rule (e.g.: the buildings belong-ing to the group must all be aggregated up during thegeneralization).

The example displayed in Fig. 3 shows how theMR acquisition procedure provides several abstractionlevels. They range from a fine geometry per building toa coarse geometry for a group of aligned buildings. Inthis example, three building groups fulfilling the sameaggregation rule have been built. More specifically,Fig. 3(d) illustrates how an aggregation rule, introducedduring the capture, can be used to aggregate objects andthen produce more abstract levels. If we refer to theCityGML initiative, Fig. 3(b) corresponds to LOD2 andFig. 3(c), (d) correspond to LOD1.

Acquisition results, involving the use of MRAPs 1,4 and 5 (cf. Fig. 2), have been obtained using ourmanual prototype (Fredericque et al., 2008) on theBeauport test site (test site description is provided atthe end of the paper, Table 1; Section 4.1). The humanoperator can select the relevant MRAP model using the

corresponding tool selection designed in the prototype.MRAPs can be grouped during the acquisition processif they have to fulfill the same generalization constraintor rule. The manual MR acquisition prototype allows fordetermining the MRAP parameters from the selection ofspecific points in the 3D ground space (e.g. the two firstpoints provide the length and the rotation angle of theMRAP). A human operator performs this selection in astereoscopic view. One of the points must be located atthe ground level. The other points must refer to specificvertices of the MRAP’s finest level. For example manualMR acquisition with MRAP number five (cf. Fig. 2)requires the selection of five points. This can involveup to seven mouse clicks if the corresponding tool mustbe selected and a building group must be defined. Thisnumber of clicks increases rapidly with the number ofMRAP parameters. The first algorithm proposed in thenext section aims to reduce the number of requiredpoints for the MRAP acquisition.

2.4. The limits of MRAP use

As mentioned before, we are aware that MRAPsare not relevant to all building geometries and thatmore flexible approaches must be considered totackle complex building representations. Some buildingelements (e.g. roof window) are not always includedat the finest level of the MRAP. It does not mean theMRAP cannot be used to extract the other buildingabstraction levels. More specifically the fact that thereis no parametric model to describe the finest leveldoes not imply that parametric models and geometricpatterns cannot be used for coarse levels. The coarserthe abstraction levels, the more relevant parametricmodels and geometric patterns are. Practically, it wouldconsist in extracting a detailed geometry through anindependent process (manual or semi-automatic) and



then linking this detailed geometry to a MRAP. The linkbetween the different coarse levels and the parameterswould be automatically computed through the MRAP.Regarding the finest level geometries, they could bedefined using different geometric structures like Brepor volumetric primitives combined in CSG. Ideally, theCSG structures should be favored for database integrityoptimization and to facilitate generalization operations(cf. 2.2.2).

To summarize, acquiring complex buildings in MRwould involve two major components (1) the detailedgeometry extraction, and (2) linking the detailedgeometry and the MRAP and then inferring the MRAPparameters. The second algorithm proposed hereafteraims at supporting detailed geometries extraction andto facilitate the matching between detailed geometriesand MRAPs.

3. Semi-automatic MR acquisition with orthogonalMRAPs

The following section deals with contexts whereMRAPs are detailed enough to represent the finestgeometry level of the buildings. The automaticdefinition of MRAP parameters (i.e. implantationparameters of the most detailed parametric model)follows the selection of the MRAP type and thebuilding’s approximate location by the human operator.This issue corresponds to the determination of theparameters of the finest parametric model encapsulatedin the MRAP. All identified MRAPs are orthogonal(i.e. they consist of primitives that are orthogonalto each other). This a priori knowledge about thevolumetric primitives’ spatial relations will be useful forthe extraction of their implementation parameters.

3.1. Works related to the parametric model parametersdetermination

Several research works have been dedicated to thedetermination of the parametric model parameters.Some of them are described hereafter. In fact, a prioriknowledge regarding building geometries can easily beformalized and introduced using parametric models. Apriori knowledge is a key element in the performance ofbuilding extraction algorithms (Baltsavias, 2004). Ourglobal strategy, involving a top-down approach and theuse of a priori knowledge, relies on this statement.

Approaches using parametric models can be classi-fied in two categories. The first consists in automati-cally identifying the parametric model type as proposedin Haala and Brenner (1999), Suveg and Vosselman

(2004), Lafarge et al. (2006) and Ortner et al. (2007).The second requires a human intervention in order tospecify the parametric model such as in Gulch et al.(1999), Vosselman and Veldhuis (1999), Rottensteinerand Schulze (2003) and in Tseng and Wang (2003).

Works of the first category look for a full automationeven if some authors also provide semi-automatic toolsto post-process failure cases. Parametric models arethen used to generate and evaluate hypotheses aboutthe building geometries. Hypotheses are evaluated bycomparing the hypothetical geometries with referencedata sources consisting mostly of aerial images andDSM. DSM are more and more used since, asmentioned in Brenner (2005), information extractedfrom DSM simplifies automatic reconstruction issues assoon as this information is already in the object space.Furthermore, the recent progress of LIDAR sensors andaerial digital cameras in the last decade has improvedthe quality of the available DSM. Vector data canbe used in addition to aerial images and DSM. InHaala and Brenner (1999) and Suveg and Vosselman(2004), building footprints are used jointly with a DSM.The proposed algorithms, using volumetric primitivecombinations, rely on the segmentation of the buildingfootprints. Gerke et al. (2001) and Vinson and Cohen(2002) proposed two automatic approaches to extractbuilding footprints. Both strategies assume that buildingfootprints can be described as rectangle combinations.Rectangles can be considered as parametric modelsrestricted to the horizontal plane. These two strategiescorrespond to a generalization of the inertial momentsbased on the Maas (1999) method.

Semi-automatic approaches using parametric mod-els, as proposed in Gulch et al. (1999), Vosselman andVeldhuis (1999), Rottensteiner and Schulze (2003) andin Tseng and Wang (2003), require a human to inter-vene at the beginning of the process. The purpose ofthis intervention is to identify the parametric model typeand to define approximate parameters for the model.Accurate parameters are then computed automatically.The human operator can also combine parametric mod-els to deal with buildings having a complex geometry.Parametric models are restricted to those having quadri-lateral footprints, generally rectangles. Approximatedparameters definition requires several clicks. For ex-ample, in Rottensteiner and Schulze (2003) three spe-cific points are needed. Fine parameters are computedthrough optimization procedures that consider consis-tency between parametric model position and referencedata. For example, in Rottensteiner and Schulze (2003)and in Tseng and Wang (2003) aerial image data is usedas reference data and parametric model wireframes are



Fig. 4. DSM generation with a max flow strategy. (a) DSM (b) source image.

projected to edge images. Maas (1999) proposed a strat-egy quickly providing approximate parameters from aDSM region, a parametric model type and a uniqueapproximated point. This strategy is based on inertialmoments and can only be used with elementary para-metric models (composed of a single volumetric prim-itive). Those cannot be used to combine primitives inCSG unless the regions of interest in the DSM, corre-sponding to each primitive, are specified earlier. Fur-thermore, as mentioned in Gerke et al. (2001), inertialmoments are unstable if the shape of a building foot-print is very similar to a square. The strategy proposedin Maas (1999) still has the major advantage of beingquick and not demanding from the human interventionstandpoint. These two aspects are particularly relevantin a semi-automatic context where response times mustbe short to keep the human operator from waiting, andwhere human intervention should be reduced as muchas possible.

Most of the existing approaches, fully or semi au-tomatic, use elementary parametric models (composedof only one volumetric primitive). They combine theseelementary parametric models to describe complex ge-ometries. Therefore, few algorithms are available todetermine the approximate parameters of complexparametric models (composed of several volumetricprimitives combined in a specific way) from a restrictednumber of approximated points. Strategies such as clus-tering or RANSAC (Fischler and Bolles, 1981) used foran elementary parametric model (Gulch et al., 1998) canhardly be used for complex parametric models as soonas the number of unknown parameters increases. This isparticularly true in a semi-automatic context (i.e. need-ing real time processing). Existing algorithms comput-ing accurate parameters from approximated values arestill relevant insofar as each primitive optimization canbe addressed independently.

3.2. Proposed algorithm for the orthogonal MRAPparameters determination

The speed of the automatic process is of greatimportance in our semi-automatic context. In fact,implementation parameters must be established on-the-fly. Several of the MRAP involve complex parametricmodels (involving several volumetric primitives). Thiscomplexity may imply many parameters to optimizeand increase the automation difficulties. Moreover, tolimit human intervention, we look for an initializationstage consisting in only introducing an approximatepoint. We restricted the spectrum of our investigationto the estimation of approximated parameters of acomplex MRAP from an approximated point. In fact,as mentioned before, several contributions alreadytackled the accurate parameter determination from anapproximated initialization.

3.2.1. Strategy for the determination of parameters:Extraction and selection of primitives

We have introduced a new approach using passiveimagery and DSM to extract the parameters oforthogonal parametric models (the more detailedparametric model included in an orthogonal MRAP).

A DSM is automatically produced from stereoscopiccouples by combining area-based matching techniques(correlation from object space) and optimal surfaceresearch in a graph (minimum cut research). Introducedby Roy (1999), this approach allows for a spatialhomogeneity notion to be included during the DSMcomputation process. Thus, it uses a priori knowledgeregarding the area continuity and allows for enhancingthe robustness of the DSM extraction. The reader canrefer to Pierrot-Deseilligny and Paparoditis (2006) toget a detailed description of this DSM productionmethod. Fig. 4 is an example of an image source andits corresponding DSM generated using the proposed



Fig. 5. Running example, (a) Source Image (b) nDSM (c) ROI (d) corresponding MRAP.

strategy. This example refers to our second test site(Charlesbourg) described in Table 1, Section 4.1.

Our strategy to automatically extract the paramet-ric model parameters from a single approximated pointdivides the optimization problem into several sub-problems to solve it quickly. This strategy considersa parametric model as a particular volumetric primi-tive combination. The parametric model parameters’ re-search is then addressed by extracting the parameters ofeach volumetric primitive. This strategy involves threesuccessive steps: (1) determining a Region Of Interest(ROI) in the ground space including the building to beextracted, (2) extracting the best volumetric primitivescombination according to the parametric model, and (3)deducing the parametric model parameters from the pa-rameters of each primitive and the connectivity relationsbetween these primitives.

The ROI is extracted using a normalized DSM(nDSM). The nDSM is computed by subtracting theDigital Terrain Model from the (DSM). The nDSMis segmented using a region-growing algorithm basedon the height analysis (Weidner, 1997; Ameri, 2000).The height threshold is defined according to the apriori knowledge regarding the minimum height of thebuilding. The DTM is deduced from the DSM throughthe application of a morphological filter (Weidner,1997; Sternberg, 1983). The DSM, nDSM and ROI aredescribed using a raster format. Examples of sourceimage, nDSM, ROI and corresponding MRAP areshown in Fig. 5. Fig. 5 will be reused in this paper asa running example to illustrate several processes.

Step 3 (i.e. deduction of parametric model parame-ters from each primitive parameters and their connec-tivity relations) is a coordinate system transformationfrom a Cartesian system to a parametric system in theterrestrial reference system. This is easily performedas soon as extracted primitives are matched with para-metric model primitives. This matching process is per-formed during step 2.

Step 2, (i.e. extraction of the best volumetricprimitive combination according to the parametricmodel) is the more complex step and is carried out on-

the-fly. The new approach we propose is a contributionto this research field. It consists of three sub-steps: (2.1)expression of the ROI as an orthogonal combinationof rectangles, (2.2) selection of the best orthogonalcombination of rectangles according to the parametricmodel footprint, (2.3) extrusion and segmentation. Sub-steps 2.2 and 2.3 are iterated if sub-step 2.3 failed and ifsub-step 2.2 succeeded.

Therefore, step 2 successively addresses the plani-metric data extraction and the altimetric data extraction.This choice is based on the fact that ROI extractionfrom nDSM is robust. Moreover, this strategy aims atdecreasing the size of the research space in order to re-duce the processing time. In fact, a short computationtime is needed to support semi-automatic data capture.The processes involved during step 2 and its sub-stepsare detailed hereafter. The global process is also synthe-sized in the UML activity diagram proposed in Fig. 6.

Sub-step 2.1 consists of converting the ROI from araster form to an orthogonal combination of rectangles.Again, the developed approach is considered as acontribution to this research field. The principle of theapproach and its two main tasks are described in thefollowing section.

3.2.2. ROI conversion strategyThe principle of the ROI conversion strategy is to

generate a collection of axes that are subsequentlyused to generate rectangles through a dilatation process.An axis is defined by a point and a direction, andthe dilatation process aims at defining the rectangledimension around this point. Starting from the fact thatthinning methods generate axes with directions that arestrongly affected by the noise of the shape boundaries(Parker, 1997) and that the positions of the axes arerelatively robust, we have proposed a mixed strategyto express the ROI as a rectangle combination. Thisstrategy uses both the raster ROI and the edge imagesto express the ROI as an orthogonal combination ofrectangles. The well-known Canny filter (Canny, 1986)is used to compute the edge images from the sourceimages. We first extract rectangle axes (defined with a



Fig. 6. UML activity diagram of the global process.

point and an orientation) and then rectangle dimensionsaround these axes (Half Forward Length (HFL), HalfBackward Length (HBL), Half Right Width (HRW),and Half Left Width (HLW)). We use the edge imagesto extract the axes direction and the ROI skeleton todefine the approximate location of the axes. The Zhang-Suen (Zhang and Fu, 1984) thinning algorithm is usedto compute the skeleton from the raster ROI.

Rectangles are then defined using these axes (toinitialize the rectangles) and the raster ROI (to delineatethe rectangles around the axis). Such a strategy isrelevant since the building direction is accurate in theedge images while the ROI boundaries are noisy. Theedge images cannot be used to improve the rectangleboundaries. In fact, it would be necessary to distinguishedges near the building boundaries from the edgesbelonging to the inner roof structure. Edge imageswill be used during the extrusion process to improvethe boundary locations of the volumetric primitives. In

fact, at the extrusion step, roof structure knowledge isavailable and it can be used to select relevant edges.The ROI conversion strategy is synthesized in theUML activity diagram of Fig. 7. Both the extraction ofdirection and the rectangle definition from the axis aredescribed hereafter.

3.2.2.1. Determination of the principal direction. Theprincipal direction is defined between 0◦ and 90◦

and corresponds to volumetric primitive directions(primitives are orthogonal). The principal direction isdeduced from the edge images using the statisticalanalysis of the edge directions included in an analysisarea. The analysis area is deduced from the ROI throughits dilation according to the DSM accuracy. In theanalysis area, edge images are orthorectified using theaverage altitude in the ROI. This strategy is usable if thecamera focal that is used for image acquisition is shortenough to confine edges inside the analysis area. In fact,



Fig. 7. UML diagram activity describing ROI conversion algorithm.

the Z-variation of the roof structure edges should notinvolve a delocalization of the building edges outsidethe analysis area. This strategy is not only very simpleto implement but it is also very fast. It is based on thea priori knowledge that most of the edges are in theprincipal direction of the building (i.e. the building roofstructures yielding to edges in the images are horizontal)and that the determination of the edge direction isrobust.

3.2.2.2. Generation of the rectangles from the approxi-mate axes. The axes are generated from both the skele-ton and the orthogonal MRAP principal direction. Foreach line of the skeleton we create two axes defined bya point and an orientation. We use the middle point ofthe skeleton line to define the axes. The orientations ofthe two axes refer to the building principal direction andto the building principal direction plus 90◦, respectively.

For each axis, a heuristic, illustrated in Fig. 8,which allows for the estimation of the largest orientedrectangle included mainly in the ROI, is used. Therectangle should include the axis point. A rectangle isdefined by a point and five other parameters: orientation,Half Forward Length (HFL), Half Backward Length

(HBL), Half Right Width (HRW), and Half Left Width(HLW).

The heuristic involves three main steps describedhereafter and is illustrated in Fig. 8:

(1) First estimation of HFL and HBL• Intersection of the axis and the ROI boundaries• Creation of segment S1 with the two shortest

points in the forward and backward directions.(2) Determination of HRW and HLW

• Statistical analysis (first quartile) of the distancesbetween the points belonging to the ROIboundaries and their orthogonal projection onsegment S1

• Note: only orthogonal projections inside segmentS1 are used.

(3) Final adjustment of HLF and HBL• Creation of segment S2 from a point (axis point),

a direction (perpendicular to S1), an Half ForwardLength (corresponding to the previous HRW) andan Half Backward Length (corresponding to theprevious HLW).

• Statistical analysis (first quartile) of distancesbetween the points belonging to the ROIboundaries and their orthogonal projection onsegment S2.



Fig. 8. Rectangle generation from axis.

Fig. 9. Illustration of selecting the best orthogonal combination of rectangles.

3.2.2.3. Selection of the best orthogonal combinationof rectangles. Finding the best orthogonal combinationof rectangles comes after the expression of the ROI asan orthogonal combination of rectangles. This researchis based on an a priori knowledge about the buildingfootprint (e.g. L shape or T shape). This a prioriknowledge, specific to the instance, is input throughthe identification of the MRAP type and through theapproximated point input. The approximated point linksthe MRAP type to the ROI.

The best orthogonal combination of rectanglesresults from matching and evaluating the combinationof rectangles with the parametric model footprint.Since graph structures are relevant when performingtopological comparisons, the matching process isperformed according to a graph-based strategy (researchof the graph and subgraph isomorphism). Eachrectangle is a graph node and each overlap betweentwo rectangles is a graph edge. This definition of thegraph edge depends on the relative positions of the axesof the two overlapped rectangles. The best subgraphmaximizes a score function and must overlap theapproximated point introduced by the human operator.

We have used the area generated by the rectangle’sgraph footprint (logical OR) as the score function.Other score functions could be used. For example, inOrtner et al. (2007), authors check the consistencybetween the theoretical roof structure (defined as avolumetric primitive) and the DSM in order to evaluatethe relevance of the solution. To identify the best sub-graph we use a heuristic assuming that the best sub-graph likely includes the biggest rectangle overlappingthe approximate point. All the possible sub-graphsincluding this node are evaluated. If no solution isfound, the second best rectangle using the approximatepoint is then used. The same principle is iterateduntil a solution is found. Fig. 9 illustrates the graphmatching process used to compute the best combinationof rectangles

3.2.3. Extrusion and segmentationThe selected combination of rectangles describes

the initial building footprint. The volumetric datais extracted during the extrusion step. It uses bothpreliminary extracted rectangles and knowledge aboutthe volumetric type. This knowledge is attached tothe rectangles during the graph-matching process



Fig. 10. Extraction of faces for a rectangle included in ROI.

(knowledge example regarding the roof structure: gableroof). On the one hand, extrusion aims at extractinginner roof structure. On the other hand, it aims atextracting building height. Additional elements, such astrees and neighboring buildings, can be included in thefootprint. They are excluded during the segmentationsteps.

Thus, each rectangle included in the selected sub-graph is extruded and segmented in order to generatea volumetric primitive. Extrusion and segmentationprocesses are specific to the volumetric type. Rectanglesare extruded according to the volumetric primitivetypes. For each type of volumetric primitive, extrusionand segmentation processes exploit faces, defined in the3D object space, that have been preliminary extracted.The faces extraction is performed in the same way foreach volumetric primitive type. A score is attached toeach face during its extraction.

Segmentation relies on topographic and radiometricdiscontinuities. In fact, the ROI analysis is not sufficientto delineate a volumetric primitive. A first segmentation,based on topographic discontinuities, is performedduring the extrusion step. Then, a second segmentation,based on the image’s edge analysis, is carried out.Rectangle position in the best sub-graph is consideredduring the segmentation in order to maintain theselected graph consistent with the parametric modelfootprint. As mentioned before, the selected graph mustdisplay the same topological structure as the parametricmodel footprint (e.g. T shape) and must overlap theapproximate point.

The extrusion and segmentation processes aredescribed hereafter. Extruded roof boundaries areprojected to the ground in order to compute thevolumetric primitive height.

3.2.3.1. Extraction of faces. All the faces are extractedfor each rectangle used to generate a volumetricprimitive. A 3D-point cloud is computed from theDSM. For this purpose, we define a regular grid insidethe rectangle to be extruded. The XY coordinatesof the points are computed through the rectanglediscretization and the Z coordinate is deduced from theDSM. The planes are computed from this point cloudusing a RANSAC strategy followed by a least squarecompensation. The face creation consists in delimitingeach plane with a polygon. The polygon is deducedfrom the oriented bounding box of the selected cloudpoints. The initial bounding box is enlarged using abuffer defined according to the DSM accuracy and thegrid-sampling step. All the extracted planes through theRANSAC strategy are kept but a score is attached toeach face depending on the face area and the number ofpoints used to extract the plane. Fig. 10 illustrates theextrusion process for a gable roof volumetric primitive.This primitive refers to the horizontal part of the runningexample T shape.

RANSAC is a reliable approach that can be easilyimplemented to extract planes. However, inadequatesettings or particular configurations can lead to methodfailure. The threshold criterion used to accept or rejecta point according to its distance to the current planemust be set carefully. In fact, a threshold that is toosmall will lead to the extraction of a lot of verticalplanes parallel to the sampling direction of the grid. Athreshold that is too high will affect the accuracy ofthe extracted plane because outlier points will be usedfor the least square adjustment of the plane equation.Moreover, these points will be rejected during the nextcomputation and they will not be involved in other planecomputations. Consequently, an efficient extraction ofthe faces relies on the knowledge of the DSM accuracy.



Fig. 11. Extrusion of a flat roof primitive. Volumetric primitive parameters are then deduced. Implementation point, orientation, width andlength are deduced from the rectangle. The selected face is then used to define building height (vertical distance between face and DSM at theimplementation point) and roof slope (face slope).

3.2.3.2. Extrusion of flat roof and pent roof primitiveand segmentation from a topographic standpoint.Extrusion of flat roof and pent roof primitives(primitives 1 and 2, Fig. 1) needs to select the bestface among all the extracted faces. This face selectiondepends on both the graph topological constraint(consistency between the parametric model and theoverlap with the approximate point) and the scoreattached to the face. If the volumetric primitive to beextruded is a flat roof primitive (primitive 1, Fig. 1), theselection is restricted to horizontal faces.

The selected face is used to segment the primitivefootprint by keeping only the overlapped area betweenthe face footprint and the initial rectangle. Thissegmentation is performed according a topographicdiscontinuity criterion. A DSM section correspondingto several flat roof primitives is shown in Fig. 11. Therectangle displayed on Fig. 11(c) represents the initialrectangle. Rectangle produced by the segmentation isshown on Fig. 11(e). This example is extracted from ourBeauport test site.

Volumetric primitive parameters are then deduced.Implementation point, orientation, width and length arededuced from the rectangle. The selected face is thenused to define building height (vertical distance betweenface and DSM at the implementation point) and roofslope (face slope).

3.2.3.3. Extrusion of a gable roof primitive and seg-mentation from a topographic standpoint. Extrusion ofa gable roof primitive (primitive 3 Fig. 1) needs to selectthe best two faces that intersect each other in a consis-tent way according to the volumetric primitive shape.In fact, the intersection of the two faces must corre-spond to a peak and not to a thalweg or a simple slopediscontinuity. This face selection depends also on boththe graph topological constraint (consistency betweenthe parametric model and the overlap with the approxi-mate point) and the score attached to the faces. The faceselection is performed through the identification of the

best peak 3D edge.3 A 3D edge is computed for each in-tersecting faces pair. These 3D edges are then classifiedinto two categories: peak and discontinuity. A 3D edgeis a peak if the Z values of its two end points are higherthan the Z values of the two faces’ centroids. All otheredges belong to the discontinuity class.

A score function, using the score attached to the twofaces and the 3D edge length is then used to select thebest peak edge. Only peak edges displaying the samedirection as the principal direction of the rectangle areconsidered. The segmentation of the footprint, based ontopographic considerations, follows the same principleas the flat roof case, where we keep only the overlaparea between the oriented bounding box of the two facesfootprint and the initial rectangle.

The selected peak edge and the segmented footprintare then used to deduce the volumetric primitiveparameters. The rectangle is used in a similar wayas for flat roof primitive. The 3D edge is used todefine roof structure parameters. The peak height isdeduced from the 3D edge middle point height and thepeak eccentricity is deduced from 3D edge distance torectangle boundaries.

3.2.3.4. Extrusion of a hip roof primitive and segmen-tation from a topographic standpoint. A hip roof prim-itive is processed as a particular gable roof primitivewith an additional parameter. This parameter defines theshift between the beginning of the peak and the primi-tive footprint. It is computed by adjusting the peak edgewith the normal faces (the orientation of the normal vec-tor of the face in the XY plane is the same as the peakedge).

3.2.3.5. Radiometric based segmentation. A volumet-ric primitive is delimited not only by its 3D shape (to-pography discontinuity in the DSM), but also by edges

3 For the sake of simplicity, we refer to 3D edge when the edge isdefined in a 3D object space and to 2D edge when this is defined in a2D space like an image.



Fig. 12. Volumetric primitive boundaries examples.

and/or changes in the spectral properties. According tothis definition, there are seven volumetric primitives inFig. 12 (considering small details in the roof boundariesas negligible). In order to acquire one of these sevenconnected buildings, we need to consider not only thetopographic discontinuities but also the radiometric dis-continuities.

Fig. 13(a) illustrates the face extracted in the contextof Fig. 12 image when the radiometric discontinuityis not considered. In order to extract the volumetricboundaries, additional processing is required afterthe extrusion and topographical segmentation. Thisprocessing consists in segmenting the volumetricboundaries using the edge images. For this purpose, welook for 3D edges in the object space that delineate theprimitive footprint as a rectangle. More specifically welook for 3D edges that are (1) oriented according to theprincipal direction, (2) on the roof 3D faces (i.e. on theextracted and selected faces when defining the primitiveroof structure at the previous step).

Using knowledge about the roof structure type, wegenerate 3D edge candidates (oriented in the principaldirection and on the roof 3D faces) by setting theirlength to the initial rectangle dimensions and bydiscretizing the candidate space according to the DSMground resolution. Fig. 13(b) and (c) illustrate the 3Dedge candidates generated in the context of Fig. 12image. The relevance of the 3D edge candidates isthen assessed using the edge image. The 3D edge is

projected on the edge image. The discrepancy betweenits position and the position of the detected 2D edges inits vicinity provides a measure of the 3D edge relevance.The number of edge pixel in this neighborhood isdirectly used as a measure function. The neighborhoodis defined according to the DSM accuracy and the edgeimage resolution.

The radiometric based segmentation then consistsin finding the 3D edges with a measure higher than afixed threshold. The research is performed according tofour directions (principal direction modulo Pi) aroundthe approximate point, starting from the nearest 3Dedge to the furthest. For each direction, the searchstops as soon as a measure value higher than thethreshold is found. If the approximated point is notinside the volumetric primitive (when the parametricmodel consists of several primitives), a point definedcontextually is used. If the roof type is gable or hipwe use a point on the peak axis (the peak was selectedduring the previous step). If the roof type is flat or pentand if the footprint is a Tshape we use a point on themedial axis of the initial rectangle, etc.

4. Experimentation

4.1. Test sites

The processes used to automate the determination oforthogonal MRAP parameters have been carried out onthree test sites in the province of Quebec. The first twosites are near Quebec City (Beauport and Charlesbourg)and the third is on the Island of Montreal. These threetest sites consist of medium-density residential areas(relatively small buildings, less than 4 floors). Thiskind of urbanization is a typical North-America suburbconfiguration. The orthogonal MRAPs are relevantin such a context. The characteristics of the aerialimages we used are described in Table 1. DSMs weregenerated with a 25 cm ground resolution because wewere only looking for approximate determination ofthe MRAP parameters. Furthermore, having the same

Fig. 13. Edge image based segmentation. (a) DSM with initial face, (b) and (c) DSM with 3D edge candidates, (d) adjusted face.



Fig. 14. Results example. (a) coarser level with building groups, (b) a coarse level, (c) fine level, (d) source image.

ground resolution for all the DSMs will facilitate thecomparison of the various results.

4.2. Results

We compared the parametric models that were au-tomatically implemented with the manual implementa-tion (using stereoscopic images) in order to validate theautomatic determination of parameters. Results are con-sidered valid when: (1) the difference between the auto-matically and the manually positioned edges are smallerthan three times the DSM resolution, and (2) the angulardifference between the two principal directions (i.e. au-tomatically and manually extracted) is smaller than 3◦.These criteria were based on our review of works re-lated to fine parameter determination from the man-ual introduction of approximated parameters (ex. resultsin Gulch et al., 1999).

Our extraction strategy of the initial MRAP’sparameters relies essentially on the DSM. Thus,the resulting accuracy is intrinsically dependent onthe DSM accuracy. The resulting accuracy couldbe improved with a better DSM. The implementedprototype involves computation times smaller than 1second per volumetric primitive using a laptop with thefollowing characteristics: Intel Centrino 1.7 GHz, 512Mo RAM. Some of the processes (e.g. face extraction,edge direction computation) should be computed duringa preliminary step in order to decrease the on-the-fly time processing. The table below synthesizes theobtained results. Fig. 14 illustrates the result obtainedon the Charlesbourg test site.

4.3. Analysis

The implemented processes dedicated to the extrac-tion of MRAPs’ parameters have been assessed on three



Table 2Results for the three test sites

Charlesbourg Beauport Montreal TotalNumber % Number % Number % %

Total number of tested buildings Line1d.

107 100 76 100 130 100 100

Failure during initial footprintextraction

ROI detection 1 0 0 4 5.3 0 0 1.3Principal direction detection 2 4 3.7 0 0 10 7.7 4.5Rectangle extraction according toparametric model (graph matching)

3 2 1.9 3 4 4 1.5 2.9

Failure during extrusion andsegmentation

DSM resolution is not enough to extractelement that are differentiable in images

4 1 0.9 0 0 0 0 0.3

Noise in the DSM surfaces 5 0 0 10 13 40 30.7 16.0Details elements (ex: windows)incoherent with MRAP

6 2 1.9 0 0 2 1.5 1.3

Abusive segmentation 7 0 0 0 0 2 1.5 0.6

Total failure cases withoutadditional knowledge

8 9 8.5 17 22.4 56 43.1 26.2

Additional knowledge aboutprincipal direction

Information about principal directionavailable through groups constitution

10 43 40.2 68 89.5 100 76.9 67.4

Information about principal directioncan be used to bypass failure inprincipal direction extraction

11 4 3.7 0 0 10 7.7 4.5

test sites. Only orthogonal MRAPs have been used.Some failures can arise during the initial footprint ex-traction (ROI extraction, conversion to a rectangle graphstructure. . . ) and during the extrusion and segmentationstep. The ROI extraction is very robust (a 99% suc-cess rate) even if the DSM quality is relatively poor.The principal errors arise when the nDSM quality doesnot allow for building detection during the ROI extrac-tion. The suggested strategy, which is dedicated to theextraction of the building principal directions and thedescription of the ROI as a rectangle graph, leads to afailure rate of less than 10% for the three tests sites.The principal cause of failure during the direction ex-traction is tree branches. Using edge ortho-images, theautomatic determination of the principal direction couldbe improved through a 3D edges analysis instead ofa 2D edges analysis. The improvement would consistin extracting the 3D edges with a feature-based match-ing strategy and then in analyzing only the horizontal3D edge directions. Furthermore, it should be under-lined that a priori knowledge about the principal direc-tion of the building to be extracted was available in allthe failure cases (lines 10 and 11 in Table 2). We in-deed counted the number of occurrences when this kindof a priori knowledge was available. The purpose wasto evaluate whether this available information could beuseful from an automation standpoint. This informationhas been provided through the CLG specification when

a group of aligned buildings were built. Taking advan-tage of this information when available is a prospectthat should be investigated (e.g. cancellation of the edgeanalysis or exploitation of this information during thestatistical analysis of the edge directions). The match-ing process between the available rectangles and the ge-ometric pattern footprint displays a rough 3% failurerate. These failure cases occur when the DSM is toonoisy and, consequently, when the ROI boundaries aretoo poor. The footprint extraction, particularly the rect-angle graph computation, is still very efficient.

The initial footprint extraction processes are fast(first part of the Table 2), relatively robust and cantake advantage of the available a priori knowledgeabout the building directions. The extrusion andtopographical based segmentation are only based onthe DSM exploitation. Our test sites are similar froman architectural standpoint (simple building structurecompatible with parametric model concept). Thereforethe performance differences between these test sitesare attributable to DSM quality differences. In fact,noisy surface definition in DSM leads to extractingwrong faces and then leads to extrusion failure.Fig. 15 represents the shaded view of the threeDSM and illustrates how the Beauport DSM andthe Montreal DSM are noisier than CharlesbourgDSM. Several factors have an impact on DSM quality(image resolution, b/h ratio..) but we think the most



Fig. 15. Shaded sample of the DSM of the test sites. (a) Charlesbourg (b) Beauport and (c) Montreal.

deterministic factor in our case is likely the type ofcamera used (film vs. digital) that, as described inPaparoditis et al. (2006), changes drastically the signalto noise ratio and then the correlation success.

The semi-automated approach implemented withMRAP decreases up to 5 times the human interventionrequired when performing a manual MRAP implanta-tion. The proposed algorithm could be easily combinedwith other strategy used to determine accurate parame-ters like those proposed in Tseng and Wang (2003) or inRottensteiner (2001).

5. Outlook on buildings footprint extraction

5.1. Goal and strategy

This section presents some preliminary resultsregarding the extraction of complex detailed geometriesbased on the strategy described in Section 2.4. Weused the Amiens city dataset provided by IGNand the workgroup 8 of the ISPRS commission III(http://isprs.ign.fr/) to facilitate the comparison ofour results with other approaches. The algorithm wedeveloped consists in extracting the building footprintswhile expressing them as an unconstrained combinationof rectangles (rectangles are not necessary orthogonalto each other). The purpose of such an approach is(1) to support MR acquisition when MRAPs are notorthogonal, and (2) to extract the fine geometries andlink them with a MRAP when MRAPs are not detailedenough to extract the finest level geometries. In fact,preliminary footprint extraction could be combinedwith other algorithms requiring geometric footprintdescription. It could, for example, be combined eitherwith the strategies proposed in Haala and Brenner(1999) or in Suveg and Vosselman (2004), whichprovide a CSG description of the building, or with thestrategy proposed in Jibrini et al. (2000), which is moregeneric from the roof structures standpoint.

5.2. Building footprints algorithm

The proposed algorithm uses aerial imagery andDSM and consists of two successive steps: the

footprint expression as a graph of rectangles and thesimplification of this graph. This algorithm assumes thatthe roof structure consists mainly of 3D edges alignedin accordance with the rectangle principal direction.

The initial graph extraction is similar to theorthogonal MRAP extraction. However, we do notconsider that rectangles must be orthogonal to eachother anymore. Indeed, a ROI is extracted from nDSMand lines corresponding to the ROI skeleton are usedto create rectangle axes. The directions are computedindependently for each axis. Only the edges insidethe ROI and close to the axis point are used for thestatistical analysis whereas, in the orthogonal MRAPsconfiguration, all the edges in the ROI were used.

The final building footprint is then inferredby simplifying the graph to decrease the numberof rectangles while preserving satisfying geometryfidelity. We developed an algorithm to performthis simplification. The basic idea is to delete therectangles that do not contribute enough to the footprintdescription. We used the covered area as a contributingcriterion. The algorithm first selects the rectangleproviding the best contribution and deletes all therectangles that do not contribute enough. The secondbest rectangle is then searched following the sameprinciple and irrelevant rectangles are deleted accordingto the two first selected rectangles. This procedure isiterated while the number of remaining rectangles keepson decreasing. This algorithm is summarized in theUML activity diagram presented in Fig. 16. Fig. 17illustrates the results obtained for the two main stepsof the algorithm. The images consist of a group ofconnected buildings.

Fig. 18 corresponds to the results obtained for twoother groups of buildings. The processing time perbuilding group was about twenty seconds. We canvisually notice that results are really close to a manualacquisition based on DSM even if some rectangleconnections are not perfect. Lafarge et al. (2006)observed similar problems on the same test site whenthey used parametric models to automatically extractbuilding geometries with a marked point process. The



Fig. 16. Algorithm to simplify a graph of rectangles.

Fig. 17. Building footprint extraction process example.

authors also proposed an algorithm to correct theseartifacts.

6. Conclusion

The multi-representation acquisition specificationshave been described in this paper. The results obtainedusing our prototypes have been presented in order to

illustrate the MR acquisition process and results. Wehave introduced two new algorithms. The first one canbe used to extract the parametric model parameters witha simple one-click initialization. This new approach isrelevant to elementary parametric model (with only onevolumetric primitive) and to more complex parametricmodels (ex: T or L shapes) and then can be used tosupport MR acquisition with MRAP. The comparison



Fig. 18. Results of the footprint extraction process.

of manual MR acquisition using MRAP with theacquisition using the proposed new algorithm hasshown a 5 times reduction of the human intervention inthe process. The current performances are encouragingsince the human intervention is lessened to a single clickper building and a response time around one secondper building. The short response time is particularlyrelevant in our semi-automatic context. The analysis ofthe results has allowed us to underline the limitations ofour approach and to propose some recommendations. Inaddition, we proposed a second new algorithm aimingto extract the building footprints and to express themas an unconstrained combination of rectangles. Thepreliminary results are as also very promising as thissecond algorithm also offers a very quick responsetimes.

Acknowledgments

The authors wish to thank for its support theIndustrial research chair in geospatial database fordecision, support financed by the Natural Sciences andEngineering Research Council of Canada (NSERC),the Universite Laval, Hydro-Quebec, Research andDevelopment Defense Canada, Natural ResourcesCanada, Ministere des Transports du Quebec, KHEOPSTechnologies, Intelec Geomatique, Syntell, Holonicsand DVP-GS.

The authors wish to thank for its NSERC IndustrialPostgraduate Scholarship the Natural Sciences andEngineering Research Council of Canada (NSERC).The authors would also like to thank DVP-GS’stechnological and financial support, as well as forproviding high resolution images and vector data.

References

Ameri, B., 2000. Automatic recognition and 3D reconstructionof buildings from digital imagery. Doctor Thesis. DeutscheGeodatische Kommission, Reihe C, Nr. 52, Institute forPhotogrammetry, University of Stuttgart.

Baltsavias, E., 2004. Object extraction and revision by image analysisusing existing geodata and knowledge: Current status and stepstowards operational systems. ISPRS Journal of Photogrammetryand Remote Sensing 58 (3–4), 129–151.

Bedard, Y., 2004. Combining automatic generalization and multi-ple representations for highly interactive web-based mapping.GeoSpatial Word. Miami, FL, USA. http://sirs.scg.ulaval.ca/YvanBedard/slideshow/publication/362/357 files/frame.htm (Ac-cessed December 2, 2007).

Bedard, Y., Rivest, S., Proulx, M.-J., 2007. Spatial on-line analyticalprocessing (SOLAP): Concepts, architectures and solutions from ageomatics engineering perspective. In: Wrembel, R., Koncillia, C.(Eds.) Data warehouses and OLAP: Concepts, architectures andsolutions, IRM Press (Idea Group), pp. 298–319.

Bernier, E., Bedard, Y., Hubert, F., 2005. UMapIT: An on-demandweb mapping application based on a multiple representationdatabase. 8th ICA Workshop on generalization and multiple rep-resentation. Coruna, Spain, 8–9th July, (On CD-ROM). http://sirs.scg.ulaval.ca/Yvanbedard/article nonprotege/401.pdf (AccessedDecember 2, 2007).

Bernier, E., Bedard, Y., 2007. Data warehouse strategy for on-demand multiscale geo-information, generalisation of geographicinformation: Cartographic modelling and applications. In: Mack-aness, W., Ruas, A., Sarjakoski, T. (Eds.), Generalisation of Ge-ographic Information: Cartographic Modelling and Applications.pp. 177–198 (Chapter 9).

Brenner, C., 2005. Building reconstruction from images and laserscanning. International Journal of Applied Earth Observation andGeoinformation 6 (3–4), 187–198.

Canny, J., 1986. A computational approach to edge detection. IEEETransactions on Pattern Analysis and Machine Intelligence 8 (6),679–714.

Cardenas, A., 2004. Utilisation de Patrons Geometriques commesupport a la Generalisation Automatique. Departement desSciences Geomatiques. M.sc. Thesis. Universite Laval.

Cecconi, A., Weibel, R., Barrault, M., 2002. Improvement ofautomated generalization for on-demand mapping by multiscaledatabases. In: Richardson, D., van Oosterom, P. (Eds.), Advancesin Spatial Data Handling. Springer-Verlag, Berlin, pp. 515–532.

Fischler, M.A., Bolles, R.C., 1981. Random Sample Consensus: Aparadigm for model fitting with applications to image analysisand automated cartography. Communications of the ACM 24 (6),381–395.

Fredericque, B., Daniel, S., Bedard, Y., Paparoditis, N., 2005.Knowledge-based process management to populate databaseswith 3D multi representation of buildings. In: ISPRS HannoverWorkshop 2005 - High-Resolution Earth Imaging for GeospatialInformation, (on CD-ROM). http://www.ipi.uni-hannover.de/fileadmin/institut/pdf/087-fredericque.pdf (Accessed December2, 2007).



Fredericque, B., Daniel, S., Bedard, Y., Paparoditis, N., 2008. Saisiesemi-automatique de batiments 3D dans une base de donnees arepresentations multiples – vers une approche integree. RevueInternationale de Geomatique 18 (1), 67–94.

Gerke, M., Heipke, C., Straub, B.-M., 2001. Building extractionfrom aerial imagery using a generic scene model and invariantgeometric moments. In: Proceedings of the IEEE/ISPRS JointWorkshop on Remote Sensing and Data Fusion over Urban Areas,IEEE Piscataway, ISBN 0-7803-7059-7, pp. 85–89.

Gulch, E., Muller, H., Labe, T., Ragia, L., 1998. On the performanceof semi-automatic building extraction. International Archives ofPhotogrammetry and Remote Sensing 32 (Part 3/1), 331–338.

Gulch, E., Muller, H., Labe, T., 1999. Integration of automaticprocesses onto semiautomatic building extraction. InternationalArchives of Photogrammetry and Remote Sensing 32 (Part 4),188–196.

Haala, N., Brenner, C., 1999. Virtual city models from laser altimeterand 2D map data. Photogrammetric Engineering & RemoteSensing 65 (7), 787–795.

Jibrini, H., Paparoditis, N., Pierrot-Desseilligny, M., Maıtre, H., 2000.Automatic building reconstruction from very high resolutionaerial stereopairs using cadastral ground plans. InternationalArchives of Photogrammetry, Remote Sensing and SpatialInformation Sciences 33 (Part B3). http://www.tsi.enst.fr/maitre/hm:isprs-00.ps.gz (Accessed January 10, 2004).

Lafarge, F., Trontin, P., Descombes, X., Zerubia, J., Pierrot-Deseilligny, M., 2006. An automatic building extractionmethod: Application to the 3D-city modeling, Research Report.INRIA, France, France, May 2006, no 5925, http://hal.inria.fr/inria-00077117 (Accessed December 2, 2007).

Maas, H.-G., 1999. Closed solutions for the determination ofparametric building models from invariant moments of airbornelaserscanner data. International Archives of Photogrammetry andRemote Sensing 32 (Part B3), 193–199.

Mackaness, W., Ruas, A., Sarjakoski, T., 2007. Generalisation ofgeographic information: Cartographic modelling and applications.Published on behalf of the international cartographic associationby Elsevier.

Olteanu, A., Mustiere, S., Ruas, A., 2006. Matching imperfect spatialdata. In: Caetano, M., Painho, M. (Es.), Proceedings of 7thInternational Symposium on Spatial Accuracy Assessment inNatural Resources and Environmental Sciences. Lisbon, 7–9 Julypp. 694–704.

Ortner, M., Descombes, X., Zerubia, J., 2007. Building outlineextraction from digital elevation models using marked pointprocesses. International Journal of Computer Vision 72 (2),107–132.

Paparoditis, N., Souchon, J-P., Martinoty, G., Pierrot-Deseilligny,M., 2006. High-end aerial digital cameras and their impact onthe automation and quality of the production workflow. ISPRSJournal of Photogrammetry and Remote Sensing 60 (6), 400–412.

Parker, J.R., 1997. Algorithms for Image Processing andComputer Vision. Wiley Computer Publishers, New York,ISBN: 0471140562.

Pierrot-Deseilligny, M., Paparoditis, N., 2006. A multiresolution andoptimization-based image matching approach: An application tosurface reconstruction from Spot5-HRS stereo imagery. In: ISPRS

Ankara Workshop 2006. Topographic Mapping from Space,36 (1/W41). www.isprs.org/commission1/ankara06/makaleler/Pierrot multiresolution matching.doc (Accessed December 2,2007).

Rottensteiner, F., 2001. Semi-automatic extraction of buildings basedon hybrid adjustment using 3D surface models and management ofbuilding data in a TIS. Ph.D. Thesis. Institute ofPhotogrammetryand Remote Sensing, Vienna University of Technology.

Rottensteiner, F., Schulze, M., 2003. Performance evaluation of asystem for semi-automatic building extraction using adaptableprimitives. International Archives of Photogrammetry, RemoteSensing and Spatial Information Sciences 34 (3/W8), 47–52.

Roy, S., 1999. Stereo without epipolar lines: A Maximum-Flowformulation, international. Journal of Computer Vision 34 (2/3),147–162.

Sabo, M.N., Cardenas, A., Bernier, E., Bedard, Y., 2005a. Introductiondu concept de patrons geometriques et application aux batimentsafin de faciliter leur generalisation cartographique a la volee.Geomatica. journal of the Canadian Institute of Geomatics 59 (3),295–313.

Sabo, M.N., Bedard, Y., Bernier, E., Cardenas, A., 2005b.Methodology for developing a database of geometric patternsto better support on-the-fly map generalization. In: Proceedingsof International Cartographic Conference, 9–16 July, Coruna,Spain. http://www.cartesia.org/geodoc/icc2005/pdf/oral/TEMA9/Session%201/MAMANE%20NOURI%20SABO.pdf (AccessedDecember 2, 2007).

Sabo, M.N., 2007. Integration des algorithmes de generalisationet des patrons geometriques pour la creation des objetsauto-generalisants (SGO) afin d’ameliorer la generalisationcartographique a la vole. Ph.D. Thesis. Universite Laval.

Sternberg, S.R., 1983. Biomedical image processing. IEEE Computer6 (1), 22–34.

Suveg, I., Vosselman, G., 2004. Reconstruction of 3D building modelsfrom aerial images and maps. ISPRS Journal of Photogrammetryand Remote Sensing 58 (3–4), 202–224.

Tseng, Y.-H., Wang, S., 2003. Semi-automated building extractionbased on CSG model-image fitting. Photogrammetric Engineeringand Remote Sensing 69 (2), 171–180.

Vinson, S., Cohen, L.D., 2002. Multiple rectangle model for buildingssegmentation and 3D scene reconstruction. In: Proceedingsof 16th International Conference on Pattern Recognition, 3,pp. 623–626.

Vosselman, G., Veldhuis, H., 1999. Mapping by dragging and fitting ofWire-Frame models. Photogrammetric Engineering and RemoteSensing 65 (7), 769–776.

Weibel, R., Dutton, G., 1999. Generalising spatial data and dealingwith multiple representations. In: Longley, P.A., Goodchild, M.F,Maguire, D.J., Rhind, D.W. (Eds.), Geographic InformationSystems: Principles, Techniques, Management and Applications.John Wiley & Sons, Cambridge, pp. 125–155.

Weidner, U., 1997. Digital surface models for building extraction.In: Grun, A. (Ed.), Automatic Extraction of Man-Made Objectsfrom Aerial and Space Images, II. pp. 193–202.

Zhang, S., Fu, K.S., 1984. A thinning algorithm for discrete binaryimages. In: Proceedings of the International Conference onComputers and Applications. Beijing, China. pp 879–886.

Author's personal copy - Université Lavalyvanbedard.scg.ulaval.ca/wp-content/documents/publications/500.pdf · to populate MRDB with a photogrammetric approach using MRAP is introduced

Documents