Institute for Geodesy and Geoinformation Science November 13th 2008 Making interoperability persistent: A 3D geo database based on CityGML Alexandra Stadler,

Institute for Geodesy and Geoinformation ScienceInstitute for Geodesy and Geoinformation Science

November 13th 2008

Making interoperability persistent:

A 3D geo database based on CityGML

Alexandra Stadler, Claus Nagel, Gerhard König, Thomas H. Kolbe

Technische Universität BerlinInstitute for Geodesy and Geoinformation ScienceChair of Geoinformation Science

Stadler, Nagel, König, Kolbe: Making interoperability persistent: A 3D geo database based on CityGML

2


November 13th 2008

MotivationMotivation

Berlin: Management of 3D geo data

Collect, compare, adapt, update, exchange

Data from various sources (cadastre, planning, architecture, utility networks, etc.)

Precondtion:

Existence of a standardised data and exchange format for 3D city models

Ensures consistent and interoperable data structuring

Well-known representative: CityGML


3


November 13th 2008

Content Models all relevant parts of the virtual city according to their

semantics, geometry, topology and appearance GML application schema based on XML Data model and exchange format for virtual 3d city models

History Developed since 2002 by the Special Interest Group 3d

(NorthRhine Westphalia, Germany) Members from >70 companies, municiplaities and research institutions

lead managed by Prof. Thomas H. Kolbe (IGG TU Berlin)

Dr. Gerhard Gröger (IGG Uni Bonn)

CityGML: OverviewCityGML: Overview


4


November 13th 2008

CityGML: On its way to become an OGC standardCityGML: On its way to become an OGC standard

CityGML 0.4.0OGC Best Practices Paper

CityGML 1.0.0 (Proposal)OGC Request for Comments

CityGML 0.3.0OGC Discussion Paper

2007-05-30

2008-02-04

2006-03-06

2008-02-192008-03-20

CityGML 1.0.0OGC Implementation Specification(after final OGC TC vote)

2008-08-20

<<<<<<< Public Comment Phase >>>>>>>

International standard

CityGML Kick-Off2002-05


5


November 13th 2008

CityGML: Thematically structured into object classesCityGML: Thematically structured into object classes

ExternalReference

- informationSystem: anyURI - externalReference: ExternalObjectReferenceType

<<FeatureCollection>>CityModel

**

…

LOD 0-4 GeometryProperty

<<Geometry>>gml::_Geometry LOD 0-4 GeometryProperty

<<Feature>>_Transportation

Object

<<Feature>>_AbstractBuilding

<<Feature>>ReliefFeatur

e

<<Feature>>_WaterBody

<<Feature>>_Vegetation

<<Feature>>_CityObject

<<Feature>>gml::_Feature


7


November 13th 2008

CityGML: Coherence of semantics and geometryCityGML: Coherence of semantics and geometry

Building

BuildingPartBuildingPart

RoofSurf.

RoofSurf.

Door

WallSurf.

WallSurf.

Window

Building Installation

...

......

CompositeSolid

SolidSolid

Polygon

Polygon

Polygon

Polygon

CompositeSurface

...

......

Polygon

Semantics Geometry

CompositeSurface

Semantics

based on ISO 19109Geometry

based on ISO 19107

7


8


November 13th 2008

A 3D geo database for BerlinA 3D geo database for Berlin


10


November 13th 2008

Flexible3D geometries

Generic(prototypical)

3D objects

Buildings up toLOD 3

DigitalTerrain Models

(DTM)

Versionmanagement

Exchange ofDTM and aerial images

(WebServices)

Referencing ofexterior

data sources

(recursive)aggregationof objects

Functionality of the 3D geo databaseFunctionality of the 3D geo database

CityGML properties additions

1st

pro

ject

ph

ase

2n

d p

roje

ct

ph

ase

Complexthematicmodelling

Appearancedata

Import andexport of

CityGML files

Buildingsincludinginterior

structures(LOD 4)


11


November 13th 2008

Creation of an import and export tool for CityGML instance

documentsc

Simplification of

CityGML‘s data model a

CityGML Xsd Schema<xs:complexTypename="CityModelType"> <xs:complexContent> <xs:extension ...

UML Schema

simplifiedUML Schema

Schema-simplifi-cation

Javabinding

(JAXB)

Schema-derived classes

public class CityModel {… }

SQLqueries

(Imp/ExportTool)

Importdata

Exportdata

Derivation of the relational database schema

b

Mappingclasses to

tables

Oracledatabase

SQL DDLstatements

(JDeveloper)

Databasecreation

Relationaldiagram

Development cycleDevelopment cycle


12


November 13th 2008


documentsc

Javabinding

(JAXB)

Schema derived classes


SQLqueries

(Imp/ExportTool)

Importdata

Exportdata


b

Mappingclasses to

tables

Oracledatabase

SQL DDLstatements

(JDeveloper)

Databasecreation

Relationaldiagram

Simplification of



UML Schema

SimplifiedUML Schema

Schemasimplifi-cation

Simplification of CityGML‘s data modelSimplification of CityGML‘s data model


13


November 13th 2008

Simplification of CityGML‘s data modelSimplification of CityGML‘s data model

Thematic model covers a broad range of application fields Complex relations within thematic modules Comparable hierarchies on the geometry level

<<Geometry>>_BRepGeometry

<<Geometry>>Polygon

+isSolid : boolean [1]+isComposite : boolean [1]+isTriangulated : boolean [1]

+geometry : Polygon [1]

<<Geometry>>BRepAggregate

0..1

1..*

bRepMember <<Geometry>>_BRepGeometry

<<Geometry>>Polygon




0..1

1..*

bRepMember


14


November 13th 2008


Simplification of



UML Schema



documentsc

Javabinding

(JAXB)



SQLqueries

(Imp/ExportTool)

Import data

Export data


b

Mapping classes to

tables

Oracledatabase

SQL DDLstatements

(JDeveloper)

Databasecreation

Relationaldiagram

Derivation of the relational database schemaDerivation of the relational database schema


15


November 13th 2008

Mapping strategies for aggregation hierarchies

Derivation of the relational database schemaDerivation of the relational database schema

<<Geometry>>_BRepGeometry

<<Geometry>>Polygon




0..1

1..*

bRepMember <<Geometry>>_BRepGeometry

<<Geometry>>Polygon




0..1

1..*

bRepMember

BRepAggregateID: NUMBER <<PK>><<FK>>

Polygon

ID: NUMBER <<PK>><<FK>>GEOMETRY : SDO_GEOMETRY

BRepGeometryID: NUMBER <<PK>>IS_SOLID : BOOLEANIS_COMPOSITE : BOOLEANIS_TRIANGULATED : BOOLEAN

BRepAggregateID: NUMBER <<PK>>IS_SOLID : BOOLEANIS_COMPOSITE : BOOLEANIS_TRIANGULATED : BOOLEAN

PolygonID: NUMBER <<PK>>IS_SOLID : BOOLEANIS_COMPOSITE : BOOLEANIS_TRIANGULATED : BOOLEANGEOMETRY : SDO_GEOMETRY

BRepGeometryID: NUMBER <<PK>>TYPE : VARCHAR2(30)IS_SOLID : BOOLEANIS_COMPOSITE : BOOLEANIS_TRIANGULATED : BOOLEANGEOMETRY : SDO_GEOMETRY

Realisation of geometry model


16


November 13th 2008

Simplification of





b

Mappingclasses to

tables

SQL DDLstatements

(JDeveloper)

Database creation

Relationaldiagram


documentsc


UML Schema

Javabinding

(JAXB)



SQLqueries

(Imp/ExportTool)

Import data

Export data

Oracledatabase

Creation of an import and export toolCreation of an import and export tool


17


November 13th 2008

Static core ofthe software

Creation of an import and export tool: OverviewCreation of an import and export tool: Overview

Data import

CityGML input file____ ____________ _____ ____ _________

ReadCityGML

Features

CityModel cityModel1 = new CityModel () ; …

follows instance of

Data export

_________ _________

Oracledatabase

Databaseimport

applied

Features

CityModel cityModel1 = new CityModel () ; …

Databaseexport

appliedinstance of

CityGML output file____ ____________ _____ ____ _________

WriteCityGML

follows

Access wrapper Xsd Schema<xs:complexTypename="CityModelType"> <xs:complexContent> <xs:extension ...

Javabinding



Import functionality

Export functionality


18


November 13th 2008

Characteristics of the import/export toolCharacteristics of the import/export tool

Support for CityGML files of arbitrary size (>4GB)

Concurrency of data processing through multithreadingHigh performance on standard platforms

Use of universally unique identifiers (UUIDs)

Filter options enable user-defined import and exportGML ID, GML name

Bundled import and export for data tiling (classified by IDs or Bounding Boxes)

Selection of object classes

Published as open source software


19


November 13th 2008

Concurrency of data processingConcurrency of data processing

Queue

Thread Pool

Worker

Worker

Worker

Thread Pool

Queue

Worker

Parallel non-blocking execution of multiple interacting tasks

Decouple compute and I/O bound Degree of parallelisation depends on number of processing

cores

Attention: Context switch Thread life cycle overhead Resource thrashing

Reuse threads by applying a pooled threads modelThread creation overhead is spread over many tasks

20


November 13th 2008Stadler, Nagel, König, Kolbe: Making interoperability persistent: A 3D geo database based on CityGML

Parser Thread(1 instance)

Converter Thread

(limited numberof instances)

Data import – step 1Data import – step 1

____ _____________ _____ ____ _____________ _____ ____ ____ _____ _________

CityGMLinput file

SAX parsing

TemporaryBuffer

BufferQueue

Featurecreation

JAXBunmarshallin

g

TopLevel FeatureQueue

__________________

_________

_________

_________

____ _____________ _____ ____ _____________ _____ ____ ____ _____ _________

21



_________

With separateXLink storage(temporary)XLink Thread


Importer Thread



TopLevel FeatureQueue

commit

XLinks XLinkinformatio

n

Databaseupdate

_________ _________

Oracledatabase

SQL creaioncommit

SQL statementQueues

Geo data

Import filter

XLinkQueue

__________________

_________

_________

_________

____ _____________ _____ ____ _____________ _____ ____ ____ _____ _________

22



XLinkResolver Thread


XLinkSplitter Thread

(1 Instanz)


__________________

_________

_________

_________

____ _____________ _____ ____ _____________ _____ ____ ____ _____ _________

XLinksplitting

XLink ResolverQueue

Databaseupdate

Oracledatabase

__________________

with resolvedXLinks

SQL creation

XLinks

commit

SQL statementQueue

With separateXLink storage(temporary)

_________

_________ _________

Oracledatabase

23



Exporter Thread(limited number of instances)

Splitter Thread(1 instance)

Data export – step 1Data export – step 1

Feature

Export filter

SQL

TopLevel Feature

ID Queue

Databaseoutput

SQL

Nested feature/geometry request

__________________

Oracledatabas

e

____ _____________ _____ ____ _____________ _____ ____ ____ ____ _________

__________________

24



IO Writer Thread(1 instance)

Exporter Thread(limited number

of instances)

JAXBmarshallin

g

TemporaryBuffer

BufferQueue

____ _____________ _____ ____ _____________ _____ ____ ____ _____ _________

CityGMLoutput file

WriteCityGML

Data export – step 2Data export – step 2

____ _____________ _____ ____ _____________ _____ ____ ____ ____ _________

__________________

Feature


26


November 13th 2008

Some performance measurements…Some performance measurements…

Dataset File size

Time Features /

second

N° of worke

rs

Versio-ning

enabled

Time Features /

second

N° of worker

s

1 mio LOD1 buildings

7,5 GB

1:15h

228 4 no 38min 455 10

2:25h

121 1 no 1:57h 150 1

>10.000 LOD2 buildings

fully textured

163 MB+

57 MB

25 min

7 4 no4,3mi

n42 4

w/o textures

163 MB

6,5 min

28 4 no 2min 91 4

>200 complex LOD3 objects

40,1MB

17s 12 4 no

7s 30 103:20h

0,02 4 yes

Import Export


27


November 13th 2008

Future plansFuture plans

Matching functionality (in progress) Recognise corresponding buildings in different datasets

Link geometries and exchange attribute

Avoid redundant information

Use the tool as backend for Web Services Broaden the applicability

Replacing the graphical user interface by a Web Service interface

Web Feature Services (WFS) may rely on existing import and export functionality

Additional export formats KML etc.

Performance optimisation


28


November 13th 2008

Alexandra StadlerClaus NagelGerhard KönigThomas H. Kolbe

stadler|nagel|kolbe @ [email protected]

Technische Universität BerlinInstitute for Geodesy and Geoinformation ScienceChair of Geoinformation Science

Questions ?


29


November 13th 2008

ReferencesReferences

3D city database (2007), www.3dcitydb.org [last access: 2008-06-20].

Döllner J, Kolbe TH, Liecke F, Sgouros T, Teichmann K (2006) The Virtual 3D City Model of Berlin - Managing, Integrating, and Communicating Complex Urban Information, In: Proceedings of the 25th Urban Data Management Symposium UDMS, Aalborg 2006.

Emgård L, Zlatanova S (2007) Implementation alternatives for an integrated 3D Information Model, In: Advances in 3D Geoinformation Systems, Springer-Verlag, pp. 313-330.

GNU Lesser General Public License, http://www.gnu.org/copyleft/lgpl.html [last access: 2008-06-20].

Gröger G, Kolbe TH, Czerwinski A (2007), Candidate OpenGIS Implementation Specification (City Geography Markup Language), Version 0.4.0, OGC Doc. No. 07-062, Open Geospatial Consortium 2007.

Snowflake Software, GO Loader product page (2008), http://www.snowflake software.co.uk/products/goloader/index.htm [last access: 2008-06-20].

Institute for Geodesy and Geoinformation Science November 13th 2008 Making interoperability persistent: A 3D geo database based on CityGML Alexandra Stadler,

Documents