Institute for Geodesy and Geoinformation ScienceInstitute for Geodesy and Geoinformation Science
November 13th 2008
Making interoperability persistent:
A 3D geo database based on CityGML
Alexandra Stadler, Claus Nagel, Gerhard König, Thomas H. Kolbe
Technische Universität BerlinInstitute for Geodesy and Geoinformation ScienceChair of Geoinformation Science
Stadler, Nagel, König, Kolbe: Making interoperability persistent: A 3D geo database based on CityGML
2
Institute for Geodesy and Geoinformation ScienceInstitute for Geodesy and Geoinformation Science
November 13th 2008
MotivationMotivation
Berlin: Management of 3D geo data
Collect, compare, adapt, update, exchange
Data from various sources (cadastre, planning, architecture, utility networks, etc.)
Precondtion:
Existence of a standardised data and exchange format for 3D city models
Ensures consistent and interoperable data structuring
Well-known representative: CityGML
Stadler, Nagel, König, Kolbe: Making interoperability persistent: A 3D geo database based on CityGML
3
Institute for Geodesy and Geoinformation ScienceInstitute for Geodesy and Geoinformation Science
November 13th 2008
Content Models all relevant parts of the virtual city according to their
semantics, geometry, topology and appearance GML application schema based on XML Data model and exchange format for virtual 3d city models
History Developed since 2002 by the Special Interest Group 3d
(NorthRhine Westphalia, Germany) Members from >70 companies, municiplaities and research institutions
lead managed by Prof. Thomas H. Kolbe (IGG TU Berlin)
Dr. Gerhard Gröger (IGG Uni Bonn)
CityGML: OverviewCityGML: Overview
Stadler, Nagel, König, Kolbe: Making interoperability persistent: A 3D geo database based on CityGML
4
Institute for Geodesy and Geoinformation ScienceInstitute for Geodesy and Geoinformation Science
November 13th 2008
CityGML: On its way to become an OGC standardCityGML: On its way to become an OGC standard
CityGML 0.4.0OGC Best Practices Paper
CityGML 1.0.0 (Proposal)OGC Request for Comments
CityGML 0.3.0OGC Discussion Paper
2007-05-30
2008-02-04
2006-03-06
2008-02-192008-03-20
CityGML 1.0.0OGC Implementation Specification(after final OGC TC vote)
2008-08-20
<<<<<<< Public Comment Phase >>>>>>>
International standard
CityGML Kick-Off2002-05
Stadler, Nagel, König, Kolbe: Making interoperability persistent: A 3D geo database based on CityGML
5
Institute for Geodesy and Geoinformation ScienceInstitute for Geodesy and Geoinformation Science
November 13th 2008
CityGML: Thematically structured into object classesCityGML: Thematically structured into object classes
ExternalReference
- informationSystem: anyURI - externalReference: ExternalObjectReferenceType
<<FeatureCollection>>CityModel
**
…
LOD 0-4 GeometryProperty
<<Geometry>>gml::_Geometry LOD 0-4 GeometryProperty
<<Feature>>_Transportation
Object
<<Feature>>_AbstractBuilding
<<Feature>>ReliefFeatur
e
<<Feature>>_WaterBody
<<Feature>>_Vegetation
<<Feature>>_CityObject
<<Feature>>gml::_Feature
Stadler, Nagel, König, Kolbe: Making interoperability persistent: A 3D geo database based on CityGML
7
Institute for Geodesy and Geoinformation ScienceInstitute for Geodesy and Geoinformation Science
November 13th 2008
CityGML: Coherence of semantics and geometryCityGML: Coherence of semantics and geometry
Building
BuildingPartBuildingPart
RoofSurf.
RoofSurf.
Door
WallSurf.
WallSurf.
Window
Building Installation
...
......
CompositeSolid
SolidSolid
Polygon
Polygon
Polygon
Polygon
CompositeSurface
...
......
Polygon
Semantics Geometry
CompositeSurface
Semantics
based on ISO 19109Geometry
based on ISO 19107
7
Stadler, Nagel, König, Kolbe: Making interoperability persistent: A 3D geo database based on CityGML
8
Institute for Geodesy and Geoinformation ScienceInstitute for Geodesy and Geoinformation Science
November 13th 2008
A 3D geo database for BerlinA 3D geo database for Berlin
Stadler, Nagel, König, Kolbe: Making interoperability persistent: A 3D geo database based on CityGML
10
Institute for Geodesy and Geoinformation ScienceInstitute for Geodesy and Geoinformation Science
November 13th 2008
Flexible3D geometries
Generic(prototypical)
3D objects
Buildings up toLOD 3
DigitalTerrain Models
(DTM)
Versionmanagement
Exchange ofDTM and aerial images
(WebServices)
Referencing ofexterior
data sources
(recursive)aggregationof objects
Functionality of the 3D geo databaseFunctionality of the 3D geo database
CityGML properties additions
1st
pro
ject
ph
ase
2n
d p
roje
ct
ph
ase
Complexthematicmodelling
Appearancedata
Import andexport of
CityGML files
Buildingsincludinginterior
structures(LOD 4)
Stadler, Nagel, König, Kolbe: Making interoperability persistent: A 3D geo database based on CityGML
11
Institute for Geodesy and Geoinformation ScienceInstitute for Geodesy and Geoinformation Science
November 13th 2008
Creation of an import and export tool for CityGML instance
documentsc
Simplification of
CityGML‘s data model a
CityGML Xsd Schema<xs:complexTypename="CityModelType"> <xs:complexContent> <xs:extension ...
UML Schema
simplifiedUML Schema
Schema-simplifi-cation
Javabinding
(JAXB)
Schema-derived classes
public class CityModel {… }
SQLqueries
(Imp/ExportTool)
Importdata
Exportdata
Derivation of the relational database schema
b
Mappingclasses to
tables
Oracledatabase
SQL DDLstatements
(JDeveloper)
Databasecreation
Relationaldiagram
Development cycleDevelopment cycle
Stadler, Nagel, König, Kolbe: Making interoperability persistent: A 3D geo database based on CityGML
12
Institute for Geodesy and Geoinformation ScienceInstitute for Geodesy and Geoinformation Science
November 13th 2008
Creation of an import and export tool for CityGML instance
documentsc
Javabinding
(JAXB)
Schema derived classes
public class CityModel {… }
SQLqueries
(Imp/ExportTool)
Importdata
Exportdata
Derivation of the relational database schema
b
Mappingclasses to
tables
Oracledatabase
SQL DDLstatements
(JDeveloper)
Databasecreation
Relationaldiagram
Simplification of
CityGML‘s data model a
CityGML Xsd Schema<xs:complexTypename="CityModelType"> <xs:complexContent> <xs:extension ...
UML Schema
SimplifiedUML Schema
Schemasimplifi-cation
Simplification of CityGML‘s data modelSimplification of CityGML‘s data model
Stadler, Nagel, König, Kolbe: Making interoperability persistent: A 3D geo database based on CityGML
13
Institute for Geodesy and Geoinformation ScienceInstitute for Geodesy and Geoinformation Science
November 13th 2008
Simplification of CityGML‘s data modelSimplification of CityGML‘s data model
Thematic model covers a broad range of application fields Complex relations within thematic modules Comparable hierarchies on the geometry level
<<Geometry>>_BRepGeometry
<<Geometry>>Polygon
+isSolid : boolean [1]+isComposite : boolean [1]+isTriangulated : boolean [1]
+geometry : Polygon [1]
<<Geometry>>BRepAggregate
0..1
1..*
bRepMember <<Geometry>>_BRepGeometry
<<Geometry>>Polygon
+isSolid : boolean [1]+isComposite : boolean [1]+isTriangulated : boolean [1]
+geometry : Polygon [1]
<<Geometry>>BRepAggregate
0..1
1..*
bRepMember
Stadler, Nagel, König, Kolbe: Making interoperability persistent: A 3D geo database based on CityGML
14
Institute for Geodesy and Geoinformation ScienceInstitute for Geodesy and Geoinformation Science
November 13th 2008
SimplifiedUML Schema
Simplification of
CityGML‘s data model a
CityGML Xsd Schema<xs:complexTypename="CityModelType"> <xs:complexContent> <xs:extension ...
UML Schema
Schemasimplifi-cation
Creation of an import and export tool for CityGML instance
documentsc
Javabinding
(JAXB)
Schema derived classes
public class CityModel {… }
SQLqueries
(Imp/ExportTool)
Import data
Export data
Derivation of the relational database schema
b
Mapping classes to
tables
Oracledatabase
SQL DDLstatements
(JDeveloper)
Databasecreation
Relationaldiagram
Derivation of the relational database schemaDerivation of the relational database schema
Stadler, Nagel, König, Kolbe: Making interoperability persistent: A 3D geo database based on CityGML
15
Institute for Geodesy and Geoinformation ScienceInstitute for Geodesy and Geoinformation Science
November 13th 2008
Mapping strategies for aggregation hierarchies
Derivation of the relational database schemaDerivation of the relational database schema
<<Geometry>>_BRepGeometry
<<Geometry>>Polygon
+isSolid : boolean [1]+isComposite : boolean [1]+isTriangulated : boolean [1]
+geometry : Polygon [1]
<<Geometry>>BRepAggregate
0..1
1..*
bRepMember <<Geometry>>_BRepGeometry
<<Geometry>>Polygon
+isSolid : boolean [1]+isComposite : boolean [1]+isTriangulated : boolean [1]
+geometry : Polygon [1]
<<Geometry>>BRepAggregate
0..1
1..*
bRepMember
BRepAggregateID: NUMBER <<PK>><<FK>>
Polygon
ID: NUMBER <<PK>><<FK>>GEOMETRY : SDO_GEOMETRY
BRepGeometryID: NUMBER <<PK>>IS_SOLID : BOOLEANIS_COMPOSITE : BOOLEANIS_TRIANGULATED : BOOLEAN
BRepAggregateID: NUMBER <<PK>>IS_SOLID : BOOLEANIS_COMPOSITE : BOOLEANIS_TRIANGULATED : BOOLEAN
PolygonID: NUMBER <<PK>>IS_SOLID : BOOLEANIS_COMPOSITE : BOOLEANIS_TRIANGULATED : BOOLEANGEOMETRY : SDO_GEOMETRY
BRepGeometryID: NUMBER <<PK>>TYPE : VARCHAR2(30)IS_SOLID : BOOLEANIS_COMPOSITE : BOOLEANIS_TRIANGULATED : BOOLEANGEOMETRY : SDO_GEOMETRY
Realisation of geometry model
Stadler, Nagel, König, Kolbe: Making interoperability persistent: A 3D geo database based on CityGML
16
Institute for Geodesy and Geoinformation ScienceInstitute for Geodesy and Geoinformation Science
November 13th 2008
Simplification of
CityGML‘s data model a
SimplifiedUML Schema
Schemasimplifi-cation
Derivation of the relational database schema
b
Mappingclasses to
tables
SQL DDLstatements
(JDeveloper)
Database creation
Relationaldiagram
Creation of an import and export tool for CityGML instance
documentsc
CityGML Xsd Schema<xs:complexTypename="CityModelType"> <xs:complexContent> <xs:extension ...
UML Schema
Javabinding
(JAXB)
Schema derived classes
public class CityModel {… }
SQLqueries
(Imp/ExportTool)
Import data
Export data
Oracledatabase
Creation of an import and export toolCreation of an import and export tool
Stadler, Nagel, König, Kolbe: Making interoperability persistent: A 3D geo database based on CityGML
17
Institute for Geodesy and Geoinformation ScienceInstitute for Geodesy and Geoinformation Science
November 13th 2008
Static core ofthe software
Creation of an import and export tool: OverviewCreation of an import and export tool: Overview
Data import
CityGML input file____ ____________ _____ ____ _________
ReadCityGML
Features
CityModel cityModel1 = new CityModel () ; …
follows instance of
Data export
_________ _________
Oracledatabase
Databaseimport
applied
Features
CityModel cityModel1 = new CityModel () ; …
Databaseexport
appliedinstance of
CityGML output file____ ____________ _____ ____ _________
WriteCityGML
follows
Access wrapper Xsd Schema<xs:complexTypename="CityModelType"> <xs:complexContent> <xs:extension ...
Javabinding
Schema derived classes
public class CityModel {… }
Import functionality
Export functionality
Stadler, Nagel, König, Kolbe: Making interoperability persistent: A 3D geo database based on CityGML
18
Institute for Geodesy and Geoinformation ScienceInstitute for Geodesy and Geoinformation Science
November 13th 2008
Characteristics of the import/export toolCharacteristics of the import/export tool
Support for CityGML files of arbitrary size (>4GB)
Concurrency of data processing through multithreadingHigh performance on standard platforms
Use of universally unique identifiers (UUIDs)
Filter options enable user-defined import and exportGML ID, GML name
Bundled import and export for data tiling (classified by IDs or Bounding Boxes)
Selection of object classes
Published as open source software
Stadler, Nagel, König, Kolbe: Making interoperability persistent: A 3D geo database based on CityGML
19
Institute for Geodesy and Geoinformation ScienceInstitute for Geodesy and Geoinformation Science
November 13th 2008
Concurrency of data processingConcurrency of data processing
Queue
Thread Pool
Worker
Worker
Worker
Thread Pool
Queue
Worker
Parallel non-blocking execution of multiple interacting tasks
Decouple compute and I/O bound Degree of parallelisation depends on number of processing
cores
Attention: Context switch Thread life cycle overhead Resource thrashing
Reuse threads by applying a pooled threads modelThread creation overhead is spread over many tasks
20
Institute for Geodesy and Geoinformation ScienceInstitute for Geodesy and Geoinformation Science
November 13th 2008Stadler, Nagel, König, Kolbe: Making interoperability persistent: A 3D geo database based on CityGML
Parser Thread(1 instance)
Converter Thread
(limited numberof instances)
Data import – step 1Data import – step 1
____ _____________ _____ ____ _____________ _____ ____ ____ _____ _________
CityGMLinput file
SAX parsing
TemporaryBuffer
BufferQueue
Featurecreation
JAXBunmarshallin
g
TopLevel FeatureQueue
__________________
_________
_________
_________
____ _____________ _____ ____ _____________ _____ ____ ____ _____ _________
21
Institute for Geodesy and Geoinformation ScienceInstitute for Geodesy and Geoinformation Science
November 13th 2008Stadler, Nagel, König, Kolbe: Making interoperability persistent: A 3D geo database based on CityGML
_________
With separateXLink storage(temporary)XLink Thread
(limited numberof instances)
Importer Thread
(limited numberof instances)
Data import – step 2Data import – step 2
TopLevel FeatureQueue
commit
XLinks XLinkinformatio
n
Databaseupdate
_________ _________
Oracledatabase
SQL creaioncommit
SQL statementQueues
Geo data
Import filter
XLinkQueue
__________________
_________
_________
_________
____ _____________ _____ ____ _____________ _____ ____ ____ _____ _________
22
Institute for Geodesy and Geoinformation ScienceInstitute for Geodesy and Geoinformation Science
November 13th 2008Stadler, Nagel, König, Kolbe: Making interoperability persistent: A 3D geo database based on CityGML
XLinkResolver Thread
(limited numberof instances)
XLinkSplitter Thread
(1 Instanz)
Data import – step 3Data import – step 3
__________________
_________
_________
_________
____ _____________ _____ ____ _____________ _____ ____ ____ _____ _________
XLinksplitting
XLink ResolverQueue
Databaseupdate
Oracledatabase
__________________
with resolvedXLinks
SQL creation
XLinks
commit
SQL statementQueue
With separateXLink storage(temporary)
_________
_________ _________
Oracledatabase
23
Institute for Geodesy and Geoinformation ScienceInstitute for Geodesy and Geoinformation Science
November 13th 2008Stadler, Nagel, König, Kolbe: Making interoperability persistent: A 3D geo database based on CityGML
Exporter Thread(limited number of instances)
Splitter Thread(1 instance)
Data export – step 1Data export – step 1
Feature
Export filter
SQL
TopLevel Feature
ID Queue
Databaseoutput
SQL
Nested feature/geometry request
__________________
Oracledatabas
e
____ _____________ _____ ____ _____________ _____ ____ ____ ____ _________
__________________
24
Institute for Geodesy and Geoinformation ScienceInstitute for Geodesy and Geoinformation Science
November 13th 2008Stadler, Nagel, König, Kolbe: Making interoperability persistent: A 3D geo database based on CityGML
IO Writer Thread(1 instance)
Exporter Thread(limited number
of instances)
JAXBmarshallin
g
TemporaryBuffer
BufferQueue
____ _____________ _____ ____ _____________ _____ ____ ____ _____ _________
CityGMLoutput file
WriteCityGML
Data export – step 2Data export – step 2
____ _____________ _____ ____ _____________ _____ ____ ____ ____ _________
__________________
Feature
Stadler, Nagel, König, Kolbe: Making interoperability persistent: A 3D geo database based on CityGML
26
Institute for Geodesy and Geoinformation ScienceInstitute for Geodesy and Geoinformation Science
November 13th 2008
Some performance measurements…Some performance measurements…
Dataset File size
Time Features /
second
N° of worke
rs
Versio-ning
enabled
Time Features /
second
N° of worker
s
1 mio LOD1 buildings
7,5 GB
1:15h
228 4 no 38min 455 10
2:25h
121 1 no 1:57h 150 1
>10.000 LOD2 buildings
fully textured
163 MB+
57 MB
25 min
7 4 no4,3mi
n42 4
w/o textures
163 MB
6,5 min
28 4 no 2min 91 4
>200 complex LOD3 objects
40,1MB
17s 12 4 no
7s 30 103:20h
0,02 4 yes
Import Export
Stadler, Nagel, König, Kolbe: Making interoperability persistent: A 3D geo database based on CityGML
27
Institute for Geodesy and Geoinformation ScienceInstitute for Geodesy and Geoinformation Science
November 13th 2008
Future plansFuture plans
Matching functionality (in progress) Recognise corresponding buildings in different datasets
Link geometries and exchange attribute
Avoid redundant information
Use the tool as backend for Web Services Broaden the applicability
Replacing the graphical user interface by a Web Service interface
Web Feature Services (WFS) may rely on existing import and export functionality
Additional export formats KML etc.
Performance optimisation
Stadler, Nagel, König, Kolbe: Making interoperability persistent: A 3D geo database based on CityGML
28
Institute for Geodesy and Geoinformation ScienceInstitute for Geodesy and Geoinformation Science
November 13th 2008
Alexandra StadlerClaus NagelGerhard KönigThomas H. Kolbe
stadler|nagel|kolbe @ [email protected]
Technische Universität BerlinInstitute for Geodesy and Geoinformation ScienceChair of Geoinformation Science
Questions ?
Stadler, Nagel, König, Kolbe: Making interoperability persistent: A 3D geo database based on CityGML
29
Institute for Geodesy and Geoinformation ScienceInstitute for Geodesy and Geoinformation Science
November 13th 2008
ReferencesReferences
3D city database (2007), www.3dcitydb.org [last access: 2008-06-20].
Döllner J, Kolbe TH, Liecke F, Sgouros T, Teichmann K (2006) The Virtual 3D City Model of Berlin - Managing, Integrating, and Communicating Complex Urban Information, In: Proceedings of the 25th Urban Data Management Symposium UDMS, Aalborg 2006.
Emgård L, Zlatanova S (2007) Implementation alternatives for an integrated 3D Information Model, In: Advances in 3D Geoinformation Systems, Springer-Verlag, pp. 313-330.
GNU Lesser General Public License, http://www.gnu.org/copyleft/lgpl.html [last access: 2008-06-20].
Gröger G, Kolbe TH, Czerwinski A (2007), Candidate OpenGIS Implementation Specification (City Geography Markup Language), Version 0.4.0, OGC Doc. No. 07-062, Open Geospatial Consortium 2007.
Snowflake Software, GO Loader product page (2008), http://www.snowflake software.co.uk/products/goloader/index.htm [last access: 2008-06-20].