Linking Data: Semantic enrichment of the existing building geometry Jeroen Werbrouck Supervisors: Prof. Pieter Pauwels, Willem Bekers Counsellor: Mathias Bonduel Master's dissertation submitted in order to obtain the academic degree of Master of Science in de ingenieurswetenschappen: architectuur Department of Architecture and Urban Planning Chair: Prof. dr. ir. Arnold Janssens Faculty of Engineering and Architecture Academic year 2017-2018
146
Embed
Linking Data: Semantic enrichment of the existing building ... · Semantic Web ontologies (BOT, PRODUCT, GeoSPARQL) are combined and extended to provide an alternative, minimalist
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Linking Data: Semantic enrichment of the existing building
geometry
Jeroen Werbrouck
Supervisors: Prof. Pieter Pauwels, Willem Bekers
Counsellor: Mathias Bonduel
Master's dissertation submitted in order to obtain the academic degree of
Master of Science in de ingenieurswetenschappen: architectuur
Department of Architecture and Urban Planning
Chair: Prof. dr. ir. Arnold Janssens
Faculty of Engineering and Architecture
Academic year 2017-2018
Linking Data: Semantic enrichment of the existing building
geometry
Jeroen Werbrouck
Supervisors: Prof. Pieter Pauwels, Willem Bekers
Counsellor: Mathias Bonduel
Master's dissertation submitted in order to obtain the academic degree of
Master of Science in de ingenieurswetenschappen: architectuur
Department of Architecture and Urban Planning
Chair: Prof. dr. ir. Arnold Janssens
Faculty of Engineering and Architecture
Academic year 2017-2018
ii
iii
The author gives permission to make this master dissertation available for consultation and to copy parts of this master dissertation for personal use.
In the case of any other use, the copyright terms have to be respected, in particular with regard to the obligation to state expressly the source when quoting results from this master dissertation.
De auteur geeft de toelating deze masterproef voor consultatie beschikbaar te stellen
en delen van de masterproef te kopiëren voor persoonlijk gebruik.
Elk ander gebruik valt onder de bepalingen van het auteursrecht, in het bijzonder met betrekking tot de verplichting de bron uitdrukkelijk te vermelden bij het aanhalen van resultaten uit deze masterproef.
Jeroen Werbrouck,
Ghent, 31 May 2018
iv
v
Acknowledgements
Underlying thesis would not have been as it is without some persons that made their
time and experience available to provide feedback on the various topics that meet in
this thesis. First of all, I would like to thank prof. Pieter Pauwels for being my promotor
during this research, for his expertise and feedback during almost an entire year and
his willingness to plan feedback sessions on very short notice. Many thanks also to
Mathias Bonduel, who spent lots of time to prepare the source files for this thesis, for
his continuous guidance on both remote sensing techniques and Linked Data, the
comments on the draft and the quick answers on my mails, regardless whether during
the week or on Sundays. Thanks to Willem Bekers, who was not involved in the thesis
from the very beginning, but jumped in when the time was right to provide new
perspectives on the thesis during the feedback sessions.
I am also very grateful to prof. Jakob Beetz, who coached me during my Erasmus
Exchange at RWTH Aachen and without whom the course of the project would have
been entirely different. The insights of the M1 Projekt under his supervision proved of
great use to the research carried out in context of this thesis. Thanks, Jakob, for
sparking my interest in ‘Architekturinformatik’, teaching me how to code, explaining
the basics of BIM and for your encouragement to dive into the world of Linked Data.
Finally, I would like thank my brother Andreas for taking the time to read the draft
version of underlying text and provide it with useful comments and questions, both on
content and writing style.
Personally, I took a lot of pleasure in the diversity of the project, in which I could
obtain and combine knowledge about topics of which I had never heard of before. The
alternation between reading, writing and coding ensured the project did not bore me a
second. I hope I succeeded in presenting all these subjects in an interesting and
accessible way and that it can form a basis for future research.
vi
Linking Data: Semantic enrichment of the existing building geometry
JEROEN WERBROUCK
Supervisors: Prof. Pieter Pauwels, Willem Bekers
Counsellor: Mathias Bonduel
Master's dissertation submitted in order to obtain the academic degree of
Master of Science in de ingenieurswetenschappen: architectuur
Faculty of Engineering and Architecture – Ghent University
Department of Architecture and Urban Planning
Chair: Prof. dr. ir. Arnold Janssens
Academic year 2017-2018
Abstract Building Information Modelling (BIM) has changed the way buildings are conceived,
planned and executed. Apart from its frequent use for as-planned buildings, BIM has
now been used for some years as a useful tool for digitizing existing buildings as well.
mostly by performing a ‘scan-to-BIM’. However, some inherent characteristics of
existing buildings are complicating such a modelling process: uncertainties,
imperfections in the built object and an interdisciplinarity that may go beyond the
traditional Architecture, Engineering and Construction (AEC) topics. In this thesis, a
method is proposed to integrate the concepts of scan-to-BIM into a Linked Data
context, which could provide some answers to these complications. Current modular
Semantic Web ontologies (BOT, PRODUCT, GeoSPARQL) are combined and
extended to provide an alternative, minimalist framework for creating a semantically
enrichened building model of existing buildings, in the form of an RDF graph. This
‘scan-to-Graph’ framework combines building topology, geospatial data and geometry
with product classes, source linking and the documentation of assumptions, with a
focus on point cloud sources. An example plugin for Rhinoceros 6 has been developed
to link such semantic information to capricious geometries based on real-world objects.
Finally, as a proof of concept, the process is demonstrated in a case study of the
Presence Chamber of the Gravensteen castle in Ghent. With point clouds as a basis, a
geometric 3D model of this room was created, semantically enriched and serialized to
Appendix E: Presence chamber – without geometry ................................. 111
xviii
List of abbreviations
AEC Architecture, Engineering, Construction
ALS Airborne Laser Scanning
BIM Building Information Modelling
BOT Building Topology Ontology
BRep Boundary Representation
CAD Computer Aided Design
CHML Cultural Heritage Markup Language
CIDOC International Comittee for Documentation
CIDOC CRM CIDOC Conceptual Reference Model
DSLR Digital Single Lens Reflex-camera
E-CRM Erlangen Conceptual Reference Model
EDM Europeana Data Model
FM Facility Management
GIS Geographic Information Systems
HBIM Heritage Building Information Modelling
HDS High Definition Surveying
IFC Industry Foundation Classes
LBD Linked Building Data
LD Linked Data
LOA Level of Accuracy
LOS Lines of sight
MEP Mechanical, Electrical, Plumbing
NURBS Non-uniform Rational B-Spline
OWL Web Ontology Language
RDF Resource Description Framework
RDFS Resource Description Framework Schema
SPARQL SPARQL Protocol and RDF Querying Language
STEP Standard for the Exchange of Product Model Data
STG scan-to-Graph
TLS Terrestrial Laser Scanning
TS Total Station
.ttl Turtle file document
UI User Interface
URI Uniform Resource Identifier
URL Uniform Resource Locator
WKT Well-Known Text
1
Chapter I: Introduction
1.1 Main topics
The advent of Building Information Modelling (BIM) has brought a revolution in the
way a construction project is designed, analysed and managed. Information exchange
between the various stakeholders that are involved in such a project being its main
reason of existence, BIM addresses the need for a ‘central hub’ that contains
architectural, engineering and ‘Mechanical, Electrical, Plumbing’ (MEP) information.
The industry that combines these topics is often called the AEC Industry (Architecture,
Engineering, Construction). A BIM model (from here on: ‘a’ BIM) allows these
disciplines to communicate and interact in a much more streamlined way than before.
Less known than the usual use of BIM for planning a new building (‘as-planned’) is its
potential to manage existing buildings; for Facility Management (FM), for renovation
projects, for building demolishment and for Cultural Heritage. BIM for existing
buildings includes several categories:
- ‘As-constructed’ refers to a BIM for the construction phase of a building, typically
until completion;
- ‘As-built’ refers to an updated ‘as-constructed’ BIM based on what is actually built;
- ‘As-is’ refers to a BIM for the operational phase of an existing building. An as-is
BIM in the case of heritage is sometimes denoted as HBIM;
- (‘As-was’ refers to a BIM of a building that does not exist anymore).
This work primary focuses on ‘as-is’ BIM. The results will be applicable to both non-
heritage and heritage buildings. As there is no agreed definition on what ‘BIM’ really
is, a short explanation on the use of the term in this thesis imposes itself. In this text,
BIM refers to a semantically enrichened model which contains essentially both the
topology and geometry of a building. This very broad definition allows the modular
approach that will be used in this work to be denoted as BIM.
The procedure to create an as-is BIM is different from a ‘normal’ as-planned BIM. Since
a BIM for an existing building should reflect its real ‘out-of-plumb’ geometry,
conducting real world surveys is the first step. The primary source for creating an as-
is model is a point cloud: a huge list of 3D points made by a laser scanner or using
photogrammetry. Therefore, the process to make a BIM based on such point clouds is
called respectively ‘scan-to-BIM’ or ‘photogrammetry-to-BIM’ (often both denoted as
‘scan-to-BIM’) (Chapter II). When the point clouds have been processed, the modelling
takes place, alternating with a deviation analysis between the model and the point
cloud. For scan-to-BIM to become a frequently applied, mature process, it needs first
to overcome several obstacles.
The first obstacle is related to the nature of current BIM and CAD software packages:
they are optimized for planning new buildings, maintaining an orthogonal fashion and
2
a strong preference for ‘platonic’ bodies without irregularities. BIM (and, to a lesser
extent, CAD) workflows are not optimised for dealing with ‘out-of-plumb’ geometries
of existing buildings. This relates to the high conversion effort from captured data to
semantic BIM objects outlined in (Volk et al., 2014): as no automatic conversion
algorithms for point cloud to simple 3D geometry are present, the geometric conversion
nowadays largely happens in a manual or semi-automatic way. Further on, current
BIM packages provide only a limited amount of product classes, which can become
difficult when one needs to model rare elements that do not have a place anymore in
newly built structures (there is, for example, no such thing as ‘IfcCapital’ to classify a
column’s capital). Lastly, BIM’s focus on new buildings limits the implementation of
efficient ways to add metadata to the project: what was the modelling source? Which
assumptions were made when information was not available or uncertain?
The next challenge relates to the ecosystem of current BIM software. Because it is
optimized for the AEC industry, it is difficult for other disciplines to become part of
the BIM story. Geographical or historical data are only two examples that could
provide valuable attributes. Efforts are made to integrate GIS (Geographic Information
Systems) and BIM with one another, which would for example enable a deeper insight
in the building site, soil materials, underlying aquifers etc. Compatibility with historical
information would be a relevant feature for as-is models specifically: for example, when
making a BIM from a building that is considered built heritage, implementing its
cultural information in the model is relevant (e.g. relating building elements to
historical events that shaped and influenced the building’s appearance). This way, a
BIM of the building could serve as a central model for a ‘virtual museum’. Such
integration of other disciplines would engage in the ambition for BIM to provide a
central information hub, made it even go beyond the typical topics of the AEC industry.
In the meanwhile, independent from BIM, there emerges a global trend where various
disciplines interlink their knowledge, on a data level instead of on document level. This
interdisciplinary ‘web of linked data’, based on the principles of the ‘classic’ world wide
web, is called the ‘Semantic Web’ and uses a universal standard ‘language’ for
representing and connecting knowledge: the Resource Description Framework (RDF)
as the basis for worldwide accessible, connected graphs of data (Chapter III). Fig. 1.1
shows the ‘Linked Open Data Cloud, as an illustration of different disciplines,
interconnected by the Semantic Web. It is clear that some disciplines are already firmer
rooted in the Semantic Web than others. Although still mainly in academic
environments, there is also a trend to translate the current BIM practices of the AEC
industry into the principles of Linked Data, thereby participating in the Semantic Web,
as it could provide various advantages compared with current BIM (Pauwels et al.,
2017b).
The use of Linked Data for semantic enrichment of existing buildings (as-is) will be the
core topic of this research project.
3
Fig. 1.1: The Linked Open Data Cloud (source: http://lod-cloud.net/. s.d. Last visited on 29/05/2018)
1.2 Research questions
1.2.1 Linked Data applied to scan-to-BIM
In this thesis, the concept of scan-to-BIM will be transferred into a Linked Data
context, to investigate if Linked Data concepts could provide answers to the above
outlined research challenges for as-is BIM creation, specifically related to
interoperability, metadata implementation and product classification. A scan-to-BIM
process that is performed using Linked Data semantics rather than standard BIM data
structures could then be classified as ‘scan-to-graph’ (or ‘photogrammetry-to-graph’).
The main research question will lie in how a minimal framework for such a scan-to-
graph process could be defined, based on existing, modular Linked Data ontologies,
complemented with definitions that were made in the context of this project (Chapter
IV). The focus will lie on the building topology, the building elements and a way to
handle modelling uncertainties. Although further extension of this framework with
historical or geographic data is not a part of the research question, the different
perspectives on Linked Data of the AEC industry and Cultural Heritage will be
compared, in order to provide a critical background for developing the scan-to-graph
framework (Chapter III).
4
1.2.2 Development of a scan-to-Graph environment
Because an accessible user interface to perform such ‘scan-to-graph’ process is currently
non-existent, another ambition of this work consists in developing a prototype for such
UI, as a plugin for an existing 3D modelling environment (Chapter V). Since we
disconnect from existing BIM applications, a ‘classic’ 3D CAD package suffices for
geometry modelling. Such CAD software is even preferred, as they are optimised for
modelling 3D geometry, which will benefit the manual conversion of point cloud to 3D
objects.1
One of the goals for the end products of this thesis is to make them as ‘open-source’ as
possible. The ‘5 ★ open data’2 is used (Fig. 1.2) as a guideline for the graphs that are
produced by the plugin and for the plugin code itself. In the best case, a 5★ ‘data
openness’ is reached: non-proprietary, URI-denoted and interlinked data. Note that in
the case of building data, general access to all data seldom occurs, because of property
rights, safety etc. Therefore, the reconstructed ‘as-is’ geometry of the case study on the
Gravensteen Castle in Ghent can only be made accessible on request. Yet this has no
influence on the data formats that are used in the proposed scan-to-graph framework:
whenever possible, open data standards will be used to further enhance interoperability
and independence from proprietary formats: e57 for point clouds, STEP (Standard for
the Exchange of Product Model Data) for geometry and RDF as the intrinsically open
framework that ‘glues’ everything together. An exception on this ambition is the
dependency on and the provided compatibility with Rhinoceros: since the plugin runs
on Rhino 6, the creation of the graph is currently restricted to this particular
environment. Since the 5 ★ of open data only refer to the availability of the data, and
does not state the way it is created, this ‘dependency’ is not really problematic. All in
all, querying the graph and the extraction of geometry can both happen independently
from Rhino and the plugin will be freely available at the GitHub repository of this
project.3
Fig. 1.2: Five Star open data scheme (source: http://5stardata.info/en/, accessed 3/5/2018, updated 31/08/2015)
1★ Make your stuff available on the Web
(whatever format) under an open license;
2★ Make it available as structured data (e.g.
Excel instead of image scan of a table);
3★ Make it available in a non-proprietary
open format (e.g. CSV instead of Excel);
4★ Use URIs to denote things, so that
people can point at your stuff;
5★ Link your data to other data to provide
context.
1 (Mezzino, 2017) reports several modelling constraints when using Revit for modelling geometry, especially when modelling complex shapes. 2 http://5stardata.info/en/ 3 https://github.com/JWerbrouck/scan-to-Graph
5
1.3 Proof-of-concept: Case study
As an illustration of both the proposed scan-to-graph framework and the plugin, a case
study on the presence chamber of the Gravensteen Castle in Ghent will be carried out
(Chapter VI). The case study includes segmenting the available point clouds, as-is
modelling of the 3D model, geometric comparison (deviation analysis) with the point
cloud sources and finally creating a Linked Data graph of the model, by use of the
plugin.
1.4 Thesis Structure
Apart from the research itself, it is this thesis’ ambition to be comprehensible to anyone
with a background in construction and architecture, especially students that are writing
a master thesis about a similar subject and have to obtain a background on the topics
of remote sensing and Linked Data. Because both remote sensing and Linked Data are
typically not included in an architecture student’s curriculum, the next two chapters
offer an introduction to these topics and their applications and implications for the
AEC industry. The fourth chapter discusses the general approach and decisions that
led to the template that will be the basis for the data graphs that are generated by the
plugin. It includes ontologies, geometry and the general structure of the graphs. The
functionality of the plugin itself will be discussed in Chapter V, as well as a method to
overcome its current limitations. Chapter VI illustrates the proposed scan-to-graph
process and the plugin by a case study of the presence chamber of the Gravensteen
Castle in Ghent. As an emphasis is laid on used methods, this chapter sometimes takes
the form of a tutorial. It starts with an overview of the available point cloud sources
and the method that is used to ‘prepare’ the point clouds for modelling in Rhino. This
is followed by the modelling stage, geometric deviation analysis and finally, the
serialization into an RDF-based graph. Finally, Chapter VII combines retrospect and
prospect in a discussion about the project and some perspectives on possible future
work.
6
Chapter II: Remote Sensing
Whenever coping with existing buildings (whether in case of heritage, renovation,
education…), the access to reliable information is of the utmost importance. Although
original plans and sections mostly provide a decent idea of a building ‘as-planned’, they
do not provide information about its actual ‘as-is’ or ‘as-built’ state. To obtain the as-
is building geometry, real-world surveys are inevitable. There are a number of possible
techniques to get spatial information about a structure, which are depicted in Figure
2.1. These techniques can be grouped into two large parts: non-contact techniques and
contact-techniques. Nowadays, the most efficient methods are non-contact techniques:
they are quick, reliable and touching fragile or unreachable surfaces is not necessary.
Non-contact techniques are often referred to as ‘remote sensing’ techniques.
A total station (TS) measures exact locations of discrete points on a surface, which
can be used as ‘control points’ to reconstruct the building’s geometry digitally. Laser
scanners and photogrammetric surveys, on the contrast, return an immense amount of
data in the form of a ‘point cloud’, a digital spatial representation of the object
containing information about millions of points. Therefore, laser scanning and
photogrammetry are called High Definition Surveying (HDS) techniques. Point clouds
are de facto the most complete sources for digitalisation of real-world objects (buildings,
landscapes …) that are currently available. However, they are essentially just a
collection of vectors containing spatial coordinates, sometimes also incorporating RGB
(colouring) values, normals and an Intensity value. They do not provide any
information about the nature of the object being scanned (is it a door, a chair, a
column?), its boundaries or the membership of a larger surface: each point is just
‘hanging’ in the digital space, independent from the others. Although a point cloud
provides a lot of geometrical information, it is inherently semantically poor. This means
there is a huge amount of data being used (Gigabyte file sizes being rather rule than
exception), with no return in terms of typological information. Therefore, in many
Fig. 2.1: Systematic overview of data capturing and surveying techniques to gather existing buildings' information (source: Volk et al., 2014)
7
cases, point clouds serve as a basis for more storage-friendly and semantically rich
models. In the construction industry, this is mostly a Building Information Model
(BIM).
This chapter starts with an introduction to both Terrestrial Laser Scanning (TLS)
(section 2.1) and Photogrammetry (section 2.2). Then, some situations are discussed
in which both techniques are complementary (section 2.3). Finally, the concept of ‘Scan-
to-BIM’ is introduced: the process of modelling a BIM with point clouds as a blueprint
(section 2.4).
2.1 Laser scanning
Within the field of laser scanning, several techniques for capturing geometrical data
exist. Airborne Laser Scanning (ALS) is used mostly when digitizing entire
topographies (e.g. for geographical or archaeological surveys), since the airplane to
which the scanner is attached can fly over the landscape without any obstructions.
When the scanning positions are located on the ground, the process is called Terrestrial
Laser Scanning (TLS). Apart from ‘static’ TLS (the scanner mounted on a tripod),
there are some subtechniques such as Mobile Laser Scanning (the scanner attached to
a vehicle) or Handheld Laser Scanning, both dealing with movements of the scanner
and therefore typically less accurate than static TLS. For building projects, static TLS
is used the most.
2.1.1 Principles of Laser Scanning
The basic principle of laser scanning is pretty straightforward. Rotating around their
axes, each second thousands (up to a million and more) of laser pulses are generated,
which are then reflected on whatever object they ‘collide’ with and recaptured by the
scanner’s receiving sensor. Then, several methods can serve to determine the distance
of the scanner to the point of reflection. One of them is the so-called direct ‘time-of-
flight’ method (Fig. 2.2): in short, to obtain the distance between scanner and measured
point, it multiplies the time t between sending and receiving the pulse with the velocity
of light c, then dividing it by two, because the pulse travels twice the scanner-point
distance: d = c * t/2. The other
common method involves a phase
difference: the distance is calculated by
emitting a continuous laser beam with
a slightly changing wave phase and
measuring the phase difference between
the emitted and received wave. A
scanner that uses phase difference
typically has a shorter range, but a
higher rate than one that uses time-of-
flight (Nuttens et al., 2014).
Fig. 2.2: principle of Time-of-Flight (source: van Genechten, 2008)
8
The resulting point cloud depends greatly on the scanner being used, even when using
the same technique. Other influencing factors are the operating settings, the calibration
of the scanner, incident angle, the object’s material and the registration process. Table
2.1 shows a comparison between typical orders of magnitude for time-of-flight scanners
Minimum distance 1 – 5 m 0,3 – 0,5 m Maximum distance 300 – 6000 m 80 –180 m Precision (length) 3 – 5 mm @50 m 2 – 3 mm @50 m Precision (angle) 0,0002 – 0,01 ° 0,001 – 0,007 ° Weight 10 – 20 kg 5 – 15 kg Table 2.1: comparison between Time-of-Flight and Phase Difference scanners (source: Dubois et al., 2017)
2.1.2 Occlusions
A laser scanner relies on ‘Lines of Sight’ (LOS): it cannot ‘see’ points that lie behind
an opaque object. Zones with large ‘occlusions’ (or ‘shadows’) occur frequently within
one individual scan. Figure 2.3 shows a room scanned in a survey of the Gravensteen
Castle executed by the city of Ghent in 2008. The room was digitized with one single
scan, resulting in a point cloud with large occlusions. Taking multiple scans on different
locations reduces the occlusions, because most of the time, there will be at least one
position from where the scanner can ‘see’ the parts of the objects that were occluded in
other scans.
Fig. 2.3: Occlusions in a TLS scan of the Gravensteen by the City of Ghent, 2008. The circular occluded area indicates the scanner position. (software: Autodesk Recap Pro)
9
2.1.3 Registration
By taking several scans within one room, a dense overlap between points is created,
which can be used for aligning the scans to each other, a process called ‘registration’.
In a simplistic way: detecting zones that represent the same space in the different point
clouds allows for the spatial transformation of one point cloud to align to the other one
(using closest point algorithms etc.).4 Although manual registration is still an option in
registration software packages, registration is nowadays done mostly automatically.
After the alignment of the point clouds, the total cloud is typically georeferenced. This
means the aligned point clouds are linked to the location of the scanned object in the
world. In order to do this, this location has to be taken during the survey, e.g. by use
of a GPS.
2.2 Photogrammetry
“Photogrammetry encompasses methods of image measurement and interpretation in
order to derive the shape and location of an object from one or more photographs of
that object” (Luhmann et al., 2006). The basic concepts of photogrammetry are not
new; they exist almost as long as photography itself: in 1859, the Frenchman Laussedat
explained to a commission the Paris Academy of Sciences how one could find 3D
coordinates of an object based on two photographs (Kraus, 2012). In 1885, the German
Meydenbauer established the first photogrammetric institute for documenting heritage
objects (Albertz, 2002). The principles they used were rooted in the same mathematical
equations that are used today in digital object reconstruction based on photographs.
The greatest advantage of photogrammetry towards laser scanning is the cost of the
equipment, which is typically a lot lower than the cost of a TLS scanner (although
there exist specialized photogrammetry cameras for which the price tag approaches the
price tag of a TLS).
Although photogrammetric measurement is, given the right circumstances and to a
certain degree, possible with two images (“stereophotogrammetry”), a ‘complete’ 3D
model can only be accomplished when multiple overlapping images are combined and
oriented in a three-dimensional space. Detection of corresponding features between
pictures used to be done by hand. Now, computer vision algorithms are capable of
detecting these features both quickly and accurate.
When pixel zones with similar characteristics in different pictures are linked to each
other, their 2D coordinates in the pictures system can be used to resolve the
mathematical equations that define the photograph’s location relative to the other
photographs (‘Relative Orientation’) and in the object’s 3D system (‘Absolute
Orientation’) (Fig. 2.4). After the position of the photographs is calculated, a ray is set
4 This is the method used in a cloud-to-cloud based registration process.
10
out, defined by a pixel and the focal
point of the camera objective. The
(interpolated) intersection of this ray
with the ray coming from the
equivalent pixel in another
photograph, reconstructs the point
spatially (Fig. 2.4). Further iterations
and computer vision algorithms can
create an extremely dense network of
points, resulting in a point cloud,
comparable with those made by a
laser scanner. Figure 2.5 shows an
extremely dense photogrammetry-
based point cloud made as a test case
on a part of the Ponttor in Aachen,
Germany. Note that for making an as-is geometrical model of an entire building, such
resolution is unnecessary (and might even be very contraproductive by slowing down
the computer). Further on, the density of the points is no indication about the
(measured) accuracy of the point cloud (section 2.4.2.2).
Fig. 2.5: Dense point cloud reconstruction from a part of the Ponttor in Aachen, Germany (source: Author; camera: Pentax K-50; software: Agisoft Photoscan Professional (trial version))
In the case of laser scanning, the equipment had a reasonable influence on the resulting
point cloud. This is also the case when using photogrammetry (sensor size, objective),
but additionally, there is the impact of lighting and weather conditions, camera settings
and the quality of the feature detection and reconstruction algorithms. Additionally, a
photogrammetry-based point cloud has to be set to the correct 1:1 scale by additional
Fig. 2.4: Picture association in a multi-image setting (source: Schwermann, R., Lecture slides for the course
‘Photogrammetrie’ at RWTH Aachen, 2017-2018)
11
on-site reference measurements.5 Further on, the topics on occlusions, georereferencing
and registration stay valid for photogrammetry-based point clouds. Because this thesis
is not a work on photogrammetry, an elaborate discussion about the mathematical and
geometrical backgrounds of photogrammetry is not relevant. The interested reader can
find more information on this in (Kraus, 2012; Luhmann et al., 2014).
2.3 Combining laser scanning and photogrammetry
Although laser scanning and photogrammetry may seem competing surveying methods,
they can complement each other in many cases. The following section discusses some
differences and how they can be combined.
2.3.1 Resolution
Because the respective point clouds originate in a different way, they are not exactly
the same. This starts at resolution level: photogrammetry typically performs better
regarding resolution in function of the distance (Fig 2.6b). On the other hand, the
accuracy of a photogrammetric process decreases quadratically with the distance, while
in the case of a laser scanner, this happens linearly (Fig. 2.6a). In theory, when point
clouds are available from both laser scanner and photogrammetry, they can be
combined to achieve a point cloud that has best of both: the more accurate laser-
scanned point clouds can serve as a basis to ‘correct’ the less accurate but denser (higher
resolution) photogrammetry-based point clouds. However, in practice, it rarely happens
that the same object is scanned multiple times with different methods. Additionally,
the quality of the resulting point cloud depends on the merging algorithm being used.
Fig. 2.6a-b: Comparing TLS and photogrammetry in point deviation (left) and resolution (right) as a function of the measuring distance (source: R. Schwermann,
lecture slides for the course ‘Photogrammetrie’ at RWTH Aachen, (2017-2018))
5 Although an indication might be given if the sensor size and focal length are known, by use of trigonometry. For example, for the on-site measurements a TLS or TS can be used.
12
2.3.2 Accessibility
Because photogrammetry essentially only needs a camera on site, the equipment is
much lighter and more manageable than in the case of laser scanning.6 It can, for
example, be used for capturing information in very small rooms (e.g. winding staircases
or sanitary rooms). Further on, a camera is light enough to be mounted on a drone,
which is able to make aerial pictures, hereby collecting information about the building’s
roof or other places that would in no way be accessible with heavy TLS equipment (in
the case of Airborne Laser Scanning (e.g. for detailed geographic surveys), the scanner
is mostly mounted on a plane, although there exist exceptions7).
2.3.3 Monoplotting
Another technique that combines laser scanning and
photographs is called ‘Monoplotting’. Admitting,
monoplotting does not involve ‘real’
photogrammetry, since the pictures do not deliver
any spatial information; as has been indicated in the
previous section, recovering 3D information is only
possible when there are at least 2 different,
overlapping pictures. However, when the geometry of
the object is already known (e.g. because there is a
laser scanned point cloud or the surface is a simple
plane), having a single image can provide additional
information about the object. For example, a point
cloud originating from a laser scan only has x, y, z
coordinates and an intensity value I. ‘Plotting’ the
image from the correct angle onto the point cloud,
provides it with colouring information, valuable for visualizing, modelling or mapping
detailed multispectral information on the points for futher analysis (Liao and Huang,
2012). Most contemporary laser scanners have a built-in high-resolution camera that
takes pictures immediately after scanning, ‘automatically’ adding an RGB dimension
to the point cloud. Figure 2.7 shows a ‘Riegl VZ-2000i’ laser scanner, which has literally
a DSLR camera mounted on top (mostly, it is integrated). Correction for the different
location of scanner and camera is done afterwards.
2.3.4 Edge detection
Although laser scanning is one of the most complete surveying methods existing right
now, automatic edge detection remains an issue that is inherent to the technique.
Because the scanner identifies single points, determining an object’s edges can only be
approached. On the other hand, automatic edge detection in photographs has taken
serious improvements, so that the positions of corners and edges can be estimated on
6 Note: as indicated before, to bring the resulting point cloud to the exact 1:1 scale, additional measurement on-site is nevertheless needed. For georeferencing the point cloud, additional on-site GPS equipment is necessary. 7 https://www.3dlasermapping.com/riegl-uav-laser-scanners/, accessed 25/05/2018
a sub-pixel level.8 Situating the found edges spatially can help in segmenting the point
cloud into object regions, which in turn may be used for automatic object recognition.
A reliable object recognition for point clouds is currently still in research phase; but
along with the progress made in the field of computer vision, this holds some promising
prospects regarding automatic conversion of point clouds to Building Information
Models.
2.4 Scan-to-BIM
As indicated in the introduction of this chapter, point clouds an sich do not contain
any semantic information, as they are in fact just points that are distributed in space,
often provided with some extra dimensions (intensity, colour …). The ‘scan-to-BIM’
process involves the creation of a semantically rich Building Information Model from
an existing building, with point clouds as a basis. The creation of such an as-is BIM
opens a whole range of possibilities, regarding the management and planning of
renovation projects, cultural heritage conservation, analysis of the building life cycle,
facility management and more. Apart from serving as a blueprint for the creation of a
BIM model, the point cloud can also serve as an important instrument of control: e.g.
by regularly making scans of the project under construction, the construction process
can be monitored in real-time (Dubois et al., 2017). This section discusses the benefits
and the challenges of creating a BIM from an existing building, which may be based
on point clouds, plans, sections and other documentation that is available, such as an
as-planned BIM (which can show large differences with the actual building). In-depth
discussions regarding scan-to-BIM processes are documented by Thomson (2016) and
Mezzino (2017).
2.4.1 As-planned/as-is BIM
The profits one gains from having a BIM and its influence on a structure’s life cycle
depends on the moment it is made. When created at the early conception of the project,
a BIM helps to manage nearly all stages of the building construction, from the inception
until far in the production phase. A BIM that is an updated version of an as-planned
BIM or made from an already existing building can additionally provide an improved
building maintenance, renovation, regulations compliance and, finally, demolishing
process. Figure 2.8 shows the different ways a BIM model can be created, along with
their influence on the building life cycle. Note that renovations are included in
‘maintenance’.
8 source: R. Schwermann, notes for the course ‘Photogrammetrie’ at RWTH Aachen (2017-2018)
14
Fig. 2.8: BIM model creation processes in new or existing buildings depending on available, pre-existing BIM and LC stages with their related requirements (Source: Volk et al., 2014)
2.4.2 Challenges in creating an as-is model
Despite its usefulness, the use of BIM for existing buildings is currently limited.
According to (Volk et al., 2014), this so far scarce implementation has several reasons
(see also Chapter I):
- The costly, time-intensive and error-prone process of reverse-engineering point clouds
to an as-is BIM, which happens nowadays mostly manually or semi-automatically;
- The handling and modelling of uncertain data, objects and relations due to the lack of
information, occlusion in point clouds or information which is simply not accessible with
remote-sensing techniques (e.g. the constituent materials of a wall);
- The lack of an incentive for continuous information maintenance and assessment in
BIM.
Other challenges include the interdisciplinarity often needed in such project (heritage,
GIS, FM, etc.) and the need for an efficient classification system for non-typical
building elements. Since the amount of different products is countless, such
classification system would rather be an extendable framework than the limited amount
of modern-product classes provided by current BIM packages (section 3.6.4).
However, the need for having a tool for efficiently managing (and adapting) the existing
building stock will only continue to grow as more and more governments are restricting
the unbridled expansion of urban development (e.g. the 2040 ‘Betonstop’ planned in
Flanders), making renovations of existing buildings more relevant. Furthermore, an
efficient and ecological use of resources in the building life cycle becomes more
important. Having a centralized model of the existing building thus becomes more
interesting, despite the problems outlined above. The amount of work that goes into
such a model depends on the building age: either the building has been built quite
recently, which means there may be an already existing BIM that could be updated
(‘as-built’), or it dates from the pre-BIM epoch, which means that, in the best case,
there are other documents available (plans, sections …). If the available documents are
15
not sufficient, surveying techniques such as laser scanning and photogrammetry can fill
the void, and a scan-to-BIM process is carried out.
2.4.2.1 Intensive modelling process
The ‘intensive modelling’ is the greatest obstacle for creating as-is and as-built BIMs.
A building is never built exactly as planned and perfect orthogonality is seldom
observed in existing buildings, especially when it concerns heritage. “The challenge then
becomes how does one represent the real world out-of-plumb conditions? […] To
complicate matters further, many of today’s design software packages prefer to work
in an orthogonal fashion and are limited in their ability to accurately represent out-of-
plumb conditions” (U.S. Institute on Building Documentation, 2016). Orthogonal
modelling thus brings a lot of errors into the model, non-orthogonal modelling is much
more time-consuming.
An automatic approach that segments and labels the point cloud and efficiently and
accurately generates a BIM out of it, will not become reality in the near future,
although there is a lot of research going on to enhance (semi)-automatic object
recognition and geometry reconstruction out of segmented point clouds. The last part
of section 2.3 indicated already that there are currently no fully automatic solutions
can already significantly reduce the time that is spent on modelling, but still require a
lot of user input. Progress made in the fields of deep learning and computer vision,
along with semantic nets, already provided a basis for more semi-automated algorithms
(e.g. Bassier et al., 2017; Nan et al., 2012; Ochmann et al., 2016; Shi et al., 2016) and
is expected to further automatize this process.
2.4.2.2 LOA specification
In order to specify the accuracy requirements of the model, the ‘USIBD Level of
Accuracy (LOA) Specification Guide’ (U.S. Institute on Building Documentation,
2016), defines five LoA levels (Table 2.2), ranging from LOA10 (low accuracy) to
LOA50 (high accuracy). The distance ranges correspond with a standard deviation of
2σ, which is equal to the 95th percentile. A difference is also made between the real
object surface and the measurement representation (e.g. point clouds), since no
measuring method is exact. Therefore, the framework defines two types of accuracy:
Measured Accuracy and Represented Accuracy. Measured Accuracy is about the final
measurements, regardless of the method used to acquire those measurements.
Represented Accuracy stands for the deviation range once the measured data is
processed into some other form (e.g. a 2D drawing or 3D model). The difference
between the true object surface, measured points and represented surface is shown in
Figure 2.9.
16
The next challenge is the method to use for analysing the model’s (or its substituent
parts’) LoA, since different methods exist with different results. The simplest method
is determining the distance for each point (possibly after subsampling first) to the
closest point on the model. An extension that is proposed by (Bonduel et al. (2017))
divides the determination process in two steps, with an analysis on macro- (the entire
building) and microscale (separate building elements). The microscale method brings
occlusions into account by segmenting the point cloud in occluded and non-occluded
parts. This allows to determine the actual LoA of the building element in a quantitative
manner.
2.4.2.3 Documenting uncertainties
The uncertainties and assumptions that are made while making the digital model need
to be kept and linked to the BIM model, as well as the sources they are based on. It
would be very misleading if assumed object geometries (e.g. when the object is partly
occluded in the source point cloud) or attributes (e.g. the substituent layers of a wall)
were modelled as if they were known for sure, especially when working in a team.
Assuming the boundaries of an object that is occluded may be based on rational
thought, but it stays nevertheless an assumption. In the same way, point cloud
cluttering around glazing or other reflecting elements may prevent the correct
Table 2.2: LoA specification according to the USIBD Level of Accuracy (LOA) Specification Guide (source: U.S. Institute on Building Documentation, 2016)
Fig. 2.9: difference between the true object’s surface, the measured points and the modelled representation (source: U.S. Institute on Building Documentation, 2016)
17
modelling of the glass thickness. Such uncertainties may influence the overall outcome
of building performance analyses, structural analyses etc. Also applications therefore
need to know which data is uncertain (preferably in a quantified way), so they can take
this into account in the probability of their outcome. Keeping track of assumptions and
the sources the model is based on may also indicate the need for further surveys.
No established standards to link such assumptions to a BIM were found, as current
BIM standards mainly focus on new buildings. However, Barbosa et al. (2016) argue
they could be extended to apply to existing ones, too, by incorporating guidelines (both
for software developers and end-users) for coping with the above outlined issues. For
example, to better handle uncertainties or in providing better integration of data
formats that are used in an as-is modelling process (e.g. point clouds). Keeping
metadata about the assumptions, deviations an more may be done using Linked Data
techniques, applied to a graph-based BIM rather than to a ‘regular’ BIM (Chapter II,
III).
2.4.2.4 Information maintenance and assessment
Information maintenance and keeping the model up-to-date hugely depend on the
facility manager and the amount of work that is needed to do this. Augmented Reality
applications that link the real building to a BIM or a semantic graph could lower the
step for active participation of facility managers in keeping the model up-to-date and
also keep track of the changes that have been made earlier.9
2.5 Conclusion
This chapter introduced TLS and Photogrammetry, two remote sensing techniques
that generate a digital point cloud as output, which can be used for digital
reconstruction of buildings. Apart from the main working principles, the factors that
influence the outcome have been discussed. It has been shown that these methods can
complement each other to keep the best elements of both. Lastly, the principle of the
scan-to-BIM process was laid out, as well as the main challenges current scan-to-BIM
Together with scan-to-BIM, the focus of this thesis lies on Linked Data. This chapter
will give a layman’s introduction to the most important concepts of Linked Data. Key
concepts, such as RDF (3.2), RDFS (3.3), ontologies (3.4) and SPARQL (3.5) will be
discussed, followed by a review of some Linked Data applications in the Architecture,
Engineering and Construction (AEC) Industry, which are becoming more and more
important in current practice (3.6). The last section discusses the application of Linked
Data in built cultural heritage, using the Cultural Heritage Markup Language (CHML)
as the main example (3.7).
The basic concepts of the Semantic Web are very much the same as those for Linked
Data in general. Therefore, some of the concepts that are to be explained in this
chapter, will be more easy to grasp when explained in a Semantic Web context. There
are many opinions about the relationship between Linked Data and the Semantic
Web10. In this thesis is maintained that on the one hand, Linked Data concepts provide
the structures for a semantic web to work; and on the other hand, the information that
can be retrieved through querying ‘the’ Semantic Web, consists out of linked data. Key
for the ‘semantic web concept’ is that the data is connected to outside data.11
3.1 Note on Literature
It is neither possible nor desirable within the context of this thesis to give a full
overview of all Linked Data concepts; this would lead us too far from the actual topics
and become irrelevant soon. For an elaborate introduction to the Semantic Web (and
Linked Data), there are a number of well-written and understandable books, with lots
of examples and exercises. For an extensive and excellent covering of the topics in this
chapter, the reader can dive into (Allemang and Hendler, 2011; Segaran et al., 2009).
Following descriptions and definitions have a firm base in these books, that introduce
the concepts of Linked Data. The documentation at the website of the W3C
Consortium (www.w3.org) also gives a clear and comprehensive idea about some topics
that will be discussed. When applicable, the links to these webpages are provided as a
footnote.
10 http://linkeddata.org/faq (accessed 18/04/2018), (Pauwels et al., 2017b), … 11 The reader should note the difference between the use of ‘Linked Data’ and ‘linked data’ in this thesis: when the capitalized version is used, it refers to the framework to link data, by use of RDF. When
written lowercase, it just means ‘data that is linked’, thus talking about the data itself rather than about
the underlying structures . The same holds for ‘Semantic Web’: written lowercase, it just means an
implementation of Linked Data concepts to organize and connect larger datasets into a connected ‘web’
that can be queried for information. When capitalized, ‘the’ Semantic Web refers to the world wide
version of the previously mentioned concept, the W3C’s vision of the Web of linked data as an extension to the World Wide Web.
19
3.2 Resource Description Framework
The advent of the internet meant an immense improvement regarding document and
data exchange and collaboration around the globe. One of the keys that enable this
global network to keep growing at an unbelievable speed is the ability, and the
‘permission’ for everyone to post all things one is able to think of.12 This encouragement
for everyone to contribute to the World Wide Web is responsible for the rapid
expansion of the current internet, since everyone can state whatever he or she wants
(relevant or not) and put it online (Wikipedia, Facebook, Youtube, TripAdvisor,
personal website etc.). For an efficient information exchange (in both document web
and Semantic Web), the way this information is structured needs to obey international
agreements and rules. Note that, in this context, information exchange methods are
meant rather than the actual content of the information that is described (agreements
cannot restrain human interpretation misunderstandings). If one wants to develop or
use a Linked Data application, it is necessary to know the basic rules upon which this
application is going to be based.
As indicated on the W3C web page dedicated to the semantic web, “the ultimate goal
of the Web of data is to enable computers to do more useful work and to develop
systems that can support trusted interactions over the network.” 13 We tend to move
towards an ever-more automatized world, with information sources and stores
connected through a ‘smart web’, and more and more work is going to be done by
computers and algorithms. Information exchange is the core purpose of the Internet,
therefore, having a communication framework with which all actors can ‘speak’ with
one another is essential. Sadly, computers are not that efficient as humans in decoding
natural language into semantic meaning. While we do not perceive any difficulty in
understanding the meaning of highly complex sentences, this is an extremely
challenging, if not nearly impossible task for machines. So, for a network that is
‘understandable’ for both human beings and computer algorithms, information has to
be represented in two ways: a human-readable presentation and a machine-readable
description (Allemang and Hendler, 2011). This machine-readable description forms the
framework for the semantic web, and is standardized in the ‘Resource Description
Framework’ (RDF). 14 Currently, RDF is used widely around the world for representing,
managing and connecting information that is available on the web, either augmenting
existing web pages to be part of the Semantic Web or in constructing new applications
that need information located somewhere on a web server, with which the application
must be able to communicate unambiguously.
12 Sometimes referred to as the AAA-slogan: “Anyone can state Anything about Any topic” 13 https://www.w3.org/standards/semanticweb (accessed 20/04/2018) 14 https://www.w3.org/TR/rdf11-primer (accessed 15/05/2018)
20
3.2.1 Triples and Resources
Anything that is described in a Linked Data context is called a ‘resource’. This means
that not only ‘things’ or ‘objects’ (instances such as columns or doors), but that also
more abstract concepts (classes, values, disciplines …) and even relationships are
defined as resources. As the name already indicates, RDF provides a way to make
statements about these ‘things’ and their relationships with others.
In RDF, all information is represented by means of triples. Triples can be thought of
as very basic sentences, containing only a ‘subject’, an ‘object’ and something that
establishes a relationship between these two, referred to as the ‘predicate’. Just as in
many spoken languages around the world (including English and Dutch), one could
define RDF as a subject-verb-object (SVO) language, therefore, the word order of any
triple has a subject-predicate-object syntax (e.g. Jeroen (subject) writes (predicate)
Thesis (object)). The more triples that are defined, the more semantic meaning the
database gets: each resource becomes related to multiple others, sometimes being the
‘subject’, other times being the ‘object’. A short statement could look like this:
Subject Predicate Object
Jeroen writes Thesis
Jeroen a Student
Thesis about Linked Data
about Scan-to-Graph
…
Another way of representing RDF
information, is to visualize it into a graph.
Each node of the graph represents a
resource that relates to other resources by
edges that define their specific
relationship. These graphs are ‘directed’,
which means that the edges are like
arrows pointing from one node to the
other, making clear which one is the
object and which one the subject. By
using a graph visualisation, the
denotation as a ‘web’ of linked data
becomes obvious. Figure 3.1 displays such
a directed graph that describes a window that is located in an opening (as defined by
the Industry Foundation Classes (IFC) and its Linked Data equivalent ifcOWL).
Fig. 3.1: example of a graph describing a window in an opening (source: Pauwels et al., 2011)
21
3.2.2 Uniform Resource Identifiers and Literals
When dealing with small databases, with few persons involved, this way of defining
information through triples is fine. But if you try to scale this up to world wide web
format, some scaling issues enter. When different people are talking, it can occur that
they use different words when, in fact, they mean the same. This is a rather small issue
that can easily be overcome, since you could easily state their equivalence with another
statement. It becomes much more difficult when people use the same word for
describing things that are not the same, either with subtle nuances or huge differences.
Using the example above: there might be another ‘Jeroen’ that writes a ‘Thesis’; which
‘Jeroen’ and which ‘Thesis’ is meant? This issue is handled through the use of ‘Uniform
Resource Identifiers’, URIs in short.15 A URI provides a unique, global identifier for a
specific resource (entity or relationship). When two people (websites) want to make
sure they are referring to exactly the same topic, they refer to the unique URI of that
topic. In fact, the URLs (Uniform Resource Locators) used to refer to websites are a
specific case of URIs, that, alongside with identifying a resource, also provide the
information to locate and access this resource.16 Apart from URIs, there also exist so-
called value Literals: these contain information that is only included in the graph,
without being located somewhere else on the web. Literals can be usual datatypes such
as strings or integers, but also very specific datatypes that refer to a certain serialization
format or other datatypes. While the subject or predicate of a triple always relates to
a URI, the object might be either URI or Literal.
3.2.3 QNames
The use of URIs counters the problem of ambiguously talking about different things
using the same words. However, it makes the triples much less human-readable. An
sich, this causes no problems, since most of the time, the data is going to be read by
machines. However, when writing the triples down, “shortening” the resources into a
more easy-to-read form can be useful. The abbreviation scheme that is often used for
this, is called QNames (“Qualified Names”).17 A QName has two parts: a namespace
and an identifier; the namespace essentially just a reference to an elsewhere in the file
fully specified URI, the identifier serving for finding the exact resource present in that
namespace. For example, the URI hosting the definitions that are going to be
constructed in context of this thesis will be: ‘https://github.com/JWerbrouck/scan-to-
graph/blob/master/stg.ttl’. Now that the full URI is specified (in fact this could be
anything, as long as it is unique), we could state that when referring to concepts with
their qnames, the namespace ‘stg’ (‘scan-to-graph’) refers to this full URI. An example
of some relationship that will be defined in chapter III is the predicate
‘hasModellingRemark’. Fully, the URI would then be:
Going further, ‘door’ can be defined as a subclass of ‘building element’, which is in
turn a ‘man-made object’ etc. A similar concept exists for properties, called
rdfs:subPropertyOf. It indicates that “If a property P is a subproperty of property Q,
then all pairs of resources which are related by P are also related by Q.” 21 For example,
if ‘hasTrapDoor’ were a subproperty of ‘hasDoor’, anything that relates to something
else with ‘hasTrapDoor’, would also relate to it by ‘hasDoor’.
3.3.3 rdfs:domain and rdfs:range
Two last important concepts are rdfs:domain and rdfs:range. Both are used to make
statements about a specific property that is going to relate specific resources. The first
one, rdfs:domain, makes a statement about the class of the subject of the triple for
which the property is going to be predicate. rdfs:range does the same, but for the object
of the triple for which the property is the predicate. For example, in the Building
Product Ontology (PRODUCT) there is one object property that is defined:
product:aggregates. In the ontology description22, only the following statement is made
about product:aggregates:
Subject Predicate Object
product:aggregates a ( = ‘rdf:type’) owl:ObjectProperty ;
rdfs:domain product:Product ;
rdfs:range product:Product .
20 rdfs:class is also itself a class: it groups the resources that are a rdfs:class 21 https://www.w3.org/TR/rdf-schema/ (accessed 25/04/2018) 22 https://github.com/pipauwel/product/blob/master/prod.ttl (accessed 25/05/2018)
24
This means that, whenever two resources are related by product:aggregates, one could
derive (‘infer’) that these two resources are both instances of the class product:Product.
It is worth mentioning that domain and range are not restricting the classes of an
instance, on the contrary, they are giving implicit information about it. This is
important for ‘inferencing’, discussed later in this chapter (section 3.4.2).
3.3.4 Web Ontology Language
When the basic elements for ontology description from RDFS do not suffice, the Web
Ontology Language (OWL) provides more expressivity and a means to make logical
rules or restrictions about some resources. These rules (and proofs) can serve a
reasoning process, which deducts new relations or recognises impossible relationships.
“In short, OWL further enhances the RDFS concepts to allow making more complex
RDF statements, such as cardinality restrictions, type restrictions, and complex class
expressions. The RDF graphs constructed with OWL concepts are called OWL
ontologies.” (Pauwels et al., 2017b)
3.4 Ontologies
The Semantic Web covers an endless amount of different areas and topics, each with
its own jargon, subjects and relationships. Providing definitions that make clear how
to talk about a discipline’s topics in a Linked Data context is no luxury. Such definitions
are described in ontologies (sometimes: ‘vocabularies’). On the one hand, ontologies
define a set of concepts (classes as well as properties) that can be used within a
particular (sub)discipline. They enable stating implicit information, e.g. by setting
domain and range of a property. On the other hand, using an OWL ontology, it is also
possible to restrict the possibilities for using these concepts (disjoint classes,
cardinalities etc.).
As indicated on the webpage of the W3C about ontologies/vocabularies, “the role of
vocabularies on the Semantic Web are to help data integration when, for example,
ambiguities may exist on the terms used in the different data sets, or when a bit of
extra knowledge may lead to the discovery of new relationships. […] Another type of
example is to use vocabularies to organize knowledge.”23 Thus, ontologies do not only
define concepts ‘for internal use’, they also help in negotiating between often
contradictory information on the web, in ‘communicating’ with other ontologies.
Because of this, there is no problem in combining several ontologies in one graph: after
all, the Semantic Web is like one large graph containing all kinds of ontologies.
Therefore, smaller, modular ontologies with a specific application area are often more
manageable than large ontologies that try to be all-encompassing (and thus more
applying to practice (Rasmussen et al., 2018)). Small ontologies might be combined to
serve a specific purpose, which will be illustrated further in Chapter IV.
Just as in the case of RDFS and OWL, ontologies consist entirely out of triples
themselves; they follow the same rules as any other RDF graph and are not to be
considered an extension but rather a set of statements about certain resources. In fact,
most ontologies largely rely on RDFS and OWL: the functional parts of the definitions
are often not using much more than rdfs:subClassOf, rdfs:subPropertyOf, rdfs:domain
and rdfs:range, and then referring to Classes that were previously defined in the
ontology, or elsewhere.
3.4.1 TBox and ABox
When talking about ontologies and graphs, there is a firm distinction between resources
that are instances and resources that are concepts (classes and properties). The terms
TBox and ABox refer to this dichotomy. As TBox (‘Terminological’) refers to the
concepts, ABox (‘Assertion’) refers to the particular instances that are linked to the
concepts in a specific graph.
3.4.2 Inferencing
‘Inferencing’ is a powerful ‘reasoning’ process typical for Semantic Web applications.24
It is based on a chain analysis of triples, that enables to discover new relationships
between resources, based on the information that is present in the graph and the rules
that are specified in an ontology (or a set of ontologies). These new relationships and
information can then further enrichen the graph (or the query (section 3.5)), making it
‘more complete’ by making implicit information explicit. As indicated in the example
above, given the meanings of rdfs:domain and rdfs:range, one could define the classes
of objects and subjects, which can then in turn be used for further inferencing. A fictive
property could be prop:OpensTo, with domain product:Door and range bot:Space. The
use of a triple: ‘inst:resource1 prop:OpensTo inst:resource2’ suffices for determining
that ‘inst:resource1’ is a Product:Door and ‘inst:resource2’ a Bot:Space.25 This class
inferencing can then serve for making more information explicit.
Logical inferencing is a powerful mechanism for improving the quality of the data,
analysis and management, tracing inconsistencies in the datasets and much more.
When provided with a set of rules, semantic reasoning applications use inferencing for
rule checking, e.g. for building regulations etc. It can also be used while querying a
dataset for specific information, which is done using SPARQL queries.
24 https://www.w3.org/standards/semanticweb/inference (accessed 05/05/2018, updated 24/06/2014) 25 Note that a class inference by looking at the domain and range is by no means a restriction to the classes a resource can be an instance of: an instance can belong to several classes. The only way this can cause information
collisions is when a ‘higher’ semantic language such as OWL, in which it can e.g. be stated that 2 classes are mutually
exclusive (disjoint). Such collisions can be detected using ‘reasoning engines’, in a variant of inferencing.
26
3.5 SPARQL
A last, very important Linked Data concept is SPARQL (“SPARQL Protocol And RDF
Querying Language”).26 SPARQL is the query language used to query RDF databases
using triple patterns. In fact, a variant of Turtle is used to define SPARQL queries.
Roughly, a query consists of three parts: defining the prefixes that refer to full URIs
(cf. Turtle), then the command (‘SELECT’, ‘INSERT’, ‘CONSTRUCT’, etc.). A
SELECT query is followed by the variables that are to be ‘extracted’ (always preceded
by a ‘?’) and then the definition of the triple patterns to which the variables have to
comply (‘WHERE’). The simplest example returns all subjects, prefixes and objects in
the graph:
SELECT ?s ?p ?o
WHERE {?s ?p ?o}
Another, very simple example that queries a graph for all instances that are an
‘IfcWallStandardCase’, limiting the amount of results to maximum 10:
to enable semantic interoperability.” Rhinoceros (McNeel), which is used as the main
CAD application for this thesis, was unfortunately not included.
The ability to combine different representations of information is of great value to this
thesis, as the resulting graph will include several geometries mapped to single instances
(e.g. point clouds and STEP-files). (See Chapter IV)
Fig. 3.2: BIM Levels of Maturity (data-based BIM on the right) as defined by BSI Standards Limited, 2013 (source: Rasmussen et al., 2018)
3.6.2 Linking across domains
Currently, Building Information Modelling is the AEC industry’s main tool for
integrating architecture, construction, MEP, time and cost estimations and so forth.
However, there are still many relevant topics that are not (or insufficiently) covered by
most BIM applications, such as Geographical Information Systems (GIS), Facility
Management (FM), or heritage conservation. Due to their inherent capability to
connect information from various fields, Linked Data technologies hold promises for
applications that can integrate information coming from various disciplines, located on
various servers.
Closely related to this is the ability to search product catalogues for specific products.
For some software packages, such catalogue databases already exist and are either the
initiative of the product manufacturers themselves or third parties that provide
databases with products of different companies. An example for BIM in general is
29
BIMobject.com29, but there are also databases for very specific applications (e.g. for
Lighting Analysis), such as LumSearch30 for DIALux. The main disadvantage of these
databases is that the information they contain is mostly very software-specific. It would
save a lot of work and money when manufacturers or third parties would be able to
implement products (and the corresponding (meta)data) in a standardized procedure
that can be used by all BIM-packages. This standardized procedure can be achieved
through Linked Data.
Some initiatives have been made for providing such general building product databases.
An important one is the bsDD31 (BuildingSMART Data Dictionary), which serves as
a foundation for instance libraries and is being converted to RDF by the DURAARK
working group.32 The bsDD defines general terms that can, for example, be used for
product manufacturers to specify information about their products. Further on, its aim
is to be language independent, thus being usable worldwide. Another, smaller (and thus
more manageable) initiative for classifying building elements is the Building Product
Ontology33, which has a subontology about building elements, that are also often linked
to the bsDD or IFC classes. In this thesis, the product ontology is used as a basis for
classifying elements, because of its compactness in comparison with the bsDD and
others. A product catalogue that bundles specific products from different European
manufacturers has been constructed in context of BauDataWeb34.
According to Figure 3.3, in 2014, the state of the industry regarding product support
lied at the very beginning of ‘Level 2’, which means vendor or market specific product
libraries with downloadable components, but without standardisation. Nowadays
(2018) this state has probably shifted towards somewhere between Level 2 and the
beginning of Level 3 (this is the author’s assumption, since no up-to-date schemes were
found). As indicated in Figure 3.3, level 3 is just the beginning of open, online product
libraries, with bsDD support. While in academic environments, lots of research is being
carried out for deepening the implementation of Semantic Web technologies for the
AEC industry, everyday praxis reaching level ‘beyond level 3’ is not something that
will be achieved in the next couple of years. Note that Level 1 in Figure 3.3 corresponds
with Level 0 in Figure 3.2.
29 http://bimobject.com/en-us (BIMobject is the successor of Autodesk SEEK, which combines different kinds of BIM-families from different manufacturers) (accessed 29/05/2018) 30 http://lumsearch.com/en-US#0 (accessed 29/04/2018) 31 http://bsdd.buildingsmart.org/ (accessed 29/04/2018) 32 http://duraark.eu/tag/standardization/ (accessed 29/04/2018) 33 https://github.com/pipauwel/product (accessed 25/05/2018) 34 http://semantic.eurobau.com/ (accessed 29/04/2018)
30
Figure 3.3: Technical roadmap Product Libraries - state of the industry in 2014 (source: https://www.buildingsmart.org/standards/technical-vision/technical-roadmaps/, accessed 28/05/2018)
3.6.3 Logical inference and proof
The ability to deduce additional information from original data and the concept of
‘semantic reasoning’ can be of considerable value for querying the graph for specific
resources or rule checking environments (Pauwels et al., 2011). In context of this thesis,
inference is a more relevant feature than rule checking, since it allows to construct
powerful queries that propagate through the graph and are able to find implicit results.
An example related to this thesis could be: a user defines a SPARQL query that
propagates through the graph for all windows that have a point cloud representation
mapped to it, in order to visualize them. Inferencing that an instance of a subclass is
also an instance of the superclass, will make sure that not only the instances of the
class ‘product:Window’ will be found, but also the instances that belong to its
subclasses, e.g. ‘product:Window-LIGHTDOME’ or ‘product:Window-SKYLIGHT’.
3.6.4 About detail
Apart from the previous arguments, there is also the fact that an ontology-based BIM
approach provides a way for a virtually unlimited detailed description of a subject.
This in contrast to standard ways of classification, which are limited in their classes
(for example, there is a class ‘ifcDoor’, but not something like ‘ifcDoorHandle’ or
‘ifcDoorHinges’, which, in contrast, could be defined very easily using Linked Data).
Apart from providing advantages for as-planned BIM, this can be valuable for as-is
BIM object specification (e.g. in a heritage context: specifying a type of medieval door
knob that is, of course, not present in any manufacturer database).
31
3.7 Linked Data in Cultural Heritage
Another discipline that strongly benefits from the interconnection of data in the
semantic web is the cultural heritage sector: enhanced research collaboration and a
more interactive dissemination (e.g. in a Virtual Museum) towards an interested public
being just two examples. Worth mentioning is the Europeana Data Model (EDM), an
immense database that contains information about European cultural heritage, among
others available via a SPARQL endpoint. Another initiative is E-CRM35, an ontology
based on CIDOC Conceptual Reference Model (CIDOC CRM)36 by the university of
Erlangen-Nuremberg. CIDOC CRM is an international ISO Standard (ISO 21127) for
dealing with cultural heritage documentation in general, defining the main classes and
properties of cultural assets and the events that shaped them. Related to this is the
conversion from the XML-based “Cultural Heritage Markup Language” (CHML) to an
OWL-based Linked Data version that relies on E-CRM. CHML is a data model that
captures the process of documenting and creating a 3D reconstruction of objects that
no longer exist or never existed at all. Since this thesis relates to 3D modelling of
existing buildings, an introduction to CHML will be the main topic of this section.
3.7.1 Interpretation and reconstruction
In its focus on heritage that doesn’t exist (anymore), CHML makes a difference between
‘3D preservation’, and ‘3D reconstruction’. ‘3D Preservation’ relates to an automatic,
algorithm-driven reconstruction of faces and solids based on point clouds (for still
existing objects, which is not the topic of CHML). As we have seen in the section about
scan-to-BIM, such a fully automatic, algorithmic conversion to a semantically rich,
segmented object is yet to be developed (at least in the case of buildings). Therefore,
the ‘3D preservation’ as mentioned in (Kuroczyński et al., 2016) may probably be seen
as just a computer generated 3D model without semantically rich objects, meant for
preservation and not for querying (as this is shortly mentioned in (Kuroczyński et al.,
2016), this is an assumption made by the author of this thesis). ‘3D reconstruction’,
the main topic of CHML, is the process in which a human modeller takes an essential,
creative role in modelling something that is currently non-existent, in a scientifically
approved process based on “a broad data acquisition (primary and secondary resources),
the evaluation and interpretation of various sources, and finally digital molding, leading
to the digital 3D object” (Kuroczyński et al., 2016).
3.7.2 An event-centric model
Since CHML is based on CIDOC CRM, it takes the same event-centric approach, which
means that the focus lies on documentation of events that influenced (or were
influenced by) the object, rather than focussing on the object itself, making direct links
between the described object and its features (‘object-centric’ model, e.g. EDM)
(Kuroczynski et al., 2016). Specific for CHML, the most important ‘event’ of the
ontology is the 3D reconstruction process itself (Fig. 3.4).
‘Activities’ create and describe sources that are
used by the modeller to make the 3D
reconstruction. In CHML, the “reconstruction
activity” serves as the core event, the ‘glue of
sources, semantic objects and actors, set in
time and space’. Both physical object and 3D
reconstruction are referred to as the ‘Semantic
Object’, the abstract idea of the object, but in
the relational schema, they are only connected
indirectly, through sources and reconstruction
activities (Fig. 3.4-3.5) (Hauck and
Kuroczyński, 2014). Such a focus on sources
and the modelling activity is essential when
the object of description is non-existing at the
moment of research: it allows modelling even
when no actual object is present, when there are only 2D or textual sources (Svenshon
and Grellert, 2010). Note that such documented use of sources is also very useful for
an as-is reconstruction of an existing building. The interpretation of the sources is not
(only) the task of the modeller, but mainly of professional (art) historians or
archaeologists. The modeller, often an architect, can contribute with his expertise in
construction logic or statistics (Svenshon and Grellert, 2010).37
Figure 3.5: the CHML workflow (source: Kuroczyński et al., 2016)
37 This methodology is a general approach to 3D reconstruction of (non-existing) heritage and has no direct relationship with Linked Data
Figure 3.4: The reconstruction process as the central event in CHML. The semantic object serves as a reference to both the physical object and its 3D reconstruction.
(source: Hauck and Kuroczyński, 2014)
33
3.7.3 TYPE labelling
As the main classification system, CHML uses definitions provided by thesauri,
Wikipedia, publications etc. These definitions of an element are then bundled in a
subclass of H6_CHML_Type, which is an owl:Thing (class) itself. An H6_CHML_Type
is characterized by 4 capital letters and can be assigned to any object, source, activity,
actor, place or event, which makes its area of use very broad (e.g. ‘FIRE’ stands for
the object class ‘Fireplace’, ‘ARIL’ for ‘Archeological Illustration’ and ‘CARN’ for
‘Computer Aided Reconstruction’)38 (Hauck and Kuroczyński, 2014; Kuroczyński et al.,
2016). Some types are more specific than others; they can be linked to a certain level
of detail: a division into ‘capital-shaft-base’ is more detailed than just ‘column’). This
‘detail’ of classification is project-specific and relates to the project’s ‘Semantic LoD’ as
defined at the beginning of a project.
3.7.4 Semantic LoD
(Hauck and Kuroczyński, 2014) propose the so-called ‘Semantic LoD’ as a Level of
Detail that links the precision of classification with the building parts of a model and
an equivalent architectural scale: as a larger scale depicts more details, so does a higher
Semantic LoD relate to a finer classification. A classification level for a given Semantic
LoD is explicitly not given, only the recommendation to make the decision based on
the project’s requirements. The combination with the ‘Level of Information’ (LoI, a
quantification for the reliability of the source) leads them to define another Level, one
that copes with modelling assumptions: the ‘Level of Hypothesis’, which is defined as
the difference between the LoD and the Level of Information of the sources (Table 3.1).
Table 3.1: : Relation between 'Level of Hypothesis', 'Level of Information'
and ‘Level of Detail', connected with a traditional architectural scale.
bot:Element bot:hostsElement bot:Element 1 bot:Site, bot:Building, bot:Storey and bot:Space are all subclasses of bot:Zone Table 4.1: classes and properties of bot that will be used throughout the thesis
4.1.2 CHML and E-CRM
CHML and E-CRM form an interesting reference point when coping with 3D modelling
of heritage in a Linked Data context. However, some important remarks on its
usefulness for this thesis should be made. The first remark concerns the fact that the
core topic of the thesis is ‘as-is’ modelling of existing buildings, in a AEC-related
perspective. In contrast, E-CRM takes the historical point of view and CHMLs core
business is documenting the modelling process of buildings that do not exist (anymore).
This has important consequences, beginning with the interpretation of the sources.
When modelling a non-existent building, one has to rely on textual sources and images.
In contrast, existing buildings allow to make 3D point clouds, which are both in Table
3.1 (Section 3.7.4) and Chapter II indicated as the most reliable sources for
reconstruction. As a consequence, since for this thesis there are complete point clouds
available, the CHML core activity of interpreting sources about the nature of the
heritage object becomes redundant. The total ‘Level of Information’ as defined in
section 3.7.4 will be 9 (all sources are point clouds), hence, according to the definition,
the ‘Level of Hypothesis’ will be 0. Yet a ‘Level of Hypothesis’ as defined in (Hauck and
Kuroczyński, 2014) does not take the internal structure into account or the fact that
there are nearly always occlusions in a point cloud. The usefulness of the ‘Semantic
LoD’ scheme to this research is thus limited. CHML provides a source-labelling method
(chml:H106_Digital_Source_Type), but these are explicitly specified according to
http://cv.iptc.org/newscodes/digitalsourcetype, which includes only images and not
point clouds. A custom property for referring to point cloud sources will be defined
later in this chapter.
Also the way CHML 3D Objects are never relating directly to the object itself but
always via the reconstruction activity might a valuable concept in CH context, but
seems a bit strange when considering a BIM perspective: in architectural practice, not
38
the modelling activity is central, but the direct link between a virtual object and its
real-world counterpart. The virtual object is a very useful tool for managing, planning
and designing but nevertheless still a ‘tool’. In this context, ‘semantic object concept’
stays applicable to BIM, but the hierarchy between the real-world object and its virtual
representation is much more outspoken.
CHML and especially E-CRM provide an extensive vocabulary for modelling historical
objects, their purposes and events in which they were created. As enrichment with
historical information is outside the scope of the thesis, E-CRM and CHML definitions
are not included in the plugin’s UI. However, since such information can be very
valuable, a method to implement it nevertheless without using the plugin will be
illustrated later in the thesis (Chapter V), scratching only the mere surface of E-CRM
and CHML. A future research project could connect with this thesis in providing a
compatible UI for historical data enrichment, eventually being able to present a project
both as a BIM and as a virtual museum.
4.1.3 Classification
The CHML_TYPE classification is an interesting classification system, each type
referring to multiple online descriptions (such as the Getty Art & Architecture
Thesaurus (AAT)40, DBpedia41 etc.) and handling a system that defines part-whole
relationships and ‘broader terms’. An AEC counterpart of an extensive product
vocabulary would be the bsDD, which has been discussed previously (section 3.6.2), or
the Building Product Ontology42.
However, the scope of CHML_TYPE is very broad and is not limited to physical
building elements: it also encompasses events and activities, places, materials an
persons. Furthermore, a structured overview of defined CHML_TYPEs is currently
lacking, which makes it difficult to use. On the other hand, the bsDD is (like BIM in
its entirety) focussed on modern-day building practice and does not contain old-
fashioned building elements such as, for example, a column acanthus leave.
Furthermore, as we want to keep a modular ontology approach for describing data
about buildings, as formulated in (Rasmussen et al., 2018), and this project is mainly
a proof-of-concept, both CHML_TYPE and the bsDD are considered too heavy. For
these reasons, it is chosen to work with the modular Building Product Ontology43.
4.1.3.1 The Building Product Ontology
The Building Product Ontology has been mentioned shortly in context of Linked Data
building databases (section 3.6.2)). It is compatible with other modular W3C Linked
40 http://www.getty.edu/research/tools/vocabularies/aat/ (accessed 25/05/2018) 41 DBpedia is the Linked Data equivalent of Wikipedia 42 https://github.com/pipauwel/product (accessed 25/05/2018) 43 Note that in a real project, RDF allows to implement both CHML_TYPE and the bsDD: multiple classification systems can be used to make statements about an element.
39
Building Data ontologies (LBD)44, such as BOT. It extends BOT as it defines classes
(and subclasses) for different building elements, as well as for furniture and MEP. The
overall class in the Product Ontology is product:Product, specifically for building
elements there exists the subclass product:BuildingElement. Subclasses include
product:Beam, product:Column, product:Window etc. These are then the superclasses
for product:Beam-BEAM, product:Beam-Hollowcore etc. Adding information with the
Product Ontology gives the bot:Elements in the graph a richer semantic meaning,
distinguishing them as different types of elements. The Product Ontology only defines
one single property, being product:aggregates (domain and range both
product:Product). Therefore, the specific class of an instance should be stated explicitly,
since only the overall class product:Product can be inferred. Since one can always add
further detail without any restriction, it is obvious that not all building elements are
specified in the Product ontology. For example, there exists a class product:Column,
with subclasses like product:Column-COLUMN and product:Column-PILASTER.
There are no classes for a column’s capital, shaft or base. By consequence, there are for
example no classes for ‘acanthus leaves’ in a capital either.
If there is no Product Ontology Class for an element of the Case Study, a subclass of
product:BuildingElement will be created in a custom vocabulary ‘stgp’ (scan-to-graph
products).45 Following the example of CHML_TYPE, a definition from Getty AAT
(Art and Architecture Thesaurus) will be provided (rdfs:seeAlso). The Getty AAT
provides RDF definitions about different architectural elements and is interesting
because it is not limited to modern building practice. As an illustration, an example of
an object definition (Turtle) that relates to the Product Ontology and provides extra
GeoSPARQL is an initiative of the Open Geospatial Consortium (OGC) and provides
a vocabulary for representing geospatial data in RDF an extension to SPARQL to
query this data. Since GIS makes already quite extensively use of the Semantic Web,
GeoSPARQL is a well-maintained ontology that can be used to connect building
information with geographical information. Particularly of interest to this thesis is
44 https://www.w3.org/community/lbd/(accessed on 08/05/2018) 45 A .ttl-document can be found at https://github.com/JWerbrouck/scan-to-graph/blob/master/stgp.ttl (also added as Appendix D) 46 http://www.opengeospatial.org/standards/geosparql (accessed 05/05/2018)
40
‘geo:hasGeometry’, a property that links an instance with its geometrical
representation. Also interesting is the fact that GeoSPARQL makes integration of
georeferenced data into the graph possible, using any geographic reference system.
4.1.4.1 geo:hasGeometry
As an object can have different representations mapped to it, the concept of
geo:hasGeometry will be used as the main connection between an instance and its
geometric representation (its domain is geo:Feature, its range geo:Geometry). It does
not provide, however, any information about the format of the geometry representation,
since GeoSPARQL serializes its geometry standard into WKT Literals (‘Well Known
Text’) 47. The property to link WKT serialization with the geo:Geometry instance is
defined as ‘geo:asWKT’. This property will serve as a model to define the custom
property ‘stg:asSTEP’ that link the geo:Geometry instance to its representation in
STEP format (section 4.2-4.3).
4.1.4.2 Georeferencing
Since every existing building has a location in the world, to be complete, its virtual
model should contain a reference to this location. Multiple Geographic Coordinate
Systems (GCS) exist, both globally used, such as UTM (Universal Transverse
Mercator) or WGS84 (World Geodetic System 1984) as locally (e.g. in Belgium there
is Belgian Lambert 1972). All these systems differ in origin and their way of referring
to a geolocation. Conversion systems exist, but are seldom exact, e.g. because different
GCS use different transformation sysstems. A graph containing a georeferenced
building should thus contain the geocoordinates in its initial system, but may
additionally contain georeferences using other GRS as well. GeoSPARQL provides a
way to georeferenced geometry in a system other than its default (WGS84), which is
interesting because the point clouds for this thesis (and thus the whole resulting digital
reconstruction) use the Belgian Lambert 1972. Often in GIS, an object (point, polygon,
spatial object …) is georeferenced rather than an entire project (e.g. locations of
electricity towers are represented by a simple georeferenced point, since a more detailed
geometry is not necessary to represent it on a (digital) map). In this thesis, the building
project as a whole is referenced with one project origin that could be mapped to the
project Site instead of to each constituent object. As indicated in section 4.1.4.1,
GeoSPARQL uses WKT to link geo:Geometries to their actual representing data. The
path from the bot:Site to the WKT representing the origin in a defined GCS would
GeoSPARQL currently does not include the definition of geometry descriptors as
subclasses of geo:Geometry (point, line, polygon…).48 To specify the inst:Origin is a
project origin and not a general geo:Geometry it is chosen to define a subproperty of
geo:hasGeometry in the context of the thesis (stg:hasOrigin) (see section 4.3). The
relationship becomes then:
Subject Predicate Object
inst:Site1 stg:hasOrigin inst:Origin
inst:Origin geo:asWKT <WKTLiteral>
For referencing the project using another system than WGS84, The WKTLiteral
written out consists of a URI that refers to the GCS, the geometry type (Point) and
its coordinates between brackets49. The datatype of the Literal is, as usual, stated after
the content of the Literal itself. All GCSs that are currently in use are specified at
http://www.opengis.net/def/crs/EPSG/0. In the case of the Gravensteen,
georeferenced in Lambert72, the WKTLiteral would look like this (104500 being the
longitude, 194300 the latitude):
"<http://www.opengis.net/def/crs/EPSG/0/31370>
Point(104500 194300)"^^geo:wktLiteral
With the same method, other GCS with respective coordinates can be mapped to
denote coordinates.
48 https://www.w3.org/2015/spatial/wiki/Further_development_of_GeoSPARQL (accessed 28/04/2018, updated 26/09/2016) 49 As defined in (Perry and Herring, 2012)
42
4.2 Geometry
4.2.1 Geometry implementation in the graph
The previous sections discussed some ontologies that provide definitions to construct
an RDF graph of an as-is building. As is outlined in (Pauwels et al., 2017a) one of the
challenges of Linked Data is the inclusion of geometric data. To reduce the size and
complexity of an RDF graph, two options are outlined:
- remove the geometry from the RDF representation
- improve the RDF representation of geometry
In the case of (Pauwels et al., 2017a), the first option is dismissed because the context
of the paper is about the ifcOWL ontology, which is aimed as a reference standard for
synchronization with EXPRESS IFC (which does include the geometry). This thesis
does not aim to provide a reference standard, but removing the geometry is not
recommended as well: a key reason for performing scan-to-BIM (scan-to-graph) is to
geometrically represent the as-is circumstances, just because they show the real-world
imperfections that are neither included in the building’s topology or an as-planned BIM.
Leaving all geometry out would thus undermine the concept of making an as-is model
and mean a serious impoverishment to the graph itself. On the other hand, as such a
reconstruction project can include a lot of different geometrical representations (due to
its stack of sources: plans, point clouds, 3D-models), including them all directly into
the graph would extremely slow down querying and loading the graph and bring the
file size to multiple Gigabytes, also because some of this geometry cannot really be
improved more qua file size. For example, the point clouds that are used as a modelling
blueprint are in fact already stripped from all information that is not strictly necessary,
only encompassing coordinates, Intensity and RGB values. Their file size is inherent to
the format.
A middle ground between the two above problems could be to include URLs that link
to a web location containing geometrical data as a document; this way a connection to
the internet could provide access to the data without overloading the graph (note the
difference between URI and URL (see section 3.2.2)). This is the way very large files,
such as point clouds, are handled in this thesis: large documents that are available on
the web can be downloaded to a project folder C:/STG/Project, which forms a ‘local’
URL that is used in the graph. The graph references to both local and global URL. It
seems logic to also include some more compact geometric representation in the graph
itself: this way, one of the most important features of scan-to-BIM/graph is kept in the
graph and quickly to retrieve, also making the access to geometry independent of access
to the internet (which is often not as self-evident as it seems)). Such a compact
representation could be BRep geometry (Boundary Representation), which represents
the modelled geometry by defining its boundaries, and is the way most CAD
applications handle their native geometries.
43
4.2.2 Geometry formats in an RDF graph
The previous section argued why BRep geometry will be included directly in the graph
and point clouds will not. However, BRep is just a way for representing geometry and
does not propose a single file format. Pauwels et al. (2017a) discuss several methods to
include geometrical data in an RDF graph. To preserve the compatibility with
EXPRESS IFC and to maintain a maximum of semantic richness, the methods that
are discussed all rely entirely on RDF: single, simple geometric elements being separate
resources that are linked to Literals with their values and to each other by specific
properties. A point such as the project origin that is defined earlier using GeoSPARQL
in a WKT serialization is a very simple example of such basic geometry type. Linked
together, these resources then form more complex geometries.
However, since the use of WKT for the AEC industry is proposed in (Pauwels et al.,
2017a), which is a quite recent publication, using WKT as a way to store all geometric
information of a building project is not (yet?) optimized to cope with complex detailed
geometries in a detailed building model, let alone the out-of-plumb conditions inherent
to as-is modelling. Despite a total decomposition of a project’s geometry being
semantically the most stable strategy, it is also very intensive and cumbersome and
would need a conversion application to be compatible with most CAD packages.
Further on, as indicated, there is absolutely no aim to be a full IFC equivalent (which
is the purpose of ifcOWL), which renders more freedom to cope with geometry. Taking
the said into account and guided by the arguments listed below, a more pragmatic
approach is used for storing the modelled geometry in the graph:
- The definition of classes for geometry-types is outside the scope of this thesis, therefore
Literals are used for storing geometric information, this also provides more flexibility
in terms of describing data and use different formats;
- To keep a reasonable level of semantic richness, the breakdown of geometric structure
is limited to element level (mostly a surface or a volume (‘closed polysurface)) rather
than to absolute geometric basic types such as points and lines;
- This allows to describe complex forms with multiple double-curved surfaces, without
each of these elements becoming a graph on its own;
- Although this approach limits ‘total’ semantic connection that would allow complex
reasoning processes, it suffices for a compact representation of an existing building for
heritage and FM. In the end, all geometric information will still be present, albeit in a
document-way instead of a Linked Data way
Essential is that the geometry format that is used is an open standard, so it can be
imported in almost any CAD software. Also, the best situation would be a storage of
geometric information without information loss. Table 4.2 compares different open
export formats (.obj, .stl, .ply, .stp) according to file size and precision. BRep NURBS
(Non-Uniform Rational B-Spline) as well as Mesh geometry will be included in the
table to make the comparison. Collada DAE, also an open format, is not included in
the comparison because Rhino does not allow to import it (exporting is fine). As a
reference, the point cloud that is used for the column has a file size of 38 146 kB.
44
NURBS exports seem to take significantly less storage space than Mesh exports, which
is obvious due to their different approach (storing vectors compared with storing
sampled vertices). Although the file size of the OBJ -NURBS is only half the one of
the STEP file, its quality is visibly inferior. In Figure 4.2 a close-up of a column’s base
is depicted: the edges of the different surfaces clearly do not match. Therefore, the
approach in this thesis will be to export geometries to STEP (Standard for the
Image Detail Parameters
OBJ (Mesh export: maximum polygons)
- File size: 2853 kB (max. polygons)
1258 kB (min. polygons)
- Precision: maximum amount of polygons
means maximum precision. This also
means that the precision depends on the
size of the object: smaller objects thus
have higher precision. Less linear places
have a higher density.
OBJ (NURBS export)
- File size: 294 kB
- Precision: no polysurfaces, each surface is
imported separately. Edges do not match
(see detail image)
STL and PLY (Mesh)
- File size: 10855 kB (ASCII STL)
1582 kB (Binary STL)
3260 kB (ASCII PLY)
1429 kB (Binary PLY)
- Precision: same remark as with .obj file.
PLY meshes have reduced visibility in
Rhino
STP (NURBS)
- File size: 515 kB
- Precision: volumetric elements are kept
as a whole, seamless connection between
the curves
Table 4.2: comparing different exporting geometries by size and precision (Rhino 6)
45
Exchange of Product Model Data, ISO 10303)50 format, and
then store each resulting file (which comprises a single
(poly)surface) as a Literal (datatype stg:asSTEP). This
Literal will be linked to the corresponding geo:Geometry
instance that is part of a (sub)object. STEP is a format that
was originally developed for the manufacturing industry, but
currently, it serves a broad application area (e.g. EXPRESS
based IFC documents follow the STEP syntax). An
interesting remark is that the information in a STEP file is
also ‘linked’: each entity gets assigned a number, and relates
itself to other entities by referring to their number at specific
places. A disadvantage of formulating the geometry in text Literals instead of in triples,
is indeed that this data cannot ‘reach out’ to other geometry data; two objects may for
example connect to each other at a certain boundary curve, but in the serialization this
information may get lost. An example of a curve definition is given below, to give an
indication about the structure of a STEP file. Each comma-separated value has a
certain meaning51, for instance the tuples with the #-values refer to control points of
the surface that are defined further in the file, grouped per curve:
The previous ontologies (section 4.1) provide a broad range of classes and properties
that can be used throughout this thesis. Some fit in very well, others are interesting
but do not exactly represent the semantics we are looking for. As has been indicated
previously, RDF allows to either refine some of these concepts or define some
completely new ones. Therefore, a small ontology has been constructed, to keep track
of sources and metadata and to link to geometry (section 4.2). For quick reference,
their main purpose can be found in table 4.2.52 The prefix that will be used for the
main vocabulary is ‘stg’, which is an acronym for ‘scan-to-graph’.
Class (Property) Comment RepresentingFile (hasRepresentingFile)
Reference to a document URL that contains a geometric representation of the whole project or a part;
SourceFile
Subclass of RepresentingFile specific for representing sources and not reconstruction deliverables;
PointCloudFile (hasPointCloudFile)
Subclass of SourceFile, refers to a Point Cloud that is a source;
RhinoFile (hasRhinoFile)
Subclass of RepresentingFile, since for modelling in this thesis Rhino 6 is used, this URL refers to the original deliverable;
(hasLocalVersion) A Literal string referring to the local URL on the computer, where a larger document is located. Linked to RepresentingFile;
RhinoID (hasRhinoID)
A RhinoID (Literal string) can be used for quick visualisation and is also used as a tool for storing the geometric information correctly in the Graph;
STEPRepresentation (asSTEP)
A Literal string containing the geometric information of a (part of an) object, serialized in .stp- format;
ProjectOrigin (hasOrigin)
Subclass of geo:Geometry that refers specifically to a project’s origin. Linked to a bot:Site. Linked to the project itself.
ModellingRemark (hasModellingRemark)
A property that refers to a Literal that is a remark about modelling assumptions or metadata (domain: geo:Geometry, range: rdfs:Literal);
(denotesRemark) Links the inst:ModellingRemark to the Literal string that contains the textual remark;
OccludedGeometry (hasOcclusion)
A subclass of ModellingRemark, to indicate a geometry is (partly) occluded in the source file;
LevelOfAccuracy (hasLOA)
A class that relates to remarks concerning the (represented) LOA (section 2.4.2.2). (see: hasLOAvalue and usedDeviationAnalysis);
(hasLOAvalue) A Literal string containing the represented LOA (LOA10, LOA20, etc.). Linked to a LevelOfAccuracy instance;
(usedDeviationAnalysis) A Literal string containing the method that was used for deviation analysis (MICROSCALE or MACROSCALE). Linked to a LevelOfAccuracy instance;
InternalGeometry (via rdf:type)
Denotes whether a geometric 3D object is invisible because it is located inside another object (e.g. part of a console is considered InternalGeometry, because it is located in a wall;
(usedEquipment) Links to the equipment that was used to create the source. Table 4.2: Outline of the Classes and Properties that are defined as a part of the dissertation
52 The main classes and properties that have been constructed for this thesis can be consulted at https://github.com/JWerbrouck/Thesis/blob/master/stg.ttl (also added as Appendix C in Turtle format).
47
The first category of classes that are defined relate to the geometries that represent the
building. They are not expected to be embedded in the graph because of their typical
large file size (see section 4.3.1). Therefore, they are referenced by a URL (an URI that
provides also access to the resource, section 3.2.2) that serves to store the source file
online. As indicated in section 4.2.1, his URL can be connected to a local location on
disk, (C:/STG/Project), which will be done by use of a Literal (string).
The second category of Classes are related to the geometry challenges in section 4.2.1
and to the definition of the project origin (see section 4.1.4.2). All three are related to
specific datatypes.
Lastly, the STG ontology defines the properties to keep track of difficulties,
uncertainties and other metadata. stg:OccludedGeometry and stg:LevelOfAccuracy are
both examples of subclasses of stg:ModellingRemark. It may be clear that a lot of other
subclasses can further be defined for different types of metadata. The equipment that
was used for the survey during which the source was created is also considered
metadata. Currently, this is linked to a Literal which denotes the equipment. In the
future, this could be changed to a class that is provided by the manufacturer, which
contains a lot more details about the TLS or DSLR camera that was used for the point
cloud. Not considered metadata, an extra property was constructed to denote whether
a geometric 3D element is invisibly part of another element, because this occurred quite
frequently during the modelling of the case study (Chapter VI).
48
4.4 Graph Scheme
Now that all the ontologies and the geometry handling have been discussed, the coming
section takes a more graphical approach to lay out the structure of the graph as
generated by the plugin. This will be done for a generic building, using the ontologies
outlined in section 4.1 and 4.3. Classes will not be included in the graph, unless they
cannot be inferred from the used properties. This is the case when using
product:aggregates (with which one can only include the object and subject are
instances of the generic class product:Product). Product Classes will be depicted in the
template graphs with a red class (TBox) node labelled “product:…*”. Such nodes occur
several times in the graph because all these “product:…*” may refer to other product
classes. Full-page figures of Fig. 4.3-5 are also added to Appendix A.
4.4.1 Topology Level
Fig. 4.3 depicts a graph that shows only the abstract building components, structured
by the BOT classes and the elements classified using the Product Ontology (whether
custom-defined classes or not). The project origin as defined in section 4.1.4.2 is linked
to the bot:Site. The URL of the Rhino Document is linked to the bot:Building instance.
In theory, Rhino Documents can be linked to each place in the hierarchy, so its use is
not restricted to one. This does not cause a problem: because the IDs are unique by
definition, the chances to have an ambiguous object-ID relationship in different Rhino
documents are virtually non-existent.
Fig. 4.3: Topology of a graph constructed with the plugin (software: yED v3.18.0.2) (Appendix A)
49
4.4.2 Building Element Level
A graph that depicts the structure on Building Element level is given by Fig. 4.4 and
Fig. 4.5. Fig. 4.4 is a general illustration for how metadata is linked to a geometrical
object. An stg:ModellingRemark is linked to a geo:Geometry instance, so the same is
true for its subclasses stg:OccludedGeometry and stg:LevelOfAccuracy. In the case of
stg:InternalGeometry, the principle is different. Since this is not considered metadata
(section 4.3) but rather a kind of ‘state’ of the object, the geometry is identified as an
stg:InternalGeometry by rdf:type (alongside with the geo:Geometry, that can be
inferred from geo:hasGeometry, but is explicitly stated here for the example).
Fig. 4.5 zooms out to the level of the object that contains the geometry. It depicts the
graph of a column, to clarify the system of ‘sub-elements of sub-elements’ (which can,
in theory, continue infinitely). How far this classification detailing goes is up to the
commissioner and the modeller (as was the case with the ‘Semantic LoD’ in CHML).
This graph of a column can be generalized as a model for how other elements are
included in a graph generated by the plugin.
Every sub-object will be assigned a class of the Product Ontology or a custom extension
of the stgp vocabulary. They will be also generally identified as a bot:Element.
Geo:hasGeometry is used to connect the instances with their geometrical
representation. It serves as a hub to connect different representations to a single
(sub)object. Note that a sub-object can have different geometries, and every geometry
represents a surface or (open or closed) polysurface. It is left to the modeller to decide
how far this breakdown structure goes.53 Closed polysurfaces also contain volumetric
information, so it is recommended to use them whenever possible.54 However, it can be
required to extract a single surface, when a modelling remark applies only to this single
geometry.
Fig. 4.4: Linking metadata to a geometric instance (Appendix A)
53 This relates to the ‘sub-object of sub-object’ detailing as defined above: the amount of geometries will always be more than (or equal to) the amount of sub-objects, unless there are sub-objects defined that do not have a geometric representation and are purely conceptual instances in the graph. 54 Volumetric information is at the time of writing not stored as an individual Linked Data property.
50
Fig. 4.5: Linking building elements with geometry and modelling remarks (yED v3.18.0.2) (Appendix A)
4.5 Conclusion
This chapter illustrated the methodology that is proposed for use in a scan-to-graph
process. Small, modular ontologies (BOT, PRODUCT, GeoSPARQL) were chosen as
the backbone of the process, complemented with a custom defined vocabulary (STG)
that contains classes and properties specific for the purpose of a scan-to-graph process.
It has been shown that product classification systems can be extended for particular
projects. A way of implementing geometry in the graph or providing (local) URL
references to heavier documents has been discussed. Finally, a structure which combines
these topics in an RDF graph has been laid out.
51
Chapter V: Rhino Plugin
During the course of the thesis, a plugin for Rhino 6 (McNeel) was developed to support
the creation of a graph according to the schema described in Chapter IV.55 The main
purpose of the plugin is to be an aid for semantic enrichment of 3D geometries and
exporting the information to an RDF graph, a feature not supported by current BIM
packages. Furthermore, the plugin simplifies the import of point clouds (and relocation
to the project origin), provides subsampling support (via CloudCompare) and uses
them as a basis for both the 3D creation and the ‘semantic objects’ in the graph. It also
provides an interface to visualize geometrical results of a SPARQL query. Although
the plugin supports creation of the graph in Rhino, the resulting graph only
encompasses non-proprietary formats. This chapter describes the main functionality of
the plugin, a demonstration is given at the end of Chapter V.
5.1 Dependencies
The plugin was made using IronPython 2.7 (built-in in Rhino), Python 2.7 and Eto, a
Rhino-implemented cross-platform framework to make user interfaces in .NET (thus
also applying to IronPython). Only rdflib (v4.2.2)56, a Python package with RDF
compatibility, should be installed manually as a Python 2.7 package. Because rdflib is
not fully compatible with IronPython, the user should certainly have installed Python
2.7. 57 Further on, for subsampling point clouds, the open-source application
CloudCompare v2.8.1 Stereo58 is used ‘under the hood’. When using the subsampling
function before importing, the user should first install CloudCompare 59 Lastly,
SPARQL queries are carried out using Stardog60, a triplestore or database for Linked
Data. Stardog has a free Community Edition for non-commercial use. The user should
make sure that (1) Stardog is installed, (2) the folder ‘stardog-5.x.x\bin’ is added as a
PATH (set as an environment variable in windows), (3) the graph that corresponds
with the opened model is stored as a Stardog database (default, it is hosted at
localhost:5820) and (4) the Stardog server is running in the background at the moment
of querying. To be able to import E57 point clouds in Rhino, the extension E57 FILE
IMPORT61 (by Dale Fugier) should be installed.
Fig. 5.1: General information settings at the top of the plugin
55 The commented files from the source code are provided at https://github.com/JWerbrouck/scan-to-graph 56 https://github.com/RDFLib/rdflib (accessed 28/05/2018) 57 default location set in the source code: “C:/Python27/python.exe” 58 http://www.danielgm.net/cc (accessed 4/4/2018) 59 default location set in the source code: “C:/Program Files/CloudCompareStereo/CloudCompare.exe" 60 https://www.stardog.com (accessed 2/05/2018) 61 http://www.food4rhino.com/app/e57-file-import (accessed 12/10/2018)
52
5.2 General information
The plugin allows to define a building site, building, storeys and spaces, which contain
objects or are adjacent to them. Objects are primary characterized by a layer that
contains its representations, such as the point clouds or the reconstructed 3D geometry.
It is possible to further specify sub-objects with a main object (which has a layer) or
another sub-object as a parent. URIs are constructed primary using the name of the
instance in Rhino. In the current version of the plugin, objects and storeys should have
a unique name to avoid errors in the graph. The URI of a space is based on its own
name and the name of the storey it belongs to, so different storeys can have a space
with the same name. Sub-objects only need to have a unique name within the ‘range’
of their parent object but may share their name with a sub-object from another parent
object, e.g. two columns can each have a sub-object ‘Capital’.
At the top of the plugin, the name of the graph is to be defined (Fig. 5.1). A local
version of the graph will be stored in a project folder (C:/STG/ProjectName). The
graph will be serialized in Turtle. Previously created graphs can be loaded with the ‘…’
button. Graphs that are loaded will be serialized again, possibly overwriting the original
file. The textbox below serves for stating an overall URI, which serves as a basis for
giving all instances their own URI (it will be used as a prefix ‘inst:’ in the Turtle files).
In the case study, this URI will be given by: ‘https://github.com/JWerbrouck/scan-
to-graph/casestudy/ ’, but in fact, any URI would be valid, as long as it is unambiguous.
5.3 Project Info Tab
In this tab (fig. 5.2), the overall topology of the building is laid out, by use of bot:Zone
data instances and a coordinate system. Although a bot:Site can contain multiple
bot:Building elements in theory, in the plugin this is currently restricted to only one.
If it would nevertheless be necessary to define multiple buildings sharing one site, this
is possible by either storing them in the same database or merging the graphs together
outside the plugin environment. Building storeys and their spaces of the building are
also part of the building topology and are defined at the bottom of the tab.
Since the point clouds of the case study are referenced using Lambert72, this is
currently the default option for georeferencing. Using another GCS is possible by
copying its openGIS URI (hyperlink by “Coordinate System”) into the coordinate
system widget. Georeferenced point clouds that are imported using the plugin will be
translated to the origin using the project coordinates; large geocoordinates often result
in the project’s canvas becoming too big and unfit for modelling (see further). Note
that this only works with cartesian coordinates, since the Rhino canvas uses a cartesian
system. In the current plugin version, it is only possible to include only one
georeferencing system, although it is in theory possible to add multiple (section 4.1.4.2).
The Project Info Tab also includes a checkbox for including STEP representations in
the graph as Literal strings. The checkbox is checked by default.
53
Fig. 5.2: Plugin tab to define the topology and geographic location of the building (additionally: the option to include geometric information directly in the graph)
54
5.4 Point Clouds Tab
When the basic project topology is defined, one can start modelling the as-is geometry
based on point clouds (or other sources). The plugin streamlines this importing process
for as-is modelling by several steps (Fig. 5.3). First of all, the large file format of huge
detailed point clouds can give performance problems. Therefore, subsampling the point
clouds by calling CloudCompare before importing them is given as an option. When
using the subsampling option, the user locates a folder which contains only the files
that should be subsampled (when it contains other files or folders, it is not sure whether
the import will happen without problems, results may vary). The subsampled point
clouds are then stored in a separate subfolder and imported. Subsampling options
include: octree, spatial and random, as performed by CloudCompare. Each point cloud
is imported as a separate layer, which takes the name of the point cloud and serves as
the representation of the ‘semantic object’. Apart from the point cloud, the layer also
contains the modelled representation of the element. If well-chosen, the name of the
point cloud can give an indication for later object classification: i.e. if it contains a
Product Ontology class (e.g. column-COLUMN_Column1_PresenceChamber.e57 will
give a classification suggestion for product:Column-COLUMN).
Fig. 5.3: Plugin tab for importing and subsampling of point clouds
55
5.5 Element Tab
This tab contains the core of the plugin (Fig. 5.4). The ‘Main Objects’, which may have
a point cloud representation, refer to a layer of the document (and take its name as
the element’s name). An object is located in a space, which is part of a storey, both
defined in the ‘Project Info’ tab. The default relationship between a zone and an object
is ‘bot:containsElement’. The checkbox ‘Adjacent’ provides the option to override this
default relationship and set it as ‘bot:adjacentElement’. In this case, it is possible to
define the zone on the other side, in the plugin limited to a space. A ‘Main Object’ may
be hosted by another element (bot:hostsElement), e.g. a door can be hosted by a wall.
As indicated in the section about the Point Cloud Tab, the element’s type is guessed
from the name of the point cloud (strictly spoken, from the name of the Layer), but
can be changed without consequences. A widget to link the geometries to the
represented LoA after a deviation analysis is provided. Although this widget is
currently mapped to an entire object in the User Interface, in the graph, this LoA will
be linked to each individual geometry, except those that are denoted as occlusions.
An object may have sub-objects, either hosted or aggregated. Apart from this parent-
child relationship, a sub-object has a name and a type. An option is provided to set a
sub-object as an aggregate/hosted element of another sub-object, hereby enabling the
core Linked Data property of virtually unlimited detail. Note that, from a certain point,
the object types will have to be custom-defined, since they will not be included in the
Building Product Ontology anymore. In the current version, this can only manually be
done by adding the URIs of these object to the .csv-file ‘custom_types’, located at the
plugin’s installation folder.
Each object intrinsically aggregates 3D geometry that its corresponding layer contains.
This is denoted by the item ‘self’ in the ‘sub-objects’ list. These geometries are visible
in a list below the sub-object’s properties (“Sub-Object has Geometries:”). When such
an element-ID is selected in this list, the corresponding object will also be selected in
the viewer. Further on, selecting an element makes it possible to add one or more
written notes to this element in the form of a modelling remark (a string linked to
stg:ModellingRemark) or an occlusion (stg:OccludedGeometry), in the plugin both
listed in the lowermost list of the Tab. When ‘self’ is selected in the sub-object list, the
geometry-list also contains the point clouds that are stored in the layer, which are,
unlike 3D objects, not serialized as a geo:Geometry, but as stg:PointCloudFile. When
selecting another sub-object, there is an option to pick the 3D objects (surfaces or
polysurfaces/volumes) in the viewer that assemble this sub-object (restricted to the
geometries in the Object Layer and Default layer), again displayed in the listbox.
To check if the information has been stored correctly, a button at the bottom of the
tab is provided to print all information that is currently stored on the command line,
in a schematic way. This includes the Project Topology, geolocation, the assumptions
that are mapped to object IDs and all Main Object attributes (‘point cloud’, ‘type’,
hosted or not …), their sub-objects and the attributes of these sub-objects.
56
Fig. 5.4: Plugin tab for assigning object attributes, sub-objects and geometries, as well as labeling 3D objects with modelling remarks
57
5.6 SPARQL query tab
The last tab (Fig. 5.7) is a tab for performing SPARQL queries and visualizing
geometry that matches the query. The graph should be loaded in an active Stardog
Database before running a query. To enable reasoning, the corresponding ontologies
should also be loaded into this Stardog database.62 This can be done by going to the
local server (default at localhost:5820, username: admin, password: admin), opening
Database > Browse > Data > +Add. A local graph file (ttl; rdf; owl …) containing the
ontology definitions can then be uploaded. To prevent that all graphs in the database
are also queried when inferencing is not enabled, the ontologies can be added as ‘named
graph’. In our case, this will be done for BOT, Product, GeoSPARQL and the custom
ontology STG.
Fig. 5.5: uploading ontologies as named graphs
The Query tab itself contains an input box where the name of the Stardog database
should be stated. Below, there is the SPARQL query input box63, with the query button
and a function to enable/disable reasoning.64 Results are displayed at the table “Query
Results”, each variable a column. When the query results contain Rhino IDs, the overall
display mode changes to ‘Wireframe’, the objects related to the IDs in the results are
individually set to a ‘Rendered’ mode, which makes them clearly visible in the viewport
(Fig. 5.6). Selecting a row in the results that contains a Rhino ID will also select the
corresponding item in the viewport.
Fig. 5.6: highlighting and selecting objects via a SPARQL query
62 […]>stardog-admin db create -o spatial.enabled=true -n “DB-name” “Path-to-DB” 63 Apparently, when querying a Stardog Database via the command line with “SELECT ?s ?p ?o WHERE {?s ?p ?o}”, Literals with a certain length (e.g. the Literal strings containing STEP geometry) are not interpreted correctly and are split at random places. Results may vary. 64 Make sure, when entering a query in the plugin interface, that a space is added between the command (SELECT, INSERT) and WHERE, also when working with multiple lines.
58
Fig. 5.7: Plugin tab for querying with SPARQL; visualization of the results as a table and in the active viewport.
59
5.7 Additional content
The Plugin provides a UI for constructing graphs with the structure outlined in Chapter
III. However, it is probable that some additional content needs to be added to the
initial graph. The plugin does not provide a UI to add ‘additional content’, since this
is too generic: a UI that uses Linked Data but does not expect Linked Data expertise
from its user should focus on documenting only a part of the information, related with
specific ontologies. This UI thus deals with providing the basic topology and geometry
of an existing building. A future research project could focus on the development of an
interface to implement historical information (e.g. using the structure of E-CRM and
CHML as a basis). When a UI for the type of information that has to be included in
the graph (historical, technical, geographical …) is yet to be developed, this issue can
be solved by performing SPARQL INSERT queries, stating the information in
SPARQL. Stardog (or another RDF store) can be used for this. For example, if someone
wants to add some historical information about the purpose of the Presence Chamber
in the Gravensteen during the Middle Ages. The integration of such information in the
graph (using E-CRM) takes the following steps:
(1) E-CRM has a property ‘P103_was_intended_for’ (domain: E71_Man-Made_Thing;
range: E55_Type)65. The subject of the relationship will be:
‘inst:Ground_Floor_Presence_Chamber’, which is already present in the graph as a
bot:Space, but is hereby also classified as a ‘E71_Man-Made_Thing’. The object will
have to be defined newly, as ‘inst:giving_audience’, now implicitly classified as an E-
CRM ‘E55_Type’. The INSERT query will thus be as follows:
In this chapter, it was shown that the plugin provides an interface for importing point
clouds for a scan-to-graph process; for defining the topology, the geometry and the
semantics of a project and for querying a graph, able to visualize the resulting
geometries. The scope of this thesis limits to this functionality, but a way to
nevertheless add extra information has been explained. In Chapter V, this functionality
will be used in the case study of performing a scan-to-Graph process for the Presence
Chamber in the Gravensteen Castle, Ghent.
61
Chapter VI: Case Study
In this chapter, the scan-to-Graph approach is demonstrated by a case study on the
Presence Chamber of the Gravensteen Castle in Ghent. First, the point clouds that
served as a blueprint for the geometric reconstruction are introduced. After a short
outline of the available point cloud sources (section 6.1.1), it is discussed how these will
be segmented into meaningful parts that will be automatically connected to individual
Linked data objects by use of the plugin (section 6.1.2). Cutting the extremely large
point cloud into smaller subclouds avoids importing entire point clouds in Rhino 6.
This is not only very demanding for the computer hardware but is also very impractical
for modelling. The extraction of such smaller point clouds will happen in Autodesk
Recap Pro v4.2.2.15 (student license). A similar procedure can be followed using
CloudCompare, but there might occur some problems when importing very large point
clouds. The results will have the e57 extension, an open format for point clouds.
The second part of the chapter involves creation of an as-is model in Rhino 6 (section
6.2). It starts with a discussion of the modelling techniques that were used for ‘reverse
engineering’ the Presence Chamber in the House of the Count as part of the
Gravensteen Castle (section 6.2.1). As commercial reverse engineering software is often
costly, part of this thesis was finding efficient techniques for (manual) modelling based
on point clouds without using such applications. BIM tools such as Revit have limited
built-in functionality for reverse engineering, but several semi-automatic plugin
applications exist. However, Revit is not a reverse engineering tool, for which a more
intuitive environment for 3D drawing and -sculpting is needed. As explained in the
introduction, the amount of possible modelling environments grows since we are not
limited to regular BIM applications; Rhinoceros 6 was used as the main modelling tool,
for its modelling versatility as well as for its support for small macros that accelerate
the modelling process and for the development of custom plug-ins.66
When an object’s geometry is modelled, it is compared with the original point cloud,
to estimate the modelling precision by making a deviation analysis (see Chapter II).
Such analysis will be done using CloudCompare. A micro-scale analysis will be
performed on all objects but the doors and windows; these are included in the deviation
analysis of the wall. If the represented Level of Accuracy is insufficient, the 3D
geometry will be corrected, otherwise, the next object can be modelled.
The third part of the chapter illustrates how the plugin can be used for semantic
enrichment of the geometries (section 6.3). It is chosen to model the objects first and
then perform the graph creation, because then, the graph creation can happen more
quickly, without interruption. However, this working sequence is no prerequisite:
starting with creating the graph is also possible. Finally, when the graph has been
generated by the plugin, following the method explained in section 5.7, some additional
information will be added using SPARQL INSERT queries (section 6.4).
66 Rhino is not a dedicated reverse engineering tool either, examples of specific reverse engineering Software for Rhino are RhinoResurf (http://www.resurf3d.com/products.htm (accessed 27/05/2018)) and Rhinoreverse
To make the importing easier and to provide the Rhino plugin with a basic structure
to work with, the elements to be modelled are first extracted from the entire point
cloud using Autodesk Recap Pro v4.2.2.15(student license). As indicated before, the
2017 TLS point cloud from KU Leuven (6.1.1.1) is by far the most precise. The ‘TLS
and photogrammetry’ scan (6.1.1.2) has the least occlusions, since the photogrammetric
approach complements the laser scanned point cloud by adding information about
locations that are not ‘visible’ for the TLS. However, as outlined in section 6.1.1.2, this
point cloud is less accurate. Point cloud 6.1.1.1 will be used to model single objects
(separate as well as hosted elements). Because it contains some occlusions in the walls
(mainly at the southern façade and window sills, Fig. 6.7-6.8), the modelled walls are
the only geometry that will be based on point cloud 6.1.1.2.
Because the modelling of the point cloud to geometry will be restricted to the presence
chamber, the first thing to do is to look what distinct elements are available in this
room and its boundaries. Figure 6.5 and 6.6 give an overview of the elements to model
(apart from walls, vault and floor). Furniture, heraldic shields etc. will not be modelled.
Fig. 6.5-6.6: selection of the elements to model (Autodesk Recap Pro): 1-2: columns; 3: Front door; 4-5: upper windows front; 6: lower window front; 7: fireplace; 8a-b-c: side windows; 9: internal door
3
4
3
8a-b-c
2
6
5
7
1
.
1
.
8c
9
.
65
Fig. 6.7 (left): southern (front) façade partly occluded by a bush (Autodesk Recap Pro) Fig. 6.8 (right): additional photogrammetric point clouds reduces the occlusion zone (Autodesk Recap Pro)
6.1.2.1 Selecting the points
The next step involves selecting the points that belong to the column. For reasons of
visibility, the Limit Box in Autodesk Recap is set to encapsulate only the necessary
points. To make sure the entire column will be exported, the Limit Box should contain
a little information from the elements’ surroundings (in this case the floor and a part
of the vault) (Fig. 6.9). Then, either the visible parts that do not belong to the column
or the ones that do are selected and respectively dismissed from the view or serve as a
basis for a new view, as is done with the part of the vault that is still visible and the
separation between floor and column. Because this part relies mainly on visually
determining the boundaries of the element, these boundaries can be made more explicit
by changing the color of the points (e.g. Elevation instead of RGB) (Fig 6.10).
Nevertheless, the selection of the points that belong to a specific element is not exact,
since it is a manual process. Because such process is very difficult to quantify, only one
general remark on the boundary determination can be given: an overlap between the
elements should be the case rather than selecting too few points. This way, it is made
sure that there are no ‘forgotten’ points. A slight overlap in the point cloud segments
Fig. 6.10 (right): Changing point colour as an aid for selecting the element’s boundaries (Autodesk Recap Pro)
66
6.1.2.2 Creating a Region
The definition of the points that belong to an element can be done in several stages.
When (part of) the column is selected (e.g. the base), the user clicks on ‘REGION’,
defines the name of the region, in this case the column. In this step of defining regions
within Autodesk Recap, we can already anticipate on the graph construction. The
Rhino plugin will try to match classes of the loaded ontology to the element by
comparing the items in the list of available classes with the name of the point cloud.
Since we know this element is a column in the presence chamber of the castle, and the
Building Product Ontology we are going to use for classifying the elements contains
the class ‘Column’ (and, more specific, a subclass ‘column-COLUMN’) we assign to this
region the following name: ‘column-COLUMN_Column1_PresenceChamber’. Although
the place of the type in the region’s name does not really matter, in this thesis, the
structure ‘Type_Name_Room’ is maintained. Actually, since there is only one room to
be modelled, the room does not have to be included in the name, but it remains a good
practice when dealing with multiple rooms. The name of the corresponding instance in
the graph can be derived from the point cloud’s name. This is just a practical step to
reduce time spent on namegiving and type assignment, however, the plugin allows for
changes to type and instance name and does not require the name to follow this syntax
(section 5.4). Figure 6.11 shows the resulting region when selected in the overview panel
at the bottom right corner.
Fig.
6.11: Column 1 selected in the overview panel (Autodesk Recap Pro)
6.1.2.3 Exporting the point clouds to .e57-format
When the desired regions are defined with corresponding names, the last step in this
section is exporting them to the e57-format. This can be done easily in Recap itself, in
the overview panel bottom right, under ‘Scan Regions’ (“Export Regions”). Exporting
happens after defining the file format and the folder.
The other elements displayed in Figures 6.5-6.6 were also exported to both serve as a
blueprint for the Rhino geometry and be the point cloud representation of the elements
in the Graph. Walls were exported with mitred corners, using point cloud 6.1.1.2 (TLS
and Photogrammetry 1). Hosted elements such as doors and windows were nevertheless
extracted from point cloud 6.1.1.1, as well as the vault, which is the largest point cloud
segment of the project. The height of the vault was determined by extracting the floor
of the upper storey, contained in point cloud 6.1.1.3.
67
6.2 Modelling the Presence Chamber
This section discusses the 3D modelling process itself, including the deviation analysis.
After some general differences with ‘regular’ scan-to-BIM environments are made, some
general options in Rhino that make the modelling process more easy are introduced.
Then, the modelling of the objects themselves is discussed: vault, column, hosted
elements (window, door), walls. Details that are considered ‘mobile’ (e.g. furniture) or
that are not considered relevant for the overall geometry (e.g. lighting fixtures) are not
included in the modelling process. When an object has been modelled, its deviation
with the point cloud is calculated and serves as an indication for the deviation with
the real-world object. The creation of a ‘perfect’ model is not the core goal of the thesis,
and is not possible anyway. As the model mainly serves as an illustration of the scan-
to-graph process, the ambition is to achieve at least LOA20, i.e. 95% (2σ) of the
deviations should be lower than or equal to 50 mm.
Despite its versatility in modelling, Rhino is not an automatic reverse engineering tool,
and professional reverse engineering tools are often too costly and still require a
considerable amount of user input. The built-in tools that are provided for reverse
engineering (‘ExtractConnectedMeshFaces’, ‘Contour/Section’ etc.) are useful for
modelling small objects based on meshes with a limited amount of faces; applying them
to large point clouds often causes a software crash or the results are not satisfying.
Therefore, only manual, regular modelling methods were applied.
Typical scan-to-BIM modelling either happens directly in BIM packages (such as Revit)
or indirectly, e.g. by creating orthophotos, tracing them in a CAD package (such as
AutoCAD) and then importing the vector drawings. The last method is in general less
accurate, because a lot of information is lost. Alternatively, 2D profiles are traced in a
CAD environment, brought into the BIM software, converted to a 3D volume (e.g. by
extrusion) and made a Family (Mezzino, 2017). By working in only one modelling
environment, such constant change of applications is avoided.
The approach that is taken here differs from a BIM modelling approach in that in a
regular BIM workflow, object classification typically happens already before modelling
(i.e. in most cases you choose first what to model, then you model it). In this approach,
classification only takes place in the end, in the graph creation process. Detaching
modelling from classification means at the same time an advantage and a disadvantage.
Not bound to ‘rules’ about building elements, there is an absolute modelling freedom,
which is a reasonable plus considering as-is irregularities and non-modern, project
specific building elements. On the other hand, this also means a lack of geometrical
constraints (e.g. ‘watertight’ element connections or closed volumes are not self-
evident), control mechanisms and modelling suggestions.69 However, in practice one
69 It also means, unlike in classic BIM, that all semantics will have to be added later on, but since this is one of the basic assumptions of the thesis (and of non-geometric nature), this is not considered an obstacle in this context.
68
also sees that in as-is modelling in Revit, custom Families have to be developed to
overcome such class-bound restrictions; further on, due to the limited amount of classes
(Categories in Revit), classification can be inaccurate (e.g. vaults being classified as
‘roof’). Lastly, the advantages of parametric modelling/families are less useful for as-is
modelling, because (especially for pre-industrial buildings)70 each element has unique
distortions.
6.2.1 Preparation
6.2.1.1 View settings
Some preparation had to be set up in order to make the modelling process more smooth.
Modelling on a point cloud heavily relies on visual analysis and less on mathematical
objects or primitive modelling. This is one of the reasons why a scan-to-BIM process is
typically considered very labour-intensive (section 2.4.2.1). Often, the modelling
technique is based on so-called ‘wireframe modelling’: boundary points are identified,
constructed and connected with control point curves or (poly)lines. Because of this ‘3D
drawing’, it is recommended to change some visibility options before starting to model.
Object surfaces, together forming the solid object, will in most cases be based on
NURBS curves that are defined by control points. Using the standard settings, curves
and their control points are often very difficult to trace, hidden behind the point cloud;
changing their visibility significantly enhances the process. This is done in the options
dialog: ‘options – view – display modes’. A copy of the display style ‘Shaded’ and
‘Ghosted’ was made and then adapted to change the curve options (colour to red and
‘width’ to 3) and control point options (size to 8) (Fig 6.13).
6.2.1.2: Modelling aids
To display only specific parts of the working
space, Rhino provides so-called ‘Clipping
Planes’. Their purpose is similar to Revit’s
‘Section Box’ or Recap’s ‘Limit Box’, but in
Rhino, the sections are made by an infinite
plane instead of by a box, which renders
more orientation freedom but also makes
them more difficult to handle (Fig. 6.12).
Combined with the _MPlane command, which allows to pick a planar object and set it
as a working plane that moves along with the object, a Clipping Plane provides a
practical way for modelling non-orthogonal objects. MPlane also proves very useful
applied to other objects than clipping planes; they allow to ‘draw’ on any planar surface.
It may seem obvious, but the flexible use of layers (“clipping planes”, “construction
planes” …) further smoothens the modelling process, as well as enabling the ‘Record
70 A large difference is present between the modelling of pre-industrial and industrial/modern buildings. For the case study of an ancient temple in (Mezzino, 2017), a different approach was used than for the modelling of an office building in (Thomson, 2016). Modelling techniques of course also differ from the person who performs the activity.
Fig. 6.12: Clipping Plane (Rhino 6)
69
History’ button, which keeps the relationship between derived objects and their parent
object.
6.2.1.3 Importing Point Clouds
It has been mentioned that an entire import of the point cloud that is the most precise
(6.1.1.1) is not possible, due to software restrictions. The previous part of this chapter
discussed the segmentation of the point cloud into multiple ‘meaningful’ point clouds
that represent the objects to be modelled. One of the tabs of the thesis plug-in provides
an interface to import .e57-point clouds in different layers. These layers can be thought
of as a ‘semantic container’ that contains all geometric representations of the object, in
this case: the point cloud and the NURBS geometry. Although subsampling was
initially assumed a necessary step, importing all object-pointclouds separately without
subsampling also succeeded. This was probably because the separate point clouds are
not that large individually, and the fact that point clouds that are imported by the
plugin are immediately disabled until needed for modelling.
Fig. 6.13: setting a visibility schema for reverse engineering in Rhino 6
70
6.2.2 Modelling
6.2.2.1 Ceiling
The ceiling of the Presence Chamber is the largest object of the project. It consists of
6 smaller vaults that are supported by the walls (via consoles) and the two central
columns. Each vault is created separately and then connected to the other ones. Since
the vault is highly irregular, creating intersecting extrusions is no option; each surface
and rib is modelled individually.
Tracing the constituting lines in an orthogonal top view is the starting point of each
vault. (Fig. 6.14). These lines are then extruded to planes (Fig 6.15). The MPlane
proved most useful here: setting the extrusion planes one by one as the main working
plane allowed to draw planar ‘control point curves’ very quickly, the opaque (or semi-
transparent) planes clearly displaying the ‘intersection’ between point cloud and plane.
Very roughly, the control points are set out, using 3-4 control points per curve. Then,
the control points are moved in the plane, to align curve and object edge as exact as
possible. This method primary involves visual alignment, but proved quite intuitive
and exact. This procedure is followed for all edges of the vault; the ribs as well as the
surface edges. Surfaces are then constructed between these curves: for the ribs an ‘edge
surface’ or ‘planar surface’ (defined by 4 curves) is used. Each vault surface was divided
in two parts for better precision (Fig. 6.16). Different options were tested for the vault
surfaces (table 6.1). The ‘Sweep2’ command, which sweeps a cross section curve along
2 rail curve, gives the most satisfying results and will be used for constructing the vault
surfaces. Enabling the ‘Record History’ button connects the control points of the edge
curves to the surface, allowing to refit them and see the result immediately (Fig. 6.16).
The total ceiling surface is depicted in Figure 6.17.
Fig. 6.14: orthogonal projection of the defining curves of a vault (Rhino 6) Fig. 6.15: modelling the curves by changing control point location in the working plane (Rhino 6) Fig. 6.16: Record history enables edge curve manipulation after construction of the surface (Rhino 6)
71
Geometry With Point Cloud
1) Patch surface:
- UV surface that divides its curves into
segments: No exact matching between
original curves and surface edges.
Connection with other surfaces will
therefore be an issue.
- At first sight quite close to the point
cloud, however generally below the
points
2) Edge Surface
- Curves seem to fit quite well
- Unwanted convexity sets a clearly
visible deviation from the point cloud.
3) Network Surface
- Better edge matching options
(tolerance) than patch surface, but
still a UV surface that doesn’t
guarantee a watertight connection
with other surfaces.
- At first sight quite close to the point
cloud, however generally above the
points. Also slightly convex close to
the front curve.
4) Sweep2
- Edges match perfectly
- At first sight close to the point cloud,
with a majority of the points above
the surface close to the edges, the
opposite in the centre of the surface.
(this depends mainly on the edge
curves)
Table 6.1: Different surface generation techniques (Rhino 6)
72
Fig. 6.17: total surface of the ceiling (Rhino 6)
The height of the ceiling mass is acquired by importing the floor of the above storey,
extracted from point cloud 6.1.1.3. The vault’s outermost edges are extruded in z-
direction to connect the ceiling surface with the 1st floor. The result is depicted in figure
6.18.
Fig. 6.18: total ‘volume’ of the vault (Rhino 6)
Since only the scanned surface of the vaulted ceiling is relevant for deviation analysis,
this part will be imported into CloudCompare. In a separate Rhino document, the vault
volume is first exploded and the parts that are covered by the point cloud are exported
to an .obj mesh. In general, the point cloud covers the ceiling pretty well: small
occlusions will hardly influence the overall LoA of the ceiling. Therefore, the total
ceiling surface is exported to an .obj-file and imported in CloudCompare, along with
the original point cloud of the ceiling.
Two comparison methods exist for such deviation (Bonduel et al., 2017). The first
method involves involves subsampling the mesh into a ‘model point cloud’ (including
occluded areas) and perform a nearest neighbour calculation with the original point
cloud (which is set as the reference point cloud). Occluded areas will become clearly
73
visible, because the distances of the subsampled points in these areas will typically be
significantly larger than with non-occluded areas. These zones can then be extracted
from the original mesh and a second analysis (without occlusions) can be performed.
Alternatively, occlusions are removed before the first deviation analysis, after a visual
analysis of the point cloud, which is the method that will be used for the deviation
analylsis of the walls (section 6.2.2.2). The other method involves measuring the
distance from the point cloud to the mesh directly. Building elements in the point cloud
that are not modelled (e.g. light fixtures at the consoles) are included in the
measurement, which makes the analysis slightly overestimates the standard deviation.
Therefore, the first option is preferred. Nevertheless, since this is the first section on
modelling, both methods are illustrated here.
In the first method, 85 000 000 points were sampled on the mesh, comparable to the
amount of points in the measured point cloud (± 84 800 000). Since the vault is a quite
large object, a first, visual check is performed to determine subzones in which the cloud-
to-cloud distance exceeds 5 cm but that are not occluded in the original point cloud. A
visual scale has been set up according to the LoA specification. The red zones indicate
either occlusions or zones which were modelled not exact enough. Note that, on the
color scale, the red part is the largest because of some extreme values, and that this is
in no way related to the amount of points in the zone, as the distribution curve at the
right side from the colour scale indicates. Alternatively, the upper boundary of the
color scale can be set to 50 mm, which would display the now red parts in grey. Fig.
6.19 shows the first analysis. There are clearly some red zones which we know to be
not occluded in the original point cloud. Therefore, the geometry in these zones is
refined in the Rhino model, exported to .obj mesh, subsampled, and a second analysis
is carried out (Fig. 6.20). A full-page figure of Fig. 6.20 is added to Appendix A.
Fig. 6.19: First Cloud-to-Cloud deviation analysis, red zones mark either occlusions or deviations in the model that exceed 50 mm (CloudCompare Stereo v2.8.1)
74
Fig. 6.20: Second Cloud-to-Cloud deviation analysis (CloudCompare Stereo v2.8.1) (Appendix A)
As we can see in Figure 6.20, the remodelling step clearly reduced the amount of zones
that exceed the 50 mm limit. The mean distance is 13.8 mm, the standard deviation σ
14.6 mm. According to the Level of Accuracy specification, we can state that the
represented LoA of the vault is LOA20 (2σ < 50 mm), which corresponds with the goal
stated at the beginning of section 6.2.
The second method renders a similar image as the first method, because of the large
coverage of the reference point cloud (Fig. 6.21). The refined model was used
immediately for this analysis. Using this method, a mean distance of -5 mm is reached
and the standard deviation σ is 24.0 mm. This corresponds again to an LOA20, albeit
more close to the 50 mm treshold (2σ = 48.0 mm). As said, the difference with the
previous method in standard deviation is due to the fact that non-modelled elements
are included in the analysis. The lower mean distance is because this method includes
negative distances, depending on the normal of the mesh triangle that is measured.
The modelling of the walls relied on point cloud 6.1.1.2 (TLS and Photogrammetry).
For each wall, at least three sections were made: one at ground level (Fig. 6.22), one
at the lowest part of the vault (following the outermost edges of the ceiling to make a
‘watertight’ connection) (Fig. 6.23) and one at the first floor, marking the end of the
storey. To make the section as clear as possible, two opposite clipping planes are
brought as close together as possible. One of the clipping planes is set as a movable
working plane (MPlane), so when reorienting the clipping planes to another section
level, the working plane changes accordingly. The first step is to model the walls as
closed solids, voids for doors and windows will be cut out later. For now, if a void or
an occlusion (e.g. because of furniture) is present at section level, existing lines are
extrapolated. Since the front façade (south) contains some extra irregularities (Fig.
6.24), these are modelled by tracing the edges on a working plane, similar to the
working planes used in the ceiling, at the two corners of the wall. The corresponding
point at the other corner is then connected by a line, forming an edge surface. This is
done until the overall form of the modelled façade matches the point cloud (Fig. 6.25).
A point can be corrected three dimensionally by holding shift (to enable ORTHO mode)
and moving it closer to its corresponding point cloud vertex.
76
Fig. 6.22: section at ground level (Rhino 6) Fig. 6.23: MPlane for section at the beginning of the ceiling. The edges of the ceiling are traced for the interior of the walls (Rhino 6)
Fig 6.24-26: Irregularities in Front and eastern façade (Rhino 6)
Then, the openings in the wall are created as solid
volumes. A Boolean Difference will later subtract
these volumes from the wall, creating the openings.
To ensure a working Boolean Difference, the
volumes stick a little out of the wall (Fig. 6.27). For
most openings, it suffices to construct parallel
working planes based on the point cloud, drawing
the section with control point curves (or if possible,
arcs) and then lofting these sections one by one.
Finally, the ‘cap’ command converts the loft to a
closed volume that can be subtracted (Fig. 6.27).
After the openings in the wall are made, the hosted elements (windows and doors) can
be modelled (based on separate point cloud segments extracted from point cloud
6.1.1.1). The geometry of the wall opening serves as the basis for constructing them.
Details like hinges and window patterns will not be included in this model. Again, the
boundaries of the elements are drawn and serve as the edges for the volumes. The doors
are modelled very modestly: their contour on the inside and outside is traced and lofted.
For the windows, the frame is traced on the inner side of the wall hole and then
extruded, the depth based on a top view of the point cloud. Since the walls are massive
stone walls, it is assumed that the frames start where the wall ends, i.e. that there are
Fig. 6.27: Closed 'void' volumes will be subtracted from the wall solid (Rhino 6)
77
no parts of the frame that are hidden inside the wall. The lower part of the window
frames in the western façade is located on the outside (Fig. 6.26a-b).
Fig. 6.28a-b: window in the western wall: frame (red) glass (yellow) (Rhino 6) Fig. 6.29: bay window with separate panes (Rhino 6)
The thickness of the glass panels is impossible to determine
from the point cloud, because glass and other transparent
materials reflect and scatters the lasers from the scanner
(Fig. 6.30). Therefore, the glass gets an assumed thickness
of 3 mm, which is a usual thickness for stained glass.71 All
panes are modelled as one volume, except the bay window
at the front (southern) façade, in which individual panes are
separated by a wooden frame (Fig. 6.29).
Lastly, the consoles that transfer the weight of the vault to the walls are modelled
(point cloud 6.1.1.1). Since all consoles in the room are different and highly irregular,
modelling them relies mostly on using Mplanes and visual alignment with their
respective point clouds in a wireframe modelling process (Fig. 6.31). With these curves
as a basis, ‘edge surfaces’ (surfaces defined by three or four nonplanar curves) can be
constructed (Fig. 6.31). Sometimes, when a console-surface was quite orthogonal, an
extrusion was made instead of an edge surface. These surfaces are joined until they
formed a closed polysurface. It is made sure that the solid model perpetrates the wall,
so a Boolean Difference can later make a perfect alignment with the wall surface (Fig.
This concludes the section about the graph creation. Further information could be
added similarly, either with SPARQL queries, manual or by coding another plugin
(extension) to perform such enrichment in a more user friendly environment.
6.5 Conclusion
This chapter illustrated the workflow of the proposed scan-to-graph process with a case
study. The first part did not differ substantially with a regular scan-to-BIM process:
the available point clouds were analysed and segmented as a preparation for modelling.
3D modelling was done in Rhinoceros 6, not a BIM- but a CAD-environment, which
(probably) helped to cope with irregular geometries. Comparing the 3D models of the
elements with the point clouds is an essential feedback step for any as-is geometry
reconstruction method: for scan-to-graph as well as for scan-to-BIM and regular reverse
engineering processes. The last step involved semantic enrichment of the model, using
the plugin and beyond, resulting in an RDF graph that includes general semantic
information, metadata about occlusions, modelling remarks and the used point clouds,
and (optionally) also the 3D geometries. Only the graph without STEP representation
of the geometries is added as Appendix E. Due to property rights, the graph that
contains STEP representations, the .3dm Rhino file and the segmented point cloud
objects are not openly accessible. Images of the total model are added to Appendix A.
92
Chapter VII: Discussion and conclusion
An alternative approach for scan-to-BIM processes was developed in this master thesis.
It was investigated how Linked Data techniques might be used to cope with several
challenges that prevent a more frequent application of modelling an as-is BIM of an
existing building. The research resulted in a proposed variant of scan-to-BIM, called
‘scan-to-graph’. The scan-to-graph process is embedded in the Resource Description
Framework (RDF), the basic ‘language’ to connect information in a Linked Data
context. This final chapter evaluates the research regarding several criteria.
It can be concluded that a main advantage of a using Linked Data approach instead of
a ‘classic’ scan-to-BIM process, is that such approach enables to deal more efficiently
with metadata handling (section 7.1). Further, it has been shown implicitly that the
general advantages of Linked Data in the AEC industry as outlined by Pauwels et al.
(2017b) also apply to digitalisation of existing buildings. In this thesis, this was mainly
related to the there defined ‘Interoperability’ and ‘Linking across domains’ (section 7.2).
To be able to perform such a scan-to-graph process, a basic plugin for Rhino 6 was
developed to connect semantics with geometry and serialize the total into an RDF
graph (reviewed in section 7.3). As a consequence of detaching from ‘conventional’ BIM,
there was a greater freedom to choose the main 3D modelling environment. In the
modelling phase of the case study, the use of Rhino 6 for as-is modelling has been tested
(reviewed in section 7.4).
7.1 Metadata handling
Point clouds, made using TLS or photogrammetry, currently serve as the main basis
for the creation of a 3D model of an existing building. Although point clouds cover a
great part of the building geometry, remote sensing techniques will never reveal all
information about a building. Even a total coverage of the visible geometry does not
contain any information about the inner structure of walls, floors etc. Besides, in reality,
such a total coverage is seldom the case. Therefore, a ‘scan-to-graph’ vocabulary was
defined as a main deliverable of this research. It contains specific Linked Data classes
and properties for keeping track of sources, modelling remarks and occlusions. These
modelling remarks are currently bundled in a very general class. In the case study, their
use was restricted for linking comments about geometric assumptions (plus stating that
a geometry is ‘internal’). One subclass has already been defined, for making statements
about occlusions. In future research, subclasses specific for assumptions on inner
structure or other uncertainties could be developed.
Another metadata-related definition included in the scan-to-graph vocabulary keeps
track of the (represented) Level of Accuracy of the 3D reconstructed objects, after a
geometric deviation analysis has been carried out. This way, an easy check on the
93
accuracy of the model could be carried out by a simple SPARQL query, e.g. for better
monitoring of the 3D modelling process by the client or when a modelling team is
working together on one project.
7.2 General LD benefits applied to existing buildings
7.2.1 ‘Interoperability’
In Pauwels et al. (2017b), ‘Interoperability’ relates to the use of Linked Data for
developing a common data model that can use the same content in different
applications, while preserving the actual information. In the context of this research,
this ambition was interpreted as the implementation of parallel representations of the
same geometry, using open formats for use in various software packages. Current
geometry formats that are linked are E57 for the source point clouds and STEP for the
3D representation: E57 is linked as a URL reference (a URI that also represents the
location of the document) both local and on the web, STEP geometries are made
explicit in the graph by embedding them in the graphs as Literal strings. It has been
mentioned that other geometry types (e.g. meshes) might be included in a similar way.
7.2.2 ‘Linking across domains’
Another promising Linked Data consequence outlined in Pauwels et al. (2017b) is
‘Linking across domains’. Examples are the combination of ontologies, combining
product data with building data and connecting BIM with other disciplines. The
proposed scan-to-graph vocabulary engages in the principles for a semantic web-based
AEC Industry, which rely on the combination of simple, modular and extensible
ontologies (as outlined in Rasmussen et al., 2018) rather than large ontologies that try
to be all-encompassing. Describing information by use of ontologies allows the
information to be structured in a centralized model, the modularity as a means to keep
it at manageable size.
The combination of product data with building data is mentioned in Pauwels et al.
(2017b) in the case of product manufacturer data. In the context of as-is buildings, this
can be broadened to also implement historical products. As an illustration, the product
classes that were developed for specific elements in the case study are an extension to
the Building Product Ontology, but also link to RDF-based definitions of these
elements on the Art and Architecture Thesaurus (AAT). This way, extra ‘product
information’ is implicitly added to the product classes.
Connecting BIM with other disciplines is an important concept in the case of as-built
semantic 3D models. Cultural Heritage was mentioned in the text as such a discipline
(e.g. for extending a BIM’s usage to a virtual museum), other examples could be
Facility Management or methods to map energy analysis results of existing buildings
to the model. The core idea of the scan-to-graph process ‘as proposed’ is to serve as a
94
general basis for such existing-building-related disciplines to go further on this trend of
modular ontologies that are small and easy to connect.
7.3 Plugin
Since no software for performing a scan-to-graph process exists (nor a 3D modelling
software that uses Linked Data in general), a large part of the thesis consisted in coding
such application as a plugin for Rhino 6. The functionality of the plugin is limited to
building topology, product classification and the core domain of the scan-to-graph
vocabulary: mapping sources and modelling uncertainties to geometry. Going further
on section 7.2, future research might extend the plugin with methods for enriching the
geometry with information related to heritage, structure, building physics etc.,
ultimately providing a Linked Data-based BIM environment.
Although the graphs that are generated by the plugin only consist in open-formats, the
plugin itself has a major dependency on the commercial Rhino software environment.
This was considered a necessary step, because this dependency was outweighed by the
benefits: a well-supported interface to write custom plugins and the versatility of Rhino
for modelling distorted geometries.
7.4 CAD vs BIM for as-is modelling
Apart from the plugin development, the modelling phase of the case study was the
most time-intensive part of the research. Indicated by Volk et al. (2014), the demanding
process of modelling of out-of-plumb geometries is one of the core reasons that prevent
a more frequent application of BIM for existing buildings. A consequence of
programming the plugin in Rhino was that a CAD environment was used for modelling
instead of a BIM environment. As only one environment was used and an in-detail
comparison between reverse engineering functionality of different packages was out of
the scope of this thesis (and would be influenced anyway by the experience of the
modeller), only some general remarks on the use of Rhino for as-is modelling can be
made, without comparing it to other environments.
Generally, the Rhino interface provided an easy to use workflow, providing quick
perspective changes in 3D as well as orthogonal views. 2D drawing on reference planes
in a 3D perspective proved a valuable technique for pinpointing quite precisely the
curvature of different surfaces. A flexible use of orientable section planes and layers
significantly improved visibility and alignment to the point cloud underlay.
The greatest remark is that every object was modelled by drawing curves, and
constructing surfaces between these curves, thus literally defining an object by its
boundaries and then converting it to a closed polysurface. This was necessary because
95
solid primitives are too platonic to represent irregular as-is geometries. Probably, this
was the greatest strength as well as the greatest issue: using edges as the basic
components of each object significantly eased the implementation of geometric
irregularities. On the other hand, every object had to be converted to a closed solid by
hand, and its edges should be totally ‘watertight’. This manual control step is very
intensive and sensible for errors. Checking for invalid geometries or ‘capping’ indicated
openings that were often invisible, mathematical inconsistencies, cancelled out much of
the time that was gained by Rhino’s advanced modelling workflow.
7.5 Conclusion
It may be clear that the possibilities of using Linked Data for semantic enrichment of
existing buildings are not limited to the topics of this dissertation. Eventually, Linked
Data can serve as a unifying framework to combine as-is BIM, as-planned BIM,
Cultural Heritage, GIS and so on, in a truly centralized semantic model that links all
the available information about a building: its geometry, its structure, its history and
all the other disciplines that are concerned with existing buildings. With such prospect
in mind, it is hoped that this research project can indeed serve as a background for
further research regarding the implementation of these topics.
96
Bibliography
Abdul-Ghafour, S., Ghodous, P., Shariat, B., Perna, E., 2007. A common design- features ontology for product data semantics interoperability, in: Proceedings of the IEEE/WIC/ACM International Conference on Web Intelligence. IEEE
Computer Society, pp. 443–446.
Albertz, J., 2002. Albrecht Meydenbauer - Pioneer of photogrammetric documentation of the cultural heritage. Int. Arch. Photogramm. Remote Sens.
Spat. Inf. Sci. 34, 19–25.
Allemang, D., Hendler, J., 2011. Semantic web for the working ontologist: effective modeling in RDFS and OWL. Elsevier. USA.
usage for existing building interventions. Struct. Surv. 34, 168–190.
Bassier, M., Vergauwen, M., Van Genechten, B., 2017. Automated classification of heritage buildings for as-built bim using machine learning techniques. https://doi.org/10.5194/isprs-annals-IV-2-W2-25-2017
Bonduel, M., Bassier, M., Vergauwen, M., Pauwels, P., Klein, R., 2017. Scan-to- bim output validation: towards a standardized geometric quality assessment of building information models based on point clouds. ISPRS - Int. Arch.
Dubois, S., Vanhellemont, Y., de Bouw, M., 2017. WTCB Innovation Paper: Geometrische opmeting in hoge resultie - 3D digitalisering in het BIM-tijdperk. WTCB.
Hauck, O., Kuroczyński, P., 2014. Cultural Heritage Markup Language, in: Proceedings of the 20th International Conference on Cultural Heritage and New Technologies 2015, Museen der Stadt Wien, Vienna.
Kraus, K., 2012. Photogrammetrie: Geometrische Informationen aus Photographien und Laserscanneraufnahmen. Walter de Gruyter, Germany.
Kuroczyński, P., Hauck, O., Dworak, D., 2016. 3D Models on Triple Paths-New Pathways for Documenting and Visualizing Virtual Reconstructions, in: 3D
Research Challenges in Cultural Heritage II. Springer, pp. 149–172.
Liao, C.-T., Huang, H.-H., 2012. Classification by Using Multispectral Point
Luhmann, T., Robson, S., Kyle, S.A., Boehm, J., 2014. Close range photogrammetry: principles, techniques and applications. Walter de Gruyter, Germany.
Mezzino, D., 2017. Cultural built heritage’s tangible & intangible dimensions and digitalization challenges (PhD Thesis). Politecnico di Torino-Carleton University.
Nan, L., Xie, K., Sharf, A., 2012. A Search-Classify Approach for Cluttered Indoor Scene Understanding. https://doi.org/10.1145/2366145.2366156
Nuttens, T., Stal, C., Wisbecq, J., Deruyter, G., De Wulf, A., 2014. Field comparison of pulse-based and phase-based laser scanners for civil engineering applications, in: 14th International Multidisciplinary Scientific Geoconference (SGEM). pp.
169–176.
97
Ochmann, S., Vock, R., Wessel, R., Klein, R., 2016. Automatic reconstruction of parametric building models from indoor point clouds. Spec. Issue
Perry, M., Herring, J., 2012. OGC GeoSPARQL - A Geographic Query Language for RDF Data. (OGC 11-052r4). Open Geospatial Consortium.
Pauwels, P., Krijnen, T., Terkaj, W., Beetz, J., 2017a. Enhancing the ifcOWL ontology with an alternative representation for geometric data. Autom. Constr.
Pauwels, P., Van Deursen, D., Verstraeten, R., De Roo, J., De Meyer, R., Van de Walle, R., Van Campenhout, J., 2011. A semantic rule checking environment
for building performance checking. Autom. Constr. 20, 506–518.
Pauwels, P., Zhang, S., Lee, Y.-C., 2017b. Semantic web technologies in AEC
industry: A literature overview. Autom. Constr. 73, 145–165. https://doi.org/10.1016/j.autcon.2016.10.003
Rasmussen, M., Pauwels, P., Lefrançois, M., Schneider, G.F., Hviid, C., Karlshøj, J., 2017. Recent changes in the building topology ontology, in: LDAC2017-5th Linked Data in Architecture and Construction Workshop.
Rasmussen, M., Pauwels, P., Hviid, C., Karlshøj, J., 2018. The BOT ontology: standards within a decentralized web-based AEC Industry. Under review.
Schwermann, R., 2017. Lecture slides for the course ‘Photogrammetrie’ at RWTH Aachen. RWTH Aachen, Aachen.
modeling for 3D scene understanding. Comput. Graph. 55, 55–67. https://doi.org/10.1016/j.cag.2015.11.003
Svenshon, H., Grellert, M., 2010. Rekonstruktion ohne Befund? in: Befund und
Rekonstruktion. Mitteilungen der Deutschen Gesellschaft für Archäologie des Mittelalters und der Neuzeit, 22. Paderborn.
Thomson, C.P.H., 2016. From Point Cloud to Building Information Model: Capturing and Processing Survey Data Towards Automation for High Quality 3D Models to Aid a BIM Process (Doctoral). UCL (University College London).
U.S. Institute of Building Documentation, 2016. Level of Accuracy (LOA) for Building Documentation. Document C120 Version 2.0.
van Genechten, B., 2008. Theory and practice on Terrestrial Laser Scanning. Universidad Politecnica de Valencia Editorial. Valencia, Spain.
Volk, R., Stengel, J., Schultmann, F., 2014. Building Information Modeling (BIM) for
existing buildings - Literature review and future needs. Autom. Constr. 38,