Topics In-memory Spatial Database based on key-value pair Building Information Model based on relationship inference.

IN-MEMORY SPATIAL DATABASE BASED ON KEY-VALUE PAIR

Topics

In-memory Spatial Database based on key-value pair

Building Information Model based on relationship inference

In-memory Spatial Database based on key-value pair

Background

Architectural Overview

Features

Background

This research is supported to solve

kernel issues during the State Grid information construction.

The achievements have been applied to the power information systems.

Deploy view

State Grid

Shanghai Grid

…

User1

User2

…

Hubei Province

Grid

Beijing Grid

User1

User2

…

User1

User2

…

User1

User2

…

Network

Storage

Software …

Physical

Resource

Graphics

…Vector

Database

ManageAnalysi

s…Spatial

Power Grid Services

ERPMarkin

gGIS …

Systems

PlanningSchedul

eSale …

Businesses

SecurityData

Network

System

…

Application IntegrateProduct

Monitor

Emergency

…

Technical Architecture

Background

Store Data

Organize Data

Access Data

Others

Large scale dataKey-valueTransaction Locks Lazy write to DBMS

High concurrency High throughout Batch to submit data long period, offline data edit Increment data query Can’t be accessed directly by user hybrid index based on spatial and topology

Change notify Reuse existing power database, such as Oracle Spatial, SDE, DB2 Spatial Extender Data synchronize …

Key problems

Partition based on spatial and topology logical data model Version History Database

Background

Why not traditional spatial database?

Why not DBMS, such as Oracle?

Why not SDE or ArcServer?

Why not object-oriented, such as Oracle spatial?

Why not traditional spatial index, such as Quad-tree, R*-tree?

Bad I/O throughout Bad performance when high concurrency and batch editCan’t provide necessary features Bad performance There are some bugs in LinuxCan’t provide necessary features

More slow than raw data type, such as BLOB Spatial index has bugs and has to be rebuilt at fixed periodSpatial index performance is badPrecise filter is even slow, such as sdo_relateSpatial function is not necessary

No. We need a hybrid index, including spatial, topology and attributes information

Background

Why in-memory spatial database on key-pair

Why store data in memory? How is the data persistence?

Why use key-pair? What is your spatial model?

Do you use spatial index?

Do you support condition query like SQL?

in-memory database is much fast than disk database The changed data will be written into DBMS

Simple means high performance! In-memory database doesn’t need complex search

We use the hybrid index, including spatial, topology and attribute information

We support simple compare function ,such as greater, equal, less…

httpd

Server computer

http

d

Serv

er

comp

uter

httpd

Server

computer

httpd

Server computer

http

d

Serv

er

comp

uter

httpd

Server

computer

Data synchronize

Data synchronize Data synchronize

cluster cluster

Lazy wri te Lazy

wri

te

DBMS

Architectural-Logical View

Apache httpd

Server computer

In-memory Database

Architectural-Framework

Memory management

Cache

Memory schedule

Page management Index library

Access API

Thread management

Persistence Pool refresh

Connect pool

Index

Hash Index

Spatial Index

In-memory database

Data edit

Pool FillQuery

HTTP protocolData

synchronize

Features-Data Model

Indeed, every record is a memory block!

Typical data struct

Index for attributesB-tree index, Hash index

Spatial index Spatial index is a hybrid index: B-tree + grid index + topology based index

protocol head description field value 1 field value 2 … field value n

Features-Cluster In my research, cluster is a group of virtual, loosely

coupled in-memory database that work together closely.

In a cluster, all in-memory databases have the same data.

For different group, maybe they have intersected data.

How to partition the power data for cluster is based on the spatial , topology and attribute information

When some data are updated in one in-memory database, other in-memory database which has these data will be synchronized almost real-time.

Features-Change notify

When data is update, clients will receive the notify.

The notification granularity is based on object, not record, which help to

The notify is based on subscribe & notify model.

User can only download changed records.message server

2012年2月21日 2012年8月21日

2012年4月1日 2012年7月1日

Snapshot 1 Snapshot 2

Ti me axi s

Features-Version & History DB Provide version mechanism and history

database based on time axis .

Sometimes, work maybe needs several days, even a month to finish. These data will be managed as a version and other client can’t see them before finishing the work.

At any time, in-memory can create database snapshot for archive or future comparative analysis.

confl ict resolve

confl ict resolve

commit commit

version 3version 2

version 1

BASE

BIM based on relationship inference

Background

Roadmap

Background

Design

Management

Construction

In common, the life cycle of a building includes 3 phases:

IFC is a vendor-neutral BIM data repository for the semantic information of building objects, including geometry, associated properties, and relationships, to facilities.

IfcPropertyDefinition

1

*IfcTypeObject

HasProperty Sets *L[1:?]

(INV) Def inesTy pe S[0:1]

(ABS)

IfcPropertySetDefinition

(INV) PropertyDef initionOf

S[0:1]

IfcRelDefines (ABS) (ABS)

IfcRelDefines ByType

IfcRelDefines ByProperties

1

RelatingTy pe

RelatingProperty Def inition

(INV) Ty peObjectOf

S[0:1]

IfcPropertySet IfcXxxProperties

1

IfcObject (ABS)

RelatedObjects S[1:?]

come from T.E. El-Diraby

(ABS)

IfcRelationship

1

IfcRelAssigns

IfcRelConnects

IfcRelDecomposes

IfcRelAssociates

IfcRelDefines

(ABS)

(ABS)

(ABS)

(ABS) (ABS)

come from T.E. El-Diraby

I f cRoot

I f cObj ect

I f cRel ati onshi p

I f cProperty

0 200 400 600 800

IFC1.0

IFC1.5

IFC2.0

IFC2x

IFC2…

number of classes in IFC

BackgroundImpression

IFC is a complex, relatively complete and huge specification.

From the OO point of view, IFC is a well designed and loose coupled class set.

Background

Problems and Limitations for IFC

For OO design, message or event driven is an import design pattern, but it seems there is no definition about it.

IFC doesn’t define something about analysis, which is important for management phase during the life cycle of a building.

The ability of relationship description is much weak and can not describe the implicit relationship automatically, which is import for analysis and modeling.

GIST

belong to

OTB

Liubelong

toGIST

Liubelong

toOTB

Roadmap

Goals

Provide the ability of relationship inference.

Improve the degree of automation during the process of building information modeling.

Provide a mechanism for building information analysis

Roadmap-relationship inference

I FC(I f cRel ati onshi p)

Formal Descri pti on

OWL

Rel ati onshi p Operator(aggregati on, group, order, add, subtract, . . )

I f cXXXRel ati ons (deri ved f rom I f cRel ati onshi p)

Bui l di ng Model i ng Bui l di ng Anal ysi s

rel ati onships

Cl assi fi cati on

Property

Constrai n

Classification based on the number of relation

unaryBinary triple

Classification based on type

subsumption(subClass-of, superClass-of, …)mereology(is-part-of,) topology(next-to, touch-with, …)….

transitive

symmetric

inverse

equality

hasValue

exists at

occur at

maximum minimum

Roadmap-analysis based on event driven

bui l di ng i nformati on pi pel i ne

I fcObject 1 IfcObject 2 IfcObject . . .

analysi s analysi s analysi s

Initial thoughts

Information transmission is modeled a virtual pipeline.

Every object will get information from pipeline and analyze them. The result is also pushed into the pipeline.

The purpose: 1) object is self-contain; 2) object is independent of other object; 3) combine different analysis

Thank you