7/30/2019 Database Driven 3D Content Management Systems
1/45
Database Driven3D Content Management
Systems
By Tim Child
3DMashUp
7/30/2019 Database Driven 3D Content Management Systems
2/45
Outline
Biography
3D Content Applications
3D Content Categories
3D Geometry
3D Samples
3D UDTs & Functions System Architecture
3D Queries
Demo
GPGPU Acceleration
Web 2.0 Integration
Performance Challenges Further Development
Summary
Q & A Session
7/30/2019 Database Driven 3D Content Management Systems
3/45
Biography
Tim Child
35 years experience of software development
Formerly
VP Oracle Corporation VP BEA Systems Inc.
VP Informix
Leader at Autodesk, Navteq, Intuit,
30 + years experience in 3D, CAD, GIS and DBMS
Built >10 Spatial DBMS Applications
7/30/2019 Database Driven 3D Content Management Systems
4/45
3D CMS Issues
Ad Hoc
File/SCM based
Large number for file formats Files dispersed across the users network
Hard to Sync content between files
Poor Web 2.0 Integration Need of Real-Time Display
7/30/2019 Database Driven 3D Content Management Systems
5/45
3D Content Applications
Augmented/Virtual Reality
Medical
10x growth in 5yr - Baptist Hospital
AEC / BIM
Arup predicts 70% growth in BIM data/yr
GIS / Earth Sciences /Environmental Sciences
CAD / CAM
Games / 3D Video
7/30/2019 Database Driven 3D Content Management Systems
6/45
3D Content
Catagories
Vector Graphics Geometry
Mathematical Representations
Topological Rules
Many varieties of Objects
Many File Formats
Raster Graphics Samples
Measurements
Location X,Y,Z
Time t
Channels
R,G,B
Intensity levels
Discreet/Quantized
Noisy Need Calibration
Scales N * N * N
7/30/2019 Database Driven 3D Content Management Systems
7/45
Representing
3D Geometry
Point
Line
Triangle
Polygon
Mesh
7/30/2019 Database Driven 3D Content Management Systems
8/45
Representing 3D Samples
Pixel/Voxel
Quantized Intensity (x,y,z,t)
Point Cloud
Voxel Array
7/30/2019 Database Driven 3D Content Management Systems
9/45
PostgreSQL
Advanced Capabilities Functions
Methods executing with in the server
Consider them a Delegate Methods Take the method to the data
Languages PGSQL
C (UDF)
Java Others,
Types Complex types and Arrays
Used Defined Types (UDT)
Examples include
XML, JSON, PostGIS
Indices BTree 1 Dimensional
RTree Multi Dimensional
Inverted Index Text searches
GIST Roll your own
User Defined Aggregate (UDA) Supports Sum, Min, Max, Average,
7/30/2019 Database Driven 3D Content Management Systems
10/45
System Overview
Authentication
Service
HTTP
Server DBMS
Server
Web Pages
LDAPActive
Directory
DCC
Tools
(COLLADA)
SQL Table
Storage
WebDavFile
Storage
Web Browser
View
(WebGL)
Plug-
In
Existing
CMS/SCM
Web / File
Crawler
File
StorageFile
Storage
3D
MashUp
Service
3D
Analytics
Service
7/30/2019 Database Driven 3D Content Management Systems
11/45
System Architecture
7/30/2019 Database Driven 3D Content Management Systems
12/45
3D UDTs & Functions
Current 3D Schema
75+ Functions
15+ UDTs
20 + Tables
7/30/2019 Database Driven 3D Content Management Systems
13/45
Recap
Native SQL representation of 3D objects
Supports
Vector Graphics
Raster Graphics 3D Content Aware
Vector math
4 x4 Homogenous coordinate transformations
Polygon to Polygon Intersections
Image processing
7/30/2019 Database Driven 3D Content Management Systems
14/45
IMPORTANT
3D QUERIES
DBMS Goal:
Provide sufficient data to rapidly render the scene with acceptable fidelity
7/30/2019 Database Driven 3D Content Management Systems
15/45
Camera View
Simplify Camera
Field of View to
a Cone
7/30/2019 Database Driven 3D Content Management Systems
16/45
Spatial Cone Query
7/30/2019 Database Driven 3D Content Management Systems
17/45
Spatial Indexing
Spatial Indexing allows scalable queries from > 100M Objects
7/30/2019 Database Driven 3D Content Management Systems
18/45
View Coherent
Queries
7/30/2019 Database Driven 3D Content Management Systems
19/45
Mesh Size
Stanford Bunny
70K Triangles
39K Vertices
XYZ Dragon
7.2M Triangles
3.6M Vertices
No point in using 7M triangles to render 64 x 64 pixels
7/30/2019 Database Driven 3D Content Management Systems
20/45
Mesh Spatial
ProcessingEach mesh subdivided into1024 x 1024 x 1024 cells Each triangle centroid is
sorted on a Z Coordinate
0-
10
23
Morton Z Coordinates
Each 10 bit coordinate
Is bit interleaved
(Z9,Y9,X9, Z0,Y0,X0)
7/30/2019 Database Driven 3D Content Management Systems
21/45
Spatially Clustered
Mesh Storage
PostgreSQL
Server
8 KB DBMS Pages
Each page Contains
50 100Mesh Triangles
Spatially Clustered
Spatially clustered meshes optimizes DBMS access!
Queries
7/30/2019 Database Driven 3D Content Management Systems
22/45
LOD Processing
On loading a Mesh
Compute and Store
Reduced LOD
Representations
7/30/2019 Database Driven 3D Content Management Systems
23/45
Triangle Decimation
Edge
Collapsing
Collapsing Cost
based on
Area and
Angle Between
Cost based algorithm
7/30/2019 Database Driven 3D Content Management Systems
24/45
LOD Queries
Document
Table
Mesh
Table
Triangle
Cluster
LOD 0
Triangle
Cluster
LOD N
LOD Determined by Query
Based on Object Size
and Query Distance
Query pathfor
Detailed Objects
Query pathfor
Coarse Objects
7/30/2019 Database Driven 3D Content Management Systems
25/45
PostgreSQL
Rules and Triggers
Select, Update, Delete Rules
Do ALSO Command, Command, Extends SQL operations
Do INSTEAD Command, Command, Replaces SQL operations
Insert, Update, Delete Trigger Events
Before Event Command
Or
After Event Command
SQL Views and Rules System hides implementation complexity
7/30/2019 Database Driven 3D Content Management Systems
26/45
Recap
Camera SQL Queries Selects only whats visible
View coherent queries Optimizes scene data retrieval for camera motions
2 Level Spatial Indexing RTree Supports large number of (>100M) objects
Z-Ordering improves finer access for large meshes
Spatially Clustered Storage Optimizes partial mesh access
LOD processing Optimizes speed versus size for large meshes
PostgreSQL
UDTs and UDF encapsulate functionality in server
Rules System hides implementation complexity
7/30/2019 Database Driven 3D Content Management Systems
27/45
Demo
7/30/2019 Database Driven 3D Content Management Systems
28/45
GPGPUACCELERATION
7/30/2019 Database Driven 3D Content Management Systems
29/45
GPGPU
Goals
OpenCL a PostgreSQL Procedural Language
OpenCL Kernels execute as SQL UDFs
Type Mapping, support common data types Vectors, Arrays, Images
Sorting
Used GPGPU based sorts in SQL queries
7/30/2019 Database Driven 3D Content Management Systems
30/45
WEB 2.0
INTEGRATION
7/30/2019 Database Driven 3D Content Management Systems
31/45
Web 2.0 Goals
Provide a Rich Web 2.0 UI
Integration with Web 2.0 Apps
Programmability
Tracking & Analytics
7/30/2019 Database Driven 3D Content Management Systems
32/45
Rich UI
HTML 5
2D Canvas
WebGLDrag & Drop
.
Others?Silverlight for IE9
7/30/2019 Database Driven 3D Content Management Systems
33/45
Web 2.0 Integration
Linking
Embedding
Social Bookmarking
Tagging
Drag & Drop
Annotations
Mash-Ups
7/30/2019 Database Driven 3D Content Management Systems
34/45
Programmable
REST APIs Supports Browsers & other Web 2.0 Apps
SQL Queries Supports LAMP, RoR, POJ, ASP.Net,
XML Input / Output support Document Exchange and Import / Export
JSON Output provides Easy integration with JavaScript
7/30/2019 Database Driven 3D Content Management Systems
35/45
Tracking & Analytics
Whos looking at what?
HTTP
Server
DBMSLogging
Service
Analytics
Service
Web Browsers
Content
QueriesHTTP
Request /
Response Log
Queries
Analytics
Report
Queries
View Web
Analytics
Reports
7/30/2019 Database Driven 3D Content Management Systems
36/45
3D MashUp
Service
URILook-Up URIRequest
BrowserContent
Filter
Transform
ClipMerge
URI
Request
Web Site
A
Web Site
B
MashUp
Meta Data
HTTP
Request /
Response
HTTPRequest /
Response
HTTP
Request /Response
3D MashUp Engine
Acts as Proxy between Browser and Web Sites
Filtering, Transforming, Clipping and Merging
Content Responses
7/30/2019 Database Driven 3D Content Management Systems
37/45
CONTENT
MANAGEMENT
7/30/2019 Database Driven 3D Content Management Systems
38/45
CMS Goals
Track Revisions & Changes
Compare documents
Publish Documents
Annotation
Metadata Queries
7/30/2019 Database Driven 3D Content Management Systems
39/45
Content Versioning
Rev 0 Rev 1 Rev 2 Rev 3
Rev 1.0
Rev 4
Rev 1.1
All Database Rows include Version ID
Versioning at the File, Section, and Record Levels
7/30/2019 Database Driven 3D Content Management Systems
40/45
Contents Diff
SHA1 SHA1
SHA1 SHA1
SHA1 SHA1
SHA1 SHA1
Objects in the database for the 3D content
are signed Cryptographically
Revision N Revision M
Diff
Diff
Diff
Diff
7/30/2019 Database Driven 3D Content Management Systems
41/45
PERFORMANCE
7/30/2019 Database Driven 3D Content Management Systems
42/45
Performance Challenges
Visual Scenes
20K 5M Polygons
Near Real-Time Queries Queries 15m 30ms end user response
How much can I retrieve in 16.7 ms?
DB Size < RAM DB Size >= Ram Size && 1 T B
7/30/2019 Database Driven 3D Content Management Systems
43/45
Further Development
Areas
Performance Investigations
What are the important queries?
Where are the bottlenecks?
Picking Operations Using a Cylinder to pick from a view
Editing Operations
Updating Geometry
OpenCl as Procedural Language
Allows cleaner OpenCl integration
7/30/2019 Database Driven 3D Content Management Systems
44/45
Summary
Why a 3D in a Database? Shared
Multi-user
Scalable
Reliable
Extensible
Secure and well define Security Model
Rich Tools Set
Why choose PostgreSQL for 3D Advanced capabilities
Open and Opensource
Versatile Simple to use
Robust Developer Community
Leverages existing skill sets
7/30/2019 Database Driven 3D Content Management Systems
45/45
Q&A