Brewing the Ultimate Data Fusion

Brewing the Ultimate Data Fusion

1. Choose your Potion Ingredients ●  Spatial (GIS, CAD, BIM, raster, etc.)

●  Web

○  FTP ○  Web Services - OGC, AWS, Dropbox, etc. ○  APIs ○  HTML webpages ○  JSON, XML, RSS feeds

●  So many more!

2. Choose a Cauldron NoSQL

Relational database

File-based database

Visualization/BI tool like Tableau, Qlikmaps

Metadata catalog

Master Database vs. Metadata Catalog

Data Fusion # 1

Awesome Vector Tile World Backgrounds

Ana García de Vicuña Pablo Martínez

Vector Tiles Format in FME

Geographic information stored in GeoJSON format into an SQLite database. Different layers for each zoom level.

Raster Tiles •  Static PNG

•  Each collection only one style •  Large storage cost •  Tile generation takes time •  Painful update of data

Vector Tiles •  GeoJSON

– Dynamic PNG on the fly – Style defined in XML file

•  Each collection multiple styles •  Less storage cost •  Less generation time •  Easy to update

Raster Tiles vs. Vector Tiles

Multilevel Tile Cache Generation Engine

Vector data text labels

Vector Tile Mapnik XML Symbology

PNG from Mapnik in the Server

Tile Render

Vector and Raster Tile Composition Merge Raster Tiles (Orthoimage or Shaded Relief) with stylized Vector Tiles to get another background.

FME Workbench

Input Workbench Output

tiles

View

Get the whole story at the

Data Fusion # 2

YVR & the Indoor Mapping Experience

Carlos Silva GIS Specialist, Engineering Services

Vancouver International Airport

YVR: 4 yrs ago (pre-GIS)

CAD, 3 – 6 months post-construction

Graphic design pamphlets, PDF

How the Mapping Process Works @YVR

1st Stage "Internal", FME

2nd Stage "Public", FME

3rd Stage "Uploads" via FME

4th Stage, "Apple AVF" Hideyuki KAMO

ArcMap: Map Production; Data Verification & Edits; Map & Feature Services (Internal Only)

YVR Digital Gateway - JSON

Where is YVR now?

1) Public Terminal Maps (PDFs, Hard Copy Pamphlet, Website Maps (Leaflet with GeoJSON)

2) Internally: CAD 2 GIS via FME, accessible on

an internal GIS application

3) YVR App & Website: Data Delivery via YVR Digital Gateway

Where is YVR now?

Typical Issues/Solutions - timeliness

- spatial errors/topology - additional data points (sources) changing

- software & DB updates - I.T. updates

KEY: Automation, standards, and API access

Get the whole story at the

FMEUC

Data Fusion # 3

DRONES! Drones Everywhere!

Renato Salvaleon Alabama Power

SCS Milestones in UAS Program

•  UAS Policy in place (FAA compliance)

•  UAS certification training of 60

•  Southern Company is leading the electric utility industry in UAS

PlantFarleyUASInspec2on:NuclearIndustryfirst

UASCer2fica2onTraining

PlantBranch,GA-1000feetabovegroundlevel

Geospatial data in the UAS workflow

Fly

ReviewDelivery(Option)

Plan

ArcGISPla+orm

From Excel and File Structure

…is there a better way to visualize these missions and data products?

Transform missions from rows to points

… project & data locations of each mission are stored in an Excel or HTML report created by FME and stored as an

attachment on each mission point feature

From Excel into a Dashboard in Portal

Every Mission delivered as feature layer in Portal

… mission data delivered as a feature layer by FME rendered identically using FME Python Caller and standard ESRI layer file. A web map is created in Portal,

every picture is a point feature on the map and is hyperlinked for review purposes

UAS Data Products that can be processed

NADIR

Oblique

Video

Metadata

Log files

MEANINGFUL, RELIABLE and TIMELY DATA

Data Fusion # 4

P. Veenstra, TRC

Pipelines and the AttributeCreator:

Crunchy Magic for Accessing Previous and Next Rows

How do I… without programming? Challenge 1:

Access the values in the previous or subsequent rows in a data set without scripting or programming?

•  Read data •  Load data into a record-set or in-memory data set (array, list, dictionary) •  Access data •  Process data

Challenge 2:

Do this in a workspace that takes a set of points in the field (X,Y,Distance) and create a M-aware polyline with cumulative measure set for each vertex along the line?

•  Use the ‘AttributeCreator’

The Workspace •  CSV file containing ID, X, Y, and Distance •  Transform into M-Aware (measure) points and a polyline route

•  Each point feature will have a M value assigned to the vertex •  Each polyline feature will have M values assigned to each

vertex in the line AND will store the begin distance = 0 and the cumulative total distance = ?

•  Utilize AttributeCreator to access previous points in data set to calculate cumulative distance

•  Utilize MeasureSetter to make geometry M-Aware •  PointConnector preserves the M values of the points and applies

them to the polylines

The Workspace

Other magical artefacts …

•  Calculate true rhum (geodesic) distance between points … •  Calculate simple 3D pythagorem distance between points … •  Calculate the angle left or right from one vector to another … •  Other than the expression editor in FME, it makes math for points on

a line easy …