CONNECT. TRANSFORM. AUTOMATE. Using FME to find out… … what we already know Roland Martin Senior GIS Consultant
Jun 21, 2015
CONNECT. TRANSFORM. AUTOMATE.
Using FME to find out… … what we already know
Roland Martin Senior GIS Consultant
Using FME to find out… …what we already know
! Everyone deals in data ! We all tread a line between:
! Reinventing the wheel, and ! Doing things the old way
! It helps if you are aware of what you already know about a particular location
! A global, integrated, multidisciplinary firm of professionals working together to tackle complex planning, infrastructure and building design challenges
! 11,000 engineers, designers, planners, management consultants and economists
! 90 offices in 37 countries
! Full service from concept through completion
! Dedicated to delivering value through expertise, global resources and local delivery
The kind of questions GIS people normally get asked
! Source: made up
CONNECT. TRANSFORM. AUTOMATE.
10 GIS? What’s that?
9 What do you spend all your time doing anyway?
8 Don’t you just draw maps all the time?
7 Is GIS really as good as AutoCAD / MicroStation / BIM?
6 Is it really worth using GIS when I can do it in CAD?
5 Can you get the data out of this scanned PDF for me?
4 Can you show me my project in Google Maps?
3 Can you draw me a map of my project location?
2 I need something that looks “cool” or “cartoon-like”. Can you help?
1 What data do we already have for my project location?
What kinds of things might we already know about a location?
! Base mapping ! Topography ! Land ownership ! Site photos ! Site surveys ! Aerial imagery ! Borehole locations ! Project reports
© O
penS
tree
tMap
con
trib
utor
s, e
Map
Site,
CC-B
Y-SA
How would we normally find out what information we already have?
! Much of the time, it simply isn’t possible ! Occasionally, someone will remember something…
http
s://
ww
w.f
lickr
.com
/pho
tos/
galo
poul
os/
Can we actually reuse any of the data?
Probably
Possibly, but data is often licensed to a particular client
Unlikely – data is normally licensed for a particular project
© O
penS
tree
tMap
con
trib
utor
s, C
C-B
Y-SA
What do we want to do with the data?
! For a decent, modern web map, you probably want: ! Leaflet (or OpenLayers) on the front-end ! PostGIS (or CouchDB) on the back-end,
generating GeoJSON directly ! Something in between to serve the data (we went
for a PHP-based API) ! FME Server, to automate the data loading process
Building a database of existing GIS data
! Office locations ! Previous projects ! Site photos ! Site surveys
Map
dat
a ©
Ope
nStr
eetM
ap c
ontr
ibut
ors,
CC-B
Y-SA
Mapping office locations
© O
penS
tree
tMap
con
trib
utor
s, C
C-B
Y-SA
Mapping project locations
© O
penS
tree
tMap
con
trib
utor
s, C
C-B
Y-SA
Enriching the data
! Weed out the obvious errors by overlaying the locations on country outlines
! Use a geocoding service to add project locations where addresses are known
! Project-specific datasets may provide the clue to more locations…
Mapping site photos
! Thousands of photos are taken for projects
! But everyone forgets they exist!
Changing technologies
! When people come back from a site visit, they mark up a map with the photo locations
! Wouldn’t it be easier if they could go straight on a web map?
Stealth GIS data collection
! The smartphone revolution ! Photos are still being taken –
but now they include location information!
Searching for existing photos
! The “Directory and File Pathnames” (PATH) reader – lets you build a simple file spider
! You can also extract additional properties (i.e. who last modified the file, and when)
Adding more photos
! Use FME Server’s scheduling capabilities to regularly check a shared folder location
! Use the notifications in FME Server to allow people to email photos directly at the server
Spatial photo library ©
Ope
nStr
eetM
ap c
ontr
ibut
ors,
CC-B
Y-SA
Spatial photo library ©
Ope
nStr
eetM
ap c
ontr
ibut
ors,
CC-B
Y-SA
Collecting site surveys
Spiderer – loops through the drive looking for particular file types
! We searched our drives for files matching the patterns: ! *SV*.dgn or *SV*.dwg ! *survey*.dgn or *survey*.dwg
PostgreSQL
Collecting site surveys
SurveyReader – reads file locations from PostgreSQL and writes footprints into PostGIS
Collected site surveys
Collected site surveys
! Survey data may not be reusable on other projects
! Surveys in some locations go out of date very quickly
! But… this is still an extremely valuable resource
Bringing it back together
© O
penS
tree
tMap
con
trib
utor
s, C
C-B
Y-SA
Thank You!
! Questions?
! For more information: ! Roland Martin – [email protected] ! Arup – www.arup.com
CONNECT. TRANSFORM. AUTOMATE.