The FME Oven: Never Too Many Ingredients

The FME Oven – Never Too Many Ingredients

Kurt HartmanDirector of Technology, Accurate Assessment Group Ltd.

Agenda

Introduction to Accurate Assessment Group Case Study – Video Logging Case Study – ERCB Data Questions

Introduction – Client Map

Introduction – Webmap Clients

Urban ClientsCity of Wetaskiwin

Town of Barrhead

*Town of Beaverlodge

*Town of Edson

*Town of Fox Creek

Town of High Level

Town of Peace River

Town of Redwater

*Town of Sexsmith

Town of Stettler

*Town of Two Hills

*Town of Wembley

*Town of Valleyview

*Village of Derwent

*Village of Myrnam

*Village of Willingdon

* Regional Sites

Rural ClientsBrazeau County

County of Athabasca

Camrose County

*County of Grande Prairie No. 1

County of Minburn

County of St. Paul

County of Stettler

*County of Two Hills No. 21

County of Wetaskiwin

Kneehill County

Lamont County

*Municipal District of Greenview No. 16

Municipal District of Opportunity No. 17

Rural Municipality of Wood Buffalo

Smoky Lake County

Westlock County

Wheatland County

Woodlands County

*Yellowhead County

Introduction – Municipal Information Integration

Video Logging

Video logging: a method of displaying video data within a GIS

Video is captured using a vehicle equipped with digital video cameras, precision GPS and on-board computers

Video Logging

Precise digital images are captured at regular intervals from GPS-equipped vehicles traveling at regular road speeds up to 100 km/ hr.

Video is post-processed using asset extraction/ identification software.

Assets can be located 80-100 meters from the vehicle.

Video Logging – Ingredients...

5,100,000 images 1,700,000 points

X,Y,Z GPS heading Image name Clip name Date/Time

Road network

Video Logging – Out of the oven...

4,800,000 images 1,600,000 linear referenced events

Linear reference values/keys Direction of travel Image name Image path

Batch files to create necessary directories Batch files to rename and move image files

Video Logging – Challenge 1

Attaching points to the correct road

Video Logging – Solution 1

Use Labeller to determine the orientation of the road in the vicinity of the point

Compare road orientation to GPS heading using Expression Evaluator

Video Logging – Solution 1 (Cont’d)

Using a Tester, determine if the difference between the road and heading azimuths are acceptable

Depending on the results from the Tester, assign the direction of travel for the point

Video Logging – Solution 1 (Overview)

Video Logging – Challenge 2

More than one pass on the same road

Blue and black points are going the same direction

Video Logging – Solution 2

Use StatisticsCalculator to determine for each video clip/road combination: Smallest linear reference value Largest linear reference value Total number of points

Video Logging – Solution 2 (Cont’d)

Use ExpressionEvaluator to determine the coverage that each clip has per road

Video Logging – Solution 2 (Cont’d)

Use a series of 3 Testers to validate which records should be included in the final dataset:

Test 1: If the point is part of the only video clip on that

road and it covers more than 10% of the road If it passes, include it If it fails, forward it on to Test 2

Video Logging – Solution 2 (Cont’d)

Test 2 If the point is part of a clip that covers more

than 25% of the road and the total coverage on the road is less than 110% This would handle scenarios where more than one clip

is needed to cover a road If it passes, include it If it fails, forward it on to Test 3

Video Logging – Solution 2 (Cont’d)

Test 3 If the point is part of a clip that covers more

than 75% of the road If it reaches this test, then it is likely a road that has

more than one pass To determine which of the passes gets included we

include additional variables Largest amount of coverage Most images Most recent date

If it fails, forward it on to the Unused feature

Video Logging – Solution 2 (Overview)

Video Logging – Challenge 3

Around 5,000,000 images (about 1.4 TB) requires intelligent file management

Developed a file structure that takes into account: Year of image Road name Alberta Township Survey township identifier Which camera (front, side, rear)

Video Logging – Solution 3

Points that are to be included in the final dataset are also forwarded to the Create Batch File process

Using a series of Testers, Concatenators and StringReplacers the Create Batch File: Create batch files that make the necessary

directory structure Creates batch files that move and rename the

image files Pushes the new image name and path back

into the final dataset

Video Logging – Solution 3 (Overview)

Video Logging – Final Translation

Video Logging – Finished Product

Video Logging – Benefits

Predictable result Reproducible result 36 person hours to create translation 1 person hour to run and validate 170 steps completed with 1 mouse click

ERCB Data

ERCB – Energy Resources Conservation Board Maintains Oil & Gas data for Alberta

Wells Pipelines Facilities

ERCB Data – Ingredients…

2 shape files 9 text files Cryptic field names Uses a lot of codes and abbreviations

Eg: Pipeline material type = “G”

ERCB Data – Out of the Oven…

4 feature classes Meaningful field names User-friendly data structure Replace codes and abbreviations with

“English” descriptions Eg: Pipeline material type = “Composite”

ERCB Data

Transformers used (118 in total): Joiner StringConcatenator SubstringExtractor FeatureMerger AttributeValueMapper PointConnector AttributeRenamer Tester

ERCB Data - Overview

ERCB Data - Benefits

Create user-friendly dataset Predictable result Reproducible result 118 steps completed with 1 mouse click

ERCB Data – Finished Product

Thank You!

Questions?

For more information: Kurt Hartman [email protected] Accurate Assessment Group Ltd. www.aag-gis.com