2013 Geospatial Data and Project Management Track, Building Better Data: The Foundation of the Future by Ben Metcalfe

Building Better Data:The Foundation for the Future

Thursday, October 10, 13

About me:

Studied GIS and Computer Science (M.Sci.)

9 years of GIS experience

Application Development, Programming, Cartography


Field Data Collection

Data Model development

Utilities, Forestry, Agriculture


Collect around 200,000 point locations per year.

Average around 6 million pieces of data to populate, process, and do Quality Control.

What is Data?

Why or How is it useful?

Why do we need accurate data?

What is “accurate data”?

How do we get it?


Topics that I will cover

What is Data?

Any piece of information

Qualitative or Quantitative

In GIS it can be Raster of Vector

Large collections form a database

Huge collections of databases becomes “BIG DATA”


Data can be just about anything

Text, numbers, symbols.

Big Data?Large and complex datasets requiring special tools to manage and process.

Working in to every corner of our world.

Finance

Government

Environmental Management

Utilities


Big data is here. If you havent learned how to handle it, it will bury you.

Its not what it is but how you use it...


Data is the foundation of the pyramid.

Every inventory and every piece of data we have collected, and there has been a lot, is part of a foundation that we construct.

This foundation is built upon extremely accurate data so that as your needs evolve you know that what you have built will work because the foundation is rock solid. We know the value of having accurate information; that it can save enormous amounts of energy, time, and money.

Google Maps


Story about the development of Google Maps

What was before google maps?

When is a map more than a map?

Google Map started in 2005.

The “Hello World” project was simple a pannable zoomable map.


Added Directions and road networks

Integrated satellite imagery in to a hybrid view

Added geocoding functionality

Deeper and more immersive content


Google kept working to update and improve upon their maps.

In 2008, started crowdsourcing business information.

Refine searches based on user ratings.

2009, Street View was launched.

Map Maker allowed user changes to be implemented in near real-time.

Made Google Maps the ubiquitous data source.


2008 - Google began using user inputs to update their maps. Leveraging the drive of other map users to have accurate data.

Street View now lets you tour inside some buildings.


You can tour CERN, the Burj Khalifa tower in Dubai, The Grand Canyon, Everest Base Camp, and the Kennedy Space Center... although now all the door are locked and the lights are off.

You can even go to the Moon!


“The never-ending quest for the perfect map.”

Brian McClendon, VP of Engineering, Google Maps


Efficiency.

Cost Savings.

Data driven world.

Why do we need accurate data?


Utility Industry Example

Electrical Networks are becoming increasingly complex and automated.

Outage Management Systems rely on accurate base data to localize power disruptions.

Bad data means the lights stay off longer.


Good data:

Reduces operating costs.

Speeds repairs.

Better maintenance.

Smarter Capital Investments


Having an accurate dataset is crucial to avoid compounding problems.

With more and more data available, the base data becomes increasingly important.

Can you make informed decisions with spotty data?


So What is Accurate Data?


Data accuracy is often difficult to quantify.

Some acceptable error rate is usually established.

For some applications, 90% accurate will work

For others accuracy must be 99.5%+


Take a large data set of 1 million values.

95% accuracy

You have 50,000 errors in your data.

The more data you have the more it becomes an issue.


Is 100% Possible?

Cost?

Time?

How do we make the most accurate data we can?


We have worked with numerous data sets that take more time and money to FIX that it would cost to just recollect the data.

They use datasets with accuracy closer to 75%. Make multi-million dollar capital expenditures on this data.

They attempt to run advanced analysis tools that dont work because the data is so bad.

Band-Aid on a bullet wound.

How do we get accurate Data?


Where does the GIS data we have come from?


Converted from Paper to CAD to GIS

You inherit their accuracy

You are limited by their completeness

Legacy Data is from digitized paper maps


Remote SensingLidar

Aerial Imagery

Multispectral/Hypersepctral

Limits? Cost?

Does it address all the needs.


Lidar can provide incredible data

Imagery can be as detailed as a few millimeters.

Data Capture Devices

GPS- Record Accuracy

Photos

Quality Control

Field Data Collection


Multitude of GPS units, Data input devices.

Photos in your system can be used to create your own google street view of an area or interest.

Quality control of data collection is key. Its long, tedious, repetitive but has to be nearly perfect.

Data Capture DevicesSpatial data created with GPS receivers

Spatial Accuracy?

Signal Strength

Multipathing

Data entry accuracy

Speed and ease of use


Concerns about varying levels of location accuracy.

Data entry set up as pick lists as often as possible to avoid typing errors

Fields are limited to a specific data types (string, numbers, etc..)

Field ImageryAdding imagery

Geotagged with GPS position

Linked and populated with object IDs

Integrate with Street View or 3D


Quality ControlVerify field data collection completeness.

Topology rules do automated error checking.

Field check the data in a second pass through the field.


Topology is where the work gets done.

Check phasing, linkages

Verify all the fields are properly filled out

Can flag spatial errors

Thank you!Ben Metcalfe

Global Mapping Solutions

email: [email protected]

phone: (541) 913-5116


mailto:[email protected]

mailto:[email protected]

2013 Geospatial Data and Project Management Track, Building Better Data: The Foundation of the Future by Ben Metcalfe

Technology