2006 ESRI International Users ConferenceAugust 8, 2006 Spatial Data Infrastructure and Data Preservation in North Carolina Jefferson F. Essic, Robert Farrell,

2006 ESRI International Users Conference August 8, 2006

Spatial Data Infrastructure and Data Preservation in North Carolina

Jefferson F. Essic, Robert Farrell, Steven P. Morris, and James TuttleNorth Carolina State University Libraries

2

Overview

Spatial Data Preservation: Values and ConsiderationsNC Geospatial Data Archiving Project

Approaches to PreservationChallengesWorkflow

NC Spatial Data InfrastructureNC OneMapRegional/Local Partnerships and Data SharingCoordinated Content Transfer

Historic and Geologic Map Preservation Project

3

Today’s geospatial data as tomorrow’s cultural heritage

Future uses of data are difficult to anticipate (as with Sanborn Maps).

4

Temporal Data Supports Decision Making

Land use change analysisReal Estate trend analysisSite selection (past uses?)

Forecasting

Parcel Boundary Changes 2001-2004

North Raleigh, NC

5

Time series – Ortho imageryVicinity of Raleigh-Durham International Airport 1993-2002

6

Geospatial Data: Risks

Producer focus on current dataFuture support of data formats in questionShift to web services- and API-based accessInadequate or nonexistent metadataIncreasing use of spatial databases for data management

Many digital archiving challenges …

7

Geospatial data types: Aerial imagery

85+ NC counties with orthophotos1-5 flights per county30-300 gb per flight

8

Geospatial data types: Vector & Tabular

Economic, infrastructure, and ethnographic data

9

Geospatial data types: Cartographic Project Files

Counterpart to the map is not just the dataset but also models, symbolization, classification, annotation, etc.

10

How would you describe your current geospatial archive?

Last week’s set of nightly tape backups

Several boxes of CD’s and DVD’s

Bob’s hard drive

A collection of files in our “GIS Folder”

A stand-alone spatial database

The data back-end for our internet mapping application

An enterprise GIS

11

NC Geospatial Data Archiving Project (NCGDAP)

Partnership between university library (NCSU) and state agency (NCCGIA), with Library of Congress under the National Digital Information Infrastructure and Preservation Program (NDIIPP)One of 8 initial NDIIPP partnershipsFocus on state and local geospatial content in North Carolina (state demonstration)Tied to NC OneMap initiative, which provides for seamless access to data, metadata, and inventoriesObjective: engage existing state/federal geospatial data infrastructures in preservation

Serve as catalyst for discussion within industry

12

Different Ways to Approach Preservation

Technical solutions: How do we archive acquired content over the long term?

ToolsHardwareSoftware

Cultural/Organizational solutions: How do we make the data more preservable—and more prone to be archived—from point of production?

CollaborationEducationFeedback

13

Technical Approaches

Receive data as is – variety of distribution methodsMigration of some at-risk formatsMetadata remediation, standardization, and synchronizationDistilling complex objects into repository ingest items (not easy)Using DSpace for demonstration purposesIn the development: use METS record as dormant item “brain” within the repository

Some unsustainable activities – for learning experience

14

Cultural/Organizational Approaches

Feedback to metadata outreach programFeedback to coordinating bodies on adherence to content standardsEngage existing spatial data infrastructure in archiving and preservationEngage software vendors and standards communityCross-fertilize with other national archiving efforts

Current use and data sharing requirements – not archiving needs – drive improved preservability of content and improvement of metadata

15

Challenge: Vector Data Formats

No widely-supported, open vector formats for geospatial dataSpatial Data Transfer Standard (SDTS) not widely supportedGeography Markup Language (GML) – diversity of application schemas and profiles threatens permanent access

Spatial DatabasesThe sum is more than the whole of the parts, and the sum is very difficult to preserveCan export individual data layers for curationSome thinking of using the spatial database as the primary archival platform

16

Challenge: Geospatial Web Services

• How to capture records from decision- making processes?• Possible: Atlas collections from automated image capture• Web 2.0 impact: Emerging tiling and caching schemes (archive target?)

17

General Workflow

1) Receive Data from Agency

2) Copy data from agency source to NCSU workstation

3) Create Dspace collection “space” for the data

4) Create administrative metadata

5) Process geospatial metadata

6) Scan geospatial formats and migrate to archival format

7) Ingest original and archival data objects, and geospatial

administrative metadata to Dspace

18

NCGDAP Leveraging Existing SDI

NC OneMap: "Historic and temporal data will be maintained and available,” RAMONA Metadata outreach and content standardsRegional PartnershipsWGRT and other Coordination EffortsData Sharing AgreementsFrequent communication and discussion among geospatial data community

19

NC Spatial Data Infrastructure: NCOneMap

NC OneMap is a next generation mechanism to coordinate and disseminate geographic information in North Carolina and interact with the NSDI.

Objectives:

• Build a common understanding of North Carolina data resources

• Enable widespread access and distribution of geospatial data

20

NC OneMap Viewer

21

NC OneMap

Objectives (cont.):

• Develop ongoing data inventory for all geospatial data holdings RAMONA – http://nc.gisinventory.net

• Develop content standards for key data themesNC Geographic InformationCoordinating Council (GICC)

One of the defined characteristics of NC OneMap is that “Historic and temporal data will be maintained and available”.

22

Emerging Regional Partnerships

Focused on development of shared infrastructure for cultivating access to data

Becoming test beds for innovation in the area of data sharing and data management, including archiving

23

Local Govt. Data Sharing

Becoming more open, fewer agreements to sign

Recent survey: over 20 state and federal agencies use

local data

Problem of local governments being swamped by

requests

Many requests are more compelling than “archiving”

Content transfer is non-trivial – large dataset sizes,

small rural staffs, technical limitations

24

Coordinated Content Transfer

Will allow one data snapshot to be accessible by multiple agenciesQuestion: Capture frequency of data snapshot?

Survey in-the-works to identify local government best practices, consumer agencies needs

Working Group for Roads and Transportation (WGRT)Stakeholder group working to build data depository for statewide local road dataFirst serious effort to develop a plan for local-to-state data sharing on a regular basis

25

Preservation of Digital Geologic and Historic Maps

Georeferenced over 450 maps scanned by NC Geologic SurveyMaps are available for download at http://wfs.enr.state.nc.us/NCGeologicMaps

15-min topo maps

1:500,000 – 1:2.5 M1:31,680 – 1:430,000

1,200 – 24,000

26

Questions?

Jeff Essic Steve MorrisData Services Librarian Head, Digital Library InitiativesNCSU Libraries NCSU LibrariesPh: (919) 515-5698 Ph: (919) [email protected] [email protected]

http://www.lib.ncsu.edu/gis http://www.lib.ncsu.edu/ncgdap