Integration and Management of Diverse Environmental Data Sets

Integration and Management of Diverse Environmental

Data Sets

Cameron Kiddle

Research Fellow

Grid Research Centre

University of Calgary

Outline Data Challenges GeoChronos (Spectral Libraries) Spectral Library Demonstration Cloud Services for Water Management Cloud Services for Water Management

Demonstration

Summit 09 Oct. 14, 2009

2

Data Challenges - Acquisition Many different data sources

Different regulations/mechanisms for accessing data

Lack of automation

Finding the right data

Summit 09 Oct. 14, 2009

3

Data Challenges - Management Scattered and unorganized data

Inadequate tools for recording/maintaining metadata Data without metadata is meaningless Lack of suitable metadata standards Validation of metadata

Tracking provenance of data

Summit 09 Oct. 14, 2009

4

Data Challenges – Pre-processing Raw data typically cannot be directly

analyzed

Significant amount of time spent preparing data for analysis

Lack of automation

Summit 09 Oct. 14, 2009

5

GeoChronos Partners

CANARIE (NEP-1) Center for Earth Observation Sciences,

University of Alberta Cybera Grid Research Centre, University of Calgary

Summit 09 Oct. 14, 2009

6

GeoChronos An on-line platform (http://geochronos.org/)

For: Earth Observation Scientists

Facilitating: Collaboration between scientists Application access, management and sharing Data access, management and sharing

Leveraging: Web 2.0 and social networking technologies Cloud computing technologies Semantic Web technologies

Summit 09 Oct. 14, 2009

7

GeoChronos Data Solutions - Spectral Libraries

Store, share and browse spectral data View spectral plots, metadata, ancillary data and maps Manage and generate metadata for spectra Create and share metadata schemas

Technology iRODS (http://www.irods.org/) for data storage/management Semantic Web technologies such as RDF (Resource

Description Framework) to link/relate data Next Steps

Generalization of spectral library solution - acquire, store, manage, browse and share other types of data (i.e., satellite, flux, phenology, meteorological, etc.)

Automate data workflows (i.e., mosaic, reproject and subset MODIS data) using cloud-based services

Summit 09 Oct. 14, 2009

8

Spectral Library - Demonstration

Summit 09 Oct. 14, 2009

9

Spectral Library - Browse

Summit 09 Oct. 14, 2009

10

Spectral Library - Metadata

Summit 09 Oct. 14, 2009

11

Spectral Library - Ancillary Data

Summit 09 Oct. 14, 2009

12

Cloud Services for Water Management

Partners Alberta Advanced Education and Technology Alberta WaterSMART Cybera Geosensor Web Lab, University of Calgary Grid Research Centre, University of Calgary Tesera Systems Inc

Summit 09 Oct. 14, 2009

13

Cloud Services for Water Management In preliminary stages of project Explore use of cloud services to store,

manipulate and expose data related to water management

Link and correlate a wide variety of data from a large number of sources

Cloud-based analysis and visualization tools Investigate integration with existing Alberta

WaterPortal (http://www.albertawater.com/)

Summit 09 Oct. 14, 2009

14

Cloud Services for Water Management - Demonstration

Summit 09 Oct. 14, 2009

15

Data Types and Locations Displayed on Google Earth

Summit 09 Oct. 14, 2009

16

Water Flow Data Shown using Google Visualization API

Summit 09 Oct. 14, 2009

17

Integrating and Visualizing Different Data Sets From Calgary’s Flood in 2005 on Google Earth

Summit 09 Oct. 14, 2009

18

Contact Information

Summit 09 Oct. 14, 2009

Cameron [email protected]://pages.cspc.ucalgary.ca/~kiddlec/

http://grid.ucalgary.ca/

19