Changing the Curation Equation: A Data Lifecycle Approach to Lowering Costs and Increasing Value Jim Myers 1 , Margaret Hedstrom 1 , Beth A Plale 2 , Praveen Kumar 3 , Robert McDonald 4 , Rob Kooper 5 , Luigi Marini 5 , Inna Kouper 4 , Kavitha Chandrasekar 4 [email protected]1 School on Information, University of Michigan, Ann Arbor, MI, United States. 2 School of Informatics and Computing, Indiana University, Bloomington, IN, United States. 3 Civil and Environmental Engineering, University of Illinois, Urbana-Champaign, IL, United States. 4 Data To Insight Center, Indiana University, Bloomington, IN, United States. 5 National Center for Supercomputing Applications, University of Illinois, Urbana-Champaign, IL, United States.
29
Embed
Changing the Curation Equation: A Data Lifecycle Approach to Lowering Costs and Increasing Value
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Changing the Curation Equation: A Data Lifecycle Approach to
Lowering Costs and Increasing Value
Jim Myers1, Margaret Hedstrom1, Beth A Plale2, Praveen Kumar3, Robert McDonald4, Rob Kooper5, Luigi Marini5, Inna Kouper4, Kavitha Chandrasekar4
1 School on Information, University of Michigan, Ann Arbor, MI, United States. 2 School of Informatics and Computing, Indiana University, Bloomington, IN, United States. 3 Civil and Environmental Engineering, University of Illinois, Urbana-Champaign, IL, United States. 4 Data To Insight Center, Indiana University, Bloomington, IN, United States. 5 National Center for Supercomputing Applications, University of Illinois, Urbana-Champaign, IL, United States.
• Quick Project Intro• What is SEAD? (Stop by the SEAD booth!)• Why is SEAD? • How does SEAD work?• Future active and social curation work
SEAD: Sustainable Environment -Actionable Data
• An NSF DataNet project started in October, 2011
• An international resource for sustainability science
• A provider of light-weight Data Services based on novel technical and business approaches:– Supporting the long-tail of research– Enabling active and social curation– Providing integrated lifecycle support for data
http://sead-data.net/
Margaret Hedstrom, PIPraveen Kumar, co-PIJim Myers, co-PIBeth Plale, co-PI
Sustainability Research• Central to solving many of society’s most critical
challenges• An exemplar of modern research
– Local processes aggregating to produce global consequences– Multiple time scales– Coupling of natural and human systems– Interacting systems-of-systems requiring multidisciplinary
understanding • Environmental – Economic - Social
Science
Technology
Economics
Poverty & Justice
Policy
Cooperation
SEAD is:
• Data discovery• Project workspaces• A data-aware community network• Curation and preservation services that link to multiple archives and discovery services
SEAD is:• Secure project spaces where teams can:– Gather reference data– Upload and share new results– Annotate– Relate– Organize– Publish
Project Dashboard
SEAD is:• An active repository that creates data pages with– Previews– Extracted Metadata– Overlays– Tags– Comments– Provenance– Use information– Download/Embed
• A tool for community exploration:– Personal and Project Profiles– Publications and Data Citations– Co-author, co-investigator graphs– Temporal analysis
SEAD is:
SEAD is:
• A way to preprint and publish data:– Branded interface– Discovery metadata– Drill-down• Sub-collections• Data Pages
– Submit for curation and preservation The National Center for Earth Surface Dynamics
~1.6 TB, 450K files (2.2 M objects) representing 10 years of research by multiple teams
SEAD is:• A community platform for reference data:– Research Object management– Inference– Curation– Preservation– ID assignment– Catalog Registration– Discovery– Citation Generation
SEAD’s Virtual Archive allows curators to access, assess, enhance, package, and submit data from SEAD project repositories for long-term storage in SEAD-managed storage or external institutional repositories and cloud data services.
Semantic Content Middleware over Scalable File System and Triple Store
Flickr-style web management of dataGeospatial, social network mash-ups, workflows and services
Curation Services to harvest and package specific data sets
Federation of OAI repositories for long-term preservation
Sensor data
– Apps read what they need and write what they know– Curation snapshots meaningful Research Objects– Multiple ROs can be defined/managed re-using the same underlying ‘living’ content– The larger graph can be ~reassembled w/o the ongoing cost of managing at the item level
Why is SEAD needed for curation?
• The nature of modern research• The nature of the data documentation problem• Artificial limitations derived from historical
practice
Unless these issues are addressed (in addition to sheer scale), data curation will remain too cumbersome and expensive for ubiquitous use…
Data Challenges in Sustainability Research• Many dimensions, many coordinate systems, many scales,
many formats, a long-tail of providers and users, …• Managing this data is a drag on productivity…
The Long Tail in Research• Individuals/small groups where:– Scale of research prohibits traditional CI
development, dedicated IT support, full-time curator…
– shared data but multiple disciplinary views – Projects involve reference data from external
sources– Project Team does not control formats and
vocabularies These are not just “challenges for the “future
Analyzing the curation/preservation problem…
• Data and Metadata are known well during the project
• Producers actually memorize or record metadata already, and then spend precious time transferring that between people and systems
• Data users manually assemble missing data/metadata but don’t often have a way to share that with others
• Repositories struggle to attain the domain understanding needed to go beyond basic bibliographic info– Repositories only use metadata to help
with data discovery and internal curation decisions
Bill Michener – DataONE
Who knows what?When do they know it?Why will they tell you?
Producers Users
Jim Myers - SEAD
Our collective legacy• Data can only be in one place…• Data transfer is costly…• Mistakes are costly…• Only the future needs well-organized data
(questionable assumptions)• Curation only happens at data/project/center end-of-life• Submission events must be formal and complete• Only cross-trained professionals are capable of getting it right• Researchers should see curation only as a public service
What’s different for users?• When you add a file:
– You can get it back, from anywhere– You can see your video, zoom in on images, overlay spatial data
on maps and retrieve them from an OGC service endpoint– You see the metadata hidden in the file– You can add titles, descriptions, locations, tags later, not as
required parts of a long submit form, and• When you do, they are search terms and ways to create custom maps
– You can add good data and bad, and figure out which data to keep later (using provenance to guide you)
– Users of your data can add metadata, comments, and derived datasets that improve quality, adapt the data for new purposes, etc.
What’s different for curators• Curation starts with data and metadata in hand, not as a
search through dusty disks• Curators can embed with project teams• Data comes with