DDI 101 DDI 101 Presented to the Presented to the: Ontario DLI training session Ontario DLI training session Queens Queens Kingston, Ontario Kingston, Ontario February 11, 2004 February 11, 2004 Carol Perry Carol Perry And And Ernie Boyko Ernie Boyko April 2004
23
Embed
DDI 101 Presented to the : Ontario DLI training session Queens Kingston, Ontario Presented to the : Ontario DLI training session Queens Kingston, Ontario.
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
DDI 101DDI 101DDI 101DDI 101
Presented to thePresented to the::
Ontario DLI training sessionOntario DLI training sessionQueensQueens
Kingston, OntarioKingston, Ontario
Presented to thePresented to the::
Ontario DLI training sessionOntario DLI training sessionQueensQueens
Kingston, OntarioKingston, Ontario
February 11, 2004February 11, 2004
Carol PerryCarol PerryAndAnd
Ernie BoykoErnie BoykoApril 2004
OutlineOutlineOutlineOutline
What is this all about?What is this all about? Why is it important?Why is it important? Metadata and XML Metadata and XML What is DDI?What is DDI? <<ddiddi> : A Metadata Framework> : A Metadata Framework STC’ Plans for DDISTC’ Plans for DDI Where to from here?Where to from here?
What is this all about?What is this all about? Why is it important?Why is it important? Metadata and XML Metadata and XML What is DDI?What is DDI? <<ddiddi> : A Metadata Framework> : A Metadata Framework STC’ Plans for DDISTC’ Plans for DDI Where to from here?Where to from here?
What is this all What is this all about?about?
What is this all What is this all about?about?
Data Documentation Initiative (DDI)Data Documentation Initiative (DDI) Another flavour of information Another flavour of information
managementmanagement Not unlike cataloguing informationNot unlike cataloguing information Think AACR2/MARC or Dublin CoreThink AACR2/MARC or Dublin Core But taking into the needs of dataBut taking into the needs of data And taking advantage of new And taking advantage of new
technologytechnology
Data Documentation Initiative (DDI)Data Documentation Initiative (DDI) Another flavour of information Another flavour of information
managementmanagement Not unlike cataloguing informationNot unlike cataloguing information Think AACR2/MARC or Dublin CoreThink AACR2/MARC or Dublin Core But taking into the needs of dataBut taking into the needs of data And taking advantage of new And taking advantage of new
Converting data to Converting data to knowledgeknowledge
True data Liberation!True data Liberation!
Converting data to Converting data to knowledgeknowledge
True data Liberation!True data Liberation!
100110100101110110011001
Data
Brainware+
Knowledge
=
Software
Metadata and XMLMetadata and XMLMetadata and XMLMetadata and XML
A markup language for documents A markup language for documents containing structured informationcontaining structured information
Provides a facility to define tags and Provides a facility to define tags and the structural relationships between the structural relationships between themthem
Created so that richly structured Created so that richly structured documents could be used over the Webdocuments could be used over the Web
Has become the de-facto exchange Has become the de-facto exchange format on the Webformat on the Web
Provides the syntax to describe a Provides the syntax to describe a metadata framework, like metadata framework, like <ddi><ddi>
A markup language for documents A markup language for documents containing structured informationcontaining structured information
Provides a facility to define tags and Provides a facility to define tags and the structural relationships between the structural relationships between themthem
Created so that richly structured Created so that richly structured documents could be used over the Webdocuments could be used over the Web
Has become the de-facto exchange Has become the de-facto exchange format on the Webformat on the Web
Provides the syntax to describe a Provides the syntax to describe a metadata framework, like metadata framework, like <ddi><ddi>
What is <ddi>? What is <ddi>? What is <ddi>? What is <ddi>?
The Data Documentation Initiative The Data Documentation Initiative ((<ddi>) <ddi>) is an international effort to is an international effort to establish a standard for technical establish a standard for technical documentation describing social documentation describing social science datascience data
It is guided by a membership-based It is guided by a membership-based alliance that is developing/evolving the alliance that is developing/evolving the <ddi><ddi> specification which is written in specification which is written in XMLXML
See http://www.icpsr.umich.edu/ddiSee http://www.icpsr.umich.edu/ddi
The Data Documentation Initiative The Data Documentation Initiative ((<ddi>) <ddi>) is an international effort to is an international effort to establish a standard for technical establish a standard for technical documentation describing social documentation describing social science datascience data
It is guided by a membership-based It is guided by a membership-based alliance that is developing/evolving the alliance that is developing/evolving the <ddi><ddi> specification which is written in specification which is written in XMLXML
See http://www.icpsr.umich.edu/ddiSee http://www.icpsr.umich.edu/ddi
What is <ddi>? What is <ddi>? (cont’d)(cont’d)What is <ddi>? What is <ddi>? (cont’d)(cont’d)
An XML structure for a codebook to An XML structure for a codebook to be:be: manipulatedmanipulated viewedviewed searched, andsearched, and employed by stat packagesemployed by stat packages
Involves diverse participants:Involves diverse participants: data producersdata producers archives/data centresarchives/data centres researchers/usersresearchers/users
An XML structure for a codebook to An XML structure for a codebook to be:be: manipulatedmanipulated viewedviewed searched, andsearched, and employed by stat packagesemployed by stat packages
Involves diverse participants:Involves diverse participants: data producersdata producers archives/data centresarchives/data centres researchers/usersresearchers/users
Brief history of <ddi>Brief history of <ddi>Brief history of <ddi>Brief history of <ddi>
Established in 1995 to create a Established in 1995 to create a universally supported metadata standard universally supported metadata standard for the social science communityfor the social science community
Initiated and organised by the Inter-Initiated and organised by the Inter-University Consortium for Political and University Consortium for Political and Social Research (ICPSR), Michigan, USASocial Research (ICPSR), Michigan, USA
Members coming from social science Members coming from social science data archives and libraries in USA, data archives and libraries in USA, Canada and Europe and from major Canada and Europe and from major producers of statistical dataproducers of statistical data
First version of the standard expressed First version of the standard expressed as an SGML-DTDas an SGML-DTD
Established in 1995 to create a Established in 1995 to create a universally supported metadata standard universally supported metadata standard for the social science communityfor the social science community
Initiated and organised by the Inter-Initiated and organised by the Inter-University Consortium for Political and University Consortium for Political and Social Research (ICPSR), Michigan, USASocial Research (ICPSR), Michigan, USA
Members coming from social science Members coming from social science data archives and libraries in USA, data archives and libraries in USA, Canada and Europe and from major Canada and Europe and from major producers of statistical dataproducers of statistical data
First version of the standard expressed First version of the standard expressed as an SGML-DTDas an SGML-DTD
Brief history of <ddi> Brief history of <ddi> (cont’d)(cont’d)
Brief history of <ddi> Brief history of <ddi> (cont’d)(cont’d)
Translated to XML in 1997Translated to XML in 1997 Extensive testing carried out Spring-Extensive testing carried out Spring-
Summer 1999Summer 1999 DDI 1.0 published Spring 2000DDI 1.0 published Spring 2000 DDI 1.1 with minor revisions and DDI 1.1 with minor revisions and
some additions published Autumn some additions published Autumn 20012001
The DDI 2.0 published Summer 2003, The DDI 2.0 published Summer 2003, including aggregate data, geographic including aggregate data, geographic elements, element formattingelements, element formatting
Translated to XML in 1997Translated to XML in 1997 Extensive testing carried out Spring-Extensive testing carried out Spring-
Summer 1999Summer 1999 DDI 1.0 published Spring 2000DDI 1.0 published Spring 2000 DDI 1.1 with minor revisions and DDI 1.1 with minor revisions and
some additions published Autumn some additions published Autumn 20012001
The DDI 2.0 published Summer 2003, The DDI 2.0 published Summer 2003, including aggregate data, geographic including aggregate data, geographic elements, element formattingelements, element formatting
Importance to Data Importance to Data ProducersProducers
Importance to Data Importance to Data ProducersProducers
Provides guidelines for documenting Provides guidelines for documenting researchresearch
Increases usefulness of the collection Increases usefulness of the collection due to standardization, increasing the due to standardization, increasing the potential for greater use by analystspotential for greater use by analysts
Provides consistent field mappings, Provides consistent field mappings, facilitating import into statistical facilitating import into statistical softwaresoftware
Enables reuse of survey componentsEnables reuse of survey components
Provides guidelines for documenting Provides guidelines for documenting researchresearch
Increases usefulness of the collection Increases usefulness of the collection due to standardization, increasing the due to standardization, increasing the potential for greater use by analystspotential for greater use by analysts
Provides consistent field mappings, Provides consistent field mappings, facilitating import into statistical facilitating import into statistical softwaresoftware
Enables reuse of survey componentsEnables reuse of survey components
Importance of the Importance of the DDI:DDI:
To ArchivistsTo Archivists
Importance of the Importance of the DDI:DDI:
To ArchivistsTo Archivists Metadata supplied in complete Metadata supplied in complete
form form Facilitates distribution of data Facilitates distribution of data
collections: codebook already collections: codebook already readily usable, and data definition readily usable, and data definition statements can be generated easilystatements can be generated easily
Facilitates online analysis and Facilitates online analysis and subsettingsubsetting
Archival formatArchival format
Metadata supplied in complete Metadata supplied in complete form form
Facilitates distribution of data Facilitates distribution of data collections: codebook already collections: codebook already readily usable, and data definition readily usable, and data definition statements can be generated easilystatements can be generated easily
Facilitates online analysis and Facilitates online analysis and subsettingsubsetting
Archival formatArchival format
Importance to UsersImportance to UsersImportance to UsersImportance to Users
Improves searching by individual Improves searching by individual field and across collections field and across collections
Makes available well-Makes available well-documented data collections documented data collections more quicklymore quickly
Potentially provides more Potentially provides more information through extensive information through extensive linking featureslinking features
Improves searching by individual Improves searching by individual field and across collections field and across collections
Makes available well-Makes available well-documented data collections documented data collections more quicklymore quickly
Potentially provides more Potentially provides more information through extensive information through extensive linking featureslinking features
Projects Using DDIProjects Using DDIProjects Using DDIProjects Using DDI
NESSTARNESSTAR Health Canada -- DAISHealth Canada -- DAIS SDA, BerkeleySDA, Berkeley University of AlbertaUniversity of Alberta University of GuelphUniversity of Guelph University of TorontoUniversity of Toronto ICPSR’s metadataICPSR’s metadata University of Minnesota University of Minnesota US Census BureauUS Census Bureau Harvard Virtual Data Center??Harvard Virtual Data Center??
NESSTARNESSTAR Health Canada -- DAISHealth Canada -- DAIS SDA, BerkeleySDA, Berkeley University of AlbertaUniversity of Alberta University of GuelphUniversity of Guelph University of TorontoUniversity of Toronto ICPSR’s metadataICPSR’s metadata University of Minnesota University of Minnesota US Census BureauUS Census Bureau Harvard Virtual Data Center??Harvard Virtual Data Center??
What are STC’s Plans for What are STC’s Plans for DDI DDI
What are STC’s Plans for What are STC’s Plans for DDI DDI
STC has purchased some NESSTAR STC has purchased some NESSTAR licenceslicences
Plan to use NESSTAR Publisher to Plan to use NESSTAR Publisher to produce standardized metadata for produce standardized metadata for master and public filesmaster and public files
Use NESSTAR Server to provide access Use NESSTAR Server to provide access across master files to support Statistics across master files to support Statistics Canada analysisCanada analysis
Disseminate Disseminate <ddi><ddi> compliant survey compliant survey files/documentation to RDCs and DLI sitesfiles/documentation to RDCs and DLI sites
STC has purchased some NESSTAR STC has purchased some NESSTAR licenceslicences
Plan to use NESSTAR Publisher to Plan to use NESSTAR Publisher to produce standardized metadata for produce standardized metadata for master and public filesmaster and public files
Use NESSTAR Server to provide access Use NESSTAR Server to provide access across master files to support Statistics across master files to support Statistics Canada analysisCanada analysis
Disseminate Disseminate <ddi><ddi> compliant survey compliant survey files/documentation to RDCs and DLI sitesfiles/documentation to RDCs and DLI sites
Provide controlled access to public Provide controlled access to public use filesuse files
Online tool for facilitating remote Online tool for facilitating remote access using synthetic filesaccess using synthetic files
Introduce students to microdataIntroduce students to microdata Archival tool for master and public Archival tool for master and public
filesfiles Develop a two-way crosswalk other Develop a two-way crosswalk other
data extractors and metadata bases.data extractors and metadata bases.
Provide controlled access to public Provide controlled access to public use filesuse files
Online tool for facilitating remote Online tool for facilitating remote access using synthetic filesaccess using synthetic files
Introduce students to microdataIntroduce students to microdata Archival tool for master and public Archival tool for master and public
filesfiles Develop a two-way crosswalk other Develop a two-way crosswalk other
data extractors and metadata bases.data extractors and metadata bases.
What could be done with What could be done with DDI/NESSTAR? DDI/NESSTAR?
What could be done with What could be done with DDI/NESSTAR? DDI/NESSTAR?
What’s next?What’s next?What’s next?What’s next?
Lets build on the work that is Lets build on the work that is already starting in Canadaalready starting in CanadaBut first, Carol will give you an But first, Carol will give you an overview of some of the ‘how overview of some of the ‘how to’s’to’s’
Lets build on the work that is Lets build on the work that is already starting in Canadaalready starting in CanadaBut first, Carol will give you an But first, Carol will give you an overview of some of the ‘how overview of some of the ‘how to’s’to’s’