Data Quality Plan Pilot Tutorial: EPA Report on the Environment

1

Data Quality Plan Pilot Tutorial: EPA Report on the Environment

Scientific Data Management Workshop Planning GroupBrand Niemann

Senior Enterprise ArchitectU.S. EPA

March 13, 2010http://semanticommunity.net/

2

Overview

• 1. Background• 2. EPA Report on the Environment

– 2.1 EPA’s Web and Electronic Versions– 2.2 My First Semantic Publishing Version (2004)– 2.3 My Second Semantic Publishing Version (2007)– 2.4 My Third Semantic Publishing Version (2010)

• 3. Suggestions• 4. Questions and Answers

3

1. Background

• 1.1 Open Government Directive• 1.2 Data Quality Viewpoints:

– 1.2.1 Federal Data Architecture Subcommittee– 1.2.2 Data Management Association– 1.2.3 Interagency Working Group on Digital Data– 1.2.4 Ken Orr– 1.2.5 Michael Daconta– 1.2.6 Mills Davis

• 1.3 Strategy• 1.4 History

4

1. Background

• 1.1 Open Government Directive, December 8, 2009:– Implement the principles of transparency, participation, and

collaboration set forth in the President’s Memorandum (January 21, 2009):

• 1. Publish Government Information Online:– Publish online in an open format at least three high-value data sets*.– Create an Open Government Webpage*.

• 2. Improve the Quality of Government Information:– Make certain that the information conforms to OMB guidance on

information quality.• 3. Create and Institutionalize a Culture of Open Government:

– Develop and publish on its Open Government Webpage an Open Government Plan*.

» Flagship Initiative: Each agency’s Open Government Plan should describe at least one specific, new transparency, participation, or collaboration initiative*.

• 4. Create an Enabling Policy Framework for Open Government:– Help OMB identify impediments to open government and to the use of

new technologies.

* Done by this pilot as of March 13, 2010.

5

OMB Guidance onHigh-value Information

• Information that can be used to increase agency accountability and responsiveness; improve public knowledge of the agency and its operations; further the core mission of the agency; create economic opportunity; or respond to need and demand as identified through public consultation.

6

OMB Guidance on Information Quality

• The Federal Government has defined quality and objectivity:– Quality is “…the encompassing term, of which

‘utility,’ ‘objectivity,’ and ‘integrity’ are the constituents.

– ‘Objectivity’ focuses on whether the disseminated information is being presented in an accurate, clear, complete, and unbiased manner, and as a matter of substance, is accurate, reliable, and unbiased.

Source: “Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by Federal Agencies” (67 FR 8452).

7

1. Background

• 1.2 Data Quality Viewpoints:– 1.2.1 Federal Data Architecture

Subcommittee:• On Authoritative Data Sources: One Data Element

at a Time, March 11, 2010, Dr. Richard Wang, Chief Data Quality Officer, Office of the U.S. Army CIO/G-6, and Director, MIT Information Quality Program:

– Conventional view: Data Quality = Accuracy– New research finding: Data Quality Goes Beyond

Accuracy.

8

1. Background

• 1.2 Data Quality Viewpoints:– 1.2.1 Federal Data Architecture

Subcommittee:• On Authoritative Data Sources: One Data Element

at a Time (continued):– Four Principles:

» 1.Understand the consumer’s information needs» 2.Manage information as the product of a well-

defined production process» 3.Manage information as a product with a life cycle» 4. Appoint an Information Product Manager to

manage the information product.

9

1. Background

• 1.2 Data Quality Viewpoints:– 1.2.1 Federal Data Architecture Subcommittee:

• On Authoritative Data Sources: One Data Element at a Time (continued):

– Solution:

» 1. Address one data element at a time using priority data elements within priority projects.

» 2. Take a first few data elements through the entire TDQM cycle to educate and illustrate value.

» 3. Establish and populate a catalog of data element quality specifications containing priority data elements for broad use.

10

1. Background

• 1.2 Data Quality Viewpoints:– 1.2.2 Data Management Association:

• One of the Ten Data Management Functions in the DAMA-DMBOK Functional Framework V3 are:

– Data Quality Management:» Planning, implementation and control activities that

apply quality management techniques to measure, assess, improve and ensure the fitness of data for use.

11

1. Background

• 1.2 Data Quality Viewpoints:– 1.2.3 Interagency Working Group on Digital

Data:• Harnessing the Power of Digital Data for Science

and Society, January 2009:– The report lays out a strategic vision for “a digital

scientific data universe in which data creation, collection, documentation, analysis, preservation, and dissemination can be appropriately, reliably, and readily managed, thereby enhancing the return on our nation’s research and development investment by ensuring that digital data realize their full potential as catalysts for progress in our global information society.”

12

1. Background

• 1.2 Data Quality Viewpoints:– 1.2.3 Interagency Working Group on Digital Data:

• Harnessing the Power of Digital Data for Science and Society, January 2009:

– The report includes three key recommendations to pursue this vision. The first is to create an Interagency Subcommittee under NSTC that will focus on goals that are best addressed through continuing broad cooperation and coordination across agencies. The second key element of the strategic framework is for departments and agencies to lay the foundations for agency digital scientific data policy and make the policy publicly available. In laying these foundations, agencies should consider all components of a comprehensive policy to address the full data management life cycle. The third key element is for all agencies to promote a data management planning process for projects that generate scientific data for preservation.

13

1. Background

• 1.2 Data Quality Viewpoints:– 1.2.3 Interagency Working Group on Digital Data:

• Subgroups on Data Management Plans and Agency Science Data Policies

• March 29-31, 2010, NIST, Gaithersburg, Maryland, 1st International Digital Preservation Interoperability Framework Symposium

– Scientific research is generally held to be of good provenance when it is documented in detail sufficient to allow reproducibility. (Source: Wikipedia)

• June 29-July 1, 2010 Workshop on Scientific Data Management, US EPA Headquarters, Federal Triangle, Washington, DC

14

1. Background

• 1.2 Data Quality Viewpoints:– 1.2.4 Ken Orr:

• Data quality can only be maintained in the long run by ensuring continuous and intensive use of both the data and the metadata (data about the data). “Use-based design means focusing on exactly how the data will be used and trying to identify inventive ways to ensure that the data are used more strenuously. In many cases, this means creatively persuading the people most knowledgeable about the data to take responsibility for it.” Taking responsibility for the data is not simply taking the blame when they are wrong, but making the data so central to one’s real job that its quality becomes important for day-to-day work (1998).

15

1. Background• 1.2 Data Quality Viewpoints:

– 1.2.5 Michael Daconta, March 10, 2010, 10 flaws with the data on Data.gov: Recently released high-value datasets reveal 10 types of deficiencies:

• 1. Omission errors.• 2. Formatting errors.• 3. Accuracy errors.• 4. Incorrectly labeled records.• 5. Access errors.• 6. Poorly structured data.• 7. Non-normalized data.• 8. Raw database dumps.• 9. Inflation of counts.• 10. Inconsistent data granularity.

– So what can we do? Here are three basics steps: Attract more citizen involvement to police the data; implement the top ideas on http://datagov.ideascale.com; and ensure agency open-government plans address, in detail, their data quality processes.

16

1. Background• 1.2 Data Quality Viewpoints:

– 1.2.6 Mills Davis, February 9, 2010, Second Open Government Deadline: US Agencies Still Flexing Their Muscles:

• The National Renewable Energy Laboratory (NREL) is developing an Open Energy Information (http://OpenEI.org) initiative. Its goal is to establish a (US and global) community-based “renewable energy data commons” based on linked open data principles.

• Right now they’re working through issues relating to how best to manage data quality across the lifecycle of data assets contributed from multiple sources. Entities aligned with Federal Agencies and Departments have multiple requirements to meet relating to quality of data and information being made public in addition to the principles for data quality principles enunciated as part of Data.gov. What is new is figuring out how to best frame data policies and align practices and procedures to fully take advantage of semantic web technologies.

• I doubt that NREL is the only group that is exploring how best to marry semantic web technologies with appropriate data quality policies, best practices, and procedures. For example, the UK has taken this path. Also, the NASCIO organization is recommending linked open data for States to share public information.

17

1. Background• 1.3 Strategy:

– Bridge across:• The Federal Cloud Computing Communities• The Interagency Working Group on Digital Data• W3C Special Interest Group on eGov (Linked Open Data)• EPA Statistical, OGD and Wiki and Blog Work Groups

– Tie Scientific Data Management (SDM) to the OGD and Data.gov/semantic:

• By implementing my “Put Your Desktop in the Cloud in Support of the OGD and Data.gov/semantic” (see Section 1.4)

– Use EPA’s “best practice” for SDM, the Report on the Environment 2008:

• Complete metadata, peer review, provenance, updating, etc.– Help Subject Matter Experts use Semantic Publishing Technologies:

• Build an ontology (see next slide)• Implement the various Data Quality Viewpoints

– The Third International Provenance and Annotation Workshop, June 15-16, 2010.• Facilitate access and reuse• Try new tools (e.g. Sheetster, Open Data Registry, etc.)

18

1. Background

• 1.3 Strategy:– Ontology, as used here, means categories of interest

in a domain and the relationships among them (note this is independent of syntax and technology) Source: John Sowa, Knowledge Representation, 2000.

• A recent book entitled "Semantic Web for the Working Ontologist: Effective Modeling in RDFS and OWL contains an excellent example of the application of ontology to Enterprise Architecture, specifically, the Federal Enterprise Architecture Reference Model Ontology (FEARMO) (see pages 248-258), as well as many other practical examples.

19

1. Background

• 1.3 Strategy:– That given a data URI, there is no standard way to

programmatically access the metadata about the resource.• Josh Tauberer: The Linked Data 'movement' started, as I recall, in

the use of http:, URIs as the convention for all RDF resources. By using http:, there's implicitly a web page that can be accessed for more information about that resource --- a page that can be in RDF format itself.

– That given a data URI there is no standard programmatic way to access a SPARQL query point for that resource and/or for associated resources.

• Josh Tauberer: See http://sw.deri.org/2007/07/sitemapextension/

– That the metadata accessed should have standard links for provenance – even very simple provenance that does not require research.

• Brand Niemann: See this pilot!

Source: W3C eGov Projects Government Linked Data: Demonstration Discussion, March 12, 2010.

20

1. Background

• 1.3 Strategy:– How do we contextualize a query such that all data

resources of interest within a certain context are included in a query, without the user having to know all the details of the data sets involved?

• No response yet.– All of the above could be accomplished with URI

conventions and supporting ontologies. My question is: What are the existing or proposed conventions and ontologies to satisfy these requirements?  Should the eGov group provide or reference such conventions for use by the government and/or within our government demos?

• Brand Niemann: See this pilot!

Source: W3C eGov Projects Government Linked Data: Demonstration Discussion, March 12, 2010.

21

1. Background• 1.4 History:

– October 28, 2009:• Federal IT Summit Cloud Computing Panel: EPA work on "a cloud

computing desktop" cited as a best practice by Peter Tseronis, Co-Chair, Cloud Computing Advisory Committee.

– December 10, 2009:• Data Architecture Subcommittee Meeting with Donna Roy (NIEM) and Jim

Hendler (Data-gov Semantic Wiki) and discussions of Data.gov CONOPS and its evolution to the Semantic Web.

– December 15, 2009:• Semantic Web Meetup: Party Cloudy with a Chance of Semantics. (Put Your

Desktop in the Cloud to Support the Open Government Directive and Data.gov/semantic!).

– December 17, 2009:• Federal Cloud Computing Advisory Council Meeting (same as above).

– December 24, 2009:• 2010 Office of Environmental Information National Symposium, May 11 – 13,

2010, Proposal for Presentation and Tutorial.

22

1. Background• 1.4 History:

– January 11, 2010:• Open Government Directive Workshop and Government Desktop in the

Cloud Breakout Session at the US Department of Transportation at the Direction of the Office of the EPA Administrator and the National Center for Innovation.

– January 14-15, 2010:• Federal CIO Council Boot Camp Training, Partnership for Public Service

SAGE Program, Washington, DC. My Suggested 2010 Plan for an Open Government Directive Roadmap Using Cloud Computing Desktop / Mobile Apps as a Member of the Federal CIO Council's Best Practice Committee.

– January 19-21, 2010:• EPA Enterprise Architecture Strategic Planning Sessions

– January 21, 2010:• Put Your Statistical Work in the Cloud in Support of the Open Government

Directive and Data.gov/semantic (EPA Statistics Users Group).– January 28, 2010:

• Tutorial for the EAWG: Solution Architecture for 2010.

23

1. Background• 1.4 History:

– February 4, 2010:• Design Suggestions for EPA’s One Wiki in Support of the EPA OGD Work

Group.– March 3, 2010:

• Joint Services / Cloud Computing Session: Semantic Cloud Computing and Linked Open Data Discussion of Three Uses Cases.

– March 3rd, 2010:• For Open Government, Technology Is The Least of Your Problems (Gartner

Blog Comment).– March 4, 2010:

• 2010 Annual Statistical Abstract of the United States: More than 1300 high-value data sets and graphs with standard metadata, and EPA Report on the Environment Indicators: More than 200 high-value data sets and graphs with standard metadata.

– March 6, 2010:• Flagship Initiative Pilot Tutorial: EPA Report on the Environment, EPA Wiki

and Blog Work Group.

24

2. EPA Report on the Environment 2.1 EPA’s Web and Electronic Versions

Home Page Chapters

25

2. EPA Report on the Environment 2.1 EPA’s Web and Electronic Versions

Chapters (expanded) Indicator (Question)

26

2. EPA Report on the Environment 2.1 EPA’s Web and Electronic Versions

Charts and Graphs References

27

2. EPA Report on the Environment 2.1 EPA’s Web and Electronic Versions

Metadata Output to PDF

28

2. EPA Report on the Environment 2.1 EPA’s Web and Electronic Versions

Results (no highlighted terms)Search

29

2. EPA Report on the Environment 2.1 EPA’s Web and Electronic Versions

• Report on the Environment:– http://www.epa.gov/roe/

• Chapters:– http://cfpub.epa.gov/eroe/index.cfm?fuseaction=list.listByQues

• Indicators:– http://cfpub.epa.gov/eroe/index.cfm?fuseaction=detail.viewInd&ch=46&lShowInd=0&subtop=341&lv=

list.listByChapter&r=209838 • Charts and Graphs:

– http://cfpub.epa.gov/eroe/index.cfm?fuseaction=detail.viewMidImg&ch=46&lShowInd=0&subtop=341&lv=list.listByChapter&r=209838

• Download Spreadsheets (xls):– http://oaspub.epa.gov/eims/eims.roereport.getfile?p_download_id=10543– http://oaspub.epa.gov/eims/eims.roereport.getfile?p_download_id=10542

• References:– http://cfpub.epa.gov/eroe/index.cfm?fuseaction=detail.viewReference&ch

=46&lShowInd=0&subtop=341&lv=list.listByChapter&r=209838 • Metadata:

– http://cfpub.epa.gov/eroe/index.cfm?fuseaction=detail.viewMeta&ch=46&lShowInd=0&subtop=341&lv=list.listByChapter&r=209838

• Create PDF:– http://cfpub.epa.gov/eroe/index.cfm?fuseaction=detail.viewPDF&ch=46&lShowInd=0&subtop=341&lv=

list.listByChapter&r=209838 • Search:

– http://nlquery.epa.gov/epasearch/epasearch?typeofsearch=area&querytext=carbon+monoxide+emissions&submit=Go&fld=nceawww1|eroe&areaname=EPA+Report+on+the+Environment&areacontacts=http://cfpub.epa.gov/ncea/cfm/nceacontact.cfm&areasearchurl=&result_template=epafiles_default.xsl&filter=sample4filt.hts

30

EPA Report on the Environment2.2 My First Semantic Publishing Version (2004)

Home Page Table of Contents

31

EPA Report on the Environment2.2 My First Semantic Publishing Version (2004)

• There have been a number of efforts to integrate environmental information and indicators during the past 20 years or so. The author has been involved in at least eight of those efforts and now has integrated examples from all of those into the re-design of the Sustainable Development Indicators Web Site which has enjoyed sustained popularity with search engines over the years.

– Integration of Environmental Information

and Indicators, December 17, 2007.http://www.sdi.gov

32

EPA Report on the Environment2.3 My Second Semantic Publishing Version (2007)

• Structure:– Introduction– What the Data Show– Limitations– Data Sources– References– Downloads– Metadata

Introduction

33

EPA Report on the Environment2.3 My Second Semantic Publishing Version (2007)

What the Data Show Limitations

34

EPA Report on the Environment2.3 My Second Semantic Publishing Version (2007)

Data Sources References

35

EPA Report on the Environment2.3 My Second Semantic Publishing Version (2007)

MetadataDownloads (did not have xls’s)

36

EPA Report on the Environment2.3 My Second Semantic Publishing Version (2007)

Expert Search Results

37

EPA Report on the Environment2.3 My Second Semantic Publishing Version (2007)

• Introduction:– http://web-services.gov/lpBin22/lpext.dll/Folder/Infobase11/1?fn=main-j.htm&f

=templates&2.0 • What the Data Show:

– http://web-services.gov/lpBin22/lpext.dll/Folder/Infobase11/1?fn=main-j.htm&f=templates&2.0

• Limitations:– http://web-services.gov/lpBin22/lpext.dll/Folder/Infobase11/1?fn=main-j.htm&f

=templates&2.0 • Data Sources:

– http://web-services.gov/lpBin22/lpext.dll/Folder/Infobase11/1?fn=main-j.htm&f=templates&2.0

• References:– http://web-services.gov/lpBin22/lpext.dll/Folder/Infobase11/1?fn=main-j.htm&f

=templates&2.0 • Downloads:

– http://web-services.gov/lpBin22/lpext.dll/Folder/Infobase11/1?fn=main-j.htm&f=templates&2.0

• Metadata:– http://web-services.gov/lpBin22/lpext.dll/Folder/Infobase11/1?fn=main-j.htm&f

=templates&2.0 • Search:

– http://web-services.gov/lpBin22/lpext.dll?f=templates&fn=searchform-frames.htm&id=10.1048/ExpertSearch&

38

EPA Report on the Environment2.4 My Third Semantic Publishing Version (2010)

Home Page Chapter

39

EPA Report on the Environment2.4 My Third Semantic Publishing Version (2010)

Indicators Structure

40

EPA Report on the Environment2.4 My Third Semantic Publishing Version (2010)

Introduction What the Data Show

41

EPA Report on the Environment2.4 My Third Semantic Publishing Version (2010)

Limitations Charts and Graphs

42

EPA Report on the Environment2.4 My Third Semantic Publishing Version (2010)

Data Sources (Provenance) References

43

EPA Report on the Environment2.4 My Third Semantic Publishing Version (2010)

Downloads Metadata

44

EPA Report on the Environment2.4 My Third Semantic Publishing Version (2010)

Site Map Search Results

45

EPA Report on the Environment2.4 My Third Semantic Publishing Version (2010)

• Home Page:– http://epaontology.wik.is/

• Chapter:– http://epaontology.wik.is/2_Air

• Indicators:– http://epaontology.wik.is/2_Air/2.2_What_Are_the_Trends_in_Outdoor_Air_Quality_and_Th

eir_Effects_on_Human_Health_and_the_Environment%3f/2.2.2_ROE_Indicators

• Structure:– http://epaontology.wik.is/2_Air/2.2_What_Are_the_Trends_in_Outdoor_Air_Quality_and_Th

eir_Effects_on_Human_Health_and_the_Environment%3f/2.2.2_ROE_Indicators/2.2.2.1_Carbon_Monoxide_Emissions

• Introduction:– http://epaontology.wik.is/2_Air/2.2_What_Are_the_Trends_in_Outdoor_Air_Quality_and_Th

eir_Effects_on_Human_Health_and_the_Environment%3f/2.2.2_ROE_Indicators/2.2.2.1_Carbon_Monoxide_Emissions/2.2.2.1.1_Introduction

• What the Data Show:– http://epaontology.wik.is/2_Air/2.2_What_Are_the_Trends_in_Outdoor_Air_Quality_and_Th

eir_Effects_on_Human_Health_and_the_Environment%3f/2.2.2_ROE_Indicators/2.2.2.1_Carbon_Monoxide_Emissions/2.2.2.1.2_What_the_Data_Show

• Limitations:– http://epaontology.wik.is/2_Air/2.2_What_Are_the_Trends_in_Outdoor_Air_Quality_and_Th

eir_Effects_on_Human_Health_and_the_Environment%3f/2.2.2_ROE_Indicators/2.2.2.1_Carbon_Monoxide_Emissions/2.2.2.1.3_Limitations

46

EPA Report on the Environment2.4 My Third Semantic Publishing Version (2010)

• Charts and Graphs:– http://epaontology.wik.is/2_Air/2.2_What_Are_the_Trends_in_Outdoor_Air_Quality_and_Their_Effects_on_

Human_Health_and_the_Environment%3f/2.2.2_ROE_Indicators/2.2.2.1_Carbon_Monoxide_Emissions/2.2.2.1.4_Charts_and_Graphs

• Data Sources:– http://epaontology.wik.is/2_Air/2.2_What_Are_the_Trends_in_Outdoor_Air_Quality_and_Their_Effects_on_

Human_Health_and_the_Environment%3f/2.2.2_ROE_Indicators/2.2.2.1_Carbon_Monoxide_Emissions/2.2.2.1.5_Data_Sources

• References:– http://epaontology.wik.is/2_Air/2.2_What_Are_the_Trends_in_Outdoor_Air_Quality_and_Their_Effects_on_

Human_Health_and_the_Environment%3f/2.2.2_ROE_Indicators/2.2.2.1_Carbon_Monoxide_Emissions/2.2.2.1.6_References

• Downloads:– http://epaontology.wik.is/2_Air/2.2_What_Are_the_Trends_in_Outdoor_Air_Quality_and_Their_Effects_on_

Human_Health_and_the_Environment%3f/2.2.2_ROE_Indicators/2.2.2.1_Carbon_Monoxide_Emissions/2.2.2.1.7_Downloads

• Metadata:– http://epaontology.wik.is/2_Air/2.2_What_Are_the_Trends_in_Outdoor_Air_Quality_and_Their_Effects_on_

Human_Health_and_the_Environment%3f/2.2.2_ROE_Indicators/2.2.2.1_Carbon_Monoxide_Emissions/2.2.2.1.8_Metadata

• Site Map:– http://epaontology.wik.is/Special:Sitemap

• Search Results:– http://epaontology.wik.is/Special:Search?search=carbon+monoxide+emissions&type=fulltext&go=Find

47

3. Suggestions

• The Scientific Data Management Workshop can:– Tie Scientific Data Management (SDM) to the OGD

and Data.gov/semantic.– Construct a set of well-defined URIs/URLs for their

best content like the third semantic publishing example.

– Use new Semantic Publishing tools to deliver their content to the Linked Open Data Web of the W3C.

– Deal with Data Quality by addressing the viewpoints contained in this tutorial.

– Try to produce the three uses cases: basic, intermediate, and advanced described elsewhere.

48

4. Questions and Answers