Health Information Technology and Open Government Policy

Case Study: An examination of Open Government initiatives at the Office of the National Coordinator (ONC) for Health IT

“Our goal is to assist with monitoring health IT adoption trends; facilitate assessment [of] progress to national and HITECH goals; enable stakeholders with data that is insightful and actionable.”-ONC Office of Economic Analysis, Modeling and Evaluation

Report SummaryMethods This is a descriptive (Yin, 2003) instrumental (Stake, 1995) case study that

sought to answer the basic question, How is ONC complying with the Obama administration’s Open Government initiatives? Goals associated with the study include:

To understand how various activities and programs align with the administration’s goals of promoting more transparent, participatory and collaborative government

To identify definitional themes of “Open Government” as illustrated by stakeholders inside and outside of ONC

To explore how data are being used “downstream” to:o Inform policy decisions;o Enhance research; and / or o Promote innovation

To propose a conceptual framework through which ONC and other government agencies could assess current and future Open Government initiatives

Projects A review of applicable projects was undertaken to examine how ONC projects were aiding Department of Health & Human Services-wide Open Government compliance / goals. ONC programs and projects include:

Transparency / Collaboration / Participationo Health IT Dashboard

Regional Extension Center Dashboardo Raw Dashboard Datasetso Data Briefs / Reports

Collaborationo Investing in Innovation (i2) Initiative

Software Development Challenges Participation

o Standards & Interoperability (S&I) Frameworko Federal Advisory Committeeso Planning Room

Data & Analysis

A series of in-person and telephone discussions with program administrators and staff informed the content of this study, including:

1

Director, Office of Economic Analysis, Modeling & Evaluation, ONC Staff, Office of Economic Analysis, Modeling & Evaluation, ONC Director, Regional Extension Center (REC) Programs, ONC Staff, Regional Extension Center (REC) Programs, ONC Program Analyst, i2 Initiative, ONC Director, Office of Science & Technology, ONC

Key Takeaways

ONC has a host of programs and projects that meet Open Government criteria established by the Obama administration – some expressly, others implicitly

o ONC should perform an internal review of all programs to better understand their utility vis-à-vis the administration’s Open Government initiatives

Programs and projects that display all three attributes of Open Government are rare, but a balanced portfolio negates the need to meet all criteria within a single policy or program

o ONC should consider the Open Government Assessment Tool to assess current initiatives and identify future opportunities to integrate Open Government elements into policies and programs

Identifying return on investment (ROI) for Open Government projects is a difficult task, though this does not obfuscate the need

o ONC should develop ROI criteria for Open Government initiatives and prioritize future projects, especially around challenge / prize competitions

Origins of Open Government

The open government or open data movement can most readily be traced back to 1966

and passage of the Freedom of Information Act (FOIA). The Freedom of Information Act

(FOIA) provides that any person has a right, enforceable in court, to obtain access to federal

agency records, except to the extent that such records (or portions of them) are protected from

public disclosure by one of nine exemptions or by one of three special law enforcement record

exclusions.1 Although FOIA has been amended several times during the last 47 years, the core

premise of the rights granted under the Act has remained: to provide citizens access to executive

branch information and to make public and accessible all bureaucratic and technical procedures

for applying for documents from a federal agency.

2

Modern enhancements of FOIA have generally included amendments to make

information available electronically and made federal agencies more accountable for their

responses to FOIA requests.2 Specifically, the Openness Promotes Effectiveness in Our National

Government (OPEN Government) Act of 2007 established an Office of Government Information

Services (OGIS) in the National Archives and Records Administration to review agency

compliance with FOIA, codified annual reporting requirements for each agency's FOIA program

and specifically addresses data sources used to generate those reports by including a directive

that agencies “shall make the raw statistical data used in its reports available electronically...”3

Despite the legal mechanisms established by FOIA and enhancements made to modernize and

reinforce its foundational precepts, it is – at its core – a reactionary mechanism. FOIA gives

citizens the right to ask, and requires the government to respond, but it does not require

government to proactively disseminate data and information that may have value to the public.

With passage of the E-Government Act of 2002, the administration of George W. Bush edged

federal agencies towards a more proactively open government.4 The main goal of this legislation

was “To enhance the management and promotion of electronic Government services and

processes… and by establishing a broad framework of measures that require using Internet-based

information technology to enhance citizen access to Government information and services.”5

This charge was in recognition of the ways modern communications could be leveraged by

citizens and government to be more efficient, transparent and collaborative. While not

recognized at the time, perhaps due to the lexicon of 2002, this was the nation’s first, modern

open data legislation. The E-Gov Act was not a performance management bill, like 1993’s

Government Performance Results Act (GPRA) nor was it focused on improving FOIA, as the

OPEN Government Act did. It was a hybrid that put in place agency and government-wide

3

policies that would enable improved performance, greater access to information and improve

citizen participation with government.

Of the several important provisions in the E-Gov Act, were three that set firmer

foundations for open government: Purpose (2) “To promote use of the Internet and other

information technologies to provide increased opportunities for citizen participation in

Government,” Purpose (8) “To promote access to high quality Government information and

services across multiple channels,” and Purpose (11) “To provide enhanced access to

Government information and services…”6 In addition to these two provisions, the E-Gov Act

helped propel federal agency technology initiatives that are central to 21st century

communications by mandating the creation of the nation’s first Federal Chief Information

Officer, the creation of a CIO Council to coordinate agency technology efforts, and provided

funds for coordination of such government-wide technology initiatives.7 While issues of

government-wide coordination of information technology procurement and programs was an

important facet of the E-Gov Act, the provisions highlighted above make a strong case that

notions of information access and citizen participation were important goals as well.

Over the next six years, agencies progressed towards meeting the goals outlined in the E-

Gov Act to varying degrees. Officials who worked to implement the bill recounted both surprise

and disappointment in how federal agencies were able to comply with the E-Gov Act, but most

agreed that the legislation moved the federal government firmly into new territory – one that was

more “citizen centric.”8 Little more than a month after this assessment of the E-Gov Act was

levied by officials from the Bush administration, newly-elected President Barak Obama signed

an executive order that would seek to magnify the notion of citizen-centric government many

times over.

4

Obama’s Version of E-Government: Open Government

The Obama administration has supplemented the E-Gov Act in two ways – it has sought

to continue the federal governments push into web-based information / service delivery and it

has sought to utilize technology as means to make government more transparent, more

participatory and more collaborative. On January 21, 2009 – Mr. Obama’s first day in office –

he signed the Memorandum on Transparency and Open Government. This memo outlined

President Obama’s intention to use openness to “strengthen our democracy and promote

efficiency and effectiveness in Government.”9 In order to achieve these goals, the administration

emphasized a three-legged framework that included the pillars of Transparency, Participation

and Collaboration.10 Notions of transparency centered on the need for federal agencies “to

disclose information rapidly in forms that the public can readily find and use,” using modern

technologies to publish information online. “Knowledge is widely dispersed in society, and

public officials benefit from having access to that dispersed knowledge,” the memo said; thus,

the theme of participatory government was identified, instructing agencies to “offer Americans

increased opportunities to participate in policymaking.” Finally, the President’s memo directed

federal agencies to engage in more multi-sector collaboration, “to use innovative tools, methods,

and systems to cooperate among,” various stakeholders. All three of these mechanisms were

accompanied by a requirement that public input be sought to help agencies accomplish these

aims, and a preliminary timeline was established to get agencies moving.11 Included in this

memo’s timeline was a requirement that the Director of the Office for Management and Budget

5

issue a directive instructing “executive departments and agencies to take specific actions

implementing the principles set forth in this memorandum.”12

Roughly 330 days later, on December 8, 2009, OMB Director Peter Orszag issued the Open

Government Directive.13 The Open Government Directive established deadlines for action,

which included four areas: (1) Publish government information online; (2) Improve the quality of

government information; (3) Create and institutionalize a culture of open government; and (4)

Create an enabling policy framework for open government. Under each of these areas were

specific ways that agencies should accomplish the aims.

For instance, the Directive set forth a mantra that agencies adopt a “presumption of

openness,” publishing information in a timely and in an “open format” in the section on

publishing government information online.14 The Directive also introduced the concept of “high-

value data sets” in this section and compelled each agency to utilize government-wide open data

clearing houses, such as Data.gov, Recovery.gov and USASpending.gov.15 “High-value

information,” the Directive says in attachment section 3.a.i is, “information that can be used to

increase agency accountability and responsiveness; improve public knowledge of the agency and

its operations; further the core mission of the agency; create economic opportunity; or respond to

need and demand as identified through public consultation.” And this information must not

currently be available on agency websites.16 Most of the instruction in the first area was meant to

be on-going initiatives that agencies undertake; but the high-value data sets needed to be

identified within 45 days and registered via Data.gov.

The primary concern of area two, improve the quality of government information, was on

the “quality and objectivity” of agency spending information. According to the Directive, the

6

federal government defines quality and objectivity as follows: Quality is “…the encompassing

term, of which ‘utility,’ ‘objectivity,’ and ‘integrity’ are the constituents,” and objectivity

“focuses on whether the disseminated information is being presented in an accurate, clear,

complete, and unbiased manner, and as a matter of substance, is accurate, reliable, and

unbiased.”17,18 This section gave agencies 45 days to identify and appoint a senior official to

oversee the quality and objectivity of agency spending data; 90 days for OMB to develop offer

guidance on a framework for the quality of federal spending information for public

dissemination; and 120 days for OMB to devise a longer-term comprehensive strategy for federal

spending transparency.

The third and forth focus areas of the Directive outline how OMB wants agencies to

make open government an enduring part of their mission and operations. Area 3 seeks to

institutionalize a culture of openness by requiring each agency to develop and publish an Open

Government Plan that describes the agency’s intentions to integrate transparency, public

participation and collaboration into its activities. This area also directs the federal Chief

Information Officer and federal Chief Technology Officer to create an Open Government

Dashboard; establish a working group that focuses on transparency, accountability, participation,

and collaboration within the federal government; and develop a framework for how agencies can

use challenges, prizes, and other incentive-backed strategies to find innovative or cost-effective

solutions to improving open government.19 These mandates were accompanied by 60-, 45-, 90-

and 120-deadlines, respectively.

The final section of the Directive outlines the administration’s acknowledgement that

emerging technologies and strategies to utilize those technologies will need an enabling

framework. So in section 4.a. the Directive says that within 120 days the Administrator of the

7

Office of Information and Regulatory Affairs (OIRA), in coordination with the federal CIO and

CTO will review existing OMB policies to “identify impediments to open government and to the

use of new technologies…”20.

Implementation & evolution of the Open Government Initiative

In the months after unveiling the Open Government Directive, the Obama administration

received praise21, followed by tones of disillusionment. Watchdog organizations22, government

technology trade publications and academics bemoaned the missed deadlines, laggard agencies,

and misapplied technologies23.

The Directive contained numerous deadlines (the Directive itself was a deadline overdue

by more than 200 days) outlining various steps agencies needed to take. These deadlines gave

watchdog groups a measuring stick with which to evaluate agency efforts. The Sunlight

Foundation in 2011 said that results were “decidedly mixed.” Introducing work based on a

comprehensive evaluation by OpentheGovernment.org, Sunlight Foundation Policy Director

John Wonderlich wrote, “In some cases, agencies' goals were clearly met.  Many of the datasets

planned to be released are now available on data.gov, and the projects and tools that agencies

described are underway. Often, however, agencies have failed to live up to the standards that

they set for themselves as a result of the Open Government Directive.”24 Sunlight focused on

each deadline stated by the Directive and subsequent deadlines set by individual agencies. Their

research indicated that roughly 36% of deadlines were met.1 Other prominent groups in the

“civil society” space, such as OpentheGovernment.org, followed a similar path of excitement,

turned disappointment.25

1 Author’s calculations using Sunlight / OpenCongress Wiki http://www.opencongress.org/wiki/Open_Government_Directive_Implementation_Tracking

8

Perhaps responding to these and other criticisms, the Obama administration in September

2011 released a Status Report, outlining five areas of activity that were making the federal

government more transparent, participatory and collaborative. The Report focused on progress

made in the following categories: Freedom of Information, specifically FOIA; Open

Government Directive and Agency Plans; Data.gov and Data-Driven Innovation; Spending

Transparency and Sensitive Government Information.26 And for the first time, it referred

collectively to the administration’s activities, OMB guidance and agency plans as the Open

Government Initiative.27

The administration said that demonstrable progress had been made in these areas and

several examples from across the federal government were highlighted. Several important trends

were mentioned that signaled an evolution in how Open Government was being viewed by the

administration. And a new type of vernacular was being embraced to explain that evolution.

FOIA requests became “proactive” and federal datasets were called “national assets” by the

Status Report; the word “platform” was used eight times to describe federal websites and

initiatives, such as Data.gov or Challenge.gov; and the term “prize competition” received special

attention as a way agencies could encourage public use of their data to address specific needs,

under the America COMPETES Reauthorization Act of 2010.28 These new descriptions were

more than simple rebranding of old government terms or processes; they represented a new

orientation towards federal technology usage, raw data dissemination and procurement process –

especially done at such a scale.

For the first time, modern technologies, simple and user-friendly, were being used to (1)

affirmatively disclosed information that is traditionally sought under the FOIA; (2) make new

information available to the public proactively in anticipation of many FOIA requests prompted

9

by some external event; and (3) make information available proactively simply because open

government entails a commitment to disclosure quite apart from what the FOIA requires and

because federal data can be an untapped “national asset” if looked at through lenses from outside

government. According to the Status Report, federal agencies grew the stockpile of data sets on

Data.gov from 272,000 to 389,000 from May 2010 to September 2011. And the “platform” of

Data.gov had received 200 million hits with over 2 million downloads of federal data in the first

two years of its existence.29 The Report’s accounting also noted that, “In its first year,

Challenge.gov featured more than 100 challenges from more than 25 agencies across the

executive branch.”30

Another document added to this dynamic understanding of Open Government: The Open

Government Partnership and the National Action Plan for the United States of America.31 The

National Action Plan, released the same month as the Status Report, outlined a number of

initiatives and priorities for the administration’s Open Government agenda moving forward. The

“National Plan builds on, but does not replace, the Open Government Initiative inaugurated by

the President’s Memorandum on Transparency and Open Government,” the report said.32 The

Plan committed to use open government towards the accomplishment of three broad goals: to

increase public integrity; to manage public resources more effectively; and to improve public

services. In all twenty-four new activities and initiatives were announced in support of achieving

those goals.

The commitment to use open government to increase public integrity entailed activities

that would further the three familiar aims of more citizen participation, increased levels of

transparency and further collaboration among government entities.33 Launching a petition

platform, called “We the People,”2 continuing to improve FOIA administration and monitoring

2 https://wwws.whitehouse.gov/petitions

10

agency implementation of the plans are some of the indicative activities mentioned in support of

the public integrity goal.

The commitment to use open government to manage public resources more effectively

focused on ways to improve spending data transparency and government performance

improvement. Some of the hallmark initiatives outlined in this section deal with expanding

lessons learned from tracking Recovery Act spending to all parts of the federal government and

making improvements to Performance.gov. 34 One notable aspect of this section is its mention of

improving government performance and accountability through open government. Specifically,

the National Action Plan said it that “Performance.gov provides a window on the

Administration’s approach to improving performance and accountability,” and that the

administration would “continue to improve the website, including adding data on other

governmentwide management initiatives…to meet the requirements of the recently enacted

Government Performance and Results Modernization Act, which requires regular progress

updates on the top agency-specific performance goals.” Although the President’s Memo focused

heavily on the need to use open government as a way to “and promote efficiency and

effectiveness in Government,”35 this marries – in part – the administration’s open government

agenda with its performance agenda.

The commitment to use open government to improve public services included promises

to seek more public participation in rulemaking; link more experts with government officials and

update policies that govern federal website management, look and feel, and structure. The

National Action Plan pledged to update Regulations.gov, further develop Data.gov and its “data

communities” and enforce Executive Order 13571, “Streaming Service Delivery and Improving

Customer Service,” to continue ongoing reform of government websites.36

11

One final component of President Obama’s vision of Open Government can be gleaned

from a memorandum entitled, “Building a 21st Century Digital Government,” and the

administration’s implementation roadmap for the memo, “Digital Government: Building a 21st

Century Platform to Better Serve the American People.” The memo said “The innovative use of

technology is fundamentally transforming how the American people do business and live their

daily lives,” and called on agencies to do more to harness increases in computing power, the rise

of high-speed networks and the growing mobile revolution to serve American citizens better.37

Released on the same day as the memo, the federal Chief Information Officer and the federal

Chief Technology Officer unveiled the Digital Government Strategy, which gave agencies a 12-

month roadmap that focuses on several priority areas.38

The Digital Government Strategy is meant to accomplish three aims that:

Gives American citizens access to high-quality digital government information

“anywhere, anytime and on any device”

Modernizes procurement and device management in secure and affordable ways

Unlocks the power of government data to spur innovation.

The Strategy outlines nearly thirty milestones, covering ten different categories of

activity – all of which are to be completed within a year. Of particular import for the concepts

related to open government were objectives listed under Part A of the Strategy’s Principles:

using and “information-centric” approach. According to the Strategy, this entails moving the

government from “managing ‘documents’ to managing discrete pieces of open data and content,

which can be tagged, shared, secured, mashed up and presented in the way that is most useful for

12

the consumer of that information.” The Strategy operationalizes this vision in the following

way:

This policy will leverage central coordination and leadership to develop guidelines, standards, and

best practices for improved interoperability. To establish a “new default,” the policy will require

that newly developed IT systems are architected for openness and expose high-value data and

content as web APIs at a discrete and digestible level of granularity with metadata tags. Under a

presumption of openness, agencies must evaluate the information contained within these systems

for release to other agencies and the public, publish it in a timely manner, make it easily accessible

for external use as applicable, and post it at agency.gov/developer in a machine-readable format.

The Strategy tasked agencies with accomplishing four specific duties under this principle,

including the identification of at least two major customer-facing systems that contain high-value

data and content; displaying this information through web APIs to the appropriate audiences;

apply metadata tags in compliance with the new federal guidelines; and publishing a plan to

transition additional systems as practical.

Some contextual factors make this document unlike anything else produced through the

federal government of its type. First, the plan was unveiled by the federal CIO and CTO at a

major gathering of technologists and developers during TechCrunch Disrupt New York.3 This is

hardly a typical venue for major government announcements, especially those that are largely

dedicated to making government more efficient and responsive to its citizens. The TechCrunch

Disrupt series of conferences is held in New York City and San Francisco and meant to attract

3 http://techcrunch.com/events/disrupt-ny-2012/

13

developers and start-up technology companies to showcase innovative new products or highlight

emerging technology trends.

Second this document continues a trend towards integrating a vernacular to describe the

function of government that has little precedent. The term platform is in the title of the

document and it used thirty-two times throughout; federal government information is again

referred to as a “national asset” and with mandates to use Application Programming Interfaces

(APIs) there is an expectation that providing access to high-value government datasets (defined

in the Open Government Directive) will become not just be proactive, but automated.

Views on implementation

As stated previously, many civil society groups and government transparency watchdogs

followed an arc of excitement, which transitioned to sentiments of disillusionment. But in the

three years since the Open Government Directive was published many civil society groups have

grown to have a more positive outlook on the administration’s progress.

OpentheGovernment.org said that “The US met most of its 2011 commitments to make the

government more open and accountable.” The evaluation was a multi-organization effort,

distributing evaluation duties, using consistent metrics, across non-profits and academia with

experience working with agencies and evaluating information policies.39 Not all like-minded

watchdog groups feel the same. During the recent 2013 “Sunshine Week” – a week dedicated to

openness and transparency in government – many civil society groups issued statements of

praise for what had been accomplished, followed by calls for more action.

14

In testimony issued by Daniel Schuman, Policy Counsel and Director of the Advisory

Committee on Transparency at the Sunlight Foundation, before the House Committee on

Oversight and Government Reform, he said:

“Over the last few years, we saw the launch of the Open Government Directive, which required

agencies to create openness plans and release high value datasets to the public. We saw a memo

that enshrined a presumption of disclosure for responses to Freedom of Information Act requests.

We saw the creation of the first Federal Chief Technology Officer and the launch of Data.gov.

The newly created White House Ethics Czar fought against FOIA backlogs and implemented

greater lobbying disclosure for the Wall Street and Main Street bailouts…And, more recently,

we’ve seen the rollout of Ethics.gov, the “We the People” platform, the Open Government

Partnership, the Presidential Innovation Fellows, and FOIA Online. This is an impressive list…

Unfortunately, many of these initiatives have tapered off…The technological solutions that are

being embraced, such as the “We the People” platform, often are more centered on getting public

input than providing data about the government to the public. The brightest spot is the

international Open Government Partnership, but it contains little about ethics in government, and

many of its commitments represent longstanding pledges for incremental change.”40

Similar statements were heard from Sean Moulton, Director of Open Government Policy

at the Center for Effective Government: “The Obama administration established an impressive

array of important open government reforms,” he said in a press statement accompanying a new

report from the Center. “However, implementation has lagged at many agencies.”41 The report

itself acknowledged some accomplishments, saying, “Improving access to government data has

been a particular accomplishment of this administration… [Data.gov] now features more than

350,000 datasets, 1,200 data tools, and more than 130 mobile applications from agencies across

the federal government.” But not to be too congratulatory, the section ends by saying, “agencies

15

have lagged at releasing inventories of their datasets, which prevents users from knowing what

important datasets might still be missing from Data.gov.”42

Each of these groups had slightly different assessments, based on their groups’ primary

focus. Sunlight Foundation begrudged missed opportunities to make lobbying and donation

rules more transparent at the expense of seeking more input from citizens via the “We the

People” platform. OpentheGovernment.org saw clear improvements in agency compliance with

the Open Government Directive where the Center for Effective Government saw a need to do

more. At their core, however, even watchdog groups and civil society organizations are

advocacy groups, looking through prisms that will garner attention and donation support.

But civil society groups were hardly the only ones watching. While not a robust portfolio

in academia, there are those who focus on open data, study good government management or

focus on government performance. In reviewing various works of the academic community

(peer-reviewed and non-peer reviewed) a pattern of viewpoints emerged on what open

government was and how well the administration was achieving it. I have grouped such views

into three types: Public engagement-oriented, Performance-oriented and Platform-oriented.

Public engagement-oriented views of the administration’s open government activities

tend to focus on how readily the public is interacting in policymaking processes. This

orientation tends to believe that Open Government produces easily digestible information that is

contextual and enables public participation.

Performance-oriented views tend to focus on how open data is or is not leading to a more

efficient and competent federal government. This view sees open government as both a means

and an end towards achieving a better-performing government.

16

The third body of work tends to focus on how government can play a convening role to

enable innovation through open government. The term “government-as-a-platform” refers to a

organizational model of government service delivery that acts more like a platform from which

citizens can build from, and less like a “vending machine” through which services are bought

and delivered.

Each of these orientations will be discussed in the next section, followed by an

examination of how ONC is achieving Open Government through these various prisms.

Open Government as means toward public engagement

Little research, academic or otherwise, has focused holistically on whether agencies have

complied with the core precepts of President Obama’s Memorandum on Open Government –

transparency, collaboration and participation. While some of the most granular assessments of

agency activity have come from the “civil society” groups discussed previously, those

assessments were largely constructed along narrow advocacy lines. Others have taken a more

nuanced approach towards assessing the administration’s approach to open government. One of

the most recent efforts come from Evans and Campos (2013), who studied a handful of agencies’

efforts to be more transparent, collaborative and participatory.43 Their overall assessment is that

most agencies are engaged in efforts to be more transparent and collaborative, but few are

engaging with the public in meaningful ways. Evans and Campos argue that the combination of

broad instructions and short time horizons put many agencies at a disadvantage and may have

hindered early innovation. Because agencies had little experience and minimal precedent in

17

open government compliance, “…agencies looked to existing data and information, which did

not contain confidentiality or privacy risks, as the focus of their compliance efforts. This focus

seemed to inhibit, perhaps unintentionally, consideration of innovative ways to use data and

information to enhance public participation” (2013, 174).

Evans and Campos further argue that much of the data being made available to citizens is

lacking necessary context to make the data useful:

If the primary goal of open government is to engage citizens, then current

initiatives must be re-evaluated and new approaches explored—shifting beyond

data delivery. Releasing volumes of data on a Web site without background on

why and how it is collected, how it is organized, and its intended use, leaves

citizens with herculean tasks of determining its relevance and reliability (2013,

172).

While Evans and Campos see some agencies “aim to engage the public through

feedback tools designed to share ideas and obtain a range of recommendations rather than

limiting feedback to a predetermined set of solutions,” these efforts seem “to be in

nascent development,” and “do not convey a context for citizens to

understand pertinent policies” (2013, 177). For Evans and Campos, the ends of

Open Government are increased public engagement in the policymaking process, using

retail, or contextualized, data. Similarly, Nam (2012) sees an opportunity for Open

Government to enable more public engagement through a phenomenon he calls, “citizen-

18

sourcing,” though Nam seems less dependent on retail data.44 Nam says that through

“various platforms enabled by Web 2.0 technologies, citizens can collectively create

public information, provide service, and take part in policy processes” (2012, 12).

Nam briefly references another aim of Open Government, speaking to the opportunity to

use it for enhancing agency performance, “Pushed by the Open Government Directive of the

Obama administration,” Nam writes, “citizen-sourcing may be a new mode of government

operations in the U.S” (2012, 12).

Open Government as way to measure performance

The concept of performance measurement has long been seen as a way to ensure

accountability for individual work or organizational efforts. In modern times, President

Clinton’s Government Performance and Results Act of 1993 (GPRA) and President George W.

Bush’s Program Assessment Rating Tool (PART) are usually cited as flagship performance

initiatives for their respective administrations. For the Obama administration, some observers

are less enthusiastic about the progress made by the Obama administration in marrying

performance data with governing45 while others believe there is a “stealth revolution” in

performance management is underway.46 In taking a midterm snapshot, John Kamensky sees the

administration engaged in several efforts, several of which are not as overtly performance-

oriented as previous administrations – but with implications for “the performance community”

nonetheless.47 Kemensky’s believed that Obama’s focus was a departure from Clinton’s

“agency” focus and Bush’s “program” focus (2011, 135). Instead, Kamensky argues that

19

President Obama’s performance agenda was characterized by three key elements: having

agencies commit to a handful of high-priority performance goals; using public dashboards to

foster data-driven reviews and discussions; and using problem-solving networks (2011, 137-

137). “If successfully implemented,” Kamensky wrote, “these new ways of creating, collecting,

sharing, interpreting and reporting performance data in near real time could become the

foundation for new results-oriented governance models of the future,” (2011, 145).

Open Government as a path towards “Platform Government”

Adherents to notion that Open Government should enable a new type of government

service and information delivery are part of a group sometimes referred to as Gov 2.0. The Gov

2.0 crowd is composed of a diverse mix of public sector officials and private sector stakeholders

– most notably from the technology industry. The phrase “Government-as-a-Platform” was

coined by a guru of the internet world, named Tim O’Reilly. Writing in several places and

delivering speeches on the concept, Mr. O’Reilly often referred to the concept as a metaphor for

getting the government to think differently about how it delivered services to citizens and

fulfilled its mission. In particular, both Tim O’Reilly is fond of looking at the National Oceanic

and Atmospheric Association (NOAA) as a model “government-as-a-platform” demonstration. 48

Weather.com, the local news and nearly every weather application uses data from NOAA, which

it makes available for the private sector to use and build services through. Likewise, the military

pioneered GPS signals that then were made available for civilian use. In applying the concept to

20

health data, observers note “the potential for private enterprise to provision services using open

data from the Community Health Data Initiative could match the billions of dollars made when

the government unlocked GPS and NOAA weather data.”49

Mr. O’Reilly also drew on inspiration that has been picked up by officials at ONC and

embodies the notion that through Open Government, government can act as a platform. For

instance, the HHS Open Government Plan Version 2.0 used the phrase “rough consensus and

running code.”50 This is derived from a presentation given by noted American computer scientist

David Clark in 1992 who asked “As the Internet and its community grow, how do we manage

the process of change and growth?”51 ONC officials have used this same answer to describe the

how the agency is managing the process of change and growth in the health IT market.52

Another point of inspiration for the government-as-a-platform ethos comes from Bill Joy,

founder of Sun Microsystems, who said, “No matter who you are, most of the smartest people

work for someone else.”53 This mantra has been picked up by former HHS Chief Technology

Officer, and current Federal CTO, Todd Park, who sees prize competitions as a means to

collaborate between public and private stakeholders to create innovative health solutions.

[T]he fastest and most efficient way to generate change is to empower, fuel and

catalyze all the smart people who don’t work for you to innovate and transform

the healthcare system. We see our roll at HHS as catalysts, as data suppliers and a

celebrator of what’s happening, but we’re just one of a number of this

increasingly self-propelled highly decentralized eco-system in the nation that is

producing amazing things already that no one organization, no 10 organizations

can even conceptualize let alone execute and bring them to life.54

21

Open Government Theory and Practice

When these orientations are overlain with the three aims of the administration’s Open

Government initiatives and combined with the types of data produced by such initiatives (raw

versus retail) and the purpose of producing the data (internal versus external audience) one can

visualize an agency’s various Open Government efforts through the Open Government

Assessment Tool (Figure 1).

Figure 1

This Framework attempts to accomplish two aims. The Framework visualizes how

various activities within government can be linked to the Obama administration’s Open

22

Government Directive (and associated goals of transparency, participation and collaboration).

And it can serve as a way to assess the mix of current activities and programs. For example, if

an agency’s activities are clustered within one box (or one section of a box) managers could use

the visual tool to quickly identify a lack of diversity among activities with potential Open

Government implications and begin thinking about ways to utilize alternative tools or strategies

to: reorient data type, repurpose administrative data or develop new reporting protocols to further

government collaboration, transparency and participation.

Open Government and the Office of the National Coordinator for Health IT

The Office of the National Coordinator plays a prominent part in HHS’s overall Open

Government Plan. From the agency-wide perspective, ONC is leading the charge for HHS in

making government more participatory, transparent and collaborative. Highlighting successes

and outlining a path towards improvements, HHS’s Open Government Plan “Version 2.0,” says,

“The most ambitious project launched by any HHS agency under the new challenge competition

authority in the America COMPETES Reauthorization Act in FY2011 is the HHS Investing in

Innovation (i2) initiative, a new $5 million program to spur innovations in Health IT.”55 The i2

initiative is one of a host of programs at ONC with implications for Open Government. Each

initiative falls into one or more category of promoting transparency, participation or

collaboration; and when juxtaposed against the dominant views over what open government can

achieve, ONC has programs represented in all three vectors of the Open Government Assessment

Tool (Figure 2).

23

Figure 2

When discussing Open Government with program administrators and staff at ONC, there

are different views on what project is the flagship Open Government initiative. However, there

is one initiative that strikes a balance of incorporating different views of what Open Government

should achieve and manages, intentionally, to touch on all three aspects of the administration’s

Open Government Directive: the Health IT Dashboard.

Performance

Raw and Retail Performance Data as a Way to Make Government More Transparent,

Collaborative and Participatory

Initiative OutputsHealth IT Dashboard

A hub for both raw and retail data produced through ONC programs and evaluations

o 100,000 data points on performance 13 datasets

o Longitudinal data (XML)

24

o Most recent reporting period (CSV)o Grantee datao Private sector datao Multiple “data layers” to provide differing degrees of

contexto Geographic Information System overlay

10 data briefs (PDF) 3 “Quick Stats”

The Dashboard is perhaps ONC’s most unique Open Government initiative. While it is

primarily a public-facing performance tool, it has various layers of contextual data that can be

added or omitted based on user preference; data types range from raw XML and CSV files to

fully developed data briefs and published reports. And while the Dashboard is purposefully

public facing, it plays an important internal function in tracking performance and milestones.

According to the agency, “ONC maintains the Health IT Dashboard as a platform to distribute a

broad range of health system and grant program performance measures tracking the adoption and

meaningful use of various health information technologies (health IT), including electronic

health records (EHR) and others enabling health information exchange (HIE).”56 This tool is a

collection of dashboards that display a host of information and indicators, spanning, spanning

five categories: (1) ONC overview; (2) Health IT adoption and use; (3) Regional Extension

Center Program; (4) Health IT workforce training; and (5) Meaningful Use. These categories

utilize various types of data including administrative data generated through ONC’s portfolio of

grantees, statistics generated by other government agencies and findings from the private sector.

The Dashboard uses a range of visual tools to help add context to the raw data, which is also

provided in one area of dashboard.healthIT.gov.

ONC Overview Dashboard

25

The ONC Overview Dashboard includes summary information about ONC’s eight

HITECH grant programs4, including their location, funding amount and contact information.

The predominant way that this information is displayed is through a map of the United States and

Territories, which is known as a geographic information system (GIS) mashup. The spending

data is also displayed as a pie graph, outlining major spending categories, such as State Health

Information Exchange grants, Regional Extension Center grants and Beacon Community grants.

The final piece of the ONC Overview Dashboard is a relational graphic that contextualizes how

each HITECH program contributes to “improved individual and population health outcomes,

increased transparency and efficiency and improved ability to study and improve care delivery,”

as described by former National Coordinator for Health Information Technology Dr. David

Blumenthal.57 Several options are available to users that layer additional spending information

on the map pertaining to one or more of the eight HITECH grant recipients.

Health IT Adoption & Use Dashboard

The Health IT Adoption & Use Dashboard presents state-level estimates for the rate of

EHR adoption among office-based physicians and hospitals. This dashboard is perhaps the most

interesting because it relies primarily on data sourced from other government and non-

government entities. There are six different maps available through this dashboard, including

EHR adoption by office-based providers, EHR adoption by non-federal acute care hospitals,

physicians actively using an EHR to e-prescribe, number of health care providers, number of

4 Programs include Health IT Extension Centers Program (aka Regional Extension Centers Program); State Health Information Exchange Program; Beacon Communities Program; Strategic Health IT Advanced Research Projects (SHARP) Program; Community College Consortia to Educate Health IT Professionals Program; Program of Assistance for University-Based Training; Curriculum Development Centers Program; and the Competency Examination Program

26

primary care providers and community pharmacies actively e-prescribing. These different views

rely on data from:

The Centers for Disease Control and Prevention (CDC) National Ambulatory Medical

Care Survey (NAMCS) EHR Supplement;

The American Hospital Association Annual Survey, IT Supplement;

ONC analysis of e-prescribing data procured from Surescripts; and

ONC analysis of a health care market research dataset procured from SK&A.

This dashboard also contains summary graphs depicting national rates of office-based

provider EHR adoption, hospital EHR adoption and community pharmacies actively e-

prescribing rates, from 2008 to 2011. As with the other maps, spending data from ONC grantees

can be overlaid the various health IT adoption and use maps.

Regional Extension Center Program Dashboard

The Health IT Regional Extension Center (REC) Program Dashboard tracks monthly

progress to the programs primary care provider EHR adoption goals. There are 62 RECs across

the country and the program has a performance-based reimbursement structure that compensates

REC grantees for assisting primary care providers through three milestones along the path to

meaningfully using EHRs. The performance milestones that qualify an REC for grant payment

are: (1) a health care provider enrolls to receive assistance from a REC; (2) the provider “goes

live” with an electronic health record (EHR) that has e-prescribing and quality reporting

functionalities enabled; and (3) the provider or REC attests that the provider has met the

Medicare and Medicaid EHR Incentive Program criteria for meaningful use of an EHR. This

27

dashboard is the most complex of the five because it includes over sixty custom reports, covering

over thirty measures at the national, grantee, state, and county level. There are nine different

REC grantee maps that focus on national trends of milestone attainment – enrollment, EHR go-

lives and demonstrating Meaningful Use. Each milestone is further broken down to indicate goal

by REC coverage area, percent progress by REC coverage area and milestone attainment by state

or county. In all, there are twelve maps, updated on a nearly-monthly basis. At both the national

and the state/county level of information there are two graphs. One is a line graph indicating

where the country (or the state) is in program-to-date enrolment, go-lives and meaningful users.

The other is a bar graph which has the same three metrics, but breaks them down by primary care

provider type, including adolescent med and pediatrics, family practice, general practice,

geriatrics, gynecology and OBGYN, internal medicine and other. These metrics and breakdowns

are the same for both national views and state/county views.

Health IT Workforce Dashboard

The Health IT Workforce Dashboard includes summary information about the

Community College Consortia to Educate Health IT Professionals Program, including location of

grantees, and progress to health IT workforce development goals. There are more than 80

community college grantees split into five regions that cover the northwest, southwest, Midwest,

south and northeast part of the country. This dashboard contains four different views of

performance, including health IT professionals trained (by grantee and state), and active

enrollments in health IT workforce programs (by grantee and state). There are three graphs that

accompany both types of GIS mashups, including a line graph indicating actual health IT

professionals trained versus quarterly goals; a pie graph indicating a breakdown of rural and

28

urban enrollee backgrounds (by zip codes provided from students); and a bar graph indicating

program enrollments and program completions by quarter. Data is available on a national basis

and a regional basis.

Meaningful Use Dashboard

The Meaningful Use Dashboard presents data from the most recent CMS public use

reports regarding EHR Incentive Program payments to eligible hospitals and professionals. The

summary data in the Health IT Dashboard depicts national trends since the program's inception

as well as current snapshots for each state. This dashboard presents eight different layers that

capture (1) number of payments and (2) payment amounts to eligible professionals (EPs) eligible

hospitals (EHs) and a combined measure (eligible providers). The two additional layers provide

views of Mediare payments to eligible providers and Medicaid payments to eligible providers.

National level payment numbers and amounts are available, as are state and county data.

Additionally, individual EHs that have received Incentive Payments are identified at the

state/county level for each state. At the national-level view, cumulative payments to EHs and

EPs are displayed in bar charts. Similarly, at the state level, two pie charts display Medicare

versus Medicaid payments to EHs and EPs.

The Health IT Dashboard, serves as both a hub for both raw and retail data produced

through ONC programs and evaluations. According to officials interviewed from the Office of

Economic Analysis, Evaluation and Modeling, since creation of the Health IT dashboard, we

have prioritized dissemination [of other ONC data] through the project’s own website. Broadly,

the goals of open government and the Dashboard within ONC are to assist with monitoring

29

health IT adoption trends; facilitate assessment to progress to national and HITECH goals;

enable stakeholders with data that is insightful and actionable. In speaking with ONC staff, the

dominant orientation towards the Health IT Dashboard’s value is one of performance. Providing

actionable data to program directors, staff, grantees and the general public using an easy to

understand framework has been a priority.

Transparency / Performance

The Health IT Dashboard, serves as both a hub for both raw and retail data produced

through ONC programs and evaluations. According to officials interviewed from the Office of

Economic Analysis, Evaluation and Modeling, since creation of the Health IT dashboard, we

have prioritized dissemination [of other ONC data] through the project’s own website. Broadly,

the goals of open government and the Dashboard within ONC are to assist with monitoring

health IT adoption trends; facilitate assessment to progress to national and HITECH goals;

enable stakeholders with data that is insightful and actionable. In speaking with ONC staff, the

dominant orientation towards the Health IT Dashboard’s value is one of performance. Providing

actionable data to program directors, staff, grantees and the general public using an easy to

understand framework has been a priority. The core measures for the Dashboard were

determined first by measures important to HHS’s Strategic Plan58 and the President’s Budget

request to Congress for FY2012-2014 (http://www.healthit.gov/policy-researchers-

implementers/onc-budget-documents-and-performance-information). Measures were also

created to aid in the evaluation of the American Recovery and Reinvestment Act (which included

HITECH Act) implementation plans. Specifically, analysis of data from the REC program was

prioritized given the opportunity to provide a large number of organizations with summarized

30

and visualized data tracking their performance in the ONC grant program, program staff said.

This approach also helped ensure REC grantees would not duplicate investments in data analysis

and visualization tools, according to interviews. The Dashboard’s internal purpose is to assist

with monitoring HITECH programs, such as ONC Grantee data, health IT adoption trends and

CMS EHR Incentive payments. The tool is also used to brief internal stakeholders regularly, as

a standardized presentation, and enable rapid visualization of routine data analysis projects.

Collaboration / Platform

ONC Staff said the data compiled on the Dashboard has several uses external to ONC,

including other federal agencies, state and local governments as well as the private sector. Other

HHS agencies use the Dashboard to monitor the relationships of their grantees, such as Federal

Qualified Health Centers (FQHCs), Indian Health Services (IHS) facilities, Critical Access

Hospitals (CAHs) to ONC programs and research. Agencies often cite trends in health

technology derived from the dashboard on their own websites and through associated media

outlets. Some examples of federal websites that refer visitors to the Dashboard include the

National Library of Medicine (NLM), the Centers for Disease Control and Prevention (CDC),

Centers for Medicare & Medicaid Services (CMS), and the Health Resources and Services

Administration (HRSA). When working on the Dashboard launch, staff said ONC consulted

widely with leaders across HHS.  Since the launch, we have regularly shared best practices with

other agencies undertaking similar projects (CMS/CMMI, HHS-CTO are two such examples)

and we have used the website to create value in strategic collaborations with partner agencies

like CMS. One example offered by ONC was the build-out of the Health IT Dashboard to

include a Meaningful Use dashboard using CMS payment data for those eligible hospitals and

31

eligible professionals who successfully attested for Stage 1 Meaningful Use. Additionally,

several datacenters in other federal agencies regularly extract the data for their own uses,

according to site metrics, and ONC present Dashboard data to HHS evaluation, performance, and

new media communities regularly.

State and local government has also used the Dashboard data for various reasons. State

governors’ offices and local governments – through regional and county officials – use the data

to understand technology adoption trends in their area and to coordinate effectively within their

states for appropriate allocations of resources. And like some federal agencies, several state-

level data centers access the site on recurring basis and download the data.

The private sector is also a regular consumer of the data, according to staff interviews.

Consulting companies seem to be using the data to target business opportunities to geographic

areas where providers are lagging behind the state or national technology adoption trends. And

beyond market research, Dashboard information has been examined and combined with CMS

data as evident in recently published media articles59 and peer-reviewed commentaries on the

EHR market.60

Participation / Public Engagement

The Health IT Dashboard makes available several data briefs, reports and evaluations that

enable policymakers and interested civilians to become engaged with the policy process. And

the raw data is made available in different formats to allow interested individuals to download

the information. However, there doesn’t appear to be an explicit engagement piece to the

Dashboard.

32

Platform

Challenges and Prize Competitions as Open Government

Initiative OutputsInvesting in Innovations (i2) Initiative

25 challenges dating back to July 2011 Cash prizes ranging from $4,500 to $150,000

Blue Button

Used < 1M times by Veterans and Medicare beneficiaries Further iterations

o Blue Button Mash Up o Blue Button+ o Automated Blue Button Initiative

The i2 initiative is focused on developing innovations that support (1) the goals of

HITECH and addressing challenges related to the achievement of widespread health IT adoption

and meaningful use, (2) ONC’s and HHS’ programs and programmatic goals, and (3) the

achievement of a nationwide learning health system that improves quality, safety, and/or

efficiency of health care.61 According to HHS, i2 initiative challenges usually last three to six

months and offer, “a chance for external developers, many new to health care, to provide

solutions to key health IT challenges. The challenges issued by ONC have included application

development, educational videos, and clinical data mash ups, such as:

Illustrating how having access to your health record and looking it over helped improve

care through the “What’s in Your Health Record Video Challenge”62

Developing an app that allows Medicare beneficiaries to download their claims data and

combine it with other types of data that will help them make better health decisions,

improve their health or find ways to lower the cost of their health63

Creating an application that leverage the popHealth open source framework, existing

functionality, standards, and sample datasets to improve patient care and provide greater

insight into patient populations64

33

Sponsoring a developer challenge and outreach initiative to inform millions of Americans

who may be unaware they are at significant risk for cardiovascular disease, encourage

those who think they may be at risk to take action, and direct them to community

pharmacies to receive blood pressure and cholesterol screenings65

Innovators who participate in challenges are offered the opportunity to compete against

other solution providers and receive detailed feedback on how to improve their innovations.66

ONC has held over twenty-five challenges dating back to July 2011, with cash prizes ranging

from $4,500 to $150,000;67 and ONC has allocated funds for more than 30 challenges through

2014.68 The 2012 HHS Open Government Plan also indicated that ONC is leading the way in

helping other HHS agencies develop expertise in holding challenge competitions. “These

challenge competitions are being integrated into other programmatic areas to achieve continued

development and implementation of prototype and early start up technologies,” the Plan said.69

Blue Button

In October 2010, US CTO Annesh Chopra, HHS CTO Todd Park and Veterans

Administration CTO Peter Levin announced the formal launch of an initiative that would allow

all military veterans and Medicare beneficiaries access to their personal health data.70 The

initiative was dubbed “Blue Button” and it would give beneficiaries and veterans the opportunity

to view online and download all the claims processed through Medicare or the VA. “An open-

government initiative, Blue Button results from a collaboration between VA and HHS to develop

an online feature that would enable Veterans and Medicare beneficiaries to easily read, use, and

share their personal health information with providers and others they trust. The ASCII text file

34

format was selected for its ease of use by individuals, while allowing computers to easily ‘read’

the information,” they wrote in a White House blog.71 The Blue Button initiative holds a

prominent position in both the HHS Open Government Plan72 and the White House Status

Report73 on open government, having been used by over a 1 million Americans since 2010.74

Currently, Blue Button is being further developed at ONC via two initiatives known as

Blue Button+ and the Automate Blue Button Initiative. Blue Button+ gives users the ability to

get records in a human-readable and machine-readable format; and to send their records where

they choose. The implementation guide developed for Blue Button+ uses interoperable

standards that enable patients to do everything from printing a physical copy to sharing it with a

third-party application.75 Similarly, the Automate Blue Button Initiative (ABBI) is looking to

use standard protocols and specifications that would patients to not only download their health

information to their personal computer, but also to privately and securely automate the sending

of that data from their healthcare providers to their personal health records, email accounts,

health-related applications, or other preferred holding place.76

By standardizing the technical specifications for sending / receiving Blue Button data, it

is hoped that an “ecosystem” of third-party application developers will spring up to give wide

customization to patients who want to share their information. ONC has already held a

challenge, the Blue Button Mash Up Challenge, that produced three winning apps, making Blue

Button data more useful for patients and their providers.77 By taking the specifications one step

further, in the ABBI project, ONC hopes more products will come to market, while enabling

improvements in healthcare delivery and health cost containment.

35

Public Engagement

S&I Framework and Federal Advisory Committees as Open Government

Initiative Outputs

Public Comments 9 proposed regulations since 2011 Nearly 2000 comments

Federal Advisory Committees

315 meetings since January 2012 Roughly 20 meetings per month

Standards & Interoperability Framework

1200 volunteers have participated in S&I initiatives and workgroups;

Approximately 600 participants join weekly calls Mean interval between calls is around 3.5 hours Pace of activity for 2+ years

Health IT Buzz Blog

Nearly 300 blog posts by ONC officials since Oct. 2009 o Several hundred comments left by public

Federal Health IT Strategic Plan o 240 comments on Strategic Plan

Planning Room “A pilot open government project to improve public

participation in strategic planning” 150+ comments in first 60 days

Both Blue Button Plus and the Automate Blue Button Initiative are projects being

developed through ONC’s Standards and Interoperability (S&I) Framework, which is, itself, an

exercise in open government. In fact, the Blue Button initiative is considered to be the

Archetype S&I Framework project.78

In the example of Blue button, ONC’s Office of Science & Technology, which manages

the S&I Framework, has served in the role of government-as-a-convener. Blue Button +

guidance was developed though the S&I Framework with input from more than 70 volunteer

individuals and organizations79 and ABBI has 84 volunteers representing more than 76

36

organizations.80 This convener role holds true for all past and current S&I initiatives. Thus far,

more than 1200 unique volunteers have participated in S&I initiatives and workgroups;

approximately 600 participants join weekly calls, where the mean interval between calls is

around 3.5 hours, and this has been the pace at ONC’s Office of Science & Technology for the

last two years.81 To facilitate this level of activity, the S&I Framework utilizes a wiki site, which

allows for any member to edit a page and also see the history of that page. And the discussion

tabs on the wiki pages are encouraged, because e-mail does not allow for all members of the

community to be engaged with the multiple discussions occurring at any given time.82 The S&I

Framework establishes a charter for each initiative that generally seeks to find consensus on

standards needed to advance programs under HITECH and support health advances through IT.

While participants are not under direct guidance from ONC, and ONC is in no way obliged to

use standards developed through the S&I Framework, the projects generally align with priority

areas established by ONC and / or ONC Federal Advisory Committees – the Health IT Policy

Committee and the Health IT Standards Committee.

HHS has more than 270 Federal Advisory Committees, though few – if any – have been

as busy as those convened for the purpose of promoting EHR adoption. From January 2012

through March 2013, the Health IT Policy Committee, the Health IT Standards Committee and

their various workgroups, power teams and tiger teams have met more than 315 times,

translating to roughly 20 meetings per month.83 While data was not available before January

2012, this pace has been holding steady for roughly three years. According to the HHS Open

Government Plan Version 2.0, it is this sort of collaborative stance that has allowed “dramatic

37

progress in EHR adoption nationwide,” especially in designing a policy framework for the

certification and “meaningful use” of EHRs and consensus-based standards development.84

Additionally, ONC utilizes public commenting opportunities to seek engagement from

various stakeholders. As is customary for any executive branch rulemaking, ONC asks public

and private stakeholders for input on proposed rules, requests for information and requests for

comment on a regular basis. In the past two years, ONC has promulgated nine proposed

rulemakings, receiving nearly 2000 public comments via Regulations.gov.5 According to

officials at HHS, each public response is read, analyzed and placed into one of several categories

that are then addressed in the preamble of the final regulation.

ONC’s foray into blogging is consistent with that of other federal agency blogs,

providing updates from Office leadership two or three times per week. There is nothing

remarkable about the traffic volume or number of public comments received through ONC’s

Buzz Blog. But the blog has been used in the past to help facilitate public input, most notably in

helping ONC construct its Federal Health IT Strategic Plan.85 In a blog post by then ONC chief

Dr. David Blumenthal, ONC encouraged stakeholders to submit their ideas on the draft via the

comments section of the Buzz Blog.86 Over 240 comments were filed in such a way and used to

inform the final Strategic Plan. More than two years later, ONC is now engaged in a process to

update the Strategic Plan and they are using a new web-based tool called “Planning Room.”

Planning Room is meant to enable citizen feedback on various aspects of the Federal

Health IT Strategic Plan. Planning Room uses a Cornell-developed platform that helps citizens

engage in “effective commenting” that frames the Strategic Plan in themes, such as Consumer

Engagement and Health IT.87 The platform is meant to encourage comments on what should, or

should not, be done – as well as – why. It also allows citizens to “endorse” other comments.

5 Results obtained by performing searches and filtering at Regulations.gov

38

After the first commenting period, which lasted from March 25 to May 9, over 250 comments

were made to address elements of the Strategic Plans sections on Consumer e-Health.

Discussion and Recommendations

When taken holistically and when applying the Open Government Assessment Tool to

ONC’s various programs, it is quite apparent that the tenets of the Obama administration’s Open

Government Directive are met in a number of ways. While some programs or initiatives have

multiple Open Government applications, ONC has spread their efforts to cover all components of

collaborative, transparent and participatory government. The degree to which this is intentional

is not entirely clear, however. ONC does a good job intertwining elements of Open Government

into many programs, and the Office clearly embraces the administration’s principles. But if

ONC were to conduct an internal review of all programs and activities and map those against the

Open Government Assessment Tool, the Office could be more strategic with where to focus their

efforts. ONC could then look for ways to better integrate their efforts to broader efforts HHS-

wide.

Writing for the Institute of Medicine’s Learning Health System Commentary Series,

ONC chief Dr. Farzad Mostashari outlined how the Office’s orientation towards “transparent and

open governance in the public interest” had thus far yielded big returns.88 “There are now more

than 1,800 certified [electronic health record] systems,” he wrote. “Compared to 2010, venture

capital funding more than doubled both in terms of deals and dollars invested, and the health IT

sector is a bright spot in the economy with over 50,000 new IT jobs filled in the past two years.”

Identifying return on investment (ROI) for Open Government projects is a difficult task,

though this does not obfuscate the need. ONC has some quality output measures – and even

39

some outcome measures – upon which to judge their progress. But there seems to be a lack of

published data on the returns of efforts, such as i2 Initiative. As ONC looks to direct more funds

and projects towards challenge and prize competitions, it would be instructive to perform a cost-

benefit evaluation or case study to describe the programs impacts.

When answering the general question of “How is ONC complying with the Obama

administration’s Open Government initiatives?” this case study sought to achieve the following:

To understand how various activities and programs align with the administration’s goals

of promoting more transparent, participatory and collaborative government

To identify definitional themes of “Open Government” as illustrated by stakeholders

inside and outside of ONC

To explore how data are being used “downstream” to:

o Inform policy decisions;

o Enhance research; and / or

o Promote innovation

40

The conceptual framework that emerged from interviews and research is pictured in

Figure 3. This conceptual framework overlaid elements of a logic model derived from

examining components of ONC’s Open Government work, as though it was a singular policy or

program, with input from interviews asking about how Open Government fit in with ONC’s

overall mission.

Enabling insightful and actionable health IT products, services, research and policies is

the end goal for ONC in their Open Government efforts, which will lead to positive impacts in

EHR adoption rates, quality and safety of care delivery, patient privacy and public health.

41

Clearly, ONC has internalized the administration’s call to be more transparent,

collaborative and participatory. It is difficult to say whether ONC is blazing new trails with

Open Government, incorporating its tenets to a degree far surpassing other federal efforts. But it

would be equally difficult to argue that ONC sees Open Government as anything other than a

genuine opportunity to develop better public policy, enable innovation and facilitate the

modernization of America’s healthcare delivery system.

42

1 US Department of Justice, Freedom of Information Act, About FOIA accessed 23 March 13 (http://www.foia.gov/about.html)2 Openness Promotes Effectiveness In Our National Government Act of 2007, Public Law 110-175, accessed 23 March 2013 (http://www.gpo.gov/fdsys/pkg/PLAW-110publ175/html/PLAW-110publ175.htm)3 Ibid.4 E-Government Act of 2002, Public Law 107–347, accessed 23 March 2013 (http://www.gpo.gov/fdsys/pkg/PLAW-107publ347/html/PLAW-107publ347.htm)5 Ibid.6 Ibid.7 Ibid.8 Miller, J. “E-governments promise partially fulfilled,” Federal News Radio, Dec 17, 2008, accessed 23 March 2013 (http://www.federalnewsradio.com/145/1552276/E-governments-promise-partially-fulfilled) 9 White House, “Memorandum of January 21, 2009, Transparency and Open Government,” Federal Register Vol. 74, No. 15, January 26, 200910 Ibid.11 Ibid.12 Ibid.13 Orszag, P. R Office of Management and Budget, “Memorandum for the heads of executive departments and agencies Open Government Directive,” December 8, 200914 Orszag, (Pg. 2)15 Orszag, (Pg. 3)16 Orszag, (Pg. 7)17 “Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by Federal Agencies” (67 FR 8452), Federal Register Vol. 67, Issue 86, May 3, 200218 Orszag, (Pg. 3)19 Orszag, (Pg. 5)20 Orszag, (Pg. 6)21 OpentheGovernment.org “OpenTheGovernment.org Welcomes the Open Government Directive,” December 8, 2009, accessed 23 March 2013 (http://www.openthegovernment.org/node/444) 22 OpentheGovernment.org, “Audit Reveals Wide Variation in Agency Plans to Make Government More Open,” April 29, 2010, accessed 23 March 2013 (http://www.openthegovernment.org/node/445) 23 Meijer, A., Thaens, M. “Alignment 2.0: Strategic use of new internet technologies in government” Government Information Quarterly, 27, 113–121. (2010)24 Wonderlich, J., “Obama's Open Government Directive, Two Years On” Sunlight Foundation Blog, December 7, 2011, accessed 20 March 2013 (http://sunlightfoundation.com/blog/2011/12/07/obamas-open-government-directive-two-years-on/) 25 OpentheGovernment.org, “Audit Reveals Wide Variation in Agency Plans to Make Government More Open,” April 29, 2010, accessed 23 March 2013 (http://www.openthegovernment.org/node/445)26 White House, “Obama Administrations Commitment to Open Government: A Status Report,” September 2011 accessed 20 March 2013 (http://www.whitehouse.gov/blog/2011/09/16/status-report-administration-s-commitment-open-government)27 Ibid. (Pg. 13)28 America COMPETES Reauthorization Act of 2010, Public Law No: 111-358, accessed 20 March 2013 (http://thomas.loc.gov/cgi-bin/toGPObss/http://www.gpo.gov/fdsys/pkg/PLAW-111publ358/pdf/PLAW-111publ358.pdf) 29 “Obama Administrations Commitment to Open Government: A Status Report,” (Pg. 17)30 Ibid. (Pg. 19)31 White House, “The Open Government Partnership and the National Action Plan for the United States of America,” September 2011 accessed 20 March 2013 (http://www.whitehouse.gov/sites/default/files/us_national_action_plan_final_2.pdf) 32 Ibid. (Pg. 1)33 Ibid. (Pg. 3-5)34 Ibid. (Pg. 6-7)35 White House, “Memorandum of January 21, 2009, Transparency and Open Government”36 Ibid. (Pg. 7-9)37 White House, “Building a 21st Century Digital Government,” (77 FR 32391), Federal Register, June 01, 2012 accessed 20 March 2013 (https://federalregister.gov/a/2012-13470) 38 White House, “Digital Government: Building a 21st Century Platform to Better Serve the American People,” May 2012 accessed 20 March 2013 (http://www.whitehouse.gov/sites/default/files/omb/egov/digital-government/digital-government.html)

39 OpentheGovernment.org, “Open Government Plan Audits, About this Project” accessed 05 March 2013 (https://sites.google.com/site/opengovtplans/home/about-this-project) 40 Schuman, D., Testimony before the House Committee on Oversight and Government Reform on March 13, 2013, accessed 20 March 2013 (http://sunlightfoundation.com/blog/2013/03/13/house-oversight-hearing-on-open-government/)41 “Obama's Legacy of Transparency is Unfinished,” Center for Effective Government, March 2013, accessed 22 March 2013 (http://www.foreffectivegov.org/obamas-legacy-of-transparency-is-unfinished)42 Moulton, S., Baker G., “Delivering on Open Government: The Obama Administration’s Unfinished Legacy,” March 2013, accessed 22 March 2013 (http://www.foreffectivegov.org/files/info/obama-first-term-transparency-report.pdf)43 Evans, A., Campos, A., “Open government initiatives: Challenges of citizen participation” Journal of Policy Analysis and Management, Vol. 32, No. 1, 172-203 (2013)44 Nam, T., “Suggesting frameworks of citizen-sourcing via Government 2.0,” Government Information Quarterly Vol. 29, 12–20 (2012)45 Joyce, P., “The Obama Administration and PBB: Building on the Legacy of Federal Performance-Informed Budgeting?” Public Administration Review, Vol. 71, Issue 3, 2011 (356–367)46 Kettl, D., “Obama’s stealth revolution: Quietly reshaping the way government works,” Public Manager, 38(4), 39–42 (2010)47 Kamenski, J., “The Obama Performance Approach - A Midterm Snapshot,” Public Performance & Management Review, Vol. 35, No. 1, 133-148 (2011)48 Slocum, M., “What lies ahead: Gov 2.0,” O’Reilly Radar, December 31, 2011, accessed 3 April 2013 (http://radar.oreilly.com/2010/12/2011-gov.html)49 Howard, A., “Here come the healthcare apps,” O’Reilly Radar, June 11, 2010, accessed 3 April 2013 (http://radar.oreilly.com/2010/06/here-come-healthcare-apps.html)50 Department of Health and Human Services, “Open Government Plan Version 2.0” April 2012 (Pg. 45)51 Clark, D., "A Cloudy Crystal Ball – Visions of the Future.” Presentation given at the 24th Internet Engineering Task Force, July 1992 (pg. 543) accessed 1 April 2013 (http://ietf.org/proceedings/prior29/IETF24.pdf)52 Mostishari, F. “Applying Innovation to the Work of Government: A Case Study of the Office of the National Coordinator for Health IT,” Institute of Medicine, June 2012 accessed 1 April 2013 (http://iom.edu/Global/Perspectives/2012/ApplyingInnovation.aspx)53 Lakhani K., Panetta J., “The Principles of Distributed Innovation. Innovations” The Berkman Center for Internet and Society Research Paper No. 2007-7, October 2007 accessed 22 March 2013 (http://ssrn.com/abstract=1021034)54 Mosquera, M. “Q&A: Todd Park on the bridge between HHS' Health Data Initiative and meaningful use,” Government Health IT, June 27, 2011, accessed 3 April 2013 (http://www.govhealthit.com/news/qa-todd-park-bridge-between-hhs-health-data-initiative-and-meaningful-use-ehrs?page=0,1)55 Department of Health and Human Services, “Open Government Plan Version 2.0” April 2012 (Pg. 32)56 ONC “Datasets & Documentation,” accessed 20 Feb 2013 (http://dashboard.healthit.gov/data/)57 Blumenthal, D. “Launching HITECH,” New England Journal of Medicine, 2010; 362:382-385. Feb. 4, 2010 accessed 20 Feb 2013 (http://www.nejm.org/doi/full/10.1056/NEJMp0912825) 58 Department of Health and Human Services, “HHS Strategic Plan” 2012 accessed 21 March 2013 http://www.hhs.gov/secretary/about/appendixb2.html#1.F.0559 Conn, J. “Tight battles under way for inpatient EHR dominance,” Modern Healthcare, March 27, 2013 accessed 28 April 2013 http://www.modernhealthcare.com/article/20130327/NEWS/303279955/tight-battles-under-way-for-inpatient-ehr-dominance60 Vest, Yoon, Bossak “Changes to the electronic health records market in light of health information technology certification and meaningful use,” J Am Med Inform Assoc 2013; 20:227-232 http://jamia.bmj.com/content/20/2/227.abstract61 Office of the National Coordinator for Health IT, “Health IT Prizes & Challenges,” accessed 12 March 2013 (http://www.healthit.gov/policy-researchers-implementers/health-it-prizes-challenges)62 Office of the National Coordinator for Health IT, “What’s in Your Health Record Video Challenge,” July 2012, accessed 12 March 2013 (http://yourrecord.challenge.gov/)63 Office of the National Coordinator for Health IT, “Blue Button Mash Up Challenge,” September 2012, accessed 12 March 2013 (http://www.health2con.com/devchallenge/blue-button-mash-up-challenge/)64 popHealth Tool Development Challenge,” October 2011, accessed 12 March 2013 (http://challenge.gov/ONC/246-pophealth-tool-development-challenge)65 Million Hearts Risk Check Challenge,” July 2012, accessed 12 March 2013 (http://challenge.gov/ONC/398-the-million-hearts-risk-check-challenge)66 Department of Health and Human Services, “Open Government Plan Version 2.0” April 2012 (Pg. 32)67 Challenge.gov, Office of the National Coordinator for Health IT, accessed 12 March 2013 (http://challenge.gov/ONC) 68 Ibid (Pg. 32)69 Ibid. (Pg. 7)

70 Chopra, A., Levin, P., Park, T., “‘Blue Button’ Provides Access to Downloadable Personal Health Data,” White House Blog, October 2010, accessed 20 February 2013 (http://www.whitehouse.gov/blog/2010/10/07/blue-button-provides-access-downloadable-personal-health-data)71 Ibid.72 Department of Health and Human Services, “Open Government Plan Version 2.0” April 2012 (Pg. 7)73 (Pg. 19)74 US Department of Veterans Affairs, Office of Public and Intergovernmental Affairs, “Blue Button Reaches One Million Registered Patients” August 31, 2012 accessed 20 Feb 2013 (http://www.va.gov/opa/pressrel/pressrelease.cfm?id=2378)75 Blue Button+ Implementation Guide, February 2013 accessed 20 Feb 2013 (http://bluebuttonplus.org/history.html)76 Office of the National Coordinator for Health IT, “Automate Blue Button Project Charter,” accessed 20 Feb 2013 (http://wiki.siframework.org/Automate+Blue+Button+Project+Charter)77 Office of the National Coordinator for Health IT, “Blue Button Mash Up Challenge,” September 201278 Slide from Fridsma79 Blue Button+ Implementation Guide80 Office of the National Coordinator for Health IT, S&I Framework, Automate Blue Button Initiative, accessed 1 April 2013 (http://wiki.siframework.org/Automate+Blue+Button+Initiative)81 Office of the National Coordinator for Health IT, Office of Science & Technology, Interview, March 28, 201382 Office of the National Coordinator for Health IT, S&I Framework, Frequently Asked Questions, accessed 1 April 2013 (http://wiki.siframework.org/Frequently+Asked+Questions+%28FAQs%29)83 Author’s calculations based on information available at http://www.healthit.gov/policy-researchers-implementers/federal-advisory-committees-facas-landing/calendar-list/2013-03?tid=12584 Department of Health and Human Services, “Open Government Plan Version 2.0” April 2012 (Pg. 44)85Office of the National Coordinator for Health IT, Federal Health IT Strategic Plan 2011-2015, accessed 10 April 2013 http://www.healthit.gov/policy-researchers-implementers/health-it-strategic-planning86 Blumenthal, D. “ONC Seeks Comment on the Federal Health IT Strategic Plan 2011-2015” ONC Buzz Blog, March 25, 2011 accessed 10 April 2013 http://www.healthit.gov/buzz-blog/from-the-onc-desk/hit-strat-plan/87 Office of the National Coordinator for Health IT, Planning Room, Accessed 10 May 2013 http://planningroom.org/learn-more/#how-does-effective-commenting-work88 Mostashari, F. “Applying Innovation to the Work of Government: A Case Study of the Office of the National Coordinator for Health IT” Institute of Medicine, June 15, 2012 accessed 10 May 2013 http://iom.edu/Global/Perspectives/2012/ApplyingInnovation.aspx