Marti A. Hearst Professor, iSchool, UC Berkeley February 8, 2012 The views expressed in this talk are my own and do not necessarily represent those of.

HOW BILLIONS IN GOVERNMENT SAVINGS IS LIKELY TO RESULT

FROM CURRENT ADMINISTRATION IT REFORMS

Marti A. HearstProfessor, iSchool, UC Berkeley

February 8, 2012

The views expressed in this talk are my own and do not necessarily represent those of any organization.

Intro

Goal: enabling a high-performance, citizen centered, 21st century government.Govt should be capitalizing on recent IT

advances (mobile, social, etc)$80 Billion Federal Govt IT portfolio

○ Over $500 billion in contracts and orders totalMany IT projects fail; how to fix this

Talk Structure

Goal: IT to help enable a 21st Century Government

Many different parallel tracks:Citizen / customer focusUse the latest technology advancesReduce Waste and CostsImprove How Large IT Projects are Built

Improving Large IT Projects The Cost of Failed IT Why Large IT Projects Fail Generally Why they Fail in Govt in Particular How To Help Govt IT Projects Succeed How YOU Can Help

Intro

Goal: enabling a high-performance, citizen centered, 21st century government.

80 Billion IT portfolio Govt should be capitalizing on recent IT

advances (mobile, social, etc) Many IT projects fail; how to fix this

http://www.cio.gov/pages.cfm/page/Chapter-19-Information-Technology-Page-1FY 2011 President's Budget,  Analytical Perspectives, Special Topics,  Chapter 19

Costs of Failed IT

Why Software Fails, R. Charette, IEEE Spectrum Sept 2005

YEAR COMPANY SYSTEM COST (US $)

2005 Hudson Bay Co. [CA] Inventory Control $33.3 million loss

2004-05 UK Inland Revenue Tax Credit $3.45 billion overpayment

2004 Avis Europe PLC [UK] Enterprise Resource Planning $54.5 million, cancelled

2004 Ford Motor Co. Purchasing system $400 million, abandoned

2004 J Sainsbury PLC [UK] Supply-chain management $527 million, abandoned

2004 Hewlett-Packard Co. Enterprise Resource Planning $160 million loss

2003-04 AT&T Wireless Customer Relations Management $100 million revenue loss

2002 McDonald’s Corp. Purchasing System $170 million, cancelled

Costs of Failed IT

The cost itself. The opportunity cost:

FAA Air Traffic control system failure:○ $2.6 Billion Spent (1984-1994)○ Cumulative cost of delays estimated at $50 B.

Can jeopardize an organization’s future.FoxMeyer Drug Co., worth $5 billion,

plummeted into bankruptcy in 1996.

In search of the future of air traffic control, T.S. Perry, IEEE Spectrum 34(8), 1997.

Why Software Fails, R. Charette, IEEE Spectrum Sept 2005

Frequency of Failed IT 15-20% failure rate for projects with budgets > $10 million

(Charette, 2005) From Standish Group CHAOS Report 2009 (via ProjectSmart):

http://www.projectsmart.co.uk/the-curious-case-of-the-chaos-report-2009.html

Why Software Fails, R. Charette, IEEE Spectrum Sept 2005

Success rating achieved if the projects were completed on time, on budget and with required features and functions (met user requirements).Challenged rating if late, over budget and/or with less than the required features and functions.

“Adding [programmers] to a late software project makes it later.”

Frederick Brooks, The Mythical Man Month, 1975

Some cost contributors to large-scale programs, Nanus & Farr, AFIPS 1964 (exponent of 1.5)

Why Large IT Projects Fail

Unrealistic or unarticulated project goals Inaccurate estimates of needed resources Badly defined system requirements Poor reporting of the project's status Unmanaged risks Poor communication among developers, and users Use of immature technology Inability to handle the project's complexity Sloppy development practices Poor project management Stakeholder politics Commercial pressures

Why Software Fails, R. Charette, IEEE Spectrum Sept 2005

Why Did Your Project Fail? Cerpa & Verner, CACM 52(12), 2009

Large Govt IT Projects According to the Government Accountability Office

(GAO):Planned federal IT spending > $81 billion for fiscal year

2012.Many are critical to the health, economy, and security of the

nation. Unfortunately, these frequently fail or have overruns

while yielding little value. Recent Examples:

After $127M over 9 years on an outpatient scheduling system, the VA had to start over.

After $40M and 7 years, FEMA canceled insurance and claims system as it did not meet user needs.

Why Do They Fail? Oversight and funding models make modern

IT design methods nearly impossible. Problems related to contracting. Lack of IT expertise by IT project leaders. Lack of focus from executives and

changeover in leadership. Lack of a user-centered design tradition. And serious legal hurdles.

Legal Hurdles to Adopting Off-the-Shelf Well-Designed Software

TOS PII OWACS COOP 508 Cookies NARA

Legal Hurdles to Adopting Off-the-Shelf Well-Designed Software

TOS (Terms of Service) PII (Personally Identifiable Information) OWACS (Security requirements) COOP (Disaster recovery requirements) 508 (Accessibility Standards) Cookies (Persistent Cookie Restrictions) NARA (National Archives and Records)

Challenges for Large Govt IT Projects Procurement Rules & Myths Waterfall Development

Not User-Centered, Not Agile Oversight

Instead of Coaching / Advice Personnel

IT and Procurement Legacy Systems and Data

UCD Legal Hurdles Paperwork Reduction Act

Goal: ensure that information collected is useful and minimally burdensome for the public.

To ask a question of 10 or more people, must:Clear internal legal process, then60 days in the Federal Register

○ May need to iterate based on commentsSubmit documents to OMBAnother 30 day federal register notice period~60 days for OMB review

○ May need to iterate and even start again

Usually a 6 month process.

New PRA Fast Track OMB clearance in 5 days! Applies to information collections

that focus on the awareness, understanding, attitudes, preferences, or

experiences of stakeholders relating to existing or future services, products, or

communication materials. Specifically includes:

Focus groups Remote usability testing Online surveys for customer feedback purposes

However, public distribution of results cannot be intended.

Plain Writing Act of 2010

The purpose of this Act is to improve the effectiveness and accountability of Federal agencies to the public by promoting clear Government communication that the public can understand and use.

The term ‘‘plain writing’’ means writing that is clear, concise, well-organized, and follows other best practices appropriate to the subject or field and intended audience.

Applies to any document that: Is necessary for obtaining any Federal Government benefit Provides information about any Federal Government benefit or

service Explains to the public how to comply with a requirement the

Federal Government administers or enforces;

Plain Writing Act of 2010

The purpose of this Act is to improve the effectiveness and accountability of Federal agencies to the public by promoting clear Government communication that the public can understand and use.

The term ‘‘plain writing’’ means writing that is clear, concise, well-organized, and follows other best practices appropriate to the subject or field and intended audience.

Applies to any document that: Is necessary for obtaining any Federal Government benefit Provides information about any Federal Government benefit or

service Explains to the public how to comply with a requirement the

Federal Government administers or enforces;

Plain Writing Act of 2010

From www.plainlanguage.gov: BEFORE

Plain Writing Act of 2010

From www.plainlanguage.gov: AFTER

Executive Order on Streamlining Service Delivery and Improving Customer Service Follows on memos from 1993, 1995, and 1998. “With advances in technology and service delivery

systems in other sectors, the public's expectations of the Government have continued to rise.  The Government must keep pace with and even exceed those expectations.”

Customer service plans to include:At least one major initiative using IT to improve the

customer experience;Mechanisms to solicit customer feedback on

Government services and use such feedback regularly to make service improvements;

Streamlining agency processes to reduce costs and the need for customer inquiries.President's Executive Order (EO) 13571, on "Streamlining Service Delivery and

Improving Customer Service,” April 2011.

Success Factors: A New Report from the Government Accountability Office

GAO-12-7, Information Technology, Critical Factors Underlying Successful Major Acquisitions ,October 2011, http://www.gao.gov/new.items/d127.pdf

Mapping Success Factors to Current IT Reforms

1. Program officials were actively engaged with end users.2. Program staff had the necessary knowledge and skills.3. Senior department and agency executives supported the programs.4. End users were involved in the development of requirements.5. End users participated in testing of system functionality prior to formal end user acceptance testing.6. Government and contractor staff were stable and consistent.7. Program staff prioritized requirements.8. Program officials maintained regular communication with the prime contractor.9. Programs received sufficient funding.

GAO-12-7, Information Technology, Critical Factors Underlying Successful Major Acquisitions ,October 2011, http://www.gao.gov/new.items/d127.pdf

User-centered Design/Agile IT Staffing Procurement Governance/Accountability

Administration Initiated IT-Related Reforms, 2009-2011

Reforming Major Government Operations IT Reform Procurement Reform Hiring Reform Regulation Reform Waste Reduction Open Data / FOIA

A Focus on IT Emphasis on Leadership 25 Point Plan for Reforming IT Promoting New Standards

An Emphasis on Users Customer Service Executive Order Plain Language Act New PRA Guidelines Federal Government Web Site Reform Modern Communication Techniques

○ Leading with High-quality Web Site Design○ Encouraging Information Visualization○ Deploying Light-weight online discussion tools

“He’ll sit here and he’ll say ‘Do this! Do that!’ And nothing will happen.”

Harry S. Truman, On Presidential Power, quoted in Neustadt, R.R., Presidential Power, Wiley 1960.

Administration Initiated IT-Related Reforms, 2009-2011

A Focus on IT Emphasis on Leadership 25 Point Plan for Reforming IT Enabling use of SaaS and Cloud Services Encouraging Challenges and Prizes

An Emphasis on Users Customer Service Executive Order Plain Language Act and EO New PRA Guidelines Federal Government Web Site Reform Modern Communication Techniques

○ Leading with High-quality Web Site Design○ Encouraging Information Visualization○ Light-weight online discussion tools

Reforming Major Government Operations IT Reform Procurement Reform Hiring Reform Regulation Reform

Strategy

Embrace modular development, Build on open standards, and Work with Congress to modernize IT

funding rules.

Steve VanRoekel, http://www.whitehouse.gov/sites/default/files/svr_parc_speech_final_0.pdf

Fixing Govt IT Development

First deliverables produced years after work beginsBy then, program sponsors and end-user

needs have changed.Should have 18-24 month duration

25 Point Plan to Reform IT

IT Reforms

Emphasis on LeadershipElevating Federal CIO, naming CTO

25 Point Plan for Reforming Federal IT Enabling use of SaaS and Cloud

Services http://www.whitehouse.gov/21stcenturygov/ http://www.whitehouse.gov/21stcenturygov/actions/

reforming-government-contracting#ethics-menu

IT Reform (key document) http://www.whitehouse.gov/blog/2010/11/19/driving-it-reform-update Tackling the information technology gap  between the public and private sectors is one of most effective ways

we can make government work more effectively and efficiently for the American people. IT has been at the center of the private sector’s productivity gains, but for too long Federal IT projects have run over budget, behind schedule, or failed to deliver what on their promise. That’s why fixing IT is a cornerstone of the President’s Accountable Government initiative.

Second, since far too many financial system modernization projects were running behind schedule and over budget, we halted all new work on those projects pending review and approval by OMB. Across the government, over 30 financial systems projects, with budgets totaling $20 billion, were affected by this policy.

Our review of 20 agencies’ projects is now complete, and I am proud to report that we have taken steps to save $1.6 billion on these projects.

Through our reviews, we determined that half the projects were basically on track. Of the half that were not, we took the following actions:

At two agencies – the Department of Housing and Urban Development and the Environmental Protection Agency  -- we pulled forward meaningful functionality, resulting in almost $230 million in budget reductions.

At two agencies – the Department of Veterans Affairs and the Small Business Administration –  we canceled their projects as a result of the review, resulting in over $500 million in budget reductions.

At three agencies – the Departments of Homeland Security, Justice, and Health and Human Services, we are moving forward with plans to decrease the scope of and improve their financial system projects, resulting in reduced costs and a greater focus on critical business needs.  This revaluation of these projects resulted in over $680 million in budget reductions. 

An additional $200 million in budget reductions was identified in various agencies, with more to come.

  (Zients, Nov 19, 2010)

Procurement Reform http://www.whitehouse.gov/blog/2011/02/04/turning-tide-contract-spending

  For the first time in 13 years, we have reduced spending on contracting and agencies have stopped the costly upward spiral in contract growth.  In FY 2010, agencies spent nearly $80 billion less than they would have spent had contract spending continued to grow at the same rate it had under the prior Administration. 

 In his State of the Union address, President Obama said that, “we can’t win the future with the government of the past.”

instead, he said we must reform the way we do business in Washington and give the American people a government that’s not only more affordable, but also more effective and more efficient. This principle has been the cornerstone of our work on contracting and across the Accountable Government Initiative. From reforming and cutting costly IT systems, implementing unprecedented transparency and reporting efforts, buying in bulk, establishing a government-wide Do Not Pay list, or moving toward electronic government payments, we’re making real progress in changing the way government does business.

Lew on February 04, 2011 http://www.whitehouse.gov/sites/default/files/contracting_reform.pdf http://www.whitehouse.gov/omb/procurement_default/

IT Reform (key document) CLOSING THE IT GAP AND MODERNIZING GOVERNMENT Twenty years ago, when people came to work for the

government, they had access to the world’s best technology. Today, government employees often have better technology at home than at work. This gap between the public and private sectors results in billions of dollars in waste, slow and inadequate customer service, and a lack of transparency about how dollars are spent. While a productivity boom has transformed private sector performance over the past two decades, the federal government has almost entirely missed this transformation and now lags far behind on efficiency and service quality. The Obama Administration has undertaken an IT reform and government modernization effort so that the government uses IT in a more efficient and cost- effective way http://www.whitehouse.gov/sites/default/files/

Background_on_Accountable_Government_Initiative.pdf

25 Point Plan for Reforming IT Issued Dec 9, 2010

Vivek Kundra, Steven VanRoekel Main Components:

Apply “Light Technology” and Shared SolutionsStrengthen Program ManagementAlign the Acquisition and Budget Processes with

the Technology CycleStreamline Governance and Improve

AccountabilityIncrease Engagement with Industry

Procurement Improvements Dan Gordon, Administrator for Federal

Procurement Policy MythBusting:

http://www.whitehouse.gov/sites/default/files/omb/procurement/memo/Myth-Busting.pdf

FAR: Federal Acquisition Regulation

More Information

Cio.gov http://www.whitehouse.gov/omb/

procurement_default/ Whitehouse.gov/21stcentury Agency open government pages /open

Waterfall

vs Agile

Images from Effective UI, Anderson et al, O’Reilly Media, 2010.

Design happens During Development The later you make a decision in a project, the more likely it is to be the best one by

virtue of having been made from a position of greater experience. Attempting to comprehensively define the functional requirements of a product on day

zero is absurd and futile; that’s the day when you know the absolute least about the product, and any decisions you make

on that day are very likely to be incorrect Development is where the majority of design happens, and design is the activity that

discovers unknown problems and their solutions, so development should begin as soon as possible.

Functional requirements and specifications are written before development begins, so they’re immediately handicapped by having been made from the least informed perspective.

Putting together written requirements and specifications is not entirely with- out value, though, so long as it’s perceived in the right way. The goal should not be to build a definitive description of the product, but rather to do a dry run of the product design, to get the team to start thinking through the prob- lems that lie ahead.

In preparing for battle, I have always found that plans are useless, but planning is indispensable. -- Dwight Eisenhower

from Effective UI, Anderson et al, O’Reilly Media, 2010.

Project Scope

Any estimation of scope, having been defined in the “mere twilight” of the project kickoff or contract negotiation, cannot possibly be accurate. And a comprehensive description of the scope of a project is so enormously complex that it simply can’t be done; the only perfect description of a product is the product itself.

from Effective UI, Anderson et al, O’Reilly Media, 2010.

Problems with Waterfall Because each step is entirely separate, each group of contributors is

entirely siloed from the others. The people brainstorming and writing requirements for the product never collaborate with the people architecting and designing it. The software engineers never have the opportunity to collaborate with the architects and designers, let alone the business managers and stakeholders.

This approach forces the engineering and quality assurance (QA) stages to absorb almost all of the effects of the risks and unknowns that arise during the project. Since the planning, architecture, and design of the product are already ostensibly complete, there’s no option for changes to them because the money for them has already been spent and the resources have been allocated to other things. This leaves it to the engineers to figure out how to account for the inevitable unforeseen problems and unknowns—and to do it within the budget and timeline they were allocated before the problems and unknowns were identified.

from Effective UI, Anderson et al, O’Reilly Media, 2010.

Advantages of Agile

Favor:Individuals and interactions over processes and

tools Working software over comprehensive

documentation Customer collaboration over contract

negotiation Responding to change over following a plan

from Effective UI, Anderson et al, O’Reilly Media, 2010.

25 Point Plan for Reforming ITCurrent status: cio.gov/modules/itreform

25 Point Plan for Reforming IT

25 Point Plan for Reforming IT

25 Point Plan for Reforming IT

Strengthen Program Management Design an IT management career path Enable mobility across govt and industry Launch a technology fellows program Require integrated program teams

Streamline Governance

Reform existing internal reviews Redefine role of agency CIOs TechStat reviews at bureau level

Techstat: face-to-face evidence-based review of an IT program with White House and agency leadership.

Fixing Govt IT Development

First deliverables produced years after work begins By then, program sponsors and end-user needs have

changed. Should have 18-24 month duration

Solution: “modular” development Must deliver functionality to users at least every 12 months Must regularly capture and incorporate user feedback

through an iterative process that assesses user satisfaction with each release, continually refining design.

Problem: government processes do not allow for this kind of timeline

○ Planning, budgeting, procurement all require that agencies do not do develop IT in this way.

From 25 Point Plan to Reform IT

25 Point Plan for Reforming IT

Procurement & Vendors

RULES Turn into MYTHS and HABITS

Other Angles Reduce Waste Initiative Prizes, Challenges Supporting Standards Efforts Entrepreneurs in Residence

a time-limited recruitment of world-class entrepreneurs and innovators to join highly-qualified internal employees in the development of new operational procedures in areas that impact innovation; the goal is to deliver transformational change by combining the best internal and external talent in testing, validating and scaling what works

Lean Government Startup Team

How YOU Can Help

Form Coaching Councils / IT Clinics Study and Report on Vendor Activity Publish Very Short How-To’s Respond to Government RFCs Help with Future First Initiative Work in Government!

Provide Advice

Agencies IT projects have LOTS of oversight, but NO coaching.

Solution: Unbiased advisors / coaches. Outside groups have to initiate these

and offer the services to government.

Study Vendor Practices

Compile statistics on:ProtestsInterventions by CongresspersonsVendors’ Success Rates

Investigate Relevant Research Problems Study and publish on how to make large

IT projects succeed.

(From EPA)

Sample Useful How-To on Social Media Usage

Example: The National Dialogue on Improving Federal Websites

Participate in Government Dialogues

Make How-To’s Independent non-profits, bloggers, university groups can

have real impact. They do this work on their own initiative.

Examples: Technology developed by OMB Watch for

fedspending.org used in relaunch of USASpending. Sunlight Foundation compiled lists of strategies for

govt to address the OGD. UC Berkeley iSchool faculty posted guidelines on

how to improve the design of recovery.gov; had a big influence.

Open gov how-to workshops and website

Example Idea: Usability ClinicProfessors teaching usability courses

○ Have their students critique a web site as a homework exercise

○ Commit to a particular time period Organizations sign up for the clinic

○ Govt, non-profits, small businessesMay turn into longer-term projects

Participate with Classwork

Build tools that use govt dataExpose inefficienciesCreate new, useful functions

Example:Analyze hiring latency on a per-agency basisData isn’t there?

○ Comment on agencys opengov websites○ Ask for time-to-hire data for each agency○ Be persistent if necessary

Participate with Data Analysis

Example:OSTP Request for CommentsFederal Register, May 21, 2009

http://www.fas.org/sgp/news/2009/05/opengov.pdf Also on the OSTP blog

Sought advice on Open Government topics:

What alternative models exist to improve the quality of decision making and increase opportunities for citizen participation?

What are the limitations to transparency?

What strategies might be employed to adopt greater use of Web 2.0 in agencies?

What policy impediments to innovation in government currently exist?

What performance measures are necessary to determine the effectiveness of open government policies?

Participate By answering Requests For Comment

A Recent Federal Register Request For comment

A brand new program!Started Fall 2011 2 year paid fellowship Rotate among agencies

Builds on the highly successful Presidential Management Fellows program Trains leaders for Federal Government Service Usually a terrific cohort

Must apply in the Fall before you graduate with a masters or PhD

But very few opportunities for non-US citizens

Become a Presidential Technology Fellow

It is still very difficult to successfully develop IT in government.

However, many reform efforts are greatly improving the situation. An IT-savvy administration A focus on usability and citizen input. The 25 point plan Additional changes in procurement guidance.

Academics, researchers, and practitioners can all help!

Summary