Workgroup Discussion Implementation, Certification, and Testing Workgroup Elizabeth Johnson, Co-Chair Christopher Ross, Co-Chair January 15, 2015.

Workgroup Discussion

Implementation, Certification, and Testing Workgroup

Elizabeth Johnson, Co-ChairChristopher Ross, Co-Chair

January 15, 2015

Health IT Congressional Provision

• December 16, 2015 continuing resolution omnibus spending bill directs ONC to, “take steps to decertify products that proactively block the sharing of information because those practices frustrate congressional intent, devalue taxpayer investments in (certified EHR technology) and make CEHRT less valuable and more burdensome for eligible hospitals and eligible providers to use.”

• ONC Requirements– ONC plan due to congress [by March 2015]

• Detailing the data sharing including estimate of the number of vendors or eligible hospitals and professionals who block information and a plan for addressing the issue

– Health IT Policy Committee [will] submit a report by [December 2015]• …on challenges and barriers to interoperability including: the technical, operational and

financial barriers to interoperability, the role of certification in advancing or hindering interoperability across various providers, as well as any other barriers identified by the Policy Committee

22Source: http://www.ihealthbeat.org/articles/2014/12/17/obama-signs-federal-spending-package-with-health-it-implications*http://docs.house.gov/billsthisweek/20141208/CPRT-113-HPRT-RU00-HR83sa.pdf

Other Health IT Provisions within the Bill

Agency ChargeVeterans Affairs (VA) • Report to Congress on progress in updating EHR systems

• Various IT, including funding for the development of a standard data reference terminology model

• Flexibility to designate funds allocated for "Medical Services" and "Medical Support and Compliance" to enhance its VistA EHR system and interoperability

• Prohibits VA from spending more than 25% of allocated funds until it reports on its plan to achieve interoperability with DOD's health systems

Department of Defense (DoD)

• Report to Congress on progress in updating EHR systems• Like VA, will only will be able to spend 25% of the funding allocated to its

Defense Health Program until it reports on its plan to achieve interoperability with VA's health system, including the implementation timeline and cost estimate of its new EHR system

Health Resources and Services Administration (HRSA)

• Help small, rural hospitals adopt health IT• Fund telehealth

Department of Agriculture

• Fund broadband transmission in rural areas for telehealth and distance learning programs

3Source: http://www.ihealthbeat.org/articles/2014/12/17/obama-signs-federal-spending-package-with-health-it-implications*http://docs.house.gov/billsthisweek/20141208/CPRT-113-HPRT-RU00-HR83sa.pdf

December 4, 2014 Homework Assignment: Comments and feedback from the May 7, 2014 Certification Hearing

Implementation, Certification, and Testing Workgroup

January 15, 2015

Homework Assignment & Objective

• Assignment– Review the certification hearing testimony– Identify opportunities for improvement based on your own

experiences– Come to January 15, 2015 (today’s call) meeting additional,

potential ideas the workgroup could consider recommending to ONC to enhance the certification program

• Purpose– Review the feedback received in this assignment and during the

May 2014 Certification Hearings to summarize lessons learned in order to have them clearly in mind as the workgroup receives and responds to the proposed Certification Rule

5

Overview of Comments Received

• Feedback received included observations and recommendations• Topics

1. Interoperability2. Usability and Workflow3. Burden of Implementation4. Guidance5. Variation in Implementation6. Parsimony7. Testing Tools & Resources8. Testing Procedures

• Out of Scope Comments– Several comments received related to topics outside of the scope of the

Certification Program were not included

6

Observations

• Interpretation of the requirements have changed• Certification efforts have shifted in focus from their

original intent of driving adoption to an obligatory regulatory process

• The regulatory process seems to be driven by calendar rather than higher level objectives resulting in – Certification requirements and testing tools are not mature

or complete at release– Lack of visibility into future requirements which leads to

reduced engagement– Insufficient time to effectively implement

7

Recommendations

• Promote interoperability– Prioritize interoperability in certification criteria– Decrease specification on implementation and more on outcomes– Align various certification standards to support eCQM reporting– Extend interoperability focus and testing to include data exchange between acute

care, ambulatory, and LTPAC • Improve usability and workflow

– Promote certification criteria that facilitates feedback loops– Acknowledge of industry product reviews (e.g., KLAS or others)– Require EHRs to undergo standardized usability test (e.g.,

http://www.measuringu.com/sus.php), at a minimum, publish those scores – Focus certification requirements on core needs– Avoid requiring functionality that passes certification but isn’t useful to product

consumers– Explore standardized workflows– Consider certification criteria to include workflow engine functionality

8

Recommendations

• Improve guidance– Improve audit guidance– Improve implementation assistance and guidance– Review guidance with all stakeholders prior to release to determine if

any re-testing is required

• Reduce variation in implementation– Tighten certification specifications– Perform testing on real-world implementations

• Pursue parsimony– Minimize certification and attestation requirements to a few key priority

areas– ONC needs to be parsimonious in terms of being prescriptive on what a

vendor “must” do to meet certification

9

RecommendationsTesting Tools & Resources

• Expand tools– Assure there are tools or systems in place to test all options available

for EHs and EPs– Investigate the use of hubs prior to creating interoperability

requirements that might entail significant costs– Include a rotating number of currently certified EMR/EHR vendors in

pilots of new test procedures

• Automate and streamline– Increase automation of testing tools to publish thousands of test decks

per “testable procedure” in an automated fashion that were consumable

– Simplify the CHPL listing - condense the CHPL to show one listing with all of the items available in the version

10

RecommendationsTesting Procedures

• Improve efficiency– Consider deeming for functionality that is already certified elsewhere (e.g., e-Rx tools)

• Increase clarity– Review certification and attestation requirements for ambiguity and address prior to

releasing final requirements

• Enhance quality– Address data mapping issues prior to specification of code sets– Ensure accuracy, clinical validity and relevance, and efficiency of data entry to reduce

time spent prepping the database for testing and time spent actually testing– Involve all stakeholders in creation of testing scripts – Pilot test scripts, methods, data sets, procedures, and testing tools with

• Use a sample of vendors of different size, scope, and architecture.

11

RecommendationsEase Burden of Implementation

• Prioritize and understand benefits– Minimize certification and attestation requirements to a few key priority areas– Understand the business case for each requirement

• Data use, beneficiary of data collection• Improvements in efficiency or outcomes• ROI• Workflow• Unintended consequences that might occur

• Consider level of effort and product development time– Assess the impact of new requirements on documentation burden and time

impacts to enter, track, and report data– Allow adequate time for implementation– More time for vendors to develop safe and efficient products including the new

functionality– Do not require all EHR users to have to upgrade in an unrealistically short time

12

Out of Scope Comments

• Broaden MU Incentive program to include long term care and post-acute care providers

• Consider incorporating telehealth and mobile Health encounters as part of MU to support access to care

• Quality control needs to be improved by those answering the CMS e-mail questions. We have numerous examples of completely opposite answers to the same questions

• Meaningful use objectives and quality measures must be aligned with each other, with certification requirements, and with clinical practice

13

Discussion

14

Certification Hearing ResultsMay 7, 2014

December 4, 2014

1616

Certification HearingFACA member attendees

• Paul Tang, MD, MS - Chair – Vice President, Chief Innovation & Technology Officer – Palo Alto Medical Foundation

• Michael H. Zaroukian, MD, PhD, FACP, FHIMSS – Co-Chair – Vice President & Chief Medical Officer – Sparrow Health System

• Carl D. Dvorak – Chief Operating Officer – Epic Systems

• Paul Egerman – Businessman/Software Entrepreneur

• Jennie Harvell PhD – Senior Policy Analyst – Department of Health & Human Services/ Office of Disability Aging & Long-Term Care Policy

• Joe Heyman, MD – Whittier IPA

• George Hripcsak MD, MS - Chair, Department of Biomedical Informatics – Director, Medical Informatics Services, Columbia University

• David Kates - Senior Vice President, Clinical Strategy - NaviNet

• Michael Lincoln - Veterans Health Administration• Nancy J. Orvis, MHA, CPHIMS – Director, Business Architecture & Interoperability – Department of Defense

• Marc Probst – Vice President & Chief Information Officer – Intermountain Healthcare

• Donald W. Rucker, MD, MS, MBA – Associate Dean for Innovation – Ohio State University Wexner Medical Center Corporation

• John Travis, FHFMA, CPA – Senior Director & Solution Strategist, Regulatory Compliance – Cerner Corporation

• Charlene Underwood, MBA – Senior Director, Government & Industry Affairs – Siemens Medical

• Larry Wolf – Senior Consulting Architect – Kindred Healthcare

1717

Hearing Purpose

• Understand what program features work well• Assess challenges • Identify opportunities to improve the

program, leveraging lessons learned in stages 1 and 2

1818

Hearing Panels

• Panel 1: Providers• Panel 2: Vendors• Panel 3: Certification/Accreditation Bodies• Panel 4: Private Sector Representatives

1919

Panel 1: Providers

• Ginny Lorenzi, NewYork-Presbyterian Hospital• Chad Jensen, LaTouche Pediatrics, LLC• John Berneike, Utah HealthCare Institute• Colin Banas, Virginia Commonwealth University Health System• Howard Hays, Indian Health Service• Cletis Earle, CHIME

• Questions– Assuming we could design an ideal program, what is the benefit of having a certification program,

from the perspective of your organization? How does a certification program help you? What are you looking for from a certification program?

– Currently certification indicates that certain capabilities exist in an EHR, would certification ever indicate a level of quality?

– What are the challenges you have experienced with the current certification program?– How would you design a certification program that would achieve the benefits you seek, while

minimizing the burden to the participants?

2020

Key pointsPanel 1: Providers

• EHR products may meet certification criteria, but the way the functions are implemented may disrupt workflow– Some functions fulfill the letter of the criteria, but not the intent (e.g.,

clinical summaries, patient education)– Some functions are implemented as “check the box”

• May be easy for vendor, but creates burden for provider and less useful

• Providers feel constrained to use products as certified, with inefficient work flow– Vendors be given enough flexibility to meet the rules without being

constrained to a particular workflow

• Some certified products do not work, or do not work in all states

2121

Key pointsPanel 1: Providers (II)

• Certification does not adequately cover interoperability

• Certification program should be less prescriptive; focus on what and less on how

• More flexibility and time for implementation is needed

• An ideal certification program would provide product comparisons in terms of their functionality

2222

Panel 2: Vendors

• Mickey McGlynn, EHRA• Sasha TerMaat, Epic• Emily Richmond, PracticeFusion• Joseph Geretz, SRSsoft• Sarah Corley, NextGen Healthcare Systems• Marc Probst, Intermountain• John Halamka, Beth Israel Deaconess Medical Center

• Questions– Assuming we could design an ideal program, what is the benefit of having a

certification program, from the perspective of your organization?– What are the challenges you have experienced with the current certification program?– How would you design a certification program that would achieve the benefits you

seek, while minimizing the burden to the participants?

2323

Key points Panel 2: Vendors

• Complete set of requirements are not provided with adequate time for development – Requirements change from what was originally defined in the

certification rule impacting quality and usability• Certification criteria for MU objectives, reports that measure these

objectives, and the clinical quality measures are not aligned with each other and are not necessarily aligned with clinical practice

• The testing tools and associated data are not properly tested before they are rolled out for use in the vendor community, and change

• Recommend that the complexity of the program be reduced and that a Kaizen process be used to support an effective review of the certification program

• Focus certification on critical few elements (e.g., interoperability, CQMs)

2424

Panel 3: Certification/Accreditation Bodies

• Amit Trivedi, ICSA Labs• Kyle Meadors, Drummond Group - VIRTUAL• Mark Shin, InfoGard

• Questions – Assuming we could design an ideal program, what is the benefit of

having a certification program, from the perspective of your organization?

– What are the challenges you have experienced with the current certification program?

– How would you design a certification program that would achieve the benefits you seek, while minimizing the burden to the participants?

2525

Key PointsPanel 3: Certification/Accreditation Bodies

• Pilot test new procedures and test tools prior to publication• Improve consistency between testing labs. Pilot tests should be a venue

for all ATLs and ACBs to:– Observe testing to understand the expected results– Learn how the test tools operate– Provide feedback to ONC

• Testing tools need to be more automated to efficiently handle more:– Test cases, reuse test data sets, and;– Employ more robust types of testing methodologies including testing the

security of products.• Focus on certification criteria related to interoperability and security

testing• How EHRs handle various functionality should be left to developers to

innovate

2626

Panel 4: Private sector representatives

• Alisa Ray, CCHIT • David Kibbe, Direct Trust• Chris Carr, IHE• Jitin Asnaani, CommonWell • Mariann Yeager, Healtheway

• Questions– If you could design your ideal program, what would be the benefit of having

a certification program, from the perspective of your organization?– What are the challenges you have experienced with the current certification

program?– How would you design a certification program that would achieve the

benefits you seek, while minimizing the burden to the participants?

2727

Key PointsPanel 4: Private sector representatives

• Need additional up-front testing and quality assurance

• Mid-cycle revisions are disruptive to the overall program

• Need subject matter experts in program development

• Enhanced collaboration between the private sector and the federal government would help

• Focus on critical few

2828

Overall Summary

• No disagreement about intent of MU objective• Insufficient time for product development and testing• Concerns about certification include:

– Specificity of certification criteria locking in vendor-created inefficient workflows for providers

– Incompletely tested and unstable testing tools delays certification and creates rework

– Inconsistent interpretations among ATLs, ACBs, and auditors– Certification does not guarantee integrated product or interoperability

– No clearinghouse for timely feedback and response (i.e., need faster

than NPRM cycles)

• Time required for certification (or documenting certification) crowds out innovation

2929

Final Recommendations (I)Kaizen

• Motivation: current process is inefficient and burdensome• Objective: Need a coordinated, integrated, well understood

certification process with minimum of burden• Recommendation: Kaizen covering end-to-end certification

process from translation of MU objective to certification criteria to development of testing scripts to development (and QA) of testing tools to conduct of test to auditing

• Involve broad stakeholders from providers to developers to ATLs/ACBs to auditors

• Establish certification roadmap and timelines• Create a timely PDCA mechanism for feedback and

continuous improvement

3030

Final Recommendations IIFocus

• Motivation: providers and vendors (and ACBs) feel overwhelmed by the pace of changing requirements

• Objective: Focus on critical few• Limit scope of certification to:

– Interoperability– CQMs– Privacy and security

• These require– Cross-organizational collaboration (and policy interoperability)– Alignment of standards, measures, and programs– Overarching governance– Public-private collaboration