Staff Perception Survey before and after EHR/CPOE Implementation Jean Loes Marcia Ward, Douglas Wakefield, John O’Brien.

Staff Perception Survey before and after EHR/CPOE ImplementationJean LoesMarcia Ward, Douglas Wakefield, John O’Brien

Introduction

• Among the most notable challenges to implementing clinical information systems are the varying levels of acceptance and use by healthcare providers and employees: – Research has shown that experiences shape the degree to which users

will accept the technology initially (Dixon, 1999; Herbert & Benbasat, 1994).

– Research has shown that users’ attitudes regarding risks to service quality and disruptions in workflow hinder implementation (Hu, et al., 2002; Zheng, et al., 2005).

• All of the significant models of information technology use suggest that perceptions of the impact on work and outcomes are significant determinants of technology use and adoption (Kufafka, et al., 2003).

Copyright © 2006 Trinity Health2


Background on Measures

• Measures have been created to explore attitudes toward technology including perceived usefulness and ease of use (Davis, 1989).

• Recently we developed a measure of healthcare worker perceptions of the effect of clinical information systems on workflow processes and outcomes (Wakefield et al., 2007).

• This Information Systems Expectations and Experiences (I-SEE) survey was administered before and after “Go-Live” of a comprehensive system change in several hospitals.

• Analysis of the survey factor structure identified scales that assessed respondents’ perceptions related to communication changes, changes in selected work behaviors, perceptions of the implementation strategy, and the impact on quality of patient care (Wakefield et al., 2007).

Objective

• The purpose of this study is to examine changes in perceptions about quality before and after implementation of a comprehensive clinical information system.

• This study is the first to explore these changes in multiple small hospitals and across various employee categories.


Survey Methods

• The I-SEE survey instrument was modified. Respondents were asked to indicate their current perceptions at each wave.

• The survey was administered at three times:– Wave 1 – March 2007 – before any changes– Wave 2 – March 2008 – during training and preparation– Wave 3 – March 2009 – after implementation

• Implementation “Go-Live” occurred:– July 2008 for 3 hospitals– September 2008 for 4 hospitals


Survey Responses Number and (Response Rate)


Hospital

WAVE 12007

WAVE 22008

Wave 32009

Ellsworth Municipal Hospital

73 (28%) 59 (23%) 41 (16%)

Franklin General County Hospital

93 (51%) 56 (31%) 43 (24%)

Hancock County Memorial Hospital 79 (65%) 63 (52%) 34 (28%)

Kossuth Regional Health Center

54 (28%) 47 (24%) 35 (18%)

Mercy Medical Center - New Hampton

58 (50%) 62 (54%) 48 (42%)

Mitchell County Regional Medical Center 45 (21%) 49 (23%) 44 (21%)

Palo Alto County Health Systems

90 (50%) 42 (23%) 49 (27%)

Total Returned Surveys 511 (40%) 385 (30%) 305 (24%)

Survey Items with an Average Response of “Strongly Agree”

Survey ItemsOverall Mean

Response

31. Patient care is consistently given according to the “9 Rights” 5.35

25. I enjoy my job 5.30

14. Overall patient care is safe in the areas I work 5.26

1. Access to information to make good patient care decisions is available

5.12

18. I get a great deal of professional satisfaction from my job 5.12

29. Communications ensuring high quality and safe patient care routinely occur when patients are transferred to other facilities

5.05


Items that Changed across Waves


Number of Respondents in Each Staff Group


Professional/Employee Groups Number

Providers (physicians and midlevel providers)

49

Registered Nurses 343

Other-Clinical (professional, technical, patient support)

458

Other-NonClinical (clerical, senior management, marketing)

293

Comparison across Staff Groups

• To explore whether the staff groups differed, we compared their responses across the three waves.– The Other – NonClinical group was excluded because they had

minimal if any contact with the clinical information system and were not directly involved in patient care.

• Significant interactions were found for three survey items shown on the next slides. The pattern from before to after implementation was:– Other – Clinical respondents showed no change or increases in

survey responses after implementation– Registered nurses showed no change in survey responses after

implementation– Providers showed sizable decreases in survey responses after

implementation.


Significant Differences among Staff Groups






Comparison between Physicians and Mid-level Providers

• To further explore whether subsets of the provider group differed, we compared their responses across the three waves.– Physicians included 11 supervisory and 13 nonsupervisory physicians.

– Mid-level Providers included 22 nurse practitioners, physicians assistants, CRNAs, etc.

• Significant interactions were found for four survey items. As shown on the next slide, the pattern from before to after implementation was:– Mid-level Providers showed increases in survey responses after

implementation to two survey items

– Mid-level Providers showed no change in survey responses after implementation to two survey items

– Physicians showed increases in survey responses after implementation to two survey items

– Physicians showed decreases in survey responses after implementation to two survey items.


Significant Differences between Physicians and Mid-Level Providers


CPOE Rates after Implementation


Relationship between Survey Items at Wave 3 and CPOE Use Rates

• Correlations between survey items at Wave 3 and CPOE use rates indicate that staff at hospitals with higher CPOE use rates also tended to respond more in agreement to three survey items: – “Patients are rarely asked the same questions by the staff” (r=.93) – “Access to information to make good patient care decisions is

available” (r=.79) – “I get a great deal of professional satisfaction from my job” (r=.73)

• The strongest relationship indicated that staff at hospitals with higher CPOE use rates tended to strongly disagree with the survey item “Too many verbal orders are made on my unit” (r= -.96)


Relationship between Survey Items at Wave 2 and CPOE Use Rates

• Correlations between survey items at Wave 2 and CPOE use rates indicate that staff at hospitals with higher CPOE use rates tended to respond more in agreement to three survey items before Go-Live:– “I spend about the right amount of time documenting

patient care” (r=.69)– “Patients are rarely asked the same questions by the

staff” (r=.63)– “Overall patient care is safe in the areas I work” (r=.59)

• Thus, higher agreement with these items may predict CPOE use rates after implementation.


References

• Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and use acceptance of information technology. MIS Quarterly, 13 (3), 318-340.

• Dixon, D. R. (1999). The behavioral side of information technology. International Journal of Medical Informatics, 56, 117-123.

• Halbesleben JRB, Wakefield DS, Ward MM, Brokel J, Crandall D. (2009). The relationship between super users’ attitudes and employee experiences with clinical information systems. Medical Care Research & Review, 66: 82-96.

• Hebert, M., & Benbasat, I. (1994). Adopting information technology in hospitals: the relationship between attitudes/expectations and behavior. Hosp Health Serv Adm, 39 (3), 369-383.

• Hu, P. J., Chau, P. Y. K., & Sheng, O. R. L. (2002). Adoption of telemedicine technology by health care organizations: An exploratory study. Journal of Organizational Computing and Electronic Commerce, 12 (3), 197-221.

• Kukafka, R., Johnson, S. B., Linfante, A., & Allegrante, J. P. (2003). Grounding a new information technology implementation framework in behavioral science: A systematic analysis of the literature on IT use. Journal of Biomedical Informatics, 36, 218-227.

• Wakefield DS, Halbesleben JRB, Ward MM, Qiu Q, Brokel J, Crandall D. (2007) Development of a measure of clinical information systems expectations and experiences. Medical Care, 45: 884-890.

• Zheng, K., Padman, R., Johnson, M. P., & Diamond, M. S. (2005). Understanding technology adoption in clinical care: clinician adoption behavior of a point-of-care reminder system. International Journal of Medical Informatics. 74(7-8), 535-543.


Acknowledgments

• This work was supported by funding from AHRQ – 5UC1HS016156 – “EHR Implementation for the Continuum of Care in Rural Iowa”

• Mercy Medical Center – North Iowa• Mercy North Iowa Network• The University of Iowa – Center for Health Policy and

Research• The University of Missouri – Center for Health Care

Quality


Staff Perception Survey before and after EHR/CPOE Implementation Jean Loes Marcia Ward, Douglas Wakefield, John O’Brien.

Documents