Health Literate Discharge Practices in Ontario Hospitals · ii Health Literate Discharge Practices in Ontario Hospitals Jennifer Anne Innis Doctor of Philosophy Institute of Health
Post on 02-Jul-2018
216 Views
Preview:
Transcript
Health Literate Discharge Practices in Ontario Hospitals
by
Jennifer Anne Innis
A thesis submitted in conformity with the requirements for the degree of Doctor of Philosophy
Institute of Health Policy, Management and Evaluation University of Toronto
© Copyright by Jennifer Anne Innis 2016
ii
Health Literate Discharge Practices in Ontario Hospitals
Jennifer Anne Innis
Doctor of Philosophy
Institute of Health Policy, Management and Evaluation
University of Toronto
2016
Abstract
Health literate discharge practices meet the health literacy needs of patients and families
at the time of hospital discharge. This dissertation used a mixed methods, sequential design to
gain insight into the use of these practices in Ontario hospitals.
The Health Literate Care Model was used to guide the first two studies. The first study
used a Delphi panel to find the best indicators of health literate discharge practices, based on the
indicators of Project RED (Re-Engineered Discharge). This led to the development of 36
indicators, which were contextualized to Ontario, and were used to create an organizational
survey tool. This survey was sent to to nursing managers in all 143 hospitals in Ontario. There
were 99 participants from 79 hospitals (participation rate 55%). Exploratory factor analysis was
done and reliability of the survey was established.
The third study used organizational learning theory to examine how acute care hospitals
take on health literate discharge practices by interviewing managers, educators and front-line
staff in 10 hospitals that participated in the survey.
In the fourth study, multiple regression analysis was used to examine the relationship
between organizational characteristics and use of health literate discharge practices, as
determined by the survey results. Smaller hospital size was associated with greater use, and
iii
survey scores were found to be highest in the North region. A significant interaction was found
between size and location.
The development of the organizational survey tool could be used by researchers, hospitals
and policy makers to measure and monitor the use of health literate discharge practices in
Ontario hospitals. The results of the qualitative study offer insights into how organizational
learning is used by hospital managers and leaders to adopt health literate discharge practices.
iv
Acknowledgments
Thank you to my co-supervisors, Jan Barnsley and Whitney Berta, for their support,
guidance and encouragement. To Imtiaz Daniel, my committee member, thank you for your
mentorship and support. My thanks to Rhonda Cockerill and Jennifer Gibson for their thoughtful
feedback. Thank you to my external examiner, Michael Paasche-Orlow, for challenging me. I
would like to thank all of the nursing managers and health care providers who participated in this
research.
As well, I wish to thank Tyrone Perreira and Monique Herbert for their friendship,
support and camaraderie. I am grateful to my sisters, Caroline and Sarah Innis, for believing in
me. I especially want to thank my parents, Hugh and Lorraine Innis, for their encouragement to
return to school, for always listening to me and for always being supportive – I could not have
done this without you.
v
Table of Contents
Acknowledgments (if any) .............................................................................................................. ii
Table of Contents ........................................................................................................................... iv
List of Tables ................................................................................................................................ vii
List of Figures ............................................................................................................................. viiii
List of Appendices ....................................................................................................................... viii
Chapter 1: Background ................................................................................................................... 1
Chapter 2: Health Literate Discharge Practices and Project RED ............................................... 99
Chapter 3: Development of Indicators of health literate discharge practices in Ontario
Hospitals Using a Delphi Panel ................................................................................................ 29
Chapter 4: Use of Health Literate Discharge Practices in Ontario Hospitals .............................. 51
Chapter 5: Absorptive Capacity and the Uptake of New Discharge Practices in Ontario
Hospitals ................................................................................................................................... 98
Chapter 6: Relationships between Organizational Characteristics and use of Health Literate
Discharge Practices in Ontario Hospitals ............................................................................... 155
Chapter 7: Adoption and Use of Health Literate Discharge Practices in Ontario Hospitals ..... 186
References ................................................................................................................................... 199
Appendices .................................................................................................................................. 222
Copyright Acknowledgements .................................................................................................... 475
vi
List of Tables
Table Page
Table 1. Roles of Participants on the Delphi Panel 31
Table 2. Indicators of Health Literate Discharge Practices for Round 1 32
Table 3. Roles and Work Location of Participants on the Delphi Panel (self-identified),
Round 1 37
Table 4. Median Values for Health Literate Discharge Practices – Round 1 37
Table 5. Themes from Round 1 40
Table 6. Roles of participants on the Delphi panel (self-identified), Round 2 43
Table 7. Health Literate Discharge Practices – Round 2 43
Table 8. Median Values for New/Revised Health Literate Discharge Practices
– Round 2 44
Table 9. Indicators of Health Literate Discharge Practices 45
Table 10. Baseline Characteristics of Participants and Hospitals 60
Table 11. Characteristics of Hospitals with More Than One Response 62
Table 12. Survey Results 64
Table 13. Reliability Testing for 9 Factor Solution 73
Table 14. Reliability testing for Five Factor Solution 77
Table 15. Factor Correlation Matrix for 6 Factor Solution 81
Table 16. Summary of Items and Factor Loadings for Direct Oblimin Oblique
Six Factor Solution 82
Table 17. Summary of Items and Factor Loadings Greater Than .3 for Six
Factor Solution 86
Table 18. Reliability Testing for Six Factor Solution 89
Table 19. Inter-rater Reliability Testing for Hospital with 2 Responses 92
Table 20. Inter-rater Reliability Testing for Hospital with More Than 2 Responses 92
vii
Table 21. Factors from 6 Factor Solution 93
Table 22. Indicators of Health Literate Discharge Practices with Corresponding
Factors 93
Table 23. List of participants 110
Table 24. Absorptive Capacity Metaroutines and Routines Used in the
Adoption and Use of Health Literate Discharge Practices 142
Table 25. Regions in Ontario 164
Table 26. Continuous Variables 166
Table 27. Categorical Variables 167
Table 28. Characteristics of Hospitals in Each Region 167
Table 29. Summary of Hierarchical Linear Regression Analyses for Relationship
between Size and Health Literate Survey Score 172
Table 30. Relationship between Teaching Status and Health Literate Survey
Score 173
Table 31. Relationship between Region and Health Literate Survey Score 174
Table 32. Relationship between Rurality and Health Literate Survey Score 174
Table 33. Summary of Stepwise Backward Regression Analyses for Size as
Predictor of Health Literate Survey Score 175
Table 34. Summary of Stepwise Backward Regression Analyses for Variables
Predicting Health Literate Survey Score 177
Table 35. Summary of Regression Analysis for Interaction between Size and
Region as Predictors of Health Literate Survey Score 179
Table 36. Summary of Regression Analysis for Interaction between
Size and Rurality as Predictors of Health Literate Survey Score 180
viii
List of Figures
Figures Page
Figure 1. Health Literate Care Model 9
Figure 2. Health Literate Care Model with Health Literate Discharge Practices 48
Figure 3. Distribution of Health Literate Survey Scores 68
Figure 4. Absorptive Capacity Metaroutines for the Uptake of Evidence-Based
Practice 101
Figure 5. Absorptive Capacity Metaroutines for the Uptake of Health Literate
Discharge Practices 141
Figure 6. Distribution of the Health Literate Survey Scores 168
ix
List of Appendices
Appendix Page
A: Email for Round 1, Delphi Panel 222
B: Participant Information Letter, Round 1 223
C: Reminder Notice, Round 1 225
D: Email for Round 2, Delphi Panel 226
E: Participant Information Letter, Round 2 227
F: Reminder Notice, Round 2 229
G: Delphi Panel, Results of Round 1 230
H: Delphi Panel, Results of Round 2 235
I: Health Literate Discharge Practices Survey 237
J: Teaching Hospitals 243
K: Large Community Hospitals 244
L: Small Community Hospitals 246
M: Email 249
N: Participant Information Letter 250
O: Endorsement Letter from the Ontario Hospital Association 253
P: Reminder Notice 254
Q: Rurality Distributions 255
R: Descriptives, Survey Items 257
x
S: Tests of Normality of Survey Items 268
T: Data Dictionary 346
U: Correlations between Survey Items 350
V: Factor Correlation Matrix for 9 Factor Solution 383
W: Scree Plot 384
X: Summary of Items and Factor Loadings for Direct Oblimin Nine Factor
Solution 385
Y: Summary of Items and Factor Loadings Greater than .3, for Nine Factor
Solution 390
Z: Factor Correlation Matrix for 5 Factor Solution 395
AA: Summary of Items and Factor Loadings for Direct Oblimin Five Factor
Solution 396
BB: Summary of Items and Factor Loadings Greater than .3, for Five Factor
Solution 400
CC: Seven Factor Solution 403
DD: Eight Factor Solution 406
EE: Email Communication for Interview 409
FF: Informed Consent 410
GG: Interview Questions 414
HH: Test of Normality 416
II: Distribution of Health Literate Survey Scores for Categorical Independent
Variables 417
JJ: Tests of Normality for Continuous Independent Variables and Base-10
Logarithm Transformations of Independent Variables 434
KK: Scatterplots of Continuous Independent Variables 454
xi
LL: Regression Analysis for Size and Size-Squared 456
MM: Relationship between Teaching Status and Health Literate Survey Score 458
NN: Relationship between Region and Health Literate Survey Score 459
OO: Relationship between Rurality and Health Literate Survey Score 461
PP: Relationship between Organizational Size, Teaching Status and
Rurality with Health Literate Survey Score 463
QQ: Relationship between Organizational Size, Teaching Status and
Region with Health Literate Survey Score 465
RR: Post Hoc Analyses to Examine Differences in Number of Acute
Beds between Regions 470
SS: Regression Analysis to Examine Interaction between Region and
Size as Predictors of Health Literate Survey Score 473
TT: Regression Analysis to Examine Interaction between Rurality and
Size as Predictors of Health Literate Survey Score 474
1
Chapter 1: Background
1.1 Health literacy and health literate organizations
Health literacy is the ability to obtain, use and understand information to make
decisions that maintain and promote health (Berkman, Sheridan, Donahue, Halpern, & Crotty,
2011; Nielsen-Bohlman, Panzer, & Kindig, 2004). Studies that have evaluated the health
literacy levels of patients have found that a lack of health literacy is associated with poor health
outcomes (Berkman, Sheridan, Donahue, Halpern, & Crotty, 2011) as well as increased mortality
(Baker et al., 2007; Berkman, Sheridan, Donahue, Halpern, & Crotty, 2011; McNaughton et al.,
2015; Sudore et al., 2006). In addition, a low level of health literacy is associated with increased
hospital readmission and increased health care costs (Bailey et al., 2015; Berkman, Sheridan,
Donahue, Halpern, & Crotty, 2011; Clark, 2011; McNaughton et al., 2015; Mitchell, Sadikova,
Jack, & Paasche-Orlow, 2012; Moser et al., 2015).
Generally speaking, health literacy is thought of as a characteristic of an individual or a
group of people and is therefore most often referred to in the context of a patient’s health
literacy. However, the concept of health literacy has recently been applied to organizations.
“Health literate organizations” is a term created by the Institute of Medicine in 2012 to refer to
organizations that create environments which individuals can easily navigate to find the services
that they need and that make it easier for people to obtain and understand information about their
health. The concept of health literate organizations recognizes that organizations have a
responsibility to be responsive to the health literacy needs of patients (Brach et al., 2012;
Institute of Medicine, 2012). The idea that healthcare organizations have an obligation to meet
the health literacy needs of their users is a new one, and it is only beginning to be addressed by
hospitals in North America (Koh, Baur, Brach, Harris, & Rowden, 2013). By meeting this
2
responsibility, hospitals have the opportunity to improve outcomes and to reduce the rate of
hospital readmission (Berkman, Sheridan, Donahue, Halpern, & Crotty, 2011; Brach, Dreyer, &
Schillinger, 2014; Brach et al., 2012; Mitchell et al., 2012).
Upwards of sixty percent of Canadians lack an adequate level of health literacy (Murray,
Hagey, Willms, Shillington, & Desjardins, 2008; Rootman & Gordon-El-Bihbety, 2008) and the
stress of illness and hospitalization can compromise a normally adequate level (Koh, Brach,
Harris, & Parchman, 2013; Kowalski et al., 2015). There is a recognized need to ensure that
patients are given information and instructions at the time of discharge that they understand and
can use to manage their own health once they leave the hospital.
When patients return to the hospital within 30 days of discharge, this often indicates that
the discharge process was done poorly (Vashi et al., 2013). Although there is a lack of research
on discharge practices in Ontario hospitals, there is evidence that discharge processes that meet
the health literacy needs of patients and families, and that work to ensure a smooth transition of
care between the settings of hospital and home, are associated with improved health outcomes
and reduced readmission rates (Baker, 2011; Coleman, Parry, Chalmers, & Min, 2006; Jack et
al., 2009; Markley et al., 2013; Naylor et al., 2004; Naylor et al., 1994).
1.2 Hospital readmission
This dissertation is defining hospital readmission as returning to the hospital and being
admitted within 30 days of discharge. North America has rates of hospital readmission that are as
high as 25% (Bernheim et al., 2010).
3
A 2012 Canadian study found that 8.5% of all patients discharged from acute care in an
eleven month period from 2010-111 required readmission to hospital for inpatient care. During
the same time period, 13.3% of adult medical patients, that is, patients over 19 years of age
discharged with a medical diagnosis, returned to the hospital for inpatient readmission within 30
days. It was estimated that the cost of these readmissions was $1.8 billion, not including
physician fees for billed services (Canadian Institute for Health Information, 2012).
The Canadian Institute for Health Information has more recently reported that this
readmission rate increased between 2011 and 2014. In an 11 month time period from 2013-14,
8.9% of all patients discharged from acute care required readmission. In Ontario, this rate was
higher at 9.1% (Canadian Institute for Health Information, 2016).
In Ontario and Canada, readmission is highest for medical patients (Baker, 2011;
Canadian Institute for Health Information, 2012). In Ontario, the 30-day hospital readmission
rate for patients with the medical diagnoses of pneumonia, chronic obstructive pulmonary
disease (COPD), congestive heart failure (CHF), stroke, gastrointestinal disease, diabetes and
cardiac conditions (excluding myocardial infarction) was 13.5% for patients in 2013-14.
Although Ontario’s Ministry of Health and Long-Term Care (MOHLTC) has made substantial
investments to reduce hospital readmission and improve quality of care for patients (Ministry of
Health and Long-Term Care, 2013d, 2015a). the readmission rate has increased every year since
it was first measured in 2009-10 at 12% (Health Quality Ontario, 2015a).
1 11 month time period is defined by CIHI as the fiscal year between April 1 and March 1 (Canadian Institute of
Health Information 2012, 2016)
4
In the United States, studies using Medicare data have found readmission rates for
patients hospitalized with acute myocardial infarction (AMI) to range from 19.1 to 19.9%
(Bernheim et al., 2010; Krumholz et al., 2009) and for patients hospitalized with heart failure to
range from 23.6 to 24.5% (Bernheim et al., 2010; Keenan et al., 2008; Krumholz et al., 2009).
This readmission is an expensive and inefficient use of acute care resources (Baker, 2011;
Canadian Institute for Health Information, 2012, 2016; Jencks, Williams, & Coleman, 2009;
Vashi et al., 2013).
There are efforts to reduce readmission in Canada and the United States. In Ontario, the
Ministry of Health and Long-Term Care (MOHLTC) has focused on reducing avoidable
hospitalizations, as identified in the 2010 Excellent Care for All Act (Ministry of Health and
Long-Term Care, 2014a). In the United States, the 2010 Affordable Care Act has set up a
Hospital Readmission Reduction Program. In 2012, the Center for Medicare and Medicaid
Services (CMS) began reducing payments to hospitals that have high readmission rates for
patients diagnosed with congestive heart failure, acute myocardial infarction and pneumonia
(Centers for Medicare & Medicaid Services, 2016; Cloonan, Wood, & Riley, 2013; Kocher &
Adashi, 2011).
There is a lack of consensus on how to determine how many readmissions are avoidable.
A review of studies that examined avoidable readmissions found that the rate varied from 5 to
79% (van Walraven, Bennett, Jennings, Austin, & Forster, 2011). Two reasons why different
studies have found such variable rates is that they use different populations as well as different
definitions of avoidable readmission (Greenwald & Jack, 2009). As stated earlier, there is a
growing awareness that how patients are discharged contributes to their readmission, and that
meeting the health literacy needs of patients and families at the time of discharge contributes to
5
decreased readmission (Adams, Stephens, Whiteman, Kersteen, & Katruska, 2014; Jack, Chetty,
Anthony, Greenwald, Sanchez, Johnson, Forsythe, O'Donnell, Paasche-Orlow, & Manasseh,
2009; Vashi et al., 2013). Hospitals that are adept at meeting the health literacy needs of patients
and families upon discharge may be considered health literate organizations.
1.3 Health literate best practices
In health literate organizations, health literate best practices are centered on the use of
“universal precautions” (Brach et al., 2012). Using universal precautions means assuming that all
health care users have inadequate health literacy since everyone will benefit from education that
is clear and easy to understand (DeWalt et al., 2010; Paasche-Orlow & Wolf, 2008; U.S.
Department of Health and Human Services, 2010). This recognizes that health literacy is not
static, but dynamic depending on one’s circumstances. For example, a patient may generally
have an adequate level of health literacy, however the stress that accompanies a new diagnosis or
disease exacerbation can challenge one’s ability to obtain, use and understand information about
one’s health, (Koh, Brach, et al., 2013; Kowalski et al., 2015).
More specifically, universal precautions recommend that health care providers employ
teach back (see description below), use plain language, speak slowly, avoid the use of medical
jargon and use written materials to highlight important information in interactions with patients
and families (Baker, 2011; Cua & Kripalani, 2008; Harper, Cook, & Makoul, 2007; Koh, Brach,
et al., 2013; Makaryus & Friedman, 2005; Nielsen, Rutherford, & Taylor, 2009; Williams,
Davis, Parker, & Weiss, 2002).
In the use of “teach back,” patients and families are asked to restate what a health care
provider has told and/or taught them about their health, in their own words. This allows
clinicians to verify that patients and families understand information and offers an opportunity to
6
provide any needed clarification (Baker, 2011; DeWalt et al., 2010; Koh, Brach, et al., 2013;
Nielsen et al., 2009; Schillinger et al., 2003; Shojania, Duncan, McDonald, Wachter, &
Markowitz, 2001). Teach back has been found to be an effective way to confirm patient
understanding of discharge instructions (Kripalani, Jackson, Schnipper, & Coleman, 2007).
This dissertation defines health literate best practices as the use of universal precautions.
Health literate discharge practices help patients and their families to obtain, use and understand
information to manage their health at the time of discharge. In the United States, the use of a
discharge process named “Project RED” (Re-Engineered Discharge) that is based on health
literate discharge practices, has been found to be associated with decreased hospital readmission
(Adams et al., 2014; Jack, Chetty, Anthony, Greenwald, Sanchez, Johnson, Forsythe, O'Donnell,
Paasche-Orlow, & Manasseh, 2009; Markley et al., 2013). .
1.4 Care transition efforts in Ontario
There are various ongoing efforts in the community to improve care transitions (Health
Quality Ontario, 2013a, 2013b). There are 13 “Home At Last” programs in place across Ontario.
These programs help to transition older adults as well as adults with complex needs from the
hospital to the home, with the support of CCAC (Community Care Access Centres) and
community support services (Central East Local Health Integration Network, 2014; Ministry of
Health and Long-Term Care, 2014b). As an example, in an Ontario community in 2010-11,
there was a nurse practitioner-led care transition intervention that supported seniors through the
discharge process from hospital to home (Lane, Pitzul, Laporte, Nauenberg, & Wodchis, 2013).
In 2012, the MOHLTC introduced an initiative called Health Links to Ontario, which is
specifically designed to improve the coordination of care for patients with multiple complex
conditions who are high users of the health care system. At present, there are 69 community
7
Health Links in the province. Two goals of the Health Links are to reduce avoidable ED visits
and to reduce 30-day readmission rates through improving care coordination between primary,
community and acute care services (Ministry of Health and Long-Term Care, 2015c)
1.5 Discharge practices in Ontario hospitals
The use of health literate discharge practices has not been studied in Ontario hospitals,
however, there is evidence that hospitals in Ontario have begun to address the issue of health
literacy (Health Quality Ontario, 2013b; St. Michael's Hospital, 2010).
We do not know if discharge practices used in Ontario hospitals meet the health literacy
needs of patients and families. In other words, it is unknown whether hospitals in Ontario are
using discharge practices that help patients and families to obtain, use and understand
information to make decisions that maintain and promote health (Berkman, Sheridan, Donahue,
Halpern, Viera, et al., 2011; Nielsen-Bohlman et al., 2004).
1.6 Introduction to the three studies
This dissertation uses a mixed methods, sequential design to address the following
research objectives:
1) To identify the best indicators of health literate discharge practices;
2) To determine whether hospitals in Ontario use health literate discharge practices;
3) To find out how Ontario hospitals adopt and use health literate discharge practices;
4) To examine the relationship between organizational characteristics and the use of health
literate discharge practices.
The first study (chapter 3) used a Delphi panel to find the best indicators of health literate
discharge practices; this study was based on, and extended, work to date on Project RED (Jack et
8
al., 2009). In the second study (chapter 4), these indicators were used to create a survey. This
survey was distributed to all adult, acute care hospitals in Ontario that have a general medicine
inpatient unit. This study was guided by the Health Literate Care Model (Koh, Brach, et al.,
2013). Exploratory factor analysis of the survey results was completed to determine the factors
that constitute the use of health literate discharge practices in Ontario hospitals.
The third study (chapter 5) examined how acute care hospitals take on health literate
discharge practices. To examine this question, health literate discharge practices were cast as
practice innovations, and organizational learning theory was used as a framework by which to
analyze data collected from key informant interviews with a subset of clinicians who participated
in the survey. These interviews examined the relationship between the organizational
characteristics of hospitals and their use of health literate discharge practices. The fourth and
final study examined the relationship between the organizational characteristics of hospitals and
the use of health literate discharge practices.
9
Chapter 2:
Health Literate Discharge Practices and Project RED
2.1 Theoretical framework: The Health Literate Care Model
The theoretical framework used to guide the first two studies of this dissertation was the
Health Literate Care Model. The Health Literate Care Model is an adaptation of Wagner’s
Chronic Care Model (Koh et al., 2013). The Chronic Care Model is a conceptual framework that
has been widely tested and has been used to enhance patient care delivery (Barr et al., 2003;
Wagner, Austin, & Von Korff, 1996).
The Health Literate Care Model posits that health literate strategies employed by health
literate organizations lead to improved outcomes through productive interactions between the
patient and family and the health care team (Koh et al., 2013) (see Figure 1).
Figure 1. Health Literate Care Model. (Office of Disease Prevention and Health Promotion,
2015). Reprinted with permission from the US Department of Health and Human Services.
10
Health literate discharge practices may be thought of as health literate strategies for
health literate organizations (Koh et al., 2013). An example of a discharge strategy that is
consistent with meeting the health literacy needs of patients and families would be to review a
medication regimen with the use of “teach back.” As described in the introductory chapter,
patients and families would be asked to restate what a health care provider has told and/or taught
them about their medication regimen, in their own words. This allows clinicians to verify that
patients and families understand their medications and how the medications are used, and it
offers an opportunity to provide any needed clarification (DeWalt et al., 2010). Communicating
about the need for follow-up consultation appointments, and ensuring that the patient and family
understand the importance of this follow-up and can identify a plan to attend these appointments,
are other examples of health literature discharge practices. These examples are consistent with
the components of Project RED that will be described in the following sections (Jack et al.,
2013).
2.2 Introduction to Project RED
In an effort to improve the discharge process of acute care hospitals, in 2001, a team of
health care providers, researchers and administrators at Boston University and Boston Medical
Center, in conjunction with the Agency for Healthcare Research and Quality’s Developmental
Center for Patient Safety Research, began to examine factors at the time of discharge that led to
hospital readmission.
This team included a broad range of stakeholders. These stakeholders included the chief
medical officer, the directors of nursing, research, case management and quality improvement as
well as the nurse manager and clinical pharmacist from the medical wards of Boston Medical
Center. They reviewed the literature on hospital discharge practices and examined the process of
11
discharge from Boston Medical Center using a number of approaches that included probabilistic
risk assessment, failure mode and effects analysis, qualitative interviews and root cause analysis
(Anthony et al., 2005). Probabilistic risk assessment is a quantitative method for evaluating the
presence of possible risks and the consequences of these risks occurring (Freudenburg, 1988).
Failure mode and effects analysis is a qualitative method; it entails locating and naming areas of
potential failure in a process, such as a discharge process, and evaluating the possible outcomes
of the failures (Stephans, 2004). In addition, interviews with patients who were admitted to the
general medicine unit of Boston Medical Center were done. These patients had been readmitted
to hospital within 90 days of discharge. The interviews used a semi-structured interview guide
and were kept open-ended in an effort to find out about the experiences of patients with hospital
discharge and readmission (Anthony et al, 2005; Jack et al, 2008).
Following these approaches, the team of providers, researchers and administrators then
used an iterative group process to develop a process map that detailed all steps of hospital
discharge. Working in smaller groups, and as a larger team, they began to identify and describe
areas of possible failure that lead to hospital readmission, and they were then able to develop and
pinpoint ways to avoid or deter possible errors (Anthony et al., 2005; Greenwald, Denham, &
Jack, 2007; Jack et al., 2008).
This led to the development of Project RED, which consists of 12 components that
address patient education, medication regimens, follow-up appointments and diagnostic testing,
the need for written instructions, use of the teach-back method to confirm patient understanding
of the discharge plan, and the need for a discharge summary to be sent to patients’ primary care
providers (Jack et al., 2009; Jack et al., 2013):
Components of Project RED (Jack et al., 2013, page 11):
1. Ascertain need for and obtain language assistance.
12
2. Make appointments for follow up care (e.g., medical appointments, post discharge tests/labs).
3. Plan for the follow up of results from tests or labs that are pending at discharge.
4. Organize post discharge outpatient services and medical equipment.
5. Identify the correct medicines and a plan for the patient to obtain them.
6. Reconcile the discharge plan with national guidelines.
7. Teach a written discharge plan the patient can understand.
8. Educate the patient about his or her diagnosis and medicines.
9. Review with the patient what to do if a problem arises.
10. Assess the degree of the patient’s understanding of the discharge plan.
11. Expedite transmission of the discharge summary to clinicians accepting care of the patient.
12. Provide telephone reinforcement of the discharge plan.
In a randomized controlled trial, this program was found to significantly reduce acute
care service (as defined by hospital readmission and ED use within 30 days of discharge) by 30%
for patients discharged from a medical unit of a large teaching hospital (Jack et al, 2009). A 2013
study in Brownsville, Texas found that this program reduced hospital readmission rates by 50%
in a 240 bed hospital (Markley et al., 2013). In addition, a more recent study of the use of
Project RED in a rural 77 bed hospital in Pennsylvania found that it reduced hospital readmission
by 32% (Adams et al., 2014).
The practices used in Project RED have been found to be consistent with the practices of
health literate organizations (Institute of Medicine, 2012; Weidmer, Brach, Slaughter, & Hays,
2012). Project RED has been funded and endorsed by the Agency for Healthcare Research and
Quality in the United States as it has been associated with significant decreases in hospital
readmission, as well as decreased costs (Adams et al., 2014; Jack, Chetty, Anthony, Greenwald,
Sanchez, Johnson, Forsythe, O'Donnell, Paasche-Orlow, & Manasseh, 2009; Markley et al.,
2013).
2.3 Current state of hospital discharge
While the components of Project RED have been found to improve care throughout the
discharge process for patients and their families (Institute of Medicine, 2012; Jack et al., 2009;
13
Jack et al, 2013; Markley et al., 2013), there has been no widespread uptake of these components
in North American hospitals. Researchers have remarked upon the suboptimal discharge
processes in Canadian and American acute care hospitals. Unfortunately, when patients are
discharged home from an inpatient medicine ward in an acute care hospital, most are given little
if any information related to their follow-up care and their discharge medication plans, and the
information they are given is often inadequate and at times incorrect (Greenwald et al., 2007;
Kripalani et al., 2010). The day of discharge from an acute care hospital is frequently a
confusing and chaotic time (Soong et al., 2013) and patients have reported hurried processes with
a lack of attention to how their care will be managed following discharge (Hesselink et al., 2012;
Robert Wood Johnson Foundation, 2013).
Family members are defined by the patient, and they refer to those individuals who are
involved in the patient’s care. It is commonly assumed that only the patient needs to receive
education about discharge, and family members who provide care are often ignored in this
process (Neilsen et al., 2009). However, many patients rely on support that they receive from
family members once they are discharged home (Arbaje et al., 2014; Calvillo–King et al., 2013),
and it is important that these individuals be recognized and included in discharge education, as
they may be helping patients to manage their health (Coleman & Roman, 2015; Doctoroff &
McNally, 2014; Hesselink et al., 2012; Nielsen et al., 2009).
After patients are discharged home, they often have new health care issues to self-
manage, medication changes to grasp and follow-up appointments to make and attend. Often
this is at the same time that they are recovering from illness or a disease exacerbation (Dudas,
Bookwalter, Kerr, & Pantilat, 2001; Meyers et al., 2014). It is therefore not surprising that the
14
care transition from the acute care hospital to home is a period when patients may experience
increased adverse events (Coleman et al., 2013).
Even when patients are given information on their discharge plan, it is often not in a form
they understand. Due to their lack of health literacy, patients may not understand complex
medical issues and may have difficulty navigating the health care system to obtain further care.
This is in addition to the stress and anxiety that patients and families may experience when
coping with information about diagnoses, treatments and follow-up plans (Nielsen et al., 2009).
2.4 Evidence for components of Project RED
As described above, Project RED consists of 12 components. The following sections
describe evidence for each of the 12 components of Project RED.
2.4.1 Language assistance
The first component of Project RED is “ascertain need for and obtain language
assistance” (Jack et al., 2013, page 11). A barrier to understanding discharge information is
language, particularly if patients and families do not speak English or French. A recent study
examined level of understanding among patients being discharged from two urban hospitals in
California with limited English proficiency. It is not surprising that the authors found that these
patients had limited understanding of their follow-up care and their medications (Karliner et al.,
2012). Another study that took place in a teaching hospital in Massachusetts found that when
patients who had limited English proficiency were provided with translation services at
admission and at discharge, their length of stay and readmission rates were significantly lower
than patients who did not receive these services (Lindholm, Hargraves, Ferguson, & Reed, 2012)
15
During hospitalization and at the time of discharge, it is recommended that translation
services be made available for both oral and written education to ensure that patients and
families understand the discharge plan (Baker, 2011; Jack et al., 2013; Karliner et al., 2012).
2.4.2 Follow-up care: appointments and tests
The second component of Project RED is: “make appointments for follow up care” (Jack
et al, 2013, page 11). Moore, McGinn & Halm (2007) found that more than 1/3 of patients being
discharged from a medical or geriatrics service did not receive recommended follow-up care
consisting of diagnostic testing and specialist consultation. Reasons for this high rate may be that
patients are unaware that they need to make appointments and need to follow-up with testing and
consultations, or that they do not understand their discharge plan or how to navigate the health
care system to make these appointments (Jack et al., 2013).
2.4.3 Test results
The third component of Project RED is “plan for the follow up of results from tests or
labs that are pending at discharge” (Jack et al, 2013, page 11). Roy et al., (2005) found that
patients are often discharged from hospital with test results pending and with no plan for follow-
up of these results. This is particularly a problem when tests are ordered on the day of discharge
(Ong, Magrabi, Jones, & Coiera, 2012). Clearly this is a quality and safety issue as abnormal test
results may then not be seen by health care providers (Ong et al., 2012; Roy et al., 2005).
A 2013 study examined strategies used by 599 hospital in the United States in a
“Hospital-to-Home” quality campaign led by the American College of Cardiology that were
significantly associated with reduced readmission rates. Assigning hospital staff to follow up on
16
pending test results at the time of discharge was significantly associated with reduced
readmission (Bradley et al., 2013).
2.4.4 Coordination of post discharge services
The fourth component of Project RED is “organize post discharge outpatient services and
medical equipment” (Jack et al, 2013, page 11). A study that took placed in the United States
used a qualitative approach to find out about the difficulties experienced by cognitively impaired
patients and their family members in the hospital discharge process. Participants described
difficulty obtaining necessary patient equipment, and coordinating services (Naylor, Stephens,
Bowles, & Bixby, 2005).
Another study that took place in a teaching hospital in Cleveland, Ohio used a quasi-
experimental design. It introduced an intervention led by an advanced practice nurse. The nurse
met with patients who had been admitted to the intensive care unit and who had required
mechanical ventilation for at least 72 hours, and with their families. The nurse was responsible
for providing education and coordinating outpatient services. The patients who received this
intervention had higher scores on a measure of physical health-related quality of life, and also
had decreased readmission rates, compared to a control group (Douglas, Daly, Kelley, O’Toole,
& Montenegro, 2007).
A similar study took place in a Hong Kong hospital. This study examined the use of a
nurse-led intervention that provided the planning and coordination of outpatient services for
patients with end stage renal disease being discharged from hospital. The patients also received
an educational intervention. The patients who received the intervention had significantly higher
17
levels of patient satisfaction and quality of life compared to the control group (Chow & Wong,
2010).
2.4.5 Medication
2.4.5.1 Medication review
The fifth component of Project RED is “identify the correct medicines and a plan for the
patient to obtain them” and the eighth component is “educate the patient about his or her
diagnosis and medicines” (Jack et al., 2013, page 11). Ensuring that patients understand how to
obtain and use their medications is an important part of hospital discharge. Patients in the
hospital have been found to have a low level of knowledge regarding the medications that they
are being administered (Marvanova et al., 2011). Thus, it may not be surprising that the most
common post-discharge adverse events are related to medications (Forster et al., 2004; Forster,
Murff, Peterson, Gandhi, & Bates, 2003; Greenwald et al., 2007).
2.4.5.2 Medication teaching
Several studies have found that the reason for medication-related adverse events and non-
adherence to discharge medications is due to inadequate or inaccurate education about the
regimen (Forster et al., 2004; Kripalani, Jackson, et al., 2007; Lindquist et al., 2011). A number
of studies have found that medication counseling and teaching at the time of discharge are
associated with improved patient outcomes. In a quasi-experimental study, Paasche-Orlow et al
(2005) looked at the effect of a focused education session on the use of a meter-dose inhaler
(MDI) with asthma patients at the time of discharge in two urban teaching hospitals in the United
States. This study used a “teach to goal” strategy meaning that the education was repeated until
participants were able to demonstrate understanding of the medication regimen and skill in using
18
the MDI. This study found that tailored medication teaching at the time of discharge improved
understanding and use of the MDI, and that an inadequate level of health literacy was not
associated with greater difficulty learning or retaining this knowledge, compared to having an
adequate level (Paasche-Orlow et al., 2005)
A 2010 study done in the Netherlands examined the effect of medication teaching at the
time of discharge, as well as written medication instructions given to the patient. This was found
to have a significant impact on decreasing the number of medication discrepancies within 30-
days of discharge (Eggink, Lenderink, Widdershoven, & Bernt, 2010).
A similar study was completed in the UK in which patients received medication teaching
while they were hospitalized and were given a written medication list, discharge summary and
medication reminder card. These measures were found to be associated with fewer unplanned
visits to a primary care provider as well as decreased hospital readmissions (Al-Rashed, Wright,
Roebuck, Sunter, & Chrystyn, 2002).
In a study conducted in the United States, hospitalized patients received medication
counseling by a pharmacist at the time of discharge followed by a phone call 3-5 days post-
discharge. This intervention was associated with decreased adverse events (Schnipper et al.,
2006). An earlier study conducted in the United States found that when a pharmacist called
patients 2 days post-discharge, there was a reduction in 30-day ED use (Dudas et al., 2001).
These studies demonstrate that medication teaching at the time of discharge, with and without a
follow-up phone call, is associated with improved outcomes.
2.4.5.3 Medication reconciliation
19
In order to ensure that patients are taking the correct medications, it is imperative that
medication reconciliation be done upon discharge from hospital. The goal is to prevent errors by
reconciling medications, including natural remedies, which patients were taking before
hospitalization with discharge medications. This process of reconciliation allows health care
providers to ensure that patients are taking the correct medications and that any unnecessary or
incorrect ones have been discontinued (Accreditation Canada, Canadian Institute for Health
Information, & Institute for Safe Medication Practices Canada, 2012; Baker, 2011; Jack et al.,
2013).
All changes to a patient’s medication regimen need to be clearly communicated to the
patient and family to prevent confusion about which medications are to be taken once the patient
is home. Without this communication, patients may miss essential medications, may take
unnecessary ones and may have adverse events (Fallis, Dhalla, Klemensberg, & Bell, 2013;
Forster et al., 2004; Jack et al., 2013; Jencks et al., 2009; Levinson, 2008).
2.4.6 Use of guidelines
The sixth component of Project RED is “reconcile the discharge plan with national
guidelines” (Jack et al., 2013, page 11). The use of national guidelines refers to the use of
evidence-based practices (Jack et al., 2013). Evidence-based practices refer to routines or work
practices that comprise clinical practice guidelines or “care bundles”2 in health service
organizations. They are based on scientific evidence, and require the expertise of clinicians to
2 Care bundles is a term used for sets of evidence-based guidelines aimed at improving outcomes in hospitals
(Institute for Healthcare Improvement, 2014).
20
implement (Berta & Baker, 2004; Berta et al., 2005, 2010; Fineout-Overholt, Levin, & Melnyk,
2004; Graham, Mancher, Wolman, Greenfield, & Steinberg, 2011; Institute for Healthcare
Improvement, 2014). Although traditionally this has not been the case, ideally, evidence-based
practices take into consideration the choices and values of patients and families, in other words,
the end users of health care services (Fineout-Overholt et al., 2004).
The purpose of using evidence-based practices is to improve patient care and outcomes.
Evidence-based practices consist of interventions that have been studied and shown to be
effective. These interventions are standardized into work processes that help health care
providers make decisions that improve care for patients (Berta et al., 2005; Grimshaw & Eccles,
2004; Hoomans, Severens, Evers, & Ament, 2009; Tucker, Nembhard, & Edmondson, 2007).
When evidence-based practices are introduced, they have the potential to enhance the quality of
patient care and to improve organizational performance (Berta et al., 2005; Grimshaw & Eccles,
2004; Okafor & Thomas, 2008).
Despite repeated findings that the use of evidence-based practices has been associated
with improved patient care, quality and outcomes, numerous studies have found that patients do
not receive care that is based on strong, scientific evidence. It can take over a decade for
research findings to be adopted and used by healthcare organizations (Berta et al., 2005;
Dobbins, Ciliska, Cockerill, Barnsley, & DiCenso, 2002; Grol & Grimshaw, 2003; Okafor &
Thomas, 2008; Smith, Saunders, Stuckhardt, & McGinnis, 2013).
2.4.7 Use of written discharge instructions
The seventh component of Project RED is “teach a written discharge plan the patient can
understand” (Jack et al, 2013, page 11). As described earlier, the time of hospital discharge is
21
typically a chaotic one in which patients and families receive an overwhelming amount of
information (Soong et al., 2013). As it is difficult for patients and families to remember verbal
instructions at this time (Baker, 2011; Makaryus & Friedman, 2005), it may be helpful to provide
them with a written discharge plan containing the key information that patients will need to
manage their own care once they leave the hospital (Makaryus & Friedman, 2005). One study
found that providing patients over 65 years of age with a written discharge summary that
included details on their medications was significantly associated with reduced readmission and
fewer unplanned outpatient physician visits (Al-Rashed et al., 2002).
2.4.8 Patient and family teaching
The eight component of Project RED is “educate the patient about his or her diagnosis
and medicines” (Jack et al., 2013, page 11). Medication teaching is reviewed in section 2.3.5.2.
There is evidence that patients receive inadequate discharge education. A 2012 study
qualitative study examined the care patients and families receive at the time of discharge. This
study was conducted in 5 EU countries, and consisted of 192 individual interviews and 26 focus
groups with patients, family members, hospital nurses and physicians, and community nurses
and physicians. Patients and family members reported that discharge instructions were often
given with medical jargon that they could not understand. Health care providers in the hospital
and the community reported that patients often received inadequate teaching regarding their
follow up care. Both patients and providers reported that the information given was unclear, was
given in a hurried manner, and that patients and family members did not have an opportunity to
ask questions (Hesselink et al., 2012).
22
Health care providers may not provide education because they are unaware that patients
and their families may not understand health information. There is evidence that health care
providers overestimate patients’ understanding and patients’ levels of health literacy (Cua &
Kriplani, 2008; Dickens et al, 2013). In interviews that the Canadian Public Health Association
conducted with Canadians who have low levels of health literacy about their health care
experiences, participants consistently reported that they did not have enough time to ask
questions and to share concerns in their interactions with providers (Canadian Public Health
Association, 2006).
There is evidence from two studies to show that educational workshops in health literacy
can improve participants’ knowledge of effective health literacy communication techniques as
well as their confidence in using these techniques. Kriplani and colleagues (2011) looked at the
use of interactive educational sessions for medical residents in a large acute care hospital in
Atlanta, Georgia. This workshop had a small didactic portion that taught principles of health
literacy to medical residents. This was followed by a larger interactive component that allowed
the attendees to practice the teach back technique in groups of three, as described above in the
proposed workshop. It was found that these educational sessions were effective in increasing
participants' confidence in providing medication counseling and in their reported use of teach
back one month later.
Another study examined the use of similar educational workshops with social workers,
nurses, health educators, office staff and administrators in various health care settings in Texas.
It had a small didactic component that provided education on principles of health literacy. A
video in which individuals with low health literacy described their experience with the health
care system was included. This was followed by a role playing exercise that is also described by
23
the proposed workshop for the proposed strategy. Following the workshop, participants reported
increased knowledge of health literacy principles and indicated that they planned to use the
communication techniques that they had learned (Mackert, Ball & Lopez, 2011). It is not known
whether this intention translated to actual change in communication with patients and families.
A large number of studies have examined the impact of patient teaching. Disease-specific
teaching was addressed by the nurse-led intervention that took place in Hong Kong that focused
on the hospital discharge of patients with end stage renal disease. Patients received education on
disease management from study nurses, and this education, with the coordination of outpatient
services, was associated with increased patient satisfaction and quality of life (Chow & Wong,
2010).
A randomized controlled trial looked at the impact of a one-hour individualized teaching
session led by a nurse educator with heart failure patients in a Michigan teaching hospital. This
intervention was found to be associated with improved clinical outcomes and improved patient
adherence to a disease management plan (Koelling, Johnson, Cody, & Aaronson, 2005).
2.4.9 Review of potential problems
The ninth component of Project RED is “Review with the patient what to do if a problem
arises” (Jack et al., 2013, page 11). Teaching patients about health-related problems was part of
the disease and medication teaching that was evaluated in the studies that have been described.
For example, disease-specific teaching was addressed by the nurse-led intervention that
took place in Hong Kong that focused on the hospital discharge of patients with end stage renal
disease. Patients received education on disease management from study nurses, and this
24
education, with the coordination of outpatient services, was associated with increased patient
satisfaction and quality of life (Chow & Wong, 2010).
This component also includes establishing a plan for patients and families of what to do if
a problem arises (Jack et al., 2009; Jack et al., 2013).
2.4.10 Patient and family education with the use of teach back
The tenth component of Project RED is “assess the degree of the patient’s understanding
of the discharge plan” (Jack et al., 2013, page 11). As patients with heart failure have among the
highest rates of hospital readmission (Canadian Institute for Health Information, 2012; Jencks et
al., 2009), studies of hospital discharge often focus on patients with this condition. Two recent
studies examined the use of teach back in preparing patients with heart failure for discharge
home. In the first study, although the use of teach back as a single intervention was not
associated with decreased readmission, it was found to be significantly associated with
knowledge retention and it was found to be a valuable way to assess patient learning (White,
Garbez, Carroll, Brinker, & Howie-Esquivel, 2013).
The second study examined the use of teach back in heart failure patients at a large
Pennsylvania teaching hospital. In this study, 469 patients received some kind of teaching about
heart failure, and teach back was used with 180 of the patients. Those patients who received
teach back had significantly reduced readmission rates compared to the other 289 patients in the
study (Peter et al., 2015).
2.4.11 Transfer of discharge summary
25
The eleventh component of Project RED is “expedite transmission of the discharge
summary to clinicians accepting care of the patient” (Jack et al., 2013, page 11). At the time of
discharge from hospital to home, responsibility for the patient’s care is being transferred from
the inpatient health care provider team to the patient, family and the primary care provider
(Shoeb, Merel, Jackson, & Anawalt, 2012). In addition to the need for clear communication
between inpatient care providers and the patient and family, it is imperative that there be
information continuity between the inpatient team and the patient’s primary care provider. The
discharge summary is an important tool that primary care providers require to ensure continuity
of care for patients (Jack et al., 2008).
Physicians discharging patients from acute care hospitals in Ontario are legally required
to complete a discharge summary (Ministry of Health and Long-Term Care, 2006), however,
these summaries are frequently not sent to primary care providers. This means that the primary
care provider will not necessarily know about the hospitalization and the care that the patient
received which has the potential to lead to poor patient outcomes (Baker, 2011; Coleman &
Berenson, 2004; Forster et al., 2003; Jack et al., 2013; Kripalani, LeFevre, et al., 2007; Van
Walraven, Mamdani, Fang, & Austin, 2004; van Walraven et al., 2010). .
Interestingly, a 2002 Canadian study found that there was a decrease in hospital
readmissions for patients when their primary care physician had received a discharge summary
prior to seeing the patient in a follow-up encounter, although this relationship was not significant
(Van Walraven, Seth, Austin, & Laupacis, 2002). The 2013 study that examined strategies used
by hospitals in the United States to reduce readmission found that having a standardized process
to send discharge summaries to primary care providers was significantly associated with
decreased readmission (Bradley et al., 2013).
26
2.4.12 Telephone reinforcement of the discharge plan
The twelfth and last component of Project RED is “provide telephone reinforcement of
the discharge plan”. A 2006 Cochrane review examined the use of follow-up phone calls and was
unable to make any conclusions on the use of telephone reinforcement because of the wide range
of study methods and outcome measures (Mistiaen & Poot, 2006). More recent studies have
found a significant relationship between the use of hospital follow-up phone calls to patients and
decreased rates of readmission (Costantino, Frey, Hall, & Painter, 2013; Harrison, Auerbach,
Quinn, Kynoch, & Mourad, 2014). Harrison et al (2014) found a significant relationship, but
determined that the significance disappeared when the analysis controlled for patients who were
able to answer the phone. This may mean that patients who are able to answer the phone are
healthier, and have a decreased likelihood of being readmitted to hospital. Interestingly,
Costantino et al (2013) found that although Medicare patients who received a call had
significantly decreased rates of readmission and emergency department use following discharge,
they also had a significantly higher (3%) increase in physician office visits.
2.5 Need for complex interventions
A number of studies have examined the use of single interventions on hospital
readmission, such as medication teaching, or ensuring that patients receive follow-up
appointments prior to discharge. Single interventions have largely not been found to be
associated with decreased readmission. However, several studies have found that complex
interventions consisting of multiple steps are effective in reducing rates of 30 day readmission
(Baker, 2011; Coleman, Parry, Chalmers, & Min, 2006; Hansen, Young, Hinami, Leung, &
Williams, 2011; Mudge, Shakhovskoy, & Karrasch, 2013; Naylor et al., 1994; Naylor et al.,
2004; Shepperd et al., 2013).
27
A 2015 study examined the use of multiple interventions with heart failure patients in a
prospective cohort design that compared an intervention group (n=548) to an historical group
(n=485) in a teaching hospital in San Francisco. The intervention group received discharge
teaching with the use of teach-back, follow-up appointments within one week of discharge,
organization of post-discharge services and follow-up telephone calls within one week. There
were significantly reduced readmissions, which was associated with decreased hospital costs
(Howie-Esquivel et al., 2015).
A similar study took place in a Chicago teaching hospital. This study, led by a
pharmacist, examined the use of medication reconciliation with patients, development of a
patient-specific discharge plan, pharmacist counseling and three phone calls on days 3, 14 and 30
of discharge in a prospective, randomized controlled study of 278 patients. Patients who received
the intervention had significantly fewer readmissions and emergency department visits, as well
as significantly fewer adverse drug events and medication errors than the control group (Phatak
et al., 2016).
The nature of the relationship between discharge practices and readmission is a complex
and requires an approach that recognizes the complexity of this relationship. A strength of
Project RED is that it takes a complex approach and contains multiple steps that are
individualized to the needs of the patient and family. The adoption and use of the health literate
discharge practices contained in Project RED can be conceptualized using the Health Literate
Care Model that is described in section 2.1. The tenets of this framework provided a guide for
the first two studies.
2.6 Need for research
28
The use of health literate best practices has been associated with improved patient
satisfaction, increased adherence to medications, and reduced use of acute care services,
including use within 30 days of hospital discharge (Berkman, Sheridan, Donahue, Halpern,
Viera, et al., 2011; Jack, Chetty, Anthony, Greenwald, Sanchez, Johnson, Forsythe, O'Donnell,
Paasche-Orlow, Manasseh, et al., 2009). Despite the known linkage between health literate best
practices and these outcomes, high rates of readmission persist in North America. This
dissertation examines what hospitals in Ontario are doing to meet the health literacy needs of
their patients and families on discharge.
29
Chapter 3:
Development of Indicators of Health Literate Discharge Practices in
Ontario Hospitals Using a Delphi Panel
3.1 Purpose
The purpose of this study was to develop indicators of health literate discharge practices
in acute care hospitals. The research question was: What are the essential indicators of health
literate discharge practices in acute care hospitals? The 34 indicators from Project RED were
used as the initial set of elements which was then elaborated upon using the approach described
below. The purpose of developing this set of indicators was to create an organizational survey
tool to evaluate the use of health literate discharge practices in Ontario hospitals. To date, the
practices of Project RED have only been evaluated in hospitals in the United States. The use of a
Delphi panel, composed largely of experts from Ontario and Canada, would allow for the
practices of Project RED to be evaluated in the Canadian context.
3.2 Methods
A Delphi method was used to develop indicators of health literate discharge practices
(Boulkedid, Abdoul, Loustau, Sibony, & Alberti, 2011; Fink, Kosecoff, Chassin, & Brook, 1984;
Mullen, 2003). The study received ethics approval from the Office of Research Ethics at the
University of Toronto on September 15, 2014.
3.2.1 Sample
Convenience sampling was used to recruit participants from Ontario, as well as across
North America. The professional network of the investigator, which included members who had
knowledge of research, and researchers in the areas of health literacy and hospital discharge
practices, were the source of potential participants. Potential respondents were contacted by
30
email, and were informed as to why they had been selected to participate. In addition, North
American experts on health literacy and the discharge process were identified through contacts
made at health literacy and health services research conferences in Canada and the United States.
The email addresses of contacts were publicly available from conference attendee lists and from
publications.
An effort was made to include representation from “Health Links.” In 2012, the Ministry
of Health and Long-Term Care in Ontario introduced an initiative called Health Links, which is
specifically designed to improve the coordination of care for patients with multiple complex
conditions who are high users of the health care system, and who have been found to be at high
risk for hospital readmission (Ministry of Health and Long-Term Care, 2015c). As well,
representation was sought from Health Quality Ontario, a government agency that monitors and
reports on health system performance and supports quality improvement in the province of
Ontario (Health Quality Ontario, 2015b).
A total of 55 health care providers, managers and health service researchers were
contacted by email on September 23, 2014. Forty-two potential participants responded that they
would be willing to consider participation. The panel consisted of physicians (7), pharmacists
(7), a discharge planner (1), case managers (2), nurses (5), nurse practitioners (7), hospital
administrators (4) and health service researchers (9). Several panel members are considered
experts on health literacy and/or the hospital discharge process in North America. One hospital
administrator held a position that was cross-appointed to the Health Links in their region. In
addition, two panel members were representatives from Health Quality Ontario. See Table 1 for
the roles of participants.
31
Table 1.
Roles of Participants on the Delphi Panel
Role Number General
medicine
General
medicine
&
outpatient
clinic
Acute
care:
Quality
& Safety
Health
Quality
Ontario
Surgery Home
care
ED Hospital
Clinic
Primary
care
clinic
MD 7 2 3 1 1
RPh 7 3 1 1 2
Case
managers
2 1 1
RN 6 2 1 2 1
NP 7 2 1 1 2 1
Heterogeneous panels have been found to lead to stronger results than homogenous panels (Boulkedid et al., 2011), hence
heterogeneous membership of this panel, with representation of an array of stakeholders was sought.
3.2.2 Data collection
Potential respondents (n = 42) were contacted via electronic mail on October 2, 2014. They were sent a personalized letter that
contained background information about the study and details about the research objectives. The letter included a short description of the
Delphi method and recipients were told that participation would be via electronic mail. See Appendix A for this email and Appendix B for
the participant information letter.
32
In order to increase the response rate, a regular mail option was available if preferred
(Blair, Czaja, & Blair, 2014; Boulkedid et al., 2011; Burns et al., 2008; Dillman, Smyth, &
Christian, 2009). Potential respondents were informed that their consent to participate was
indicated by their choice to respond to the survey. They were asked to respond to the survey by
October 17, 2014. Reminder notices were sent out on October 15 and October 21, 2014. See
Appendix C for the reminder notice. The platform for the web-based survey was Fluid Surveys©.
The first round of the Delphi panel consisted of a list of the 34 indicators of health literate
discharge practices from Project RED (Jack et al., 2013) (see Table 2). The Delphi panel
members were asked to rate the importance of the indicators using a 5-point Likert scale as
follows: 1=not at all important, 2= somewhat important, 3=neutral, 4=important; 5=very
important (Jupp, 2006).
Prior to this first round, the set of indicators were pilot tested with both of the supervisors
and a PhD student at the University of Toronto to ensure that they were clear and that the rating
process was feasible (Powell, 2003). The set of indicators were pre-tested using a desktop
computer, a laptop computer, an iPhone and an android phone.
Table 2.
Indicators of Health Literate Discharge Practices for Round 1.3
Language Assistance
1. Language preference of patient and family is determined and documented
2 If patient and/or family members do not speak English, a translator is arranged for
on discharge
3 Indicators from Jack et al., (2013). Reprinted with Permission from the US Department of
Health and Human Services.
33
3 If patient and/or family members do not speak English, written materials are
provided in the preferred language
Post-discharge appointments, tests
4 Need for primary care and specialty care follow-up is determined
5 If patient does not have a primary care provider, one is located for patients
6 Need for future tests is determined
7 Appointments are made for patient for follow-up appointments and testing
8 If appointments are made, they are made with input from the patient/family
regarding the best time/date
9 If patient requires future diagnostic testing, patient and/or family member is
instructed on any preparation for testing
10 Importance of clinician appointments and further testing is discussed with
patient/family
11 Patient/family is asked about traditional healing practices, and there is confirmation
made that practices are complementary with patient’s discharge plan
12 There is confirmation made with patient/family that they know where to go for
further appointments and tests, and that they have a plan to get to appointments
13 Barriers to keeping appointments are addressed
Plan for follow-up of results from diagnostic lab tests or studies that are pending at
the time of discharge
14 Pending lab and test results are identified with patient/family
15 Determination is made of who will be reviewing the results, and when and how this
information will be communicated to the patient/family
Organization of post-discharge outpatient services and medical equipment if needed
16 If patient requires medical equipment on discharge, there is a process for ensuring
that the medical equipment is obtained
17 Before discharge, patient/family is given contact information for medical equipment
companies, CCAC (as needed)
Medication Review
18 Medication list is reviewed with patient/family
19 Medication reconciliation is done at the time of discharge
20 Patient/family member is given an explanation of what medications to take and
changes in the medication regimen are emphasized
21 Each medication’s purpose, administration and side effects are reviewed with
patient/family
22 Patient/family’s concerns about medication plan are assessed
Discharge plan is reconciled with use of guidelines
23 There is use of guidelines in the development and planning of the discharge process
for patients and family members
Written discharge plan
24 The patient/family is given an easy-to-understand written discharge plan that
includes medications, medical equipment, future appointments, and future
diagnostic tests to take home
25 The written plan is reviewed with the patient/family
26 Patient/family is encouraged to ask questions about the plan
Patient/family education of diagnosis
34
27 Prior to day of discharge, patient/family is met with to provide education about
patient’s diagnosis and treatment and to prepare for discharge
Patient/family’s understanding of the discharge plan is assessed
28 Patient/family is asked to explain, in their own words, the details of the discharge
plan
29 As needed, family members and other caregivers who will share in the care-giving
responsibilities are contacted
Patient/family is instructed on what to do if a problem arises
30 Patient/family is made aware of how to contact primary care provider
31 Patient/family is instructed on what constitutes an emergency and what to do in case
of emergency
Discharge summary is sent to clinicians accepting care of the patient
32 Within 24 hours of discharge, a discharge summary is provided to primary care
provider
Telephone reinforcement of the discharge plan is provided
33 Patient/family is called within 3 days of discharge to reinforce the discharge plan
and help with problem-solving
34 A help line is staffed where phone calls can be answered from patients, families
and/or other caregivers with questions about the at home care plan, hospitalization,
and follow-up plan in order to help patients transition from hospital care to
outpatient care setting
In addition, participants were asked to suggest indicators that they believed required
review in the next round. The items included in the second round were determined by the
analysis of the first round (Hasson, Keeney, & McKenna, 2000). The aim of the second round
was to incorporate indicators suggested by participants for inclusion.
For the second round, participants (n = 42) were contacted using electronic mail on
November 10, 2014. They were sent a personalized letter that contained the same background
information about the study and details about the research objectives that had been sent with the
first round. In addition, they were given a summary of the results from the end of the first round.
Once again, there was a short description of the Delphi method and they were told that
participation would be via electronic mail, and that a regular mail option was available if
preferred (Blair et al., 2014; Burns et al., 2008; Dillman et al., 2009). See Appendix D for this
35
email and Appendix E for the participant information letter. Participants were again notified that
their consent to participate in the study was indicated by their choice to respond to the survey.
They were asked to respond to the survey by November 21, 2014. Two reminder notices were
sent out on November 17 and November 24, 2014. See Appendix F for the reminder notice.
To be retained in the survey, an indicator had to receive a median rating of 3.0 or higher
during the round(s) (Fink et al., 1984). It was anticipated that this study would require two to
three rounds to reach consensus.
A response rate of 85% was required for each of the rounds. There is no established
recommended response rate for the Delphi method (Mullen, 2003). The rate of 85% appeared
reasonable in light of a recent systematic review of 80 studies that used a Delphi method which
found that, for studies reporting a response rate, the median response rates for the first, second,
and third rounds were as follows: 90%, 92% and 88% (Boulkedid et al., 2011). The purpose of
using reminders was to increase response rates (J. Van Geest & Johnson, 2011).
Using the Delphi method offered several advantages for this study. This method is
particularly good for attaining consensus in an area where there has been a lack of research and
where ambiguity exists (Boulkedid et al., 2011; Hasson et al., 2000; Powell, 2003; Skulmoski,
Hartman, & Krahn, 2007), as is the case with health literate hospital discharge practices. The
components of Project RED have been found to be associated with significantly decreased
hospital readmission (Adams et al., 2014; Jack et al., 2009; Markley et al., 2013), but it is not
known which components are the most important. In addition, as most of the panel members
were from Ontario, the Delphi method offered insight into the use of these practices in the
context of Ontario. It was anticipated that the expert opinions obtained using the Delphi method
36
would help to provide some clarity in order to develop a set of indicators of health literate
discharge practices.
The Delphi method relies on anonymity. The group members were not identified and
there was no physical meeting of the group. This was to ensure that no one participant was able
to direct the rankings of other participants (Boulkedid et al., 2011; Fink et al., 1984; Rowe &
Wright, 1999). This is particularly useful in a group consensus method as it prevents one expert
from dominating the process (Fink et al., 1984; Keeney, Hasson, & McKenna, 2006).
3.2.3 Data analysis
Median ratings for each of the indicators were computed, for each round, and summaries
of ratings these were distributed to the panel participants. Median ratings were chosen as Likert
scales were used, which provide an ordinal level of measurement (Field, 2013). In round 1,
participants were asked to suggest indicators for inclusion. The suggested indicators were
categorized using qualitative analysis, and included in material distributed in round 2.
3.3 Results
3.3.1 Round 1
3.3.1.1 Descriptive statistics
For round 1, there were a total of 37 responses (88% response rate). See Table 3.
Most of the respondents were Canadian, and 4 (11%) were from the United States. In
addition, 29 (78.4%) of the respondents worked in the acute care hospital setting. Four (10.8%)
worked in the university setting, 2 (5.4%) worked in the community and 2 (5.4%) worked at
Health Quality Ontario. There was consensus in the responses between these 4 settings.
37
Table 3.
Roles and Work Location of Participants on the Delphi Panel (self-identified), Round 1
Primary roles Total
number
Acute
care
hospital
Community Health
Quality
Ontario
University
Physician 7 6 1
Pharmacist 6 6
Discharge planner
(RN)
1 1
Case manager 2 2
Registered nurse 5 3 1 1
Nurse practitioner 6 5 1
Hospital Administrator 4 4
Researcher 6 2 4
3.3.1.2 Quantitative results
Each of the 34 indicators received a median ranking of 4.0 or 5.0, which is greater than
the median value of 3.0 that was set for inclusion in the survey (see Table 4). The quantitative
results of round 1 of the Delphi panel can be found in Appendix G.
Table 4.
Median Values for Health Literate Discharge Practices – Round 1
Item Health literate discharge practice Range Mean Median
Language Assistance
1 Language preference of patient and family is determined
and documented
3-5 4.62 5
2 If patient and/or family members do not speak English,
a translator is arranged for on discharge
2-5 4.38 5
3 If patient and/or family members do not speak English,
written materials are provided in the preferred language
2-5 4.27 4
Post-discharge appointments, tests
4 Need for primary care and specialty care follow-up is
determined
3-5 4.78 5
5 If patient does not have a primary care provider, one is
located for patients
3-5 4.68 5
6 Need for future tests is determined 3-5 4.3 4
7 Appointments are made for patient for follow-up
appointments and testing
3-5 4.57 5
38
8 If appointments are made, they are made with input
from the patient/family regarding the best time/date
1-5 4.11 4
9 If patient requires future diagnostic testing, patient
and/or family member is instructed on any preparation
for testing
1-5 4.41 5
10 Importance of clinician appointments and further testing
is discussed with patient/family
3-5 4.46 5
11 Patient/family is asked about traditional healing
practices, and there is confirmation made that practices
are complementary with patient’s discharge plan
2-5 3.83 4
12 There is confirmation made with patient/family that they
know where to go for further appointments and tests,
and that they have a plan to get to appointments
3-5 4.51 5
13 Barriers to keeping appointments are addressed 3-5 4.49 5
Plan for follow-up of results from diagnostic lab tests
or studies that are pending at the time of discharge
14 Pending lab and test results are identified with
patient/family
1-5 4.24 4
15 Determination is made of who will be reviewing the
results, and when and how this information will be
communicated to the patient/family
1-5 4.38 5
Organization of post-discharge outpatient services
and medical equipment if needed
16 If patient requires medical equipment on discharge,
there is a process for ensuring that the medical
equipment is obtained
3-5 4.54 5
17 Before discharge, patient/family is given contact
information for medical equipment companies, CCAC
(as needed)
3-5 4.46 5
Medication Review
18 Medication list is reviewed with patient/family 4-5 4.92 5
19 Medication reconciliation is done at the time of
discharge
3-5 4.92 5
20 Patient/family member is given an explanation of what
medications to take and changes in the medication
regimen are emphasized
3-5 4.97 5
21 Each medication’s purpose, administration and side
effects are reviewed with patient/family
3-5 4.57 5
22 Patient/family’s concerns about medication plan are
assessed
3-5 4.65 5
Discharge plan is reconciled with use of guidelines
23 There is use of guidelines in the development and
planning of the discharge process for patients and family
members
1-5 4.22 4
Written discharge plan
24 The patient/family is given an easy-to-understand
written discharge plan that includes medications,
3-5 4.78 5
39
medical equipment, future appointments, and future
diagnostic tests to take home
25 The written plan is reviewed with the patient/family 3-5 4.7 5
26 Patient/family is encouraged to ask questions about the
plan
3-5 4.68 5
Patient/family education of diagnosis
27 Prior to day of discharge, patient/family is met with to
provide education about patient’s diagnosis and
treatment and to prepare for discharge
3-5 4.57 5
Patient/family’s understanding of the discharge plan
is assessed
28 Patient/family is asked to explain, in their own words,
the details of the discharge plan
3-5 4.38 4
29 As needed, family members and other caregivers who
will share in the care-giving responsibilities are
contacted
2-5 4.24 4
Patient/family is instructed on what to do if a
problem arises
30 Patient/family is made aware of how to contact primary
care provider
2-5 4.46 5
31 Patient/family is instructed on what constitutes an
emergency and what to do in case of emergency
3-5 4.78 5
Discharge summary is sent to clinicians accepting
care of the patient
32 Within 24 hours of discharge, a discharge summary is
provided to primary care provider
3-5 4.73 5
Telephone reinforcement of the discharge plan is
provided
33 Patient/family is called within 3 days of discharge to
reinforce the discharge plan and help with problem-
solving
3-5 4.36 4
34 A help line is staffed where phone calls can be answered
from patients, families and/or other caregivers with
questions about the at home care plan, hospitalization,
and follow-up plan in order to help patients transition
from hospital care to outpatient care setting
3-5 4.11 4
3.3.1.3 Qualitative results
The indicators suggested by participants were reviewed and analyzed qualitatively. They
were divided into five themes: 1) discharge summaries for clinicians; 2) medication review; 3)
patient and family assessment; 4) patient and family education, and 5) follow-up after hospital
40
discharge. As a result of this analysis, two new indicators were developed, and two indicators
from Project RED were re-worded. See Table 5 for a description of the themes.
Table 5.
Themes from Round 1
Theme Suggested indicators Comments and changes made
Discharge
summaries
for clinicians
Discharge summaries are
standardized so it is easy to find
information.
This suggestion was partially addressed
by item 32: Within 24 hours of
discharge, a discharge summary is
provided to patient care provider.
An additional indicator was added:
Discharge summary has a standardized
format so that information is easy to
find.
Medication
review
Medication reconciliation
documentation is sent to
community pharmacist and primary
care clinician. This would be in
addition to the discharge summary
The post-discharge medication plan
and rationale for medication
changes are communicated with the
primary care provider and the
community pharmacist for high risk
patients on high risk medications
(e.g. warfarin, insulin, phenytoin,
digoxin).
Post-discharge medication reviews
are ordered to be completed by the
community pharmacist.
An additional indicator was added:
Patient/family is referred to community
pharmacist within 2 weeks of discharge
for a medication review.
Patient and
family
assessment
Education level:
Assessment of level of education
The patient teaching is tailored to
the patient education level,
capabilities, and preferences
Finding out about patients’ reading
capabilities is not the same as
determining whether patients
understand health information (. Institute of Medicine, 2009). There is
mixed evidence for the idea that
education level is correlated with health
literacy level. Education has been found
to have an influence on health literacy
level (Murray et al., 2008), however, an
insignificant relationship between
education and health literacy level has
41
also been found (Dickens, Lambert,
Cromwell, & Piano, 2013).
One’s level of health literacy can
depend on one’s circumstances. For
example, the stress of a new diagnosis
or disease exacerbation may
compromise one’s level of health
literacy (Koh, Brach, et al., 2013). For
these reasons, these suggestions were
not included in the second round.
Modalities of learning:
How patients’ families like or
prefer to get information (format)
Assessment of how best do patients
learn (modalities)
Use of technology
(tablets/computers) are used to
complement verbal education /
communication
Hospitals are often limited in the kinds
of educational formats that they can
provide to patients (oral, written
materials). The indicators are meant to
address practices, and not delivery
methods. As these suggestions were
outside of the scope of the study, they
were not included.
Patients’ learning goals, concerns:
What are patients learning goals.
I think it's important to elicit
patient's fears and anxieties and
address them prior to discharge. In
addition, it's important to ensure the
patients feel that their concerns for
coming to hospital have been met
(they frequently indicate that the
reason they went to hospital were
never resolved).
Patients’ learning goals are addressed
by items: 22. Patient/family’s concerns
about medication plan are assessed; 26.
Patient/family is encouraged to ask
questions about the (discharge) plan.
Patient and
family
education
A copy of (the discharge summary)
should also be provided to the
patient/family
Patients do not understand the
language in which prescriptions are
written (BID, TID, QID, OD). A
patient friendly way to write
prescription is to use “take this
medication at breakfast, lunch, and
dinner.
Any educational material that
patients are going to receive should
be written in plain language with
clear writing, an effective
organization, an inviting
appearance to ensure
understandability and actionability.
The wording of item 23: “The
patient/family is given an easy-to-
understand written discharge plan that
includes medications, medical
equipment, future appointments, and
future diagnostic tests to take home”
(Jack et al., 2013) was changed to:
The patient/family is given an easy-to-
understand written, prioritized
discharge plan that includes
medications, medical equipment, future
appointments, and future diagnostic
tests to take home. (Adapted from Jack
et al, 2013).
42
The information to be discussed is
prioritized to 3-5 points
Follow-up
after hospital
discharge
The help line should be staffed by
registered nurses who have access
to the patients’ medical records.
Help line calls should be forwarded
to primary care for communication
and relevant follow up.
A contact number for the
discharging team is always
available on discharge summaries.
The wording of item 34: “A help line is
staffed where phone calls can be
answered from patients, families and/or
other caregivers with questions about
the at home care plan, hospitalization,
and follow-up plan in order to help
patients transition from hospital care to
outpatient care setting” (Jack et al.,
2013) was changed to: Patient/family is
provided with a phone number where
they can speak with a hospital staff
member to ask questions about the at-
home care plan, hospitalization, and
follow-up plan in order to help patients
transition from hospital care to
outpatient care setting (Adapted from
Jack et al, 2013).
3.3.2 Round 2
3.3.2.1 Descriptive statistics
For round 2, there were a total of 39 responses4 (93% response rate). See Table 6.
Most of the respondents were Canadian, and 7 (18%) were from the United States. In
addition, 29 (74.4%) of the respondents worked in the acute care hospital setting. Six (15.4%)
worked in the university setting, 2 (5.1%) worked in the community and 2 (5.1%) worked at
Health Quality Ontario. There was consensus in the responses among these 4 settings.
4There were more responses in the Round 2. Each of the rounds was anonymous, and the indicators were sent to all
42 participants in the sample.
43
Table 6.
Roles of Participants on the Delphi Panel (self-identified), Round 2
Primary roles Total
number
Acute
care
hospital
Community Health
Quality
Ontario
University
Physician 6 5 1
Pharmacist 6 5 1
Discharge planner
(RN)
1 1
Case manager 3 3
Registered nurse 6 4 1 1
Nurse practitioner 6 5 1
Hospital
Administrator
4 4
Researcher 7 2 5
3.3.2.2 Quantitative results
For round 2, participants were informed that each of the 34 indicators from Round 1 had
been rated as important or very important. They were told that in response to the feedback
received in round 1, two of the indicators were reworded, and two new indicators were
added. Participants were asked to rate these four indicators using the scale from 1 (not at all
important) to 5 (very important) (see Table 7). The quantitative results of round 2 of the Delphi
panel can be found in Appendix H.
Table 7
Health Literate Discharge Practices – Round 2
New practice or rewording of practice is in italics.5
5 Indicators from Jack et al., (2013). Reprinted with Permission from the US Department of
Health and Human Services.
44
Medication Review
Patient/family is referred to community pharmacist within 2 weeks of discharge for a medication
review
Written discharge plan
The patient/family is given an easy-to-understand written, prioritized discharge plan that includes
medications, medical equipment, future appointments, and future diagnostic tests to take home
Discharge summary is sent to clinicians accepting care of the patient
Discharge summary has a standardized format so that information is easy to find
Telephone reinforcement of the discharge plan is provided
Patient/family is provided with a phone number where they can speak with a hospital staff member
to ask questions about the at-home care plan, hospitalization, and follow-up plan in order to help
patients transition from hospital care to outpatient care setting
Each of the four indicators received a median ranking of 4.0 or 5.0, which is greater than
the median value of 3.0 that was set for inclusion in the list of indicators (see Table 8)
Table 8.
Median Values for New/Revised Health Literate Discharge Practices – Round 2
Item Health literate discharge practice Range Mean Median
Medication Review
New Patient/family is referred to community pharmacist
within 2 weeks of discharge for a medication review
2-5 4.18 4
Written discharge plan
New The patient/family is given an easy-to-understand
written, prioritized discharge plan that includes
medications, medical equipment, future appointments,
and future diagnostic tests to take home
1-5 4.85 5
Discharge summary is sent to clinicians accepting
care of the patient
24* Discharge summary has a standardized format so that
information is easy to find
3-5 4.79 5
Telephone reinforcement of the discharge plan is
provided
34* Patient/family is provided with a phone number where
they can speak with a hospital staff member to ask
questions about the at-home care plan, hospitalization,
and follow-up plan in order to help patients transition
from hospital care to outpatient care setting
2-5
4.31 4
*re-worded items
45
All 36 indicators of health literate discharge practices were retained (see Table 9).
Table 9
Indicators of Health Literate Discharge Practices
The two reworded indicators and the two new indicators are italicized (see items 23, 25, 33 and
36).6
Item Discharge Practice
Language Assistance
1 Language preference of patient and family is determined and documented
2 If patient and/or family members do not speak English, a translator is arranged
for on discharge
3 If patient and/or family members do not speak English, written materials are
provided in the preferred language
Post-discharge appointments, tests
4 Need for primary care and specialty care follow-up is determined
5 If patient does not have a primary care provider, one is located for patients
6 Need for future tests is determined
7 Appointments are made for patient for follow-up appointments and testing
8 If appointments are made, they are made with input from the patient/family
regarding the best time/date
9 If patient requires future diagnostic testing, patient and/or family member is
instructed on any preparation for testing
10 Importance of clinician appointments and further testing is discussed with
patient/family
11 Patient/family is asked about traditional healing practices, and there is
confirmation made that practices are complementary with patient’s discharge
plan
12 There is confirmation made with patient/family that they know where to go for
further appointments and tests, and that they have a plan to get to appointments
13 Barriers to keeping appointments are addressed
Plan for follow-up of results from diagnostic lab tests or studies that are pending at
the time of discharge
14 Pending lab and test results are identified with patient/family
15 Determination is made of who will be reviewing the results, and when and how
this information will be communicated to the patient/family
Organization of post-discharge outpatient services and medical equipment if
needed
6 Indicators from Jack et al., (2013). (Items 23 and 33 are new. Items 25 and 36 are adapted).
Reprinted with Permission from the US Department of Health and Human Services.
46
16 If patient requires medical equipment on discharge, there is a process for
ensuring that the medical equipment is obtained
17 Before discharge, patient/family is given contact information for medical
equipment companies, home care services (as needed)
Medication Review
18 Medication list is reviewed with patient/family
19 Medication reconciliation is done at the time of discharge
20 Patient/family member is given an explanation of what medications to take and
changes in the medication regimen are emphasized
21 Each medication’s purpose, administration and side effects are reviewed with
patient/family
22 Patient/family’s concerns about medication plan are assessed
23 Patient/family is referred to community pharmacist within 2 weeks of discharge
for a medication review
Discharge plan is reconciled with use of guidelines
24 There is use of guidelines in the development and planning of the discharge
process for patients and family members
Written discharge plan
25 The patient/family is given an easy-to-understand written, prioritized discharge
plan that includes medications, medical equipment, future appointments, and
future diagnostic tests to take home
26 The written plan is reviewed with the patient/family
27 Patient/family is encouraged to ask questions about the plan
Patient/family education of diagnosis
28 Prior to day of discharge, patient/family is met with to provide education about
patient’s diagnosis and treatment and to prepare for discharge
Patient/family’s understanding of the discharge plan is assessed
29 Patient/family is asked to explain, in their own words, the details of the discharge
plan
30 As needed, family members and other caregivers who will share in the care-
giving responsibilities are contacted
Patient/family is instructed on what to do if a problem arises
31 Patient/family is made aware of how to contact primary care provider
32 Patient/family is instructed on what constitutes an emergency and what to do in
case of emergency
Discharge summary is sent to clinicians accepting care of the patient
33 Discharge summary has a standardized format so that information is easy to find
34 Within 24 hours of discharge, a discharge summary is provided to primary care
provider
Telephone reinforcement of the discharge plan is provided
35 Patient/family is called within 3 days of discharge to reinforce the discharge plan
and help with problem-solving
36 Patient/family is provided with a phone number where they can speak with a
hospital staff member to ask questions about the at-home care plan,
hospitalization, and follow-up plan in order to help patients transition from
hospital care to outpatient care setting
47
3.4 Discussion
This study used a consensus method to evaluate the use of the 34 practices contained in
Project RED as indicators of health literate discharge practices. Members of the Delphi panel
were invited to contribute additional items.
All 34 practices from Project RED were rated as “important” or “very important” using a
5-point Likert scale. The additional items that were contributed by panel members were analyzed
qualitatively. Two items from Project RED were re-worded, and two additional indicators were
added.
In this way, the indicators are contextualized to Ontario hospitals. Several participants
identified the need to refer patients to a community pharmacist upon discharge to review
medications. This may be seen as Ontario-specific measure, as the MedsCheck program was
established by the Ministry of Health and Long-Term Care in 2007. Through this program,
community pharmacists consult with patients to review medications and ensure patient
understanding of medication regimens (Ministry of Health and Long-Term Care, 2008).
In addition, several participants identified the need for patients to be given a phone
number that they can use following discharge to have questions and concerns addressed. The
original indicator from Project RED states that a help line be staffed to answer calls from patients
and family members following discharge (Jack et al., 2013). As hospitals in Ontario are not
likely to have a help line that is staffed, this indicator was adjusted, and the wording was
changed to providing patients and family members with a phone number where they can speak
with a hospital staff member to ask questions.
48
These 36 indicators can be used to evaluate the use of health literate discharge practices,
which are strategies of health literate organizations. As described in the preceding sections, the
use of these discharge practices has been associated with improved patient and system outcomes
(Adams et al., 2014, Jack et al., 2009; Markley et al., 2013). The use of the Delphi panel allowed
for the initial development of an organizational survey tool, consisting of 36 items, to assess the
use of these practices in Ontario hospitals.
To date, there are no published studies of what acute care hospitals in Ontario are doing
to manage their discharge processes, although there is evidence of some hospitals adopting
health literate practices and, in particular, of using health literate discharge practices (Health
Quality Ontario, 2013b; Markley et al., 2013).
3.5 Theoretical contribution
In the health literate care model, the indicators developed with this Delphi panel fit as
strategies for health literate organizations. See Figure 2.
Figure 2. Health Literate Care Model with Health Literate Discharge Practices. (Office of
Disease Prevention and Health Promotion, 2015). Reprinted, with adaptations, with permission
from the US Department of Health and Human Services
49
This revised framework focuses on the use of health literate discharge practices, which are
strategies of health literate organizations. They improve verbal and written communication with
patients and families; they provide links to post-discharge services such as home care, follow-up
appointments and tests, and they actively engage patients and families. Several studies from the
United States have already demonstrated that the use of health literate discharge practices lead to
the improved outcomes listed: decreased readmission rates, decreased adverse events, increased
patient satisfaction and decreased costs (Berkman, Sheridan, Donahue, Halpern, Viera, et al.,
2011; Jack, Chetty, Anthony, Greenwald, Sanchez, Johnson, Forsythe, O'Donnell, Paasche-
Orlow, Manasseh, et al., 2009). While there is a need to further examine the use of these
practices in the Ontario context, the first step in the introduction of health literate discharge
practices is to measure the current level of performance in Ontario hospitals.
3.6 Limitation
50
Although several members of the Delphi panel were from across Canada, and there were
several members from the United States, most of the panel members were from Ontario, Canada.
This list of indicators would likely require modification for use in acute care hospital settings
outside of North America.
3.7 Practice implications
Hospital managers and health care providers are increasingly aware of the need to meet
the health literacy needs of their patients and families (Koh, Brach, et al., 2013). The Institute of
Medicine has identified the need for health care organizations to become health literate (Institute
of Medicine, 2012). To date, there is no means of measuring the use of health literature
discharge practices in acute care hospitals. Examining current practices and processes is a
logical first step toward becoming a health literate hospital. Measuring current performance
identifies areas for improvement, and provides a baseline with which to check for improvements
(Kripalani et al., 2014). The indicators developed here may be used by hospitals as a means of
doing so, and may identify areas that need improvement (Kowalski et al., 2015). Together, these
indicators comprise a tool with the potential to allow policy makers and hospitals to monitor the
performance of hospital discharge, as well as their progress in using and promoting health
literate strategies.
51
Chapter 4:
Use of Health Literate Discharge Practices in Ontario Hospitals
4.1 Background
At present, there is no organizational survey to measure the use of health literate
discharge practices. The preceding chapter describes the development of a set of health literate
discharge practice indicators. The Delphi technique was used to establish construct and content
validity of the indicators with a panel of health care providers, managers and researchers who
have experience and expertise in the hospital discharge and/or health literacy. The development
of this set of indicators, through the use of the Delphi panel, was the first step in the creation of
the organizational survey consisting of 36 items. For each survey item, the respondents are asked
to rate the use of the discharge practice at their hospital with the following scale: 1=never,
2=seldom, 3= undecided, 4=often, 5=almost always (Burns et al., 2008; Jupp, 2006). See
Appendix I.
As described in Chapter 2, this survey is based on the 12 components of Project RED Jack et
al., 2013, p.11):
1. Language assistance
2. Post discharge appointments, tests
3. Plan for the follow up of results from diagnostic lab tests or studies that are pending at
discharge.
4. Organization of post discharge outpatient services and medical equipment if needed.
5. Medication review.
6. Discharge plan is reconciled with use of guidelines.
7. Written discharge plan.
8. Patient/family education of diagnosis.
9. Patient/family’s understanding of the discharge plan is assessed.
10. Patient/family is instructed on what to do if a problem arises.
11. Discharge summary is sent to clinicians accepting care of the patient.
12. Telephone reinforcement of the discharge plan is provided.
52
Each of these components contain one to ten survey items (as shown in Appendix I). The items
belong to the component with which they conceptually fit, however, this “fit” has not been
tested.
4.2 Research goals
The goals of this study were to find out about the use of health literate discharge practices
in Ontario hospitals and to determine the psychometric properties of the organizational survey as
this was the first time it was being administered to hospitals. In addition, a goal of this study was
to develop an understanding of the construct of “health literate discharge practices” in acute care
hospitals.
The research questions were:
1. To what extent do Ontario hospitals use health literate discharge practices?
2. Using the results of the survey of health literate discharge practices, what are the
factors that constitute the construct “use of health literate discharge practices” in
Ontario hospitals?
2a. What is the reliability of this survey?
2b. Do the factors that measure the construct “use of health literate discharge
practices” in Ontario hospitals match the components contained in Project RED?
4.3 Methods
This study received ethics approval from the Office of Research Ethics at the University
of Toronto on January 29, 2015. In addition, it received ethics approval from the Research
Ethics Board at St. Michael’s Hospital on June 29, 2015.
4.3.1 Sample
53
The sample consisted of the nursing managersof adult, general medicine units in the 143
Ontario acute care hospitals. There are 15 teaching hospitals, 48 large community hospitals
(hospitals with 100 beds or more) and 80 small community hospitals (hospitals with less than
100 beds) (Ministry of Health and Long-Term Care, 2013a, 2013b, 2013c). See Appendices J, K
and L for lists of the hospitals. Nursing managers were asked to complete the survey, or to share
the survey with a health care provider on their unit who had knowledge of the discharge process
at their hospitals. Nursing managers were chosen as they were considered the central point of
access to information about hospital discharge on inpatient medical units.
To identify nursing managers and to obtain their contact information, every hospital in
Ontario was contacted by phone. The name and email address of the nursing manager(s) for the
general medicine unit(s) was obtained from the inpatient medicine ward, or the locating
department of the hospital. The survey was emailed to potential participants with a request to
either complete the survey or to share it with a health care provider on the general medicine
patient care unit who has experience in the hospital discharge process.
4.3.2 Data collection
Pre-testing of the survey tool was done prior to its administration. The survey was pilot-
tested using the web based platform Fluid Surveys© with two nurses, a nurse practitioner, one
pharmacist who was also a health services researcher and a non-clinician health services
researcher. This testing was done to evaluate the survey items and to ensure that the questions
were clearly written so that they would be correctly interpreted by respondents (Burns et al.,
2008). As with the pre-testing done for the Delphi panel, the survey was pre-tested using a
desktop computer, a laptop computer, an iPhone and an android phone.
54
Potential respondents (n = 212) were contacted via electronic mail between March 17 and
August 14, 2015. There were a greater number of surveys (212) distributed than hospitals (143)
as a number of hospitals had more than one medicine unit or more than one nursing manager for
the medical unit(s). They were sent a personalized letter that contained background information
about the study and details about the research objectives. The letter included a short description
of the survey and recipients were told that participation would be via electronic mail. In order to
increase the response rate, a regular mail option was available if preferred (Blair et al., 2014;
Boulkedid et al., 2011; Burns et al., 2008; Dillman et al., 2009). See Appendix M for this email
and Appendix N for the participant information letter.
Sponsorship from a professional association has been found to be associated with
increased response rates (J Van Geest, Johnson, & Welch, 2007), and sponsorship for the survey
was obtained from the Ontario Hospital Association. See Appendix O for the letter of
endorsement that was sent to all potential respondents.
Potential respondents were informed that their consent to participate was indicated by
their choice to respond to the survey. They were asked to respond to the survey within 2 weeks.
Reminder notices were sent out after one and two weeks. As with the pilot testing, the platform
for the web-based survey was Fluid Surveys©. See Appendix P for the reminder notice.
The survey was electronic, and each question required an answer. If a participant missed
a question, the survey did not let them proceed until they had answered the question that was
missed. In the case where one participant missed a question on a paper version of the survey, the
principal investigator followed up with the participant to obtain an answer for the missing
question. Incomplete surveys were not included in this analysis.
55
4.3.3 Analysis
4.3.3.1 Characteristics of participants and hospitals
Descriptive statistics were produced to determine the baseline characteristics of
participants and hospitals. These characteristics included: role of participant and hospital group
(i.e. teaching hospital, large community hospital and small community hospital). Hospitals’
locations were categorized by the Local Health Integration Network (LHINs) in which they are
located. There are 14 LHINs in Ontario. The LHINs receive their funding from Ontario’s
Ministry of Health and Long-Term Care. They are responsibility for planning, funding and
integrating hospital, home and community services (Deber, 2014).
In addition, characteristics of the hospitals included rurality scores. The rurality score is a
composite value that reflects a community’s location with respect to population density and
travel times for basic and advanced health care services. The rurality index ranges from 0
(highest population density, lowest travel times) to 99 (lowest population density, highest travel
times) (Kralj, 2009).
4.3.3.2 Survey items and total scores
Descriptive statistics were produced for the survey items and total scores. For the survey
items, number and percentages of scores are calculated. For the total scores, the mean, standard
deviation and median values are presented as well as the distribution of the scores.
It was anticipated that there would be hospitals with multiple responses. For hospitals
with multiple responses, the median value for the health literate survey score was used.
4.3.3.3 Testing assumptions
56
Assumptions of normality for the distribution of scores was checked by examining the
distribution and the tests for skewness and kurtosis. Correlations between items were checked by
using Spearman’s rho to ensure that there was sufficient correlation between the variables to
proceed with analysis. Sampling adequacy was assessed using the Kaiser-Meyer-Olkin (Field,
2013). It is particularly important to check sampling adequacy with small samples (Gorsuch,
1997).
4.3.3.4 Exploratory factor analysis
Exploratory factor analysis was undertaken to gain an understanding of the structure of
the construct of interest, that is, the organizational use of health literate discharge practices
(Field, 2013). This was the first time this organizational survey was being administered to
hospitals. Another purpose of exploratory factor analysis is to reduce the number of items or
measures in a measurement instrument. In a survey, this could consist of reducing the number of
survey items (Field, 2013). As this was the analysis of a new survey, the purpose of this analysis
did not include reducing the number of survey items (Hinkin, 1998).
Exploratory factor analysis was done with principal axis factors extraction. Principal axis
factors extraction is recommended when results are non-normal, and it was anticipated that the
results would be non-normal. In addition, it has the benefit of being widely used and understood
(Costello & Osborne, 2005; Tabachnick & Fidell, 2007). Exploratory factor analysis was first
done without rotation, to determine the number of factors, and was then done with rotation.
Rotation is used to maximize the loading of items on as few factors as possible. This helps
improve the interpretability of the results (Meyers, Gaust, & Guarino, 2013; Tabachnick &
Fidell, 2007).
57
The assumption was made that the factors underlying the items in the survey are
correlated, as all of the survey items are measuring the use of a health literate discharge practice,
hence oblique rotation was used. As Costello & Osborne (2005) point out, in the social sciences,
some correlation between factors is expected as the items making up the factors are unlikely to
be completely independent of one another. Several authors contend that an orthogonal rotation
should only be done if there is a clear reason to do so and that otherwise, an oblique rotation is
preferred (Conway & Huffcutt, 2003; Costello & Osborne, 2005; Fabrigar, Wegener,
MacCallum, & Strahan, 1999; Floyd & Widaman, 1995; Gaskin & Happell, 2014; Tabachnick &
Fidell, 2007).
Factor correlation matrices were examined to confirm that the items were correlated
(Meyers et al., 2013). According to Meyers et al (2013), factor correlations equal to or greater
than .32 are considered appropriate for oblique rotation. When values are between .1 and .3, it is
less clear whether factors have enough correlation to rule out orthogonal rotation. These authors
state that when the results are not clear, the safest and most conservative route is to proceed with
oblique rotation. In addition, Meyers et al (2013) reinforce the notion that this decision to
proceed with oblique or orthogonal rotation should be based on the underlying theory. As stated
above, oblique rotation was chosen because the assumption is being made that the survey items
measuring the use of health literate discharge practices are correlated with each other.
Because the sample size was relatively small for performing factor analysis (less than
200), direct oblimin was chosen, as recommended by Field (2013). Promax is an alternate
method of performing oblique rotation however it is best suited to large data sets; Meyers et al.,
(2013) point out that the two methods will nearly always produce similar solutions.
58
To determine the number of factors to be considered in the analysis, both Kaiser’s
criterion7 and the scree plot were considered (Costello & Osborne, 2005; Fabrigar et al., 1999;
Field, 2013; Tabachnick & Fidell, 2007). This is consistent with the recommendation that more
than one test be used to determine the number of factors (Conway & Huffcutt, 2003; Fabrigar et
al., 1999). Parallel analysis is another technique to determine the number of variables (Costello
& Osborne, 2005; Courtney & Gordon, 2013; Gaskin & Happell, 2014). It was not performed as
this is only meant to be an initial analysis of the survey items.
The determination of the number of factors was also made based on the sample size.
Arrindell & Ende (1985) make the recommendation that the sample size should be used to
determine the number of factors. Their work demonstrates that stable factor solutions are found
when the sample size is approximately 20 times the number of factors. For example, with a
sample of 100, the authors recommend using a 5 factor solution.
Factor loadings greater than 0.3 were considered in the interpretation, based on
recommendations by a number of authors. Factor loadings are correlations of the items or
variables with the factors.(Costello & Osborne, 2005; Fabrigar et al., 1999; Field, 2013; Floyd &
Widaman, 1995; Kline, 2014; Tabachnick & Fidell, 2007). Costello & Osborne (2005) make the
point that a factor loading of .5 or higher reflects a strongly loaded item. This means that the item
is highly correlated with the factor.
4.3.3.4.1 Sample size for exploratory factor analysis
7 Kaiser’s criterion states that those factors with an eigenvalue of greater than 1 should be kept (Field, 2013)
59
When considering sample size in factor analysis, there are two sets of recommendations
to consider. One set of recommendations is focused on the size of the total sample. The second
set focuses on the subject to variable ratio.
Several authors recommend a sample of at least 300 (Comrey & Lee, 1992; Conway &
Huffcutt, 2003; Tabachnick & Fidell, 2007), although several authors note that smaller samples
can be considered if there are variables with high factor loadings and high communalities
(Fabrigar et al., 1999; MacCallum, Widaman, Zhang, & Hong, 1999; Tabachnick & Fidell,
2007). However, as Costello & Osborne (2005) note, this is an unlikely situation with data in the
social sciences.
There are two main issues with a low sample size in performing factor analysis. If the
sample is too small, the results are less likely to be generalizable to the population. Secondly, in
exploratory factor analysis, items are associated with factors, and with smaller samples, there is a
greater chance that items will be associated with the wrong factor (Costello & Osborne, 2005).
With respect to the subject to variable ratio, several authors recommend a ratio of 10:1
(Costello & Osborne, 2005; Everitt, 1975; Field, 2013; Kass & Tinsley, 1979; Nunnally, 1978).
Gorsuch (1983) states that an absolute minimum subject to variable ratio is 5:1, with a sample of
at least 100. Hatcher (1994) also recommends that the sample size be the larger of two criteria:
either 1) have a 5:1 ratio, or 2) be at least 100. Catell (1978) states that a subject to variable ratio
of 3:1 may be acceptable, but only with an absolute minimum sample size of 250 (Cattell, 1978).
Arrindell & van der Ende (1985) point out that subject to variable ratio recommendations
are inconsistent and vague. Their work found that a 1.3:1 was an acceptable subject to variable
ratio with a minimum sample size of 100, and that a 3.9:1 was an acceptable subject to variable
60
ratio with a minimum sample size of 78. As noted in the preceding section, the authors couple
this recommendation with the recommendation that the sample size should be 20 times the
number of factors (Arrindell & Van der Ende, 1985). This recommendation was considered in
determining the factor solution that had the best fit for the survey data.
4.3.3.5 Reliability testing
Reliability testing was done to assess whether the survey is consistently reflecting the
construct being measured (Field, 2013). It was first done with all 36 survey items, and then for
each factor as it is recommended that reliability testing be applied separately to subscales
(Cronbach, 1951). A reliability level of at least .7 is regarded as acceptable (Cortina, 1993; Field,
2009).
4.3.3.5.1 Inter-rater reliability testing
It was anticipated that there would be hospitals that had more than one response as there
are hospitals in Ontario with more than one medicine unit. For hospitals with 2 survey responses,
inter-rater reliability testing was done using Cohen’s kappa (Meyers et al., 2013). For hospitals
with more than 2 survey responses, inter-rater reliability testing was done using the intraclass
correlation coefficient (Field, 2013).
4.4 Results
4.4.1 Baseline characteristics of participants and hospitals
See Table 10 for baseline characteristics of participants.
Table 10.
Baseline Characteristics of Participants and Hospitals
61
Characteristic n (%)
Role Registered nurse 23 (23.2)
Nurse practitioner 6 (6.1)
Discharge planner 3 (3)
Case manager 1 (1)
Nursing manager 65 (65.7)
Pharmacist 1 (1)
Hospital
group
Teaching hospital N=15 19 (19.2)
Large community N=48 39 (39.4)
Small community N=80 41 (41.4)
LHIN Erie St Clair 7 (7.1)
South West 8 (8.2)
Waterloo Wellington 6 (6.1)
Hamilton Niagara Haldimand Brant 8 (8.1)
Central West 2 (2)
Mississauga Halton 7 (7.1)
Toronto Central 8 (8.1)
Central 7 (7.1)
Central East 4 (4)
South East 5 (5.1)
Champlain 10 (10.1)
North Simcoe Muskoka 4 (4)
North East 13 (13.1)
North West 10 (10.1)
There were 99 participants (46% response rate) from 79 hospitals. The majority of
participants were nursing managers (65.7%). Other participants included registered nurses, nurse
practitioners, discharge planners, one case manager and one pharmacist. Almost one-fifth
(19.2%) of the participants were from teaching hospitals. 39.4% were from large community
hospitals and 41.4% were from small community hospitals. The majority (81.8%) responded to
the internet-based survey. Seventeen participants (17.2%) filled out the survey by hand, and
returned it via email. One participant (1%) mailed the survey.
There was representation from each of the 14 LHINs in Ontario.
62
Multiple responses were received from 12 hospitals. See Table 11 for characteristics of
the hospitals that had more than one response.
Table 11.
Characteristics of Hospitals with More Than One Response (n=12)
Characteristic Number of
hospitals
Number of
responses
Hospital group Teaching hospital 4 11
Large community 7 19
Small community 1 2
Total 12 32
LHIN Erie St Clair 1 2
South West 0 -
Waterloo Wellington 2 5
Hamilton Niagara
Haldimand Brant
1 2
Central West 0 -
Mississauga Halton 1 5
Toronto Central 2 5
Central 1 3
Central East 1 2
South East 1 2
Champlain 1 2
North Simcoe Muskoka 0 -
North East 0 -
North West 1 4
Total 12 32
4.4.1.1 Rurality scores
The rurality scores of the 99 participants ranged from 0 (Toronto, Kingston, London,
Ottawa, Thunder Bay and Windsor) to 99 (Fort Frances). Most of the sample (51%) had a
rurality score of 10 or less. The distribution of the rurality scores was positively skewed. This
distribution is similar to the distribution of all of the acute care hospital sites in Ontario (N=143).
This distribution is also positively skewed, and a third of all hospitals (33.6%) have a rurality
63
score of 10 or less. See Appendix Q for the distribution of the rurality scores of the sample, and
for the distribution of the rurality scores of all adult, acute care hospitals in Ontario.
64
4.4.2 Survey results
The results of the survey are summarized below (n = 99) in Table 12.
Table 12.
Survey Results. Results are listed as frequencies (percentages).
Survey items 1=
never
2 =
seldom
3 =
undecided
4 =
often
5 =
almost always
1 Language preference of patient and family is
determined and documented
8 (8.1) 13 (13.1) 2 (2) 29 (29.3) 47 (47.5)
2 If patient and/or family members do not speak
English, a translator is arranged for on discharge
3 (3) 26 (26.3) 15 (15.2) 35 (35.4) 20 (20.2)
3 If patient and/or family members do not speak
English, written materials are provided in the
preferred language
17 (17.2) 39 (39.4) 12 (12.1) 22 (22.2) 9 (9.1)
4 Need for primary care and specialty care follow-up is
determined
0 5 (5.1) 1 (1) 30 (30.3) 63 (63.6)
5 If patient does not have a primary care provider, one
is located for patients
9 (9.1) 22 (22.2) 19 (19.2) 32 (32.3) 17 (17.2)
6 Need for future tests is determined 0 4 (4) 0 44 (44.4) 51 (51.5)
7 Appointments are made for patient for follow-up
appointments and testing
4 (4) 6 (6.1) 4 (4) 41 (41.4) 44 (44.4)
8 If appointments are made, they are made with input
from the patient/family regarding the best time/date
11 (11.1) 32 (32.3) 15 (15.2) 27 (27.3) 14 (14.1)
9 If patient requires future diagnostic testing, patient
and/or family member is .instructed on any
preparation for testing
2 (2) 13 (13.1) 6 (6.1) 41 (41.4) 37 (37.4)
10 Importance of clinician appointments and further
testing is discussed with patient/family
2 (2) 2 (2) 3 (3) 47 (47.5) 45 (45.5)
65
11 Patient/family is asked about traditional healing
practices, and there is confirmation made that
practices are complementary with patient’s discharge
plan
22 (22.2) 42 (42.4) 19 (19.2) 12 (12.1) 4 (4)
12 There is confirmation made with patient/family that
they know where to go for further appointments and
tests, and that they have a plan to get to appointments
0 21 (21.2) 9 (9.1) 44 (44.4) 25 (25.3)
13 Barriers to keeping appointments are addressed 5 (5.1) 26 (26.3) 20 (20.2) 40 (40.4) 8 (8.1)
14 Pending lab and test results are identified with
patient/family
2 (2) 15 (15.2) 12 (12.1) 44 (44.4) 26 (26.3)
15 Determination is made of who will be reviewing the
results, and when and how this information will be
communicated to the patient/family
4 (4) 17 (17.2) 21 (21.2) 37 (37.4) 20 (20.2)
16 If patient requires medical equipment on discharge,
there is a process for ensuring that the medical
equipment is obtained
1 (1) 0 3 (3) 19 (19.2) 76 (76.8)
17 Before discharge, patient/family is given contact
information for medical equipment companies, at-
home services (as needed)
1 (1) 0 1 (1) 16 (16.2) 81 (81.8)
18 Medication list is reviewed with patient/family 0 2 (2) 3 (3) 20 (20.2) 74 (74.7)
19 Medication reconciliation is done at the time of
discharge
3 (3) 8 (8.1) 7 (7.1) 18 (18.2) 63 (63.6)
20 Patient/family member is given an explanation of
what medications to take and changes in the
medication regimen are emphasized
1 (1) 4 (4) 3 (3) 29 (29.3) 62 (62.6)
21 Each medication’s purpose, administration and side
effects are reviewed with patient/family
1 (1) 13 (13.1) 16 (16.2) 34 (34.3) 35 (35.4)
22 Patient/family’s concerns about medication plan are
assessed
1 (1) 7 (7.1) 5 (5.1) 46 (46.5) 40 (40.4)
23 Patient/family is referred to community pharmacist
within 2 weeks of discharge for a medication review
27 (27.3) 24 (24.2) 27 (27.3) 11 (11.1) 10 (10.1)
66
24 There is use of guidelines in the development and
planning of the discharge process for patients and
family members
8 (8.1) 9 (9.1) 20 (20.2) 34 (34.3) 28 (28.3)
25 The patient/family is given an easy-to-understand
written, prioritized discharge plan that includes
medications, medical equipment, future
appointments, and future diagnostic tests to take
home
2 (2) 9 (9.1) 7 (7.1) 24 (24.2) 57 (57.6)
26 The written plan is reviewed with the patient/family 2 (2) 8 (8.1) 9 (9.1) 28 (28.3) 52 (52.5)
27 Patient/family is encouraged to ask questions about
the plan
1 (1) 5 (5.1) 5 (5.1) 33 (33.3) 55 (55.6)
28 Prior to day of discharge, patient/family is met with
to provide education about patient’s diagnosis and
treatment and to prepare for discharge
4 (4) 14 (14.1) 20 (20.2) 39 (39.4) 22 (22.2)
29 Patient/family is asked to explain, in their own
words, the details of the discharge plan
5 (5.1) 38 (38.4) 23 (23.2) 22 (22.2) 11 (11.1)
30 As needed, family members and other caregivers
who will share in the care-giving responsibilities are
contacted
0 9 (9.1) 9 (9.1) 57 (57.6) 24 (24.2)
31 Patient/family is made aware of how to contact
primary care provider
1 (1) 6 (6.1) 10 (10.1) 44 (44.4) 38 (38.4)
32 Patient/family is instructed on what constitutes an
emergency and what to do in case of emergency
0 13 (13.1) 16 (16.2) 35 (35.4) 35 (35.4)
33 Discharge summary has a standardized format so that
information is easy to find
2 (2) 4 (4) 10 (10.1) 20 (20.2) 63 (63.6)
34 Within 24 hours of discharge, a discharge summary
is provided to primary care provider
8 (8.1) 13 (13.1) 22 (22.2) 24 (24.2) 32 (32.3)
35 Patient/family is called within 3 days of discharge to
reinforce the discharge plan and help with problem-
solving
43 (43.4) 23 (23.2) 12 (12.1) 11 (11.1) 10 (10.1)
36 Patient/family is provided with a phone number
where they can speak with a hospital staff member to
ask questions about the at-home care plan,
36 (36.4) 21 (21.2) 12 (12.1) 13 (13.1) 17 (17.2)
67
hospitalization, and follow-up plan in order to help
patients transition from hospital care to outpatient
care setting
68
4.4.2.1 Distribution of survey scores
For the 99 responses, the mean health literate score was 135.75 (20.44), median 138.
With 36 variables and a 5-point Likert scale, the survey had a possible score range of 36 to 180.
As this is the first time the survey was administered, it is not yet known what constitutes an
“adequate” health literate score. The distribution is both skewed and kurtotic. The histogram
indicates that the distribution is negatively skewed. See Figure 3 for the histogram illustrating the
distribution of the health literate scores.
Figure 3. Distribution of Health Literate Survey Scores
69
The distributions of each of the 36 items of the survey were examined by reviewing the
tests for skewness and kurtosis, the Kolmogorov-Smirnov statistics, the histogram and the Q-Q
plots. Only 4 items (items 13, 15, 23 and 29) had normal distributions according to the tests for
skewness and kurtosis. However, the Kolmogorov-Smirnov statistics for each of the items
indicated that they all had non-normal distributions. The review of the histograms and the Q-Q
plots also indicated that each of the distributions, items 13, 15 23 and 29, had non-normal
distributions. See Appendix R for reports of the descriptives, and Appendix S for the tests of
normality, for the survey items.
4.4.2.2 Correlations between survey items
Correlations between items were checked by using Spearman’s rho. See Appendix T for a
data dictionary for the table of correlations, and see Appendix U for the table of correlations.
There were a number of correlations below 0.3, particularly for items 1, 2, 3, 5, 6, 17, 18, 19, 23,
25, 30, 33, 34, 35 and 36, although not all of these low correlations were significant. A large
number of low correlations may indicate that the items are measuring different aspects of health
literate discharge practices (Field, 2013).
The highest correlation was .844 (p < .001), between items 25 and 26:
Item 25. The patient/family is given an easy-to-understand written, prioritized discharge plan that
includes medications, medical equipment, future appointments, and future diagnostic tests to take
home
Item 26. The written plan is reviewed with the patient/family
Although this correlation is high, Field (2013) states that correlations above .9 indicate
multicollinearity. Bartlett’s test of sphericity was significant (p < .001), indicating sufficient
70
correlation between the variables to proceed with analysis. Factor analysis is based on
correlations between items, and if there is a lack of correlation, factor analysis should be
reconsidered. Bartlett’s test indicates that the levels of correlations between the survey items are
adequate for factor analysis to proceed (Field, 2013).
4.4.3 Exploratory factor analysis
An exploratory factor analysis of the 36 items of the survey was performed on the data
from the 99 participants, using principal axis factor analysis for the extraction, and with direct
oblimin rotation. The Kaiser-Meyer-Olkin measure of sampling adequacy was .84, which is
above the acceptable level of .5. This indicates that the data were acceptable for exploratory
factor analysis (Field, 2013). As alluded to above, Bartlett’s test of sphericity χ2 (630) = 2165.26,
p < .001 which indicated that there was sufficient correlation between the variables to carry out
the analysis. An initial analysis was completed to obtain eigenvalues for each factor. Kaiser’s
criterion advises that those factors with an eigenvalue of greater than 1 should be kept (Field,
2013).
4.4.3.1 Nine factor solution
Nine factors had eigenvalues greater than 1.00, accounting for 61.05% of the total
variance. A factor correlation matrix confirmed that the items were correlated (see Appendix V).
The presence of 9 factors was confirmed with the scree plot (see Appendix W).
See Appendix X for the factor loadings after rotation, with the communalities.
71
The results of the communalities indicate that items 1, 2, 3, 5, 11, 23, 24 and 30 have a
communality value of less than .50. This indicates that they are not substantially captured by the
factor structure and could be possible candidates for removal from the analysis (Meyers et al.,
2013). In addition, all of the items contained one, or at the most two, factor loadings greater than
.3, except for item 11. Item 11, which involves asking the patient and family about traditional
healing practices, did not have any factor loadings greater than .245 and the communality was
also slightly low at .44. As this is the initial study of a new survey, and data reduction was not a
goal, no items were removed (Hinkin, 1998). The construct and content validity of the items was
established in the development of Project RED (Anthony et al., 2005). The Delphi panel
provided additional evidence for the construct and content validity of the items (see Study 1).
Furthermore, the reliability of the survey items is discussed in a following section.
Appendix Y shows the items re-ordered, so that it is clear which items load on each of the
factors. The nine factors are: 1) medication review; 2) use of a written plan with the
patient/family; 3) co-ordination of services; 4) preparation for follow-up appointments and tests;
5) use of a discharge summary; 6) primary and specialty care follow-up; 7) patient/family
education and telephone reinforcement; 8) test results and emergencies, and 9) language
assistance and referral to community pharmacist. Not surprisingly, these factors are consistent
with the 12 components of Project RED.
The survey items that load to each of these factors will be further described in the next
section.
4.4.3.1.1 Reliability testing
72
Reliability testing was first done with all 36 survey items. Although a high level of
reliability was found, with a Cronbach’s 𝛼 = .93, it is important to note that the value of Cronbach’s
𝛼 increases with the number of survey items (Cortina, 1993).
As described, the survey has 9 factors or sub-scales. It is recommended that reliability
testing should be applied separately to each of the subscales (Cronbach, 1951). As stated in the
methods section, a reliability level of at least .7 is regarded as acceptable (Cortina, 1993; Field,
2009). As noted in the following table, each of the factors has an acceptable level of
measurement except “use of a discharge summary” and “language assistance and referral to
community pharmacist.” “Use of a discharge summary” has a level of .68, which is very close to
acceptable. The level for “language assistance and referral to community pharmacist” is below
the level of acceptability at .59. Field (2013) points out that in the early stages of questionnaire
development, a level greater than .5 is acceptable. In addition, Hull & Nie (1981) and Nunnally
(1978) point out that an alpha level of less than .7 is acceptable with new scales (Hull and Nie,
1981; Nunnally, 1978).
The low level of reliability for this factor is perhaps not surprising as the four items of
this sub-scale had low communalities. In addition, the referral to a community pharmacist is not
a natural fit with language assistance.
Table 13 lists each factor, with the survey items, their communalities, and the reliability
testing for each factor:
73
Table 13
Reliability Testing for 9 Factor Solution
Factors and Survey Items Communality Cronbach’s
𝛂 Factor 1: Medication Review
20 Patient/family member is given an explanation of what
medications to take and changes in the medication
regimen are emphasized
.832 .87
21 Each medication’s purpose, administration and side
effects are reviewed with patient/family
.744
22 Patient/family’s concerns about medication plan are
assessed
.730
19 Medication reconciliation is done at the time of
discharge
.507
18 Medication list is reviewed with patient/family .743
24 There is use of guidelines in the development and
planning of the discharge process for patients and family
members
.387
Factor 2: Use of a Written Plan with Patient and Family
25 The written plan is reviewed with the patient/family .958 .91
26 The patient/family is given an easy-to-understand
written, prioritized discharge plan that includes
medications, medical equipment, future appointments,
and future diagnostic tests to take home
.851
27 Patient/family is encouraged to ask questions about the
plan
.644
Factor 3: Coordination of Services
17 Before discharge, patient/family is given contact
information for medical equipment companies, at-home
services (as needed)
.962 .78
16 If patient requires medical equipment on discharge, there
is a process for ensuring that the medical equipment is
obtained
.647
31 Patient/family is made aware of how to contact primary
care provider
.569
10 Importance of clinician appointments and further testing
is discussed with patient/family
.509
30 As needed, family members and other caregivers who
will share in the care-giving responsibilities are
contacted
.348
Factor 4: Preparation for Follow-up Appointments and Tests
9 If patient requires future diagnostic testing, patient
and/or family member is instructed on any preparation
for testing
.750 .87
74
12 There is confirmation made with patient/family that they
know where to go for further appointments and tests,
and that they have a plan to get to appointments
.699
13 Barriers to keeping appointments are addressed .676
11a Patient/family is asked about traditional healing
practices, and there is confirmation made that practices
are complementary with patient’s discharge plan
.440
Factor 5: Use of a Discharge Summary
33 Discharge summary has a standardized format so that
information is easy to find
.702 .68
34 Within 24 hours of discharge, a discharge summary is
provided to primary care provider
.528
Factor 6: Follow-up Primary and Specialty Care
6 Need for future tests is determined .537 .77
7 Appointments are made for patient for follow-up
appointments and testing
.661
4 Need for primary care and specialty care follow-up is
determined
.704
8 If appointments are made, they are made with input from
the patient/family regarding the best time/date
.514
5 If patient does not have a primary care provider, one is
located for patients
.488
Factor 7: Patient/Family Education and Telephone Reinforcement
36 Patient/family is provided with a phone number where
they can speak with a hospital staff member to ask
questions about the at-home care plan, hospitalization,
and follow-up plan in order to help patients transition
from hospital care to outpatient care setting
.670 .75
35 Patient/family is called within 3 days of discharge to
reinforce the discharge plan and help with problem-
solving
.550
29 Patient/family is asked to explain, in their own words,
the details of the discharge plan
.537
28 Prior to day of discharge, patient/family is met with to
provide education about patient’s diagnosis and
treatment and to prepare for discharge
.625
Factor 8: Test Results and Emergencies
15 Determination is made of who will be reviewing the
results, and when and how this information will be
communicated to the patient/family
.844 .79
14 Pending lab and test results are identified with
patient/family
.599
32 Patient/family is instructed on what constitutes an
emergency and what to do in case of emergency
.534
Factor 9: Language Assistance & Referral to Community Pharmacist
75
23 Patient/family is referred to community pharmacist
within 2 weeks of discharge for a medication review
.456 .59
2b If patient and/or family members do not speak English, a
translator is arranged for on discharge
.419
1 Language preference of patient and family is determined
and documented
.361
3 If patient and/or family members do not speak English,
written materials are provided in the preferred language
.248
aItem 11 had a loading of .245 on Factor 4
bSurvey item 2 had two factor loadings greater than >.3: for Factor 4 it had a loading of .420,
and for Factor 9 it has a loading of .361. Because use of a translator is assisting with language,
the decision was made to keep survey item 2 with Factor 9.
4.4.3.2 Five factor solution
As described earlier, Arrindell & Ende (1985) make the recommendation that the sample
size should be used to determine the number of factors. Their work demonstrates that stable
factor solutions are found when the sample size is approximately 20 times the number of factors.
For example, with a sample of 100, the authors recommend using a 5 factor solution.
As stated earlier, the organizational survey has 36 items, and there were 99 responses.
The subject to variable ratio was 2.7:1. In the light of Arrindell & van der Ende’s (1985) work,
this ratio is acceptable. In keeping with the recommendation of these authors, a five factor
solution was explored in this analysis, as 20*5= 100, which is close to the actual sample size of
99. In other words, using this recommendation means that the sample of 99 would be limited to
five factors.
An exploratory factor analysis of the 36 items of the survey was repeated on the data
from the 99 participants, once again using principal axis factor analysis for the extraction, and
with direct oblimin rotation. The factor correlation matrix confirmed that the items were
76
correlated (see Appendix Z). This factor correlation matrix confirms that there was enough
correlation between the factors to use direct oblimin rotation (Meyers et al., 2013).
The number of factors was limited to five, and this accounted for 49.72% of the variance.
Appendix AA displays the factor loadings after rotation, with the communalities. Items with
factor loadings greater than 0.3 are highlighted.
The results of the communalities indicate that 19 of the 36 items have a communality of
less than .50. This included the same 8 items from the nine factor analysis: items 1, 2, 3, 5, 11,
23, 24 and 30, and in addition, items 6, 8, 9, 14, 16, 17, 19, 29, 34, 35 and 36. The low
communalities indicates that the items are not substantially captured by the factor structure and
could be possible candidates for removal from the analysis (Meyers et al., 2013).
In the nine factor solution, item 11 did not contain any factor loadings greater than .3
(patient/family is asked about traditional health practices). In the five factor solution, although it
still had a low communality, item 11 did have a factor loading that was greater than 0.3 (factor
3).
In the five factor solution, two items did not have factor loadings greater than .3: item 2
(arrangement of a translator on discharge) and item 31 (patient/family is made aware of how to
contact primary care provider).
Appendix BB shows the items re-ordered, so that it is clear which items collect on each
of the factors. The five factors are: 1) follow-up appointments and tests, and coordination of
services; 2) use of a written plan with the patient/family; 3) patient/family education and
language assistance; 4) medication review, and 5) discharge summary and primary care provider
77
contact information. The survey items that load to each of these factors will be further described
in the next section.
4.4.3.2.1 Reliability of five factor solution
Reliability testing was applied separately to each of the factors (Cronbach, 1951). As noted
previously, a reliability level of at least .7 is regarded as acceptable (Cortina, 1993; Field, 2009).
The following table demonstrates that each of the factors has an acceptable level of measurement.
Like the 9 factor solution, the factor “use of a discharge summary” has a level of .68, which is
close to acceptable.
Table 14 lists each factor, with the survey items, their communalities, and the reliability
testing for each factor.
Table 14
Reliability testing for Five Factor Solution
Factors and Survey Items Communality Cronbach’s
𝛂
Factor 1: Follow-up Appointments and Tests, and Coordination of Services
8 If appointments are made, they are made with input from
the patient/family regarding the best time/date
.485 .89
7 Appointments are made for patient for follow-up
appointments and testing
.520
9 If patient requires future diagnostic testing, patient
and/or family member is .instructed on any preparation
for testing
.460
4 Need for primary care and specialty care follow-up is
determined
.554
6 Need for future tests is determined .304
13 Barriers to keeping appointments are addressed .586
16 If patient requires medical equipment on discharge, there
is a process for ensuring that the medical equipment is
obtained
.425
78
17 Before discharge, patient/family is given contact
information for medical equipment companies, at-home
services (as needed)
.495
12 There is confirmation made with patient/family that they
know where to go for further appointments and tests,
and that they have a plan to get to appointments
.543
10 Importance of clinician appointments and further testing
is discussed with patient/family
.509
15 Determination is made of who will be reviewing the
results, and when and how this information will be
communicated to the patient/family
.553
5 If patient does not have a primary care provider, one is
located for patients
.296
30 As needed, family members and other caregivers who
will share in the care-giving responsibilities are
contacted
.343
31a Patient/family is made aware of how to contact primary
care provider
.525
Factor 2: Use of a Written Plan with Patient and Family
26 The written plan is reviewed with the patient/family .860 .91
25 The patient/family is given an easy-to-understand
written, prioritized discharge plan that includes
medications, medical equipment, future appointments,
and future diagnostic tests to take home
.837
27 Patient/family is encouraged to ask questions about the
plan
.651
Factor 3: Patient/Family Education and Language Assistance
29 Patient/family is asked to explain, in their own words,
the details of the discharge plan
.486 .8
32 Patient/family is instructed on what constitutes an
emergency and what to do in case of emergency
.520
36 Patient/family is provided with a phone number where
they can speak with a hospital staff member to ask
questions about the at-home care plan, hospitalization,
and follow-up plan in order to help patients transition
from hospital care to outpatient care setting
.448
11 Patient/family is asked about traditional healing
practices, and there is confirmation made that practices
are complementary with patient’s discharge plan
.437
28 Prior to day of discharge, patient/family is met with to
provide education about patient’s diagnosis and
treatment and to prepare for discharge
.500
35 Patient/family is called within 3 days of discharge to
reinforce the discharge plan and help with problem-
solving
.292
79
1 Language preference of patient and family is determined
and documented
.319
14 Pending lab and test results are identified with
patient/family
.469
3 If patient and/or family members do not speak English,
written materials are provided in the preferred language
.186
2b If patient and/or family members do not speak English, a
translator is arranged for on discharge
Factor 4: Medication Review
20 Patient/family member is given an explanation of what
medications to take and changes in the medication
regimen are emphasized
.799 .85
21 Each medication’s purpose, administration and side
effects are reviewed with patient/family
.734
22 Patient/family’s concerns about medication plan are
assessed
.710
19 Medication reconciliation is done at the time of
discharge
.456
18 Medication list is reviewed with patient/family .702
23 Patient/family is referred to community pharmacist
within 2 weeks of discharge for a medication review
.341
24 There is use of guidelines in the development and
planning of the discharge process for patients and family
members
.333
Factor 5: Discharge Summary and Primary Care Provider Contact Information
33 Discharge summary has a standardized format so that
information is easy to find
.611 .68
34 Within 24 hours of discharge, a discharge summary is
provided to primary care provider
.417
a Item 31 had a loading of .234 on Factor 1
b Item 2 had a loading of .235 on Factor 3
4.4.3.3 Considering other factor solutions
The five factor solution did not capture item 2: If patient and/or family members do not
speak English, a translator is arranged for on discharge. As translation services have been found
to be key in ensuring patients and families understand discharge plans (Karliner et al., 2012;
80
Lindholm et al., 2012), the factor analysis was repeated, to assess solutions with 6, 7 and 8
factors.
The best solution was found with the six factor solution. It was determined to be the best
of these possible solutions as it captured all of the survey items, except for item 31:
Patient/family is made aware of how to contact primary care provider. While intuitively, this
may seem to be an important point, there is a lack of empirical evidence on the relationship
between patients and families being aware of the contact information of their primary care
provider, and patient and hospital outcomes.
The 7 factor solution did not capture item 15: Determination is made of who will be
reviewing the results, and when and how this information will be communicated to the
patient/family. This survey item is important, as it is a quality and safety issue when abnormal
tests results are not assessed by health care providers (Ong et al., 2012; Roy et al., 2005). See
Appendix CC for the 7 factor solution.
The 8 factor solution did not capture item 11: Patient/family is asked about traditional
healing practices, and there is confirmation made that practices are complementary with patient’s
discharge plan. This survey item is important, as traditional healing practices need to be assessed
in case they interfere with patients’ discharge plans (Gardiner, Filippelli, Sadikova, White, &
Jack, 2015; Jack et al., 2013). See Appendix DD for the 8 factor solution.
4.4.3.4 Six factor solution
An exploratory factor analysis of the 36 items of the survey was repeated on the data
from the 99 participants, once again using principal axis factor analysis for the extraction, and
81
with direct oblimin rotation. The factor correlation matrix confirmed that the items were
correlated. See Table 15.
Table 15
Factor Correlation Matrix for 6 Factor Solution
Factor 1 2 3 4 5 6
1 1 0.113 -.082 -.325 .289 .381
2 -.113 1 .218 .273 -.186 -.196
3 -.082 .218 1 .168 -.150 -.269
4 -.325 .273 .168 1 -.302 -.315
5 .289 -.186 -.15 -.302 1 .288
6 .381 -.196 -.269 -.315 .288 1
The number of factors was limited to six, and this accounted for 52.8% of the variance.
Table 16 displays the factor loadings after rotation, with the communalities. Items with factor
loadings greater than 0.3 are highlighted.
82
Table 16.
Summary of Items and Factor Loadings for Direct Oblimin Oblique Six Factor Solution
Survey items
Factor Loading Communality
1 2 3 4 5 6
1 Language preference of patient and family is
determined and documented
.337 .282 -.047 -.106 .269 .037 .324
2 If patient and/or family members do not speak
English, a translator is arranged for on discharge
.495 -.089 -.100 .009 -.100 .003 .258
3 If patient and/or family members do not speak
English, written materials are provided in the
preferred language
.396 .100 .046 -.128 -.015 .001 .197
4 Need for primary care and specialty care follow-up
is determined
-.039 -.154 -.106 -.205 .010 .625 .635
5 If patient does not have a primary care provider,
one is located for patients
.203 .069 .100 -.204 .025 .366 .326
6 Need for future tests is determined -.201 -.036 .063 -.017 .130 .762 .559
7 Appointments are made for patient for follow-up
appointments and testing
.011 -.140 -.127 .009 -.024 .686 .589
8 If appointments are made, they are made with input
from the patient/family regarding the best time/date
.226 .200 -.053 .080 -.027 .632 .517
9 If patient requires future diagnostic testing, patient
and/or family member is .instructed on any
preparation for testing
.532 -.094 -.296 .177 .032 .206 .531
10 Importance of clinician appointments and further
testing is discussed with patient/family
.209 -.070 -.252 -.304 .029 .248 .508
11 Patient/family is asked about traditional healing
practices, and there is confirmation made that
practices are complementary with patient’s
discharge plan
.414
-.070 .231 -.168 .056 .251 .453
83
12 There is confirmation made with patient/family that
they know where to go for further appointments and
tests, and that they have a plan to get to
appointments
.511 -.129 -.148 -.020 .091 .216 .563
13 Barriers to keeping appointments are addressed .576 .032 -.104 -.116 .026 .232 .605
14 Pending lab and test results are identified with
patient/family
.263 .005 -.051 .015 .404 .245 .467
15 Determination is made of who will be reviewing the
results, and when and how this information will be
communicated to the patient/family
.388 -.030 -.130 .058 .364 .212 .547
16 If patient requires medical equipment on discharge,
there is a process for ensuring that the medical
equipment is obtained
.228 -.011 -.627 -.107 .007 .080 .579
17 Before discharge, patient/family is given contact
information for medical equipment companies, at-
home services (as needed)
.092 -.054 .639 -.189 -.044 .129 .608
18 Medication list is reviewed with patient/family -.016 -.143 -.219 -.530 -.138 .384 .705
19 Medication reconciliation is done at the time of
discharge
-.120 -.225 .001 -.584 -.078 .166 .485
20 Patient/family member is given an explanation of
what medications to take and changes in the
medication regimen are emphasized
-.051 -.149 -.272 -.754 -.021 .030 .795
21 Each medication’s purpose, administration and side
effects are reviewed with patient/family
.026 -.076 -.038 -.784 .077 .012 .734
22 Patient/family’s concerns about medication plan are
assessed
.147 -.083 -.204 -.692 .039 -.018 .721
23 Patient/family is referred to community pharmacist
within 2 weeks of discharge for a medication
review
.185 .179 .134 -.478 .085 .036 .347
24 There is use of guidelines in the development and
planning of the discharge process for patients and
family members
.092 -.181 .028 -.386 .161 .010 .336
84
25 The patient/family is given an easy-to-understand
written, prioritized discharge plan that includes
medications, medical equipment, future
appointments, and future diagnostic tests to take
home
-.017 -.893 .033 -.108 -.032 .022 .842
26 The written plan is reviewed with the patient/family .032 -.906 .010 -.060 -.015 .016 .859
27 Patient/family is encouraged to ask questions about
the plan .099 -.669 -.065 -.086 .154 .044 .650
28 Prior to day of discharge, patient/family is met with
to provide education about patient’s diagnosis and
treatment and to prepare for discharge
.352 -.060 .037 -.341 .213 .046 .496
29 Patient/family is asked to explain, in their own
words, the details of the discharge plan
.518 -.215 .059 -.168 .228 -.142 .515
30 As needed, family members and other caregivers
who will share in the care-giving responsibilities
are contacted
.066 -.242 -.309 .179 .308 .095 .354
31 Patient/family is made aware of how to contact
primary care provider
.239 -.259 -.275 -.245 .200 .013 .554
32 Patient/family is instructed on what constitutes an
emergency and what to do in case of emergency
.354 -.265 .142 -.217 .305 -.057 .511
33 Discharge summary has a standardized format so
that information is easy to find
-.255 .120 -.406 -.260 .497 .048 .569
34 Within 24 hours of discharge, a discharge summary
is provided to primary care provider
-.171 -.012 -.212 -.082 .576 .056 .398
35 Patient/family is called within 3 days of discharge
to reinforce the discharge plan and help with
problem-solving
.067 -.054 .151 .099 .537 .141 .350
36 Patient/family is provided with a phone number
where they can speak with a hospital staff member
to ask questions about the at-home care plan,
hospitalization, and follow-up plan in order to help
patients transition from hospital care to outpatient
care setting
.084 -.114 .229 -.132 .588 .072 .523
85
The results of the communalities indicate that 12 of the 36 items have a communality of
less than .50. This is less than the 5 factor solution, which found that 19 of the 36 items had a
communality of less than .50.
The 12 items include the same 8 items that are included in the 9 factor and the 5 factor
solutions: 1, 2, 3, 5, 11, 23, 24 and 30. In addition, it included 4 items that were included in the 5
factor solution: 14, 19, 34, and 35. (Item 28, with a communality of .496 is not included, as this
is very close to .5). As stated earlier, when items have a low communality, it indicates that the
items are not substantially captured by the factor structure and could be possible candidates for
removal from the analysis (Meyers et al., 2013).
With respect to factor loadings, in the nine factor solution, item 11 did not contain any
factor loadings greater than .3 (patient/family is asked about traditional health practices). In the
five factor solution, although it still had a low communality, item 11 did include an acceptable
factor loading.
In the five factor solution, two items did not have factor loadings greater than .3: item 2
(arrangement of a translator on discharge) and item 31 (patient/family is made aware of how to
contact primary care provider).
Table 17 shows the items re-ordered, so that it is clear which items collect on each of the
factors.
86
Table 17
Summary of Items and Factor Loadings Greater Than .3 for Six Factor Solution
Survey items Factor Loading Communality
1 2 3 4 5 6
13 Barriers to keeping appointments are addressed .576 .605
9 If patient requires future diagnostic testing, patient
and/or family member is instructed on any preparation
for testing
.532 .531
29 Patient/family is asked to explain, in their own words,
the details of the discharge plan
.518 .515
12 There is confirmation made with patient/family that
they know where to go for further appointments and
tests, and that they have a plan to get to appointments
.511 .563
2 If patient and/or family members do not speak English,
a translator is arranged for on discharge
.495 .258
11 Patient/family is asked about traditional healing
practices, and there is confirmation made that practices
are complementary with patient’s discharge plan
.414 .453
3 If patient and/or family members do not speak English,
written materials are provided in the preferred
language
.396 .197
15 Determination is made of who will be reviewing the
results, and when and how this information will be
communicated to the patient/family
.388 .364* .547
32 Patient/family is instructed on what constitutes an
emergency and what to do in case of emergency
.354* .305 .511
28 Prior to day of discharge, patient/family is met with to
provide education about patient’s diagnosis and
treatment and to prepare for discharge
.352* -.341 .496
1 Language preference of patient and family is
determined and documented
.337 .524
87
26 The written plan is reviewed with the patient/family -.906 .859
25 The patient/family is given an easy-to-understand
written, prioritized discharge plan that includes
medications, medical equipment, future appointments,
and future diagnostic tests to take home
-.893 .842
27 Patient/family is encouraged to ask questions about the
plan
-.669 .650
17 Before discharge, patient/family is given contact
information for medical equipment companies, at-
home services (as needed)
-.639 .608
16 If patient requires medical equipment on discharge,
there is a process for ensuring that the medical
equipment is obtained
-.627 .579
30 As needed, family members and other caregivers who
will share in the care-giving responsibilities are
contacted
-.309 .308 .354
31 Patient/family is made aware of how to contact
primary care provider
.239a -.259* -.275* -.245* .200* .013* .554
21 Each medication’s purpose, administration and side
effects are reviewed with patient/family
-.784 .734
20 Patient/family member is given an explanation of what
medications to take and changes in the medication
regimen are emphasized
-.754 .795
22 Patient/family’s concerns about medication plan are
assessed
-.692 .721
19 Medication reconciliation is done at the time of
discharge
-.584 .485
18 Medication list is reviewed with patient/family -.530 .384 .705
23 Patient/family is referred to community pharmacist
within 2 weeks of discharge for a medication review
-.478 .347
24 There is use of guidelines in the development and
planning of the discharge process for patients and
family members
-.386 .336
88
10 Importance of clinician appointments and further
testing is discussed with patient/family
-.304 .508
36 Patient/family is provided with a phone number where
they can speak with a hospital staff member to ask
questions about the at-home care plan, hospitalization,
and follow-up plan in order to help patients transition
from hospital care to outpatient care setting
.588 .523
34 Within 24 hours of discharge, a discharge summary is
provided to primary care provider
.576 .398
35 Patient/family is called within 3 days of discharge to
reinforce the discharge plan and help with problem-
solving
.537 .350
33 Discharge summary has a standardized format so that
information is easy to find
-.406 .497 .569
14 Pending lab and test results are identified with
patient/family
.404 .467
6 Need for future tests is determined .762 .559
7 Appointments are made for patient for follow-up
appointments and testing
.686 .589
8 If appointments are made, they are made with input
from the patient/family regarding the best time/date
.632 .517
4 Need for primary care and specialty care follow-up is
determined
.625 .635
5 If patient does not have a primary care provider, one is
located for patients
.366 .326
*These are factor loadings of items that are cross-loaded. Items marked with an asterisk were not used. They were assigned to the
factor with which they fit best conceptually.
aItem 31 did not have a factor loading greater than .3. However, it was included in this analysis and had the best conceptual fit with
factor 1: Follow-up appointments and tests, and coordination of services.
89
The six factors are: 1) preparation for follow-up appointments and tests, and language
assistance; 2) use of a written plan with the patient and family; 3) coordination of services and
contact of informal caregivers; 4) medication review and patient education, 5) discharge
summary and follow up care, and 6) arrangement of follow up appointments and tests. The
survey items that load to each of these factors will be further described in the next section.
4.4.3.4.1 Reliability testing of six factor solution
As in the preceding sections, reliability testing was applied separately to each of the factors
(Cronbach, 1951). As noted previously, a reliability level of at least .7 is regarded as acceptable
(Cortina, 1993; Field, 2009).
As noted previously, the 9 and 5 factor solutions had reliabilities of less than .7. The
following table demonstrates that each of the factors of the 6 factor solution have an acceptable
level of measurement, ranging from .71 to .91. Table 18 lists each factor, with the survey items,
their communalities, and the reliability testing for each factor.
Table 18
Reliability Testing for Six Factor Solution
Factors and Survey Items Communality Cronbach’s
𝛂
Factor 1: Patient preparation for Follow-up Appointments and Tests, and Language
Assistance
13 Barriers to keeping appointments are addressed .605 .84
9 If patient requires future diagnostic testing, patient
and/or family member is instructed on any preparation
for testing
.531
29 Patient/family is asked to explain, in their own words,
the details of the discharge plan
.515
90
12 There is confirmation made with patient/family that they
know where to go for further appointments and tests,
and that they have a plan to get to appointments
.563
2 If patient and/or family members do not speak English, a
translator is arranged for on discharge
.258
11 Patient/family is asked about traditional healing
practices, and there is confirmation made that practices
are complementary with patient’s discharge plan
.453
3 If patient and/or family members do not speak English,
written materials are provided in the preferred language
.197
15 Determination is made of who will be reviewing the
results, and when and how this information will be
communicated to the patient/family
.547
1 Language preference of patient and family is determined
and documented
.524
31a Patient/family is made aware of how to contact primary
care provider
.554
Factor 2: Use of a Written Plan with Patient and Family
26 The written plan is reviewed with the patient/family .859 .91
25 The patient/family is given an easy-to-understand
written, prioritized discharge plan that includes
medications, medical equipment, future appointments,
and future diagnostic tests to take home
.842
27 Patient/family is encouraged to ask questions about the
plan
.650
Factor 3: Coordination of Services, Contact of Informal Caregivers
17 Before discharge, patient/family is given contact
information for medical equipment companies, at-home
services (as needed)
.608 .71
16 If patient requires medical equipment on discharge, there
is a process for ensuring that the medical equipment is
obtained
.579
30 As needed, family members and other caregivers who
will share in the care-giving responsibilities are
contacted
.354
Factor 4: Medication Review and Patient Education
21 Each medication’s purpose, administration and side
effects are reviewed with patient/family
.734 .86
20 Patient/family member is given an explanation of what
medications to take and changes in the medication
regimen are emphasized
.795
22 Patient/family’s concerns about medication plan are
assessed
.721
19 Medication reconciliation is done at the time of
discharge
.485
18 Medication list is reviewed with patient/family .705
91
23 Patient/family is referred to community pharmacist
within 2 weeks of discharge for a medication review
.347
24 There is use of guidelines in the development and
planning of the discharge process for patients and family
members
.336
10 Importance of clinician appointments and further testing
is discussed with patient/family
.508
28 Prior to day of discharge, patient/family is met with to
provide education about patient’s diagnosis and
treatment and to prepare for discharge
.496
Factor 5: Discharge Summary and Follow-up Care
36 Patient/family is provided with a phone number where
they can speak with a hospital staff member to ask
questions about the at-home care plan, hospitalization,
and follow-up plan in order to help patients transition
from hospital care to outpatient care setting
.523 .73
34 Within 24 hours of discharge, a discharge summary is
provided to primary care provider
.398
35 Patient/family is called within 3 days of discharge to
reinforce the discharge plan and help with problem-
solving
.350
33 Discharge summary has a standardized format so that
information is easy to find
.569
14 Pending lab and test results are identified with
patient/family
.467
32 Patient/family is instructed on what constitutes an
emergency and what to do in case of emergency
.511
Factor 6: Arrangement of Follow-up Appointments and Tests
6 Need for future tests is determined .559 .77 7 Appointments are made for patient for follow-up
appointments and testing
.589
8 If appointments are made, they are made with input from
the patient/family regarding the best time/date
.517
4 Need for primary care and specialty care follow-up is
determined
.635
5 If patient does not have a primary care provider, one is
located for patients
.326
a Item 31 had a loading of .239 on Factor 1.
4.4.3.5 Inter-rater reliability
92
Seven of the participating hospitals had 2 response. Inter-rater reliability was tested using
Cohen’s kappa (Meyers et al., 2013). Table 19 lists the Kappa values for each of the hospitals
that had two responses. These values indicate low levels of inter-rater reliability.
Table 19
Inter-rater Reliability Testing for Hospital with 2 Responses
Hospital LHIN Kappa
Teaching HNHB .18
Teaching South East .56
Large community Toronto Central .12
Large community Erie St. Clair .12
Large community Central East .06
Small community Waterloo Wellington .28
Small community Champlain .15
Five of the participating hospitals had more than 2 response. Inter-rater reliability was
tested using the intraclass correlation coefficient (ICC) (Field, 2013). Table 20 lists the intraclass
correlation coefficient values for each of the hospitals that more than two responses. Two of the
hospitals have ICC values that indicate acceptable levels of reliability (.71 and .77)
Table 20
Inter-rater Reliability Testing for Hospital with More Than 2 Responses
Hospital LHIN No. of
responses
ICC
Teaching Toronto Central 3 .71
Teaching North West 4 .37
Large community Waterloo Wellington 3 .32
Large community Central 3 .58
Large community (2 sites) Mississauga Halton 5 .77
4.5 Discussion
As reviewed in the introduction, the survey, based on Project RED is divided into 12
components. The exploratory factor analysis examined solutions that consisted of 5, 6, 7, 8 and
93
9 factors. The factor solution that appears to have the best fit and highest reliability is the 6 factor
solution, as listed in Table 21.
Table 21
Factors from 6 Factor Solution.
1 Patient preparation for follow-up appointments and tests and
language assistance
2 Use of a written plan with patient and family
3 Coordination of services, contact of informal caregivers
4 Medication review and patient education
5 Discharge summary and follow up care
6 Arrangement of follow up appointments and tests
The following table (Table 22) contains the original 12 components of the survey, which
is based on Project RED. Each of the survey items is matched (in the last column) to the factor
with which it best loaded or fit.
Table 22
Indicators of Health Literate Discharge Practices with Corresponding Factors
The two reworded indicators and the two new indicators are italicized (see items 23, 25, 33 and
36).8
Item Discharge Practice Factor
Language Assistance
1 Language preference of patient and family is determined and documented 1
2 If patient and/or family members do not speak English, a translator is arranged
for on discharge
1
3 If patient and/or family members do not speak English, written materials are
provided in the preferred language
1
8 Indicators from Jack et al., (2013). (Items 23 and 33 are new. Items 25 and 36 are adapted).
Reprinted with Permission from the US Department of Health and Human Services.
94
Post-discharge appointments, tests
4 Need for primary care and specialty care follow-up is determined 6
5 If patient does not have a primary care provider, one is located for patients 6
6 Need for future tests is determined 6
7 Appointments are made for patient for follow-up appointments and testing 6
8 If appointments are made, they are made with input from the patient/family
regarding the best time/date
6
9 If patient requires future diagnostic testing, patient and/or family member is
instructed on any preparation for testing
1
10 Importance of clinician appointments and further testing is discussed with
patient/family
4
11 Patient/family is asked about traditional healing practices, and there is
confirmation made that practices are complementary with patient’s discharge
plan
1
12 There is confirmation made with patient/family that they know where to go for
further appointments and tests, and that they have a plan to get to
appointments
1
13 Barriers to keeping appointments are addressed 1
Plan for follow-up of results from diagnostic lab tests or studies that are pending
at the time of discharge
14 Pending lab and test results are identified with patient/family 5
15 Determination is made of who will be reviewing the results, and when and
how this information will be communicated to the patient/family
1
Organization of post-discharge outpatient services and medical equipment if
needed
16 If patient requires medical equipment on discharge, there is a process for
ensuring that the medical equipment is obtained
3
17 Before discharge, patient/family is given contact information for medical
equipment companies, home care services (as needed)
3
Medication Review
18 Medication list is reviewed with patient/family 4
19 Medication reconciliation is done at the time of discharge 4
20 Patient/family member is given an explanation of what medications to take
and changes in the medication regimen are emphasized
4
21 Each medication’s purpose, administration and side effects are reviewed with
patient/family
4
22 Patient/family’s concerns about medication plan are assessed 4
23 Patient/family is referred to community pharmacist within 2 weeks of
discharge for a medication review
4
Discharge plan is reconciled with use of guidelines
24 There is use of guidelines in the development and planning of the discharge
process for patients and family members
4
Written discharge plan
95
25 The patient/family is given an easy-to-understand written, prioritized
discharge plan that includes medications, medical equipment, future
appointments, and future diagnostic tests to take home
2
26 The written plan is reviewed with the patient/family 2
27 Patient/family is encouraged to ask questions about the plan 2
Patient/family education of diagnosis
28 Prior to day of discharge, patient/family is met with to provide education
about patient’s diagnosis and treatment and to prepare for discharge
4
Patient/family’s understanding of the discharge plan is assessed
29 Patient/family is asked to explain, in their own words, the details of the
discharge plan
1
30 As needed, family members and other caregivers who will share in the care-
giving responsibilities are contacted
3
Patient/family is instructed on what to do if a problem arises
31 Patient/family is made aware of how to contact primary care provider 1
32 Patient/family is instructed on what constitutes an emergency and what to do
in case of emergency
5
Discharge summary is sent to clinicians accepting care of the patient
33 Discharge summary has a standardized format so that information is easy to
find
5
34 Within 24 hours of discharge, a discharge summary is provided to primary
care provider
5
Telephone reinforcement of the discharge plan is provided
35 Patient/family is called within 3 days of discharge to reinforce the discharge
plan and help with problem-solving
5
36 Patient/family is provided with a phone number where they can speak with a
hospital staff member to ask questions about the at-home care plan,
hospitalization, and follow-up plan in order to help patients transition from
hospital care to outpatient care setting
5
As this table demonstrates, there is congruence between the 12 components and the 6 factor
solution. As illustrated, of the 12 components, 6 fit together on factors: (language assistance;
organization of post-discharge outpatient services and medical equipment if needed; medication
review; use of a written discharge plan; discharge summary is sent to clinicians accepting care of
the patient; telephone reinforcement of the discharge plan is provided).
In addition, reliability of the organizational survey was established. The survey, as a
whole, has a high level of reliability (Cronbach’s 𝛼 = .93), although it is acknowledged that the
96
value of Cronbach’s 𝛼 increases with the number of survey items (Cortina, 1993). Each of the
factors or “sub-scales” of the 6 factor solution had acceptable levels of reliability, ranging from
.71 to .91.
The inter-rater reliability testing indicated low levels of inter-rater reliability. When there
were multiple responses from hospitals, participants typically had different roles, or were
working on different units of the hospital. This may account for the differences in their ratings.
4.6 Limitations
A limitation to this analysis was the small sample size. The small size limits the ability to
generalize the findings to all hospitals in Ontario (Field, 2013). There is a need to replicate this
analysis with a larger sample size, to see if the factor solution remains stable.
As reported in the descriptive statistics section, there were 12 hospitals that had more
than one response. This may have influenced the results and this study did not examine the
impact of having multiple responses from hospitals.
4.7 Theoretical contribution
In Chapter 3, the Health Literate Care Model was revised to focus on the use of health
literate discharge practices as strategies of health literate organizations. This exploratory factor
analysis was done to understand the construct “use of health literate discharge practices.” This
analysis contributes to our understanding of the factors that constitute health literate discharge
practices.
4.8 Conclusion
97
The indicators that were developed in the first study were used to create an organizational
survey tool to measure the use of health literate discharge practices in Ontario hospitals. These
indicators were based on Project RED, and with the use of a Delphi panel, 2 indicators were
reworded, and 2 indicators were added. The administration of this 36-item survey offers insight
into the wide range of use of health literate discharge practices in Ontario. In addition,
psychometric testing was done to establish the reliability of this survey.
This survey could be used by Ontario hospitals to measure and monitor their use of health
literate discharge practices, and to identify areas for improvement. In addition, this survey could
be tested in other provinces and territories in Canada. It could be used in benchmarking, to
compare hospitals’ progress in using these practices. The use of health literate discharge
practices have been associated with decreased readmission rates and improved care for patients
and families, and by measuring and adopting these practices, hospitals have the opportunity to
reach these important outcomes.
98
Chapter 5:
Absorptive Capacity and the Uptake of New Discharge Practices in
Ontario Hospitals
5.1 Introduction
Most of the research on health literacy to date has focused on the rates of low health
literacy among patients and their consequent poor health outcomes (Berkman, Sheridan,
Donahue, Halpern, & Crotty, 2011; Nielsen-Bohlman et al., 2004) or has examined the ways in
which health care providers do not meet the health literacy needs of patients and families (Cua &
Kripalani, 2008; Schillinger et al., 2003). Rather than focusing solely on the group levels of
patients and providers in hospitals, it is important to extend our lens to the organizational level
where there are both barriers and facilitators that will influence whether or not new knowledge,
such as health literate discharge practices, is utilized (Damschroder et al., 2009; Grol, Bosch,
Hulscher, Eccles, & Wensing, 2007). This study uses an organizational learning lens to examine
how Ontario hospitals adopt and use health literate discharge practices,
5.2 Organizational learning
Health literate discharge practices, like other evidence-based practices, may be
conceptualized as an innovation in that knowledge is encountered by an organization for the first
time (Damanpour, Walker, & Avellaneda, 2009; Panzano & Roth, 2006). When an organization
such as a hospital adopts and implements an innovation, it is engaging in organizational learning.
Organizational learning theory is a meta-theory that places an emphasis on the
organizational context in which learning about an innovation occurs. The organizational context
includes those facilitators and barriers to learning that exist at the individual, organizational and
99
environmental levels (Argote, 2012; Berta, Ginsburg, Gilbart, Lemieux-Charles, & Davis, 2013;
Berta et al., 2010; March, 1991; Nonaka, 1994). In order to successfully introduce innovations, it
is imperative that organizations pay attention to these contextual factors (Argote, 2012).
5.2.1 Absorptive Capacity
How organizations introduce, adopt and implement innovations will depend upon their
“learning” or “absorptive” capacity, that is, their ability to recognize the value of new knowledge
and to integrate it into their work (Cohen & Levinthal, 1990; Lewin, Massini, & Peeters, 2011).
This concept of absorptive capacity has been associated, in the health services literature, with an
organization’s ability to integrate innovations like best practices (Barnsley, Lemieux-Charles, &
McKinney, 1998; Berta et al., 2010).
Absorptive capacity can manifest as routines. Routines refer to the practices, procedures
and customs that are used to carry out work and make work-related decisions. Routines are a
form of knowledge and they come from inside or outside of the organization (Cohen &
Levinthal, 1990; March, 1991). There are different levels of routines.
Straightforward, lower level routines may be thought of as the everyday practices that are
carried out in the organization’s work. They are particular to the organization, and they
represent activities that can actually be observed (Lewin et al, 2011).
Metaroutines are higher-order, large scale routines that influence the ability to perform
lower level routines. As well, metaroutines influence organizational practices and how
organizations use new knowledge (Lewin et al., 2011). Lewin et al. (2011) describe metaroutines
as the basis or underpinning of absorptive capacity. Absorptive capacity metaroutines in an
organization are directed externally or internally.
100
External absorptive capacity routines can be employed by organizations to examine the
external environment to find new knowledge that may benefit it, or to find potential solutions for
existing challenges. Internal absorptive capacity routines are applied to new knowledge that is in
the organization, whether that new knowledge has been imported as a consequence of enacting
external routines, or emerged through actions carried out from within the organization. Internal
absorptive capacity metaroutines are used to consider new practices to implement, to exchange
ideas and information between organizational members, and to reflect on, and revise, established
practices (Lewin et al., 2011; Peeters, Massini, & Lewin, 2014). Over time, both external and
internal routines may be modified in response to new challenges, new practices and
organizational learning, or they may be abandoned for new routines. As routines change, develop
and grow, new organizational aptitudes can develop (Lewin et al., 2011).
Peeters, Massini & Lewin (2014) point out that current models using absorptive capacity
routines have not been tested empirically. However, the construct of absorptive capacity
routines has been used to understand how health care organizations implement new practices
(Berta & Baker, 2004; Berta et al., 2013; Innis & Berta, 2016), and a purpose of this study is to
investigate the use of routines in hospitals in the uptake of health literate discharge practices.
5.2.2 Metaroutines for the uptake and use of evidence-based practices
As described above, health literate discharge practices are evidence-based practices. In
the Spring of 2014, a scoping literature review (Arksey & O'Malley, 2005) was undertaken to
gain an understanding of the factors, contexts and processes that influence the uptake and use of
evidence based practice in health care organizations (Innis, Dryden-Palmer, Perreira, & Berta,
2015). Based on the scoping review, this study chose to focus on four metaroutines when
developing the semi-structured interview questions that were used to collect data. These four
101
metaroutines - allocating resources, responding to environmental mandates, participative
decision making and scanning - were all found to be integral to the uptake of evidence based
practice (Innis et al., 2015). See Figure 4.
Figure 4.
Absorptive Capacity Metaroutines for the Uptake of Evidence-Based Practice
5.2.2.1 Allocating resources.
For implementation of an innovation to succeed, an organization needs to have resources
available to members (Argote, 2012; Berta et al., 2013). This refers to not only financial
resources, but human resources and shared knowledge (Berta et al., 2005). If an innovation is
introduced with a budget, and resources such as training opportunities have been made available,
the innovation is more likely to be integrated into the work of the organization (Greenhalgh,
Robert, Macfarlane, Bate, & Kyriakidou, 2004). The resources of time (Berta et al., 2013;
102
Fineout-Overholt et al., 2004; Panzano & Roth, 2006), education and training (Berta et al., 2013;
Ellen et al., 2013; Greenhalgh et al., 2004), information technology (Fineout-Overholt et al.,
2004; Lau et al., 1999; Lukas et al., 2007), financial resources, resources for patient care (Angus,
Hodnett, & O'Brien-Pallas, 2003; Ploeg, Davies, Edwards, Gifford, & Miller, 2007) and staff
qualifications have all been found to facilitate the adoption of innovations in health care.
Time is identified as a resource to plan and consult with stakeholders (Berta et al., 2013;
Panzano & Roth, 2006); to obtain and read research (Fineout-Overholt et al., 2004) and to
implement new practices (Kimber, Barwick, & Fearing, 2012; Plath, 2013).
In order to adopt and implement an innovation, the use of resources for training and
education of staff has been found to be a facilitating factor (Berentson-Shaw & Price, 2007;
Berta et al., 2013; Ellen et al., 2013; Greenhalgh et al., 2004). In a 2013 study of protocol
implementation in long term care homes in Ontario, it was found that for-profit homes and not-
for-profit homes provided a higher level of resources for staff in the form of in-service education
and opportunities to attend external conferences, than did government-operated homes. This
study found that providing adequate resources for staff to receive training in the use of the new
protocols was vital to the success of the protocol implementation (Berta et al., 2013).
Three studies found that the presence of IT infrastructure, including computer resources,
IT support and the availability of training, were facilitators to the adoption of evidence-based
practice (Fineout-Overholt et al., 2004; Lau et al., 1999; Lukas et al., 2007).
As staff time, education and training, and IT are all associated with enhancing the uptake
and implementation of evidence-based practice, it is not surprising that a lack of financial
resources is a barrier. A 2004 literature review identified financial resources as a facilitating
103
factor to the use of research by nurses (Olade, 2004) and two other studies identified the lack of
financial resources as a barrier. In a survey of nurses’ ability to understand and use research
findings in the United States, nurses reported that a common barriers to research utilization was
the absence of a hospital budget to fund training and educational opportunities (Cadmus et al.,
2008).
Following a largely failed attempt at implementation of evidence-based practice in ten
Ontario hospitals, a 2003 follow-up qualitative study was done to examine the reasons for the
failure at one hospital, and the limited success at a comparison hospital. Through observation
and interviews with staff nurses and nursing managers, it was discovered that the hospital where
the intervention failed had been undergoing financial difficulties at the time of the intervention,
which had actually threatened to close the hospital. The second comparator hospital was not
having financial issues (Angus et al., 2003). It is quite possible that the hospital’s failure to
implement the evidence-based practice was associated with the hospital’s lack of financial
resources.
The adoption of evidence-based practice often involves the introduction of resources that
are needed for patient care, such as new therapies. Both of the Ontario hospitals described in the
preceding paragraph were challenged to find ways to meet the language and cultural needs of
patient populations that were culturally diverse. In the first hospital, where the intervention
largely failed, it was found there were limited resources to provide translation services or
culturally appropriate care. In the second hospital, where the intervention had some success,
there were no financial difficulties and the hospital was actively engaged in the delivery of
outreach programs to several cultural community groups (Angus et al., 2003).
104
A study of Ontario hospitals, in which nursing clinical best practice guidelines had been
implemented over a two year period, found that providing patient equipment resources, such as
pressure mattresses and wound care products, was a facilitating factor to the implementation of
the guidelines (Ploeg et al., 2007).
The Contingency Model of Innovation Adoption proposes that the probability of clinical
practice guideline adoption and use increases as the number of registered, professional nursing
staff increases (Berta et al., 2005). Although this model is referring to the use of innovations in
the long term care setting and to nursing staff, there is evidence in other study settings that the
use of evidence-based practices increases as staffing qualifications increase. A 2012 study
compared the features of high- and low-performing anticoagulation clinics in Veterans’ Affairs
hospitals in the United States. It was found that in the high-performing clinics, pharmacists were
significantly more likely to have completed a pharmacy residency, which is typically a one-year
additional training period in a teaching hospital following completion of a pharmacy degree
(Rose et al., 2012).
5.2.2.2 Responding to environmental mandates
Environmental mandates refer to those factors in the environment that influence how
organizations adopt and use innovations (Berta et al., 2005). Health service organizations
respond to triggers, which may come from the internal or external environment. Examples of
external triggers are changes in regulatory or accreditation requirements or a shift in government
policy (Berta et al., 2005; Zahra & George, 2002). As an example of responding to government
policy, Ellen et al (2013) found that participants from Ontario health service organizations
reported that the establishment of quality assurance departments was partly due to the provincial
government’s emphasis on quality monitoring.
105
Regulatory requirements have been found to influence the adoption of clinical practice
guidelines (Davis & Taylor-Vaisey, 1997) as well as increase the use of research by nurses
(Gifford, Davies, Edwards, Griffin, & Lybanon, 2007). A survey of nurses from a trauma
hospital in Minnesota found that respondents stated that regulation was a negative motivator for
adopting practice change, however, the study did not examine whether this finding had actually
influenced the adoption of new practices (Gale & Schaffer, 2009).
5.2.2.3 Participative decision making to select innovations
How an organization decides to select an innovation will depend on the processes in
place to consider innovations and how resources can be used (Argote, 2012). Several studies
have found that using a participative approach to include front-line staff, management and
leadership in selecting new evidence-based practices is central to the successful implementation
of the practices (Berta et al., 2013; Leape et al., 2006; Ruffolo, Savas, Neal, Capobianco, &
Reynolds, 2009).
A 2013 study examined the organizational and contextual factors that affect adoption of
new practices in long-term care homes in Ontario. Using a participative approach that included
staff and managers in choosing clinical practice guidelines was found to be vital to the success of
a guideline’s implementation (Berta et al., 2013).
Involving staff in protocol selection was also found to be beneficial in the study of
hospitals in Massachusetts, in which hospitals were asked to select two safe practice policies to
implement in their organization. The two practices selected were protocols for medication
reconciliation and the communication of critical test results. A facilitator to the program’s
106
success was determined to be the involvement of leadership and staff, particularly hospital
physicians, in the selection process (Leape et al., 2006).
A participative approach was also demonstrated in the case study of a behavioural health
care organization in the United States that used a community-centred process to select
innovations for adoption. This selection process included a wide range of stakeholders that
included clients, families, other mental health organizations, housing support services and
substance use programs. Although this process was found to be time-consuming, it was found to
be invaluable in decreasing opposition from staff to innovations and it ultimately increased
services for families (Ruffolo et al., 2009).
5.2.2.4 Scanning
Scanning is a concept in organizational learning theory that describes the routines that
organizations employ to scan or observe the external environment for innovations that could be
used to meet challenges or improve performance (Lewin et al., 2011). Titler et al., (1994) refer
to these innovations as “triggers.” Triggers come from information that is new, or that is being
newly recognized (Titler et al., 1994).
The role of scanning in the uptake and adoption of evidence-based practice was examined
in the following two studies. A 2010 study examined the introduction of evidence-based clinical
practice guidelines in long-term care homes in Ontario. It was found that prior to the adoption of
the guidelines, there was a phase in which key members of the organization, whether asked to by
management or self-directed, scanned sources of information in the environment to find out
about current protocols and recommendations that were associated with clinical concerns (Berta
et al., 2010).
107
A second study used a mixed methods approach of surveys and interviews to examine
how public sector mental health service organizations (n=66) in Ohio made decisions to adopt
practice changes. It was found that organizations which made an effort to stay informed of new
practices by scanning the environment and examining activities in other organizations were more
likely to adopt new practices (Panzano & Roth, 2006).
5.3 Use of health literate discharge practices in Ontario
Despite the growing realization that there is a relationship between how patients are
discharged from hospital and their subsequent readmission (Vashi et al., 2013), little research has
examined how hospitals discharge patients. Given the relationship observed between the use of
health literate discharge practices and improved patient outcomes, including decreased
readmission (Adams et al., 2014; Coleman et al., 2006; Jack, Chetty, Anthony, Greenwald,
Sanchez, Johnson, Forsythe, O'Donnell, Paasche-Orlow, Manasseh, et al., 2009; Markley et al.,
2013; Naylor et al., 2004), there is a need to understand how hospitals adopt and use these
practices.
As previously described in chapter 4, a survey that measured the use of health literate
discharge practices was administered to nursing managers of general medicine inpatient wards in
the 143 adult, acute care hospital sites in Ontario, Canada in spring 2015. Medical units were
chosen as they have the highest rates of readmission (Canadian Institute for Health Information,
2012). A total of 99 nursing managers or designates responded to the survey, representing 79
hospitals (participation rate 55%). Of the 99 participants, 55 participants from 52 hospital sites
agreed to be contacted for a follow-up interview.
5.4 Research question
108
The research question for this study was: What are the absorptive capacity metaroutines
that hospitals in Ontario use in the adoption and uptake of health literate discharge practices? A
goal of this study was to use the findings to generate propositions for future study.
5.5 Methods
The study received ethics approval from the Office of Research Ethics at the University
of Toronto on June 23, 2015.
5.5.1 Sample
Key informant interviews were conducted with 20 participants from 10 hospital sites.
Stratified purposeful sampling was used (Patton, 2002) to select hospital sites for the interviews,
from among 52 hospitals where participants had agreed to be contacted. Stratification was done
according to hospital type, location, rurality, and level of health literate discharge practices as
determined by the organizational survey. These were characteristics known to influence
performance in earlier studies (Berta et al., 2013; Canadian Institute for Health Information,
2012; Damschroder et al., 2009; Greenhalgh et al., 2004), and generally in the organization
theory literature (Argote, 2012; Cohen, March, & Olsen, 1972; Cyert & March, 1963; Jiménez-
Jiménez & Sanz-Valle, 2011).
There are three types of acute care hospitals in Ontario: teaching hospitals, large
community hospitals with at least 100 beds, and small community hospitals with less than 100
beds (Ministry of Health and Long-Term Care, 2013a, 2013b, 2013c). Hospitals’ locations are
described by the Local Health Integration Network or LHINs in which they are located. There
are 14 LHINs in Ontario. The LHINs are regional health authorities. They receive funding from
109
Ontario’s Ministry of Health and Long-Term Care and are responsible for planning, funding and
integrating hospital, home and community services (Deber, 2014).
Hospitals locations are also described by their rurality. The rurality index is a composite
value that reflects a community’s location with respect to population density and travel times for
basic and advanced health care services. The rurality index ranges from 0 (highest population
density, lowest travel times) to 99 (lowest population density, highest travel times) (Kralj, 2009).
Participants from eleven hospitals were initially contacted via email (see Appendix EE).
Participants at two hospitals refused and participants from two other hospitals did not respond.
Using the stratification criteria, participants from three additional hospitals were contacted, and
these responded and agreed to participate. Each respondent was asked to identify a second
organizational member to also be interviewed. Informed consent was obtained prior to the start
of the interview (see Appendix FF).
See Table 23 for a list of the participants.
110
Table 23
List of Participants
aNM = nursing manager, RNPF = RN focused on patient flow, SM = senior manager, NE = nursing educator, SN = staff nurse, SW =
social worker bTH = teaching hospital, LCH = large community hospital, SCH = small community hospital
Type of
hospital
LHIN Survey
score
Rurality Role Interviewee codes
(Interview number,
rolea, hospitalb)
Teaching North West 121 0 Social worker 6A_SW_TH
Social worker 6B_SW_TH
RN focused on patient flow 6C_RNPF_TH
RN focused on patient flow 6D_RNPF_TH
South West 142 0 Nursing manager 10A_NM_TH
Nursing manager 10B_NM_TH
Large
community
HNHB 78 8 Nursing manager 2A_NM_LCH
Nursing manager 2B_NM_LCH
Erie St Clair 109 10 Nurse educator 9A_NE_LCH
Nurse educator 9B_NE_LCH
Central 121 4 Nursing manager 1A_NM_LCH
Nursing manager 1B_NM_LCH
Small
community
South East 111 13 Nursing manager 3_NM_SCH
Champlain 122 40 Nursing manager 5A_NM_SCH
RN focused on patient flow 5B_RNPF_SCH
Champlain 147 70 RN focused on patient flow 4_RNPF_SCH
North West 149 92 Senior manager 7A_SM_SCH
Senior manager 7B_SM_SCH
North East 123 93 Senior manager 8A_SM_SCH
Staff nurse 8B_SN_SCH
111
The participants consisted of: 8 nursing managers, 4 registered nurses who had a role focused on
patient flow, 3 senior managers, two nurse educators, two social workers and one staff nurse. Of
note, 18 of the 20 participants were nurses. All of the participants worked in unionized
environments.
5.5.2 Data collection
5.5.2.1 Interviews
Interviews were conducted in-person, at the participant’s work site in the summer and fall
of 2015 by the principal investigator. The interview questions were guided by the scoping
literature review that provided evidence for the four metaroutines: allocating resources,
responding to environmental mandates, participative decision making and scanning (Innis et al.,
2015). See Appendix GG for the interview questions that were used.
5.5.2.2 Documents
Participants were asked to provide documents used in the discharge process at their
hospital. A total of 46 documents were collected. Across all participant hospitals the following
documents were collected: patient discharge forms, discharge summary templates, medication
profiles, checklists, order sets, referral forms for consultation and home care services, and patient
education materials. Photographs (4) were taken at those hospitals where signage and white
boards were used for communication.
5.5.3 Data analysis
The analysis was guided by the research question: What are the absorptive capacity
metaroutines that hospitals use in the uptake of health literate discharge practices? Thematic
112
analysis was used in analyzing the interview data. The processes of data collection and analysis
of the interviews and documents took place concurrently, in an iterative process (Jackson &
Gillis, 2003; Teddlie & Tashakkori, 2009). This enabled the authors to identify points that
needed clarification or questions that required further exploration with study participants
(Jackson & Gillis, 2003; Sandelowski, 1995).
Data from the interviews and the documents was coded with descriptive labels to
distinguish elements that appeared relevant to the study (Braun & Clarke, 2006; Sandelowski,
1995). These codes were informed by the literature review (Innis et al., 2015), and included any
evidence regarding allocating resources, responding to environmental mandates, participative
decision making and scanning. The coding was used to form categories, and these categories led
to the development of themes (Creswell, 2008; Thomas, 2006). Both the transcripts and the
documents were coded using deductive and inductive analysis.
Deductive analysis was used to determine if there was a relationship between the
absorptive capacity metaroutines of: allocating resources, responding to environmental
mandates, participative decision making and scanning, and the uptake of health literate discharge
practices. The analysis was open to unexpected themes and inductive analysis was used to look
for other themes that emerged from the data (Thomas, 2006).
The first two interview transcripts were read and coded independently by the principal
investigator and the two co-supervisors to ensure that coding was consistent and to develop a
tentative coding scheme. The principal investigator was responsible for the bulk of reading and
coding of the transcripts. To ensure the consistency of the coding, there were coding checks and
further development of the coding scheme with the co-supervisors after 4 interviews, after 16
113
interviews, and at the conclusion of the interviews. This process was manual; the NVivo 11
software program was used for data management only.
The findings of the study were used to inform propositions. Although the initial focus of
this study was on the four metaroutines of allocating resources, responding to environmental
mandates, participative decision making and scanning, it was anticipated that other metaroutines
would arise in the interviews. A goal of the analysis was to develop propositions to further our
conceptual understanding of the uptake of health literate discharge practices in hospitals.
5.6 Results
5.6.1 Allocating resources
Participants described the importance of the resources of information technology, staff,
space, and professional development to facilitating the uptake of health literature discharge
practices.
5.6.1.1 Information technology (IT)
Two nursing managers described how IT was used to share information between the
hospital and primary care providers, as well as with home care services, while other participants
described shortcomings with IT. A nursing manager described how the IT system being used was
notifying primary care providers of patients’ hospitalization weeks after the discharge had
actually occurred: “People think that the IT piece is in there, and that it’s capable, they’re
relying on (the software), but (the software) doesn’t work like it’s supposed to
work.”(2B_NM_LCH)
114
Only one participant described the use of electronic health records (HER) to share
information between hospitals. At a small community hospital, a director of nursing described
how her hospital was connected to five others with the same EHR:
There's a group of six hospitals, and we're all connected… it’s a network of the hospitals
that we can see each other's EMR (electronic medical records), so if somebody went to (a
different hospital in the network) and they had a visit, then we can go in, look at that date
and pull up information on what happened in (that hospital). (8A_SM_SCH)
While all of the hospitals used electronic health records to some extent, the
communication between hospitals and the community health care providers was largely by
facsimile. In addition, at five of the hospitals, including one teaching hospital, the discharge
instructions or medication reconciliation record were completed using carbon paper. At two of
these hospitals, patients received the carbon copy, and not the original sheet that was filled in by
staff.
Although only one hospital, a large community hospital, used an electronic discharge
instruction form that nursing staff completed online and then printed to give to patients and
families, participants at three other hospitals described the presence of online resources for
patient discharge education. A nursing manager at a large community hospital described how the
hospital provided Lexicomp©, an online resource for medications, to provide patient education at
discharge:
We have Lexicomp. The staff know that Lexicomp has information, if there are questions
(from patients and families) about any drugs, we get information from Lexicomp, and
they (nursing staff) print out patient education. (1B_NM_LCH)
At two other small community hospitals, participants also described the use of Lexicomp©, and
Meditec© to provide patient education on medications. At one small community hospital, nursing
115
staff had access to the online platforms Mosby Nurse© and UpToDate©, to provide patients with
disease-specific education.
5.6.1.2 Space
In three of the hospitals, participants referred to the importance of space that was
designated for patient care for discharge planning activities. A discharge planner in a small
community hospital discussed working in a palliative care patient lounge. A social worker in a
teaching hospital lamented the lack of space: “There's no physical place for a person to work on
a project” (6A_SW_TH).
Having space for staff to meet and to work on projects was the norm in most of the other
hospitals. Participants described having space in the form of conference rooms, board rooms or
libraries that staff could access.
5.6.1.3 Staff
A number of interviewees noted that when work processes were adopted or changed,
there was a need to identify who would be carrying out the work. Almost all of the hospitals had
a nursing role that was dedicated to discharge planning and patient flow. The only two hospitals
that did not were small community hospitals that each had 10 acute care beds. The organizations
used different titles for this role, such as discharge planner, patient flow navigator, utilization
coordinator, flow nurse and rapid admission nurse, however, the responsibilities of this role were
similar across hospitals. A nursing manager at a teaching hospital described the role in this way:
They just stay right on top of each patient. They work alongside the teams, alongside
nursing, alongside allied health, kind of the coordinator of each kind of team is how I
would describe it. They also concentrate on facilitating discharge, and moving things
forward, and getting appropriate equipment or whatever they need from (home care) or
whatever barriers we can break down. (10A_NM_TH).
116
Participants were asked if there was enough staff available to give patients and families
information about discharge and to answer their questions and concerns. Generally speaking,
nurses in managerial roles felt that there was staff available, while nurses in non-managerial
roles did not agree. This was illustrated in a small community hospital. In response to this
question, the CEO responded: “Yes, for sure. Generally our staffing mix is constant, and if a
person calls in sick, that person will be replaced, so that would be available” (8A_SM_SCH).
When a charge nurse at the same hospital was asked about the availability of nursing staff, she
responded: “You're so short of nursing staff at the best of times” (8B_SN_SCH).
A number of participants identified workload as being a barrier to the use of new
discharge practices, and to educating patients and families about hospital discharge. The value of
the new practices was acknowledged, and it was coupled with the caveat that carrying out these
practices is difficult when nurses already have heavy workloads. A nursing educator at a large
community hospital described the difficulty in carrying out new discharge practices while facing
multiple demands on the ward. In describing the use of a new discharge teaching tool for patients
and families she stated:
We do these new initiatives, but we're not taking anything else away from the nurses. I
would say that's a big barrier. It's just we don't have the time… Just coming from a
nursing role, it's very frustrating because you want to give them the best chance that you
have, but when you've got three people who are confused, waiting to go to the bathroom
who might fall, you've got another person who's supposed to be coming up to you from
emerg, and now I'm supposed to (discharge a patient)... It's having more time. Nurses,
from my experience, genuinely want to do a better job. It's just there is so much on our
plates. (9A_NE_LCH)
Nursing managers from two different community hospitals acknowledged that nurses had a
heavy workload, and that there was not funding available to increase staffing. A manager at a
small community hospital stated that having more staff was not an option, and that the process of
117
hospital discharge needed to change. When asked if there were enough staff available to
discharge patients, she responded:
I think if you ask that question to my staff, my staff would say we need more nurses if we
really want to be able to do it effectively. In my mind, on a med-surg unit you're never
going to get a ratio better than five (patients) to one (nurse). We have that ratio straight
through until 11:00 at night. We have to find a process of doing our work better.
(5A_NM_SCH).
Patients receive nursing care on inpatient units 7 days a week, 24 hours a day. However,
participants at most of the hospitals expressed the concern that because of lack of allied health
staff on the weekends, that discharges were either delayed until Monday, or if they were
discharged over the weekend, the discharges were not done well. A social worker at a teaching
hospital described the situation:
What happens is on the weekend everything is at a standstill. And a lot of times people
are discharged (on the weekend) without the appropriate referrals…I wouldn't mind
working 5 (days) on, 2 (days) off. To cover the weekends. (6A_SW_TH)
5.6.1.4 Professional development
Participants at each of the hospitals reported that professional development was
encouraged for staff and managers. However, staff and managers had different levels of
resources, with respect to time and money, to take courses or to attend conferences. Managers
spoke of receiving funding to further their education, and being able to attend workshops and
conferences during working hours. For example, at two community hospitals (one large and one
small), nursing managers described recently attending a patient flow conference in the United
States.
Generally speaking, staff nurses did not have the same access to these resources. An
exception was in a small community hospital where a staff nurse described having the possibility
of being supported to participate in professional development:
118
If I wanted to go to a discharge planning workshop that would really benefit my work and
the work I do for the hospital, then maybe I think they would support that. I would put the
application in. I think they would support that. (8B_SN_SCH)
At other hospitals, managers described promoting professional development opportunities for
staff, while being unable to provide resources, and requiring staff to attend events on their own
unpaid time. A nursing manager in a large community hospital bluntly noted: “So we endeavor,
we endeavor to give them days off, we don’t” (1B_NM_LCH). Repeatedly, managers described
helping staff re-arrange their work schedule so that staff could attend an educational event on
their own unpaid time.
These findings lead to the following proposition:
Proposition 1. Hospitals with internal absorptive capacity metaroutines to allocate resources for
IT, space, staffing and professional development will facilitate the uptake of health literate
discharge practices.
5.6.2 Building and nurturing external relationships
5.6.2.1 Community organizations
All participants spoke of their relationships with community organizations, including
home care, rehabilitation and complex continuing care facilities, and group homes. A nursing
manager in a small community hospital stressed the importance of relationships with her
community partners:
My whole time in my position in the last three years has been linking with our
(community) partners. Having amazing relationships, coming to the table as equals.
Really putting all of our egos at the door and saying, this is my part of the job, that's your
part of the job, we're going to have some overlap in how we're going to make that as safe
as we can for patients and get them out of here. (5A_NM_SCH)
119
In a large community hospital, a nursing manager described the location of the hospital in
an area with a heavy concentration of group homes. She described the challenges of working
with a population that has a high level of mental health issues, and the regular communication
established with the group homes in the region to facilitate discharge from hospital and prevent
readmission:
We have one of the largest concentration of group homes in the province...they’re
estranged from their families, they age in place and they age younger, faster than the
normal population, so we have some real challenges with that population…We have
meetings with (the group homes) quarterly. Because mental health has challenges too.
They bounce in and out like rubber balls so we’ve been working really hard with them to
try and make that transition (from the hospital to the community). (1A_NM_LCH)
When patients require home care following hospitalization, or they require a referral to
complex continuing care or long-term care, the CCAC is responsible for arranging these services
and making these referrals. Lack of home care services from CCAC was identified as a barrier to
hospital discharge to home in half of the hospitals where interviews took place. Participants
spoke of the lack of resources that the CCAC is able to provide, with respect to nursing and
personal support worker (PSW) services, as well as supplies needed for patient care.
At the small community hospital, where the number of alternate level of care (ALC) beds
had decreased from 50% of the patients to none, the nursing manager described the importance
of mutual respect between the hospital and the CCAC, and the need to work together towards a
common goal:
What we like to do is make other people the enemy. CCAC (says) "Oh that hospital’s
discharging them too early." The hospital says, "I can't send them in the community,
CCAC can't get services, they’re never there." We've just come to the table and said,
what’s your job? This is what my job is. Can you trust me in my job? I'll trust you in your
job... We meet with CCAC regularly. I've never thrown CCAC under the bus. We're going
to link together and say, okay, this is what I'm owning, what are you owning? How are
we going to make this better? We work at making it better together for our patients
(5A_NM_SCH).
120
Participants at four of the hospitals described sending the prescriptions and the discharge
instruction sheet to the patient’s community pharmacy. The participants reported that the
communication with the community pharmacist has been helpful, and a nursing educator at a
large community hospital described how this helped to avoid medication errors:
We have lots of great pharmacists (in the community) who will call up and say, "Is this
the right dosing?" We've gotten calls going, "Oh, you gave this, but did you know this?"
Then it’s, "Oh, well, let's talk to the doctor before the patient's discharged," or to get
clarification.” (9A_NE_LCH)
5.6.2.2 Professional association
Participants from three of the hospitals surveyed spoke of the benefits of becoming a
Registered Nurses’ Association of Ontario (RNAO) Spotlight Organization. Becoming a
spotlight organization means that hospitals agree to adopt and evaluate best practice guidelines
from the RNAO. The RNAO provides training and educational resources for staff nurses to
facilitate the adoption of the best practice guidelines (Registered Nurses’ Association of Ontario,
2015). A nursing manager from a large community hospital reported:
We are actually right in the middle of being, trying to be recognized as a best practice
hospital, from the BPSO (best practice spotlight organization), from the RNAO. That’s
definitely front line staff directed, that’s taking our front line staff and making them
invested in being a better hospital, and to incorporating all of those (best practices).
(1B_NM_LCH)
5.6.2.3 Educational institutions
Participants at each of the hospitals reported that there were medical and nursing students
from universities and community colleges at their sites. Having students was identified as
contributing to a learning culture. As a nurse manager in a hospital that had recently aligned with
121
nursing and medical schools in two universities reported: “(It’s) helpful, really helpful. I think it
creates a culture of ongoing learning.” (2B_NM_LCH).
5.6.2.4 Communities of Practices
Communities of practice are informal groups of individuals who share common work.
They have been shown to facilitate learning between organizational members and to improve
organizational performance (Brown & Duguid, 1991). Three participants from small community
hospitals described having communities of practice. A discharge planner reported meeting with
other discharge planners in her area several times a year to find out about new discharge
processes and to discuss barriers and solutions to problems. Senior leaders from two small
community hospitals described participating in informal communities of practices with other
senior leaders in their regions to share ideas and work through common barriers.
These results lead to the second proposition:
Proposition 2: Hospitals with external absorptive capacity metaroutines for building and
nurturing external relationships with community organizations, educational institutions and
communities of practice facilitate the uptake of health literate discharge practices.
5.6.3 Fostering internal networks
5.6.3.1 Use of hospital rounds
Participants at five of the hospitals described weekly hospital rounds to discuss complex
discharges. These meetings were attended by nursing managers, staff nurses, other members of
the intraprofessional team, and home care service coordinators. The goals of these meetings were
to discuss complex discharge plans, to share resources and to find solutions for patients who no
122
longer needed acute care services, but could not return home. A nursing manager in a large
community hospital described the use of weekly discharge rounds:
We started it so we could review all the patients in the hospital who we were having
challenging discharges. We didn’t want to discuss the ones like Mr. Jones he’ll be fine
tomorrow, we wanted to discuss Mrs. Brown (who) lives alone, has dementia, family lives
in California, no finances, can’t reach the family, want to go to long-term care and the
wait is 3 years, so those are the ones (we discuss). (1A_NM_LCH).
5.6.3.2 Daily unit rounds
Most of the hospitals described rounds that happened every morning on the nursing unit.
These were meetings of the intraprofessional team, and included nurses, rehabilitation therapists,
social workers, dietitians, and pharmacists as well as home care service coordinators. The
meetings were short, lasting between 15-35 minutes and were used to share information about
discharge plans and to ensure that patients were receiving the services that they needed. A nurse
manager at a teaching hospital described the meetings:
They just go through every patient. It only takes about 15 minutes to go through probably
25 patients, and they just literally go through to say, "Where are we at? What are we
missing? What do they need? How are we getting them out for discharge? How are we
preparing them? What's that look like? Is it two more days? One more day? Can they go
today? What can we do today to make it happen,” those kind of things. (2B_NM_LCH).
Physicians did not usually attend these daily rounds, however it was physicians who were
responsible for writing orders to discharge patients home. This lack of communication between
physicians and other members of the intraprofessional team had the potential to lead to tension.
A nursing manager at a teaching hospital described how physicians had the authority, and the
need to maintain open, respectful communication:
Do (physicians) make the final decision? Of course, they're writing the discharge order…
I think you’ve got to be open to have the discussion, and it's, "No, we're not going to do
that, we’ve got to do this," then that's okay, but let's talk about it. I think that's where
123
we're improving is to say, “It's okay that you want to discharge today.” The allied health
or whoever is saying: “It's got to be tomorrow because of this and this.” You (the
physician) could still discharge because you’re the man. You can do it. We can do it.
Knowing that they're probably not going to be successful, they're going to come back.
(10A_NM_TH).
A nursing educator noted the link between physicians not participating in the daily unit
rounds and not having the same plans for patients as the team:
Sometimes it's the team has a plan and then the physician comes in after. In their defense,
too, if they're not part of the conversation, then they never get the context.
(9B_NE_LCH).
As described earlier, participants described how a lack of home care services from CCAC
was as a barrier to hospital discharge to home in half of the hospitals where interviews took
place. The frustrations that were expressed did not extend to relationships with CCAC
coordinators in the hospitals. At most of the hospitals, a CCAC coordinator had her or his office
on site. Participants from four of the hospitals stated that CCAC attended the daily morning
patient care rounds with the intraprofessional team. At three of the hospitals, participants
described good working relationships with the CCAC coordinators, and they described
interacting with them on a regular, daily basis.
5.6.3.3 Internal social networks
Several participants described how their work, or the work of others, depended upon their
social networks within the hospital. At a large community hospital, a nursing manager described
how the patient flow navigators had been at the hospital for years and had good relationships
with physicians as well as with managers in diagnostic imaging. They were able to leverage
these relationships to ensure that patients were seen by consultant physicians and had access to
diagnostic testing on a timely basis.
124
At the same hospital, another nursing manager described how she was able to use her
network to find out information to help patient flow: “I’ve been here a long time so I know a lot
of people. I know people in the organization, so I know where to pull resources.”
(1B_NM_LCH)
These findings suggest a third proposition:
Proposition 3: Hospitals with internal absorptive capacity metaroutines to foster internal
networks will facilitate the uptake of health literate discharge practices.
5.6.4 Standardizing processes
5.6.4.1 Discharge forms
At three of the hospitals, participants described the use of discharge forms throughout the
hospitalization, to ensure that information about the patient’s care did not get lost, and that issues
that affect discharge, such as housing or family support, were identified at the beginning of the
admission. At a large community hospital, a nursing manager described how a nurse practitioner
and two physicians on the inpatient medicine ward developed a new discharge form that had just
been implemented:
They came up with a different process to discharge our patients which provides them with
a better discharge sheet so when a patient gets admitted now in the ER the doctor writes
on the sheet why they were admitted, any co-morbidities…and on the back of that sheet is
actually their discharge summary sheet. So right from the get go, they’re starting to write
things on the discharge summary sheet and even our consulting doctors (are using it).
(1B_NM_LCH).
In a small community hospital, a nursing manager had developed a discharge process tool with
nurses on the unit. The tool identifies issues that influence hospital discharge at the time of
admission:
125
…we've color coded it (the discharge process tool) so that there's risk indicators that
maybe this is going to be a complicated discharge. If they have impaired mobility, they
have frequent falls, recurrent admissions within the past 3 months, delirium, lives alone
with few supports or evidence of a care deficit, caregiver burnout or patient and family
concerns. Those are all risk indicators that we need to know about on admission, so that
we can start addressing them. (3_NM_SCH).
The same nurse manager spoke of the need for this tool to cue staff and to ensure that all steps of
discharge planning were taking place:
It'll ensure that staff are cued to make sure that family are involved during the health
teaching and they're present, and then being able to go through "Do you understand what
your admission was about for the past few days? Did you learn anything?" A lot of
talkback type of discussion, so that we know that we're getting that information to the
patient. (3_NM_SCH).
The need for a standardized record of discharge teaching was echoed by other
participants. At a small community hospital and at a teaching hospital, participants spoke of the
need for a record of what discharge teaching had been done by other staff members. A discharge
planner at the small community hospital expressed the concern that staff were not, or did not
have a way of, recording the details of patient teaching:
I'll go in and talk to Mr. Smith, and he'll say, "Oh, nurse so and so talked to me about
that already," but I don't necessarily have any documentation. There's consistent
discharge specific documentation that should take place. (4_RNPF_SCH)
The nursing manager at the teaching hospital stated:
I think nursing teaches a lot and are doing it every day. It's just, we don't document well,
and I don't know what Sally taught, and then I don't know what Jane's going to teach at
night, and it's just crazy. I think we need better documentation to what we're actually
teaching our patients, and then we know, oh, that's covered, concentrate on something
else. We'll get more information to them if we're not repeating ourselves, or if we're just
avoiding it because we thought somebody else did it. (10A_NM_TH).
At a large community hospital, a nursing manager described how discharge practices
126
were developed and used on individual units or services, rather than being standardized across
the hospital. This led to problems when patients were bed-spaced, or transferred to other units
where there were available beds, to make room on the unit for new admissions from the
emergency department. She stated: “That’s a discharge nightmare for us. That patient will
probably come back, they were discharged from (another unit) without the right information.”
(1B_NM_LCH).
5.6.4.2 Checklists
Participants at three of the hospitals described the use of a discharge checklist by nursing
staff to ensure that patients had received the information they required. A senior manager
remarked on the value of this practice: “It all gets missed if you don't have a checklist.”
(7B_SM_SCH).
5.6.4.3 Standardized order sets
At three hospitals, participants spoke of the value of incorporating discharge planning
into the standardized order sets. At a large community hospital, a nursing manager described
how adopting an electronic health record system had required the hospital to develop
standardized order sets which led to the development of standardized discharge instructions as a
component of the order sets.
The use of checklists and order sets illustrates how standardizing processes can facilitate
the use of health literate discharge practices, and leads to the following proposition:
Proposition 4: Hospitals with internal absorptive capacity metaroutines to standardize processes
will facilitate the uptake of health literate discharge practices.
127
5.6.5 Responding to environmental mandates
5.6.5.1 Accreditation
Although the accreditation of hospitals in North America is voluntary, it has become a
standard of care (Saufl & Fieldus, 2003) and can be considered a mandate in today’s health care
environment. Participants at eight of the ten hospitals identified accreditation as a force that
drives improvements in patient care. A discharge planner at a small community hospital
remarked on the helpfulness of the accreditation process to improve practices:
Accreditation is a big one (driver), we're going through that now…Updating policies to
make sure and just going through everything to make sure that we're on par with it. I
think it's a positive thing. Big-time positive thing, because it helps us recognize where
we're potentially having those deficiencies, or we're not doing things to the best of our
ability. (4_RNPF_SCH)
A director of nursing described how preparing for accreditation had led to the new
practice of telephoning patients at home after they were discharged from hospital. At a teaching
hospital, two nursing managers spoke of using accreditation as a force to sustain new practices.
One of them stated:
We're not being accredited again for quite some time, but let's not fall off the radar. That
whole of, put your eggs all in one basket, and carry your basket, shaking it to the end of
the line, because we've just been working on it for three months rather than building and
continuing all the time. We're doing things like mock tracers that, as leads, we go around
…and see what we're missing. (10A_NM_TH)
5.6.5.2 Mandates from Ontario’s Ministry of Health and Long-Term Care
Several managers described how mandates from the Ministry of Health and Long-Term
Care (MOHLTC) were leading to improvements in the hospital discharge process. In 2010, the
MOHLTC enacted the Excellent Care for All legislation, which requires hospitals to formulate
128
quality improvement plans (Ministry of Health and Long-Term Care, 2014c). A manager in a
small community hospital described how this initiative had helped to drive a new practice of
telephoning patients within 5 days of hospital discharge:
In the Excellent Care for All legislation, every hospital has quality indicators that is
monitored by the LHIN. All of our senior leaders have paid for performance tied to these
indicators in the hospital. …Our new quality indicator plan started in April. (The
discharge follow-up phone call is) on our QIP (Quality Improvement Plan) for 2015-
2016. (5A_NM_SCH).
The influence of the MOHLTC was echoed by a discharge planner at the same hospital who
clearly voiced the sentiments of the Ministry’s 2015 Action Plan for Health Care (Ministry of
Health and Long-Term Care, 2015b):
Within the last two or three years, there's been a huge, huge cascade of change for our
discharge planning. It's been very impactful for our community and for our hospital. Our
ALC numbers have dropped dramatically…I think a lot of it comes from the legislation,
from the government saying that we're funded to have the right patient, right bed, right
time. That's our mantra that we're going to go by, is we want to make sure our patients
are the right ones in the right beds at the right time. We know hospital's not the best
place for everybody, so we try to facilitate (discharge). (5B_RNPF_SCH)
At a large community hospital, a nursing manager stated that within the first 48 hours of
a patient’s admission, there was a discussion with the patient about length of stay and hospital
discharge, in response to “Home First.” Home First is a provincial initiative to decrease the
number of inpatients with complex needs waiting for long-term care placement. Each LHIN
works with home care services to provide community supports, so that patients can return home
with the appropriate care (Ministry of Health and Long-Term Care, 2014a).
Participants from 7 of the 10 hospitals interviewed described how the introduction of
Quality Based Procedures (QBPs) had led to improvements in care for patients with the common
medical diagnoses of pneumonia, COPD, CHF and stroke. In an effort to improve health care
129
quality, and increase efficiencies in the health care system, the MOHLTC introduced Quality-
Based Procedures (QBPs) to Ontario in 2012. These refer to specific diagnoses and procedures.
Hospitals are reimbursed for the care of patients with these diagnoses, or who require these
procedures, and this reimbursement depends upon hospitals’ use of standardized, evidence-based
practices. QBPs have been released for several diagnostic groups that are commonly admitted to
inpatient medical units and that have been found to have the highest rates of readmission in
Ontario: pneumonia, COPD, CHF and stroke (Ministry of Health and Long-Term Care, 2015a).
At three hospitals, participants described developing and using patient education
materials in response to the QBP initiative. A nursing educator in a large community hospital
remarked: “The QBPs have changed the way we (discharge patients) ... Because we've pretty
much been forced. You have to do better discharge.” (9B_NE_LCH). At this hospital, nurses had
been taught the use of “teach back” in providing discharge education for patients with COPD.
Nursing staff were given “COPD Teach Back” forms. The forms contained essential teaching
points, and they asked nurses to assess the patient’s performance of teach back. This assessment
contributed to decision-making regarding the need for home care services for patients.
A nursing manager in a small community hospital spoke of how QBPs were leading to
improved care for patients, and she highlighted that QBPs are tied to funding for hospitals: “It
makes it complicated when they start tying funding to it. Although without tying funding, how do
you make anybody do anything?” (5A_NM_LCH)
5.6.5.3 Funding models
The MOHLTC introduced the Emergency Department Process Improvement Plan in
2010 to help hospitals reduce patient waiting times in the emergency department and to improve
patient flow (Ministry of Health and Long-Term Care, 2013e). Two community hospitals
130
described how pay for results had led to improved patient flow processes. At a small community
hospital, a nursing manager commented:
…things that affect us are the new funding formulas that the ministry's put out. We have
to find ways of providing high quality care more efficiently. That drives change.
(3_NM_SCH)
This manager was able to introduce a new role, the rapid admission nurse, which had been
successful in improving patient flow at the hospital that was funded by pay for results. After
analyzing data on admission and discharge times, and staffing ratios, he identified the need for a
nursing position that worked during the hours when the majority of hospital admissions occurred,
in the afternoon and early evening:
The rapid admission nurse matches those hours…It's a P4R project (having a rapid
admission nurse), but it's wonderful so far…Pay for results, the ministry gives you
funding dependent on certain measures that you work towards. One of our measures is
from time of admission to getting the patient to the bed, and if our measure is within that
standard, we get so much money for our results…Pulling your patients up and getting
them admitted is the other side of patient discharges and flow. (3_NM_SCH)
Participants spoke of the need to improve performance in an era of fiscal constraint. At a
small community hospital that had implemented a number of health literate discharge practices, a
nursing manager stated:
Our twenty nine acute care beds are less than a ward in the (urban) hospital. We have to
be amazing. Because we're at risk of the government coming in and saying, "This
actually costs us too much to keep your hospital here."…That's what I tell my teams.
We've got to freaking be amazing. (5A_NM_SCH)
As these participants describe, environmental mandates can lead to improvements in how
hospital discharge is carried out:
131
Proposition 5: Hospitals with external absorptive capacity metaroutines to respond to
environmental mandates will facilitate the uptake of health literate discharge practices.
5.6.6 Scanning
As described earlier, two nursing managers from different hospitals spoke of the value of
learning new processes for hospital discharge at an annual patient flow summit held in the
United States every spring. One of the managers, from a small community hospital, spoke of the
value of the conference:
It was amazing. I would highly recommend it. Came out with a lot of good information,
gave me a lot of ideas to think about, and look up, and look into. What happened was I
learned that one of the things that we need to do is really start our discharge at the time
of admission. What we've done is we've really implemented that, and we've really tried to
push that. (3_NM_SCH)
Several nursing managers described how they find out about new practices through reading and
through online resources. One of the managers described how she used social media to stay
abreast of current trends:
“I mind a bunch of groups on LinkedIn for healthcare executives. What's happening out
there? Are there some neat practices? How can we guide it? Can we beg, borrow and
steal from people? See what's really working great for others. Is there any way for us to
do something like that here?” (5A_NM_SCH)
This led to the sixth proposition:
Proposition 6: Hospitals with external absorptive capacity metaroutines for scanning will
facilitate the uptake of health literate discharge practices.
5.6.7 Engaging patients and families
132
The need to help patients and family find and understand the information that they need was
expressed by a nursing educator at a large community hospital:
Sometimes we find (that) families and patients (say) "You're the expert. You just tell me."
They don't want to take any ownership. I think it's overwhelming for them, too.
Sometimes they don't even know where to start to ask and they're scared to look silly or
stupid. (9A_NE_LCH)
At this hospital, a discharge checklist is currently being developed. This checklist is for patients
and families to use, to ensure that they know what questions to ask, and that they have received
the education that they need. The same nursing educator stated:
This is the discharge teaching checklist. Things I need to ask my nurse. We're trying to
put the ownership back on the patient. What kind of diet? Do I understand the care for
my ostomy? Those types of things. (9A_NE_LCH)
A similar process to increase patient and family engagement with hospital discharge was
being undertaken at a small community hospital. On the unit, signs were posted that had
questions for patients and families. The nursing manager described the purpose of the signs:
Letting them know upfront what information that we need them to know. We have posters
that are up now and it says: Patients and families, can you answer these questions before
discharge? We have it on the doors before you exit. They're up on the walls. People are
wandering around. They get kicked out of a room because care was going to be provided
or something. They read them. Their family member's going to go home and they say,
"Hey, did we answer this question? Do we know what the answer to this question is
before we can get discharged?" (5A_NM_SCH).
At five of the hospitals, participants stated that they asked patients and families about their living
situations, and their use of home care and community services on admission. Two participants
described a formal process to collect this information. At a small community hospital, a nursing
manager described the use of a paper discharge process tool that was to be used to assess and
133
plan for discharge throughout the hospitalization. At another small community hospital, a charge
nurse described how information was collected on assessment using a standardized form:
Are you going to need services in the home like home care services, CCAC (Community
Care Access Centres)? That's all part of our admission history. Also the nursing
admission history is that when you get home, do you think you're going to need services?
That's part of the admission history process. We start that process right off when we're
starting to do care planning. (8B_SN_SCH)
Participants from two large community hospitals described involving family in the
assessment of the patient’s needs and in discharge planning “when they show up.”
(1B_NM_LCH, 2B_NM_LCH). At one of these hospitals, a nursing manager described how not
assessing a patient’s family situation had led to misunderstanding:
We thought the family wasn’t involved but they all had mental health issues and the
mother did (have mental health problems) so it wasn’t that they weren’t, it was because
they had so many problems of their own, they wanted to be involved, they didn’t know
how…we labeled them as estranged and…they couldn’t be involved, just because of their
own mental health issues… there could be trauma you don’t know about, there’s lots of
stuff. (1A_NM_LCH)
Participants at only four hospitals described actively involving the family in discharge planning.
At a small community hospital, a charge nurse described how she ensured the family was
involved in this process:
The discharge planning happens each time that I go into a patient room with the doctor
… I make sure I bring it up…Usually, I hope, when the family is there then I make sure to
go and talk to them. If the family is not there in the morning when we do a round, then I
make sure I'll go in and talk to the family…If it's a caregiver of an elderly parent or
something, you want them to know what the plan is. (8B_SN_SCH)
At two of the hospital, participants spoke of the need for nurses to start bedside reporting,
that is, sharing information about the patient’s care between nurses working different shifts, at
the patient’s bedside. A nursing manager at a teaching hospital stated:
134
The verbal bedside reporting, that's one thing that we need to make more meaningful
again. I think that nurses need to hear that patient's story. (As the manager) I get the
patient's story. It comes to me, but they don't always hear that. (10B_NM_TH)
A nursing manager at a large community hospital also spoke of the need for bedside reporting, so
that the patient understands their own care:
I think bedside reporting is the way to go, making the patient be aware of everything
that’s going on, so when I go into the room, and I haven’t had you for 5 days, you the
patient can tell me what’s going on. (1B_NM_LCH)
Participants at three of the hospitals described how the demographics of the community
served by the hospital drive change. Although the largest concentration of Aboriginals in Canada
is located in Ontario (Government of Canada, 2014), only one hospital described efforts to meet
the needs of this population. Participants from this hospital spoke of the creation of an
Aboriginal Advisory Committee as part of the hospital’s strategic plan. This engagement had led
to the development of a new role within the hospital of Aboriginal Patient Navigator who was
charged with facilitating the discharge process for Aboriginal patients:
We have an Aboriginal navigator to help us with our First Nations people. They navigate
through the hospital, talk to them (Aboriginal patients), what's going on, and just helping
them discharge because a lot of the time they'll be discharged and they have not a clue of
what happened, what's wrong with them, and then they're a bounce back…It's helped us
out immensely. (6C_RNFP_TH)
A social worker at a teaching hospital described how the hospital used “Patient Family
Advisors.” She shared a document that reported the hospital has 76 Patient Family Advisors,
who have been involved in more than 200 working groups and committees. Their participation
has led to several improvement activities that included “better discharge information for
patients.” (6A_SW_TH)
These results lead to the seventh proposition:
135
Proposition 7: Hospitals with internal absorptive capacity metaroutines to engage patients and
families will facilitate the uptake of health literate discharge practices.
5.6.8 Fostering an inclusive culture
Participants from three community hospitals spoke of the tensions, misunderstandings
and different values that existed on their hospital units. An inclusive organizational culture is one
that values open communication and shares information (Schein, 2010).
Participants from two hospitals spoke of the changes in the pace of health care, and the
resulting tensions between senior and junior staff members. A manager in a small community
hospital described the changes that had been made to reduce their ALC beds from half of the
inpatient unit to being a rare occurrence, and the impact on team members
We have long term staff members here (who have) been here twenty five, thirty five years.
They have seen now a massive change in healthcare in the last few years. If we continue
to try and do our work the same way we did our work when half of our population was
ALC, versus now where we can admit and discharge three to five people a day, we need a
different way of doing our work. That's a mind shift. We also have a number of staff who
have been here three to five years…they're used to fast change, ready to go, “What's
happening, where do we go? I can handle things coming at me. I'm good. You just keep
them flowing.” I have another part of my team that's like, "Whoa. Slow down the change.
What's going on? This isn't how we used to do it." It creates a team dynamic. People get
frustrated and they struggle with that. I think because of that a barrier to discharge
becomes everybody's just in a fluster… (5A_NM_SCH)
In conjunction with the tension between generations, participants at two hospitals spoke
of the tension between the philosophy of care of senior staff members, and the current pressures
facing hospitals. A discharge planner at a small community hospital spoke of the barrier that the
mindsets of senior staff posed:
A lot of our older docs and our older nurses are still of the belief that hospital is best
place for people. I would say that's our hugest barrier. (5B_RNPF_SCH)
136
At a large community hospital, a nursing manager also spoke of the differing
philosophies of care between generations of staff:
We have a lot of, senior staff that nursed when nursing was a little bit different, and they
really feel the need, they really see, they don’t understand acute care, right? So they get
very focused on things like cutting people’s hair, and they sort of lose track of the whole
(discharge) education piece. (2B_NM_LCH)
This same manager described how front line staff was not aware of the reasons for the pressures
facing hospitals, and the need for organizational changes: “The problem is I think my personal
feeling is, we haven’t given that information to the front line.” (2B_NM_LCH)
The values and beliefs of staff members could also be seen to pose a barrier to patient
discharge. This was described by a manager at a small community hospital:
(What’s) helpful is getting the (staff) to understand the process of discharge. What we
have to do. Trying to say that a huge one for the nursing staff, and it goes against their
grain in nature, is that people have the right to live at-risk. If a person has an
understanding of the risk, and they're willing to assume that, then they're allowed to go
home and live in the condition that they've lived in. I find my staff are extremely caring,
compassionate. They don't want to see somebody that has been living in a home with dirt
floors for 20 years go back to that, but that's the way they've been living and they've
chose to live for 20 years. It's not a problem to send them back. Set up community
supports. It's not a reason for them to stay in the hospital. That's really difficult for the
nursing staff. They can actually become a barrier to discharge. (3_NM_SCH)
These findings led to the eighth proposition:
Proposition 8: Hospitals with internal absorptive capacity metaroutines to foster an inclusive
organizational culture that values open communication will facilitate the uptake of health
literate discharge practices.
5.6.9 Participative decision making to select discharge practices
137
Three participants reported that decisions were made at a senior level, and that, in the
words of a nursing manager at a large community hospital: “…front line staff sort of get told
what to do.” (1B_NM_LCH). However, the majority of participants reported that their hospital
actively sought their feedback.
Three participants described the use of unit-based councils or teams as providing
opportunities for staff to participate in making decisions. In a small community hospital, a
nursing manager used a unit-based team of nurses to develop and introduce a discharge process
tool. He attributed the success of the tool partly to a hospital culture that embraced staff
engagement: “I find that you're going to have better investment. You're going to have better
ownership if you engage the staff. A lot of times these are the experts around a lot of the issues.
(3_NM_SCH).
Participants from two hospitals reported that setting up a formal process for different
professional groups to participate in decision making was key to making effective changes to the
discharge process. A director of nursing at a small community hospital reported that involving
physicians at the outset of a discharge planning project was integral to the project’s success.
Several participants spoke of processes that were put in place to encourage staff members
to suggest new ways of working. A large community hospital had providing Lean training for
staff nurses as well as managers. A nursing educator at the hospital stated:
We have a Lean board in every unit, so sometimes its nurses saying, "Hey, this is not
(working), we could do this way better, and this is my idea," so it's trying to make things
more efficient or better based on the knowledge that you have. (9A_NE_LCH)
A similar process was described in a more informal manner in a small community hospital. A
nursing manager described meeting with staff in his large office that was located on the nursing
138
unit to discuss new ideas and ways to manage problems relating to discharge:
This is just brainstorming with the staff, and talking about changes, and how we can do
stuff. We just throw ideas on the board. We talk about it...Let's just work on this guys.
Here's a problem that I've been presented with. We need to come up with a solution, so
let's do this. (3_NM_SCH).
These findings led to the ninth proposition:
Proposition 9: Hospitals with internal absorptive capacity metaroutines to foster participative
decision making to select new practices will facilitate the uptake of health literate discharge
practices.
5.6.10 Providing leadership
The support of senior leadership to new discharge practices was seen as being key to their
success. Two nursing managers described how senior leadership supported them in discharge
planning. A nursing manager at a large community hospital described the problem of patients
waiting in hospital for a long-term care bed. Although it had not been used, a policy had been
developed to prevent this from happening, and this process depended on the support of the
executive leadership team:
…waiting in hospital for long term care bed, because that’s a big problem for us, is not
an option. You can’t wait for long term care in the hospital, it’s a process that has go up
to the senior team now, you have to be approved by the vice president, so the VP of all (of
the hospital) sites has to approve you to wait in the hospital. (2B_NM_LCH)
At a small community hospital, a nursing manager described a number of steps that had
been taken to reduce the number of ALC beds from half of the inpatient unit to being a rare
occurrence, and the support she received from senior leadership when she faced opposition from
physicians or family members to discharge plans:
139
I can tell you that if I'm in conflict with a physician, a family, my VP and my CEO are not
going back over my head. It's not that if you just are cranky enough or you jump me and
get to the CEO (that) the decision is going to be over turned. It's not. He's going to say…
go back to the team. (5A_NM_SCH)
The importance of being visible to staff and responsive to their concerns was voiced by
nursing managers who were interviewed. A nursing manager in a small community hospital
stated:
I used to be in the administration wing. I felt that part of being manager on the floor is to
be visible and accessible to the staff. I felt that was lacking…I ended up assuming this
room. It's right on the floor. Staff can come. I have an open door policy. They come in,
they come out at all times. They know they can, it's not disruptive for me. (3_NM_SCH)
These findings led to the tenth proposition:
Proposition 10: Hospitals with internal absorptive capacity metaroutines to provide leadership
that is supportive and responsive will facilitate the uptake of health literate discharge practices.
5.6.11 Evaluating
Several participants described the value of including patients and staff in evaluation
activities. Participants from four different hospitals spoke of engaging patients when evaluating
current or new processes. A manager from a small community hospital described the use of new
patient education materials on the nursing unit:
We've trialed this (patient educational resource materials) for a month. Right now, we're
going to take it down. We're going to review it. I've met with some patients already and
asked them how they feel about it. (3_NM_SCH)
A manager from a teaching hospital described the introduction of patient advisors to the
unit’s continuous quality improvement council. She described the benefits to nursing staff of
engaging patients in their work:
140
I think that nurses need to hear that patient's story…When nurses are engaged and hear
that, because we're all here to look after the patients, it makes more meaning, and they're
more engaged in the change rather than me saying, "Here you go, do this." …I think it's
hearing that patient's story is what will really make the change more meaningful and the
staff will be more engaged. (10B_NM_TH).
Participants from two hospitals spoke of the benefit of having staff engaged in evaluating
new processes. A manager of a small community hospital spoke of introducing a new program to
phone patients at home following their discharge from hospital, and ensuring that patients knew
the answers to questions about their follow-up care. She involved staff in evaluating the results
of the phone calls:
They're able to see, "Wow, why are we always missing this one question?” When you
keep seeing a tool that's going out the door and people are missing it, it makes it hit
home. You start focusing on, I need to make sure that I'm getting that question done.
Really ties it back around. It just brings a bit of ownership back home. (5A_NM_SCH)
A director of nursing at a small community hospital described the process of involving
staff in the evaluation of a new discharge pamphlet:
If we're deciding to change something, what we'll do is, we'll sit down, try to figure out
what the best way is to do it. Then, we'll involve the staff by saying this is going to be a
three month trial. Okay let’s see how you like it. If you don't we're going to sit down and
talk about what went right and what went wrong. I can tell you our discharge planning
pamphlet that we fill out has probably changed four times. Based on staff input.
(7A_SM_SCH)
These findings led to the eleventh proposition:
Proposition 11: Hospitals with internal absorptive capacity metaroutines to evaluate processes
with patients and staff will facilitate the uptake of health literate discharge practices.
The results and propositions of this study are illustrated in Figure 5.
141
Figure 5
Absorptive Capacity Metaroutines for Uptake of Health Literate Discharge Practices
5.7 Discussion
The results of this qualitative study offer insights into the absorptive capacity
metaroutines that hospital managers and leaders use to support the uptake of health literate
discharge practices, and are illustrated in the model presented (see Figure 5). This study
examined the absorptive capacity metaroutines of: allocating resources, responding to
environmental mandates, fostering participative decision making and scanning the environment.
In addition to these four metaroutines, seven new absorptive capacity metaroutines were
identified by participants: building and nurturing external relationships, fostering internal
networks, standardizing processes, engaging patients and families, fostering an inclusive
142
organizational culture, providing leadership that is supportive and responsive, and evaluating
processes.
Participants described examples of how the use of these metaroutines facilitated the
uptake of health literate discharge practices. For example, participants described how the
introduction and use of rounds to discuss patients’ discharge plans helped staff to share
information and resources to ensure that patients had the services they required. Several
participants spoke of the value of maintaining and growing their relationships with colleagues
within the hospital. These routines illustrated the metaroutine of fostering internal networks.
Table 24 illustrates the routines that participants used to carry out the metaroutines described in
this paper:
Table 24
Absorptive Capacity Metaroutines and Routines Used in the Adoption and Use of Health Literate
Discharge Practices
Metaroutines Routines
Allocating resources Using IT to share information between hospital and
community providers/services
Designating space for staff to use for discharge
planning
Establishing discharge planning staff roles
Ensuring managers & staff can take advantage of
professional development opportunities
Building & nurturing external
relationships Meeting regularly with partners in community
organizations
Promoting relationships with community colleges &
universities
Participating in communities of practice
Fostering internal networks Using weekly hospital rounds to discuss complex
patients
Using daily unit rounds to review discharge plans
Leveraging social networks within hospital
Standardizing processes Creating & using checklists
Creating & using standardized order sets
143
Responding to environmental
mandates Using Accreditation as an incentive to sustain new
practices
Working with government initiatives to improve
patient care
Scanning Encouraging members to stay abreast of new practices
through reading, attending conferences, online
resources and social media
Engaging patients & families Ensuring process exists for communication with
patients & families about their needs & experiences
Including patients & families in evaluating work of
hospital
Fostering an inclusive
organizational culture Sharing organizational goals with front line staff
Ensuring that staff have forums to communicate with
each other
Fostering participative
decision making Actively requesting feedback from all levels of the
hospital to select new practices for adoption and
implementation
Using unit-based councils and teams to provide
opportunities for staff & physicians to participate in
decision making
Providing leadership Supporting front-line staff and managers in their use of
new discharge practices
Being responsive to concerns of front-line staff and
managers
Evaluating Including patients and staff in evaluating practices
Responding to input of patients and staff
As stated above, 7 absorptive capacity metaroutines were identified by participants, in
addition to the 4 that were studied: building and nurturing external relationships, fostering
internal networks, standardizing processes, engaging patients and families, fostering an inclusive
organizational culture, providing leadership that is supportive and responsive and evaluating
processes. There is literature to support the use of these metaroutines.
5.7.1 Building and nurturing external relationships
Hospitals learn from relationships with other health service organizations, external
experts, educational institutions and professional associations. These collective relationships
144
form the hospital’s social capital. Several studies have found that a relationship with a university
(Everett & Sitterding, 2011; Olade, 2004; Plath, 2013) or with a professional association (Berta
et al., 2013; Parsons et al., 2013; Ploeg et al., 2007) was a facilitator to the use of research-based
evidence within health service organizations. In addition, two studies found that there was a
positive relationship between a hospital’s use of evidence and its teaching status (Okafor &
Thomas, 2008; Parsons et al., 2013).
In addition, both Ellen et al. (2013) and Berta et al. (2013) found that an important
resource for the introduction of evidence-based practice was the presence of external experts
who were able to teach staff and transfer knowledge into the organization.
Similar to communities of practice, the formation of collaboratives between organizations
has been found to be a facilitating factor to the uptake of new evidence. A 2003 study reported
on the development of a collaborative between multiple neonatal intensive care units in 34
different hospitals to improve quality and safety. The formation of this multidisciplinary
collaborative was found to be time-consuming, however, it was also found to help develop and
share tools and resources to make improvements (Horbar, Plsek, & Leahy, 2003). A
collaborative model was also a facilitating factor in the uptake of evidence-based safety practices
in Massachusetts hospitals. Hospital CEOs and team leaders reported that the collaborative
meetings offered the chance to learn from peer organizations (Leape et al., 2006).
Poor outcomes can result when hospitals and primary care organizations do not
communicate. Hesselink et al (2013) examined the use of evidence-based discharge practices in
five European countries. It was found that one of the barriers to providing optimal care
transitions between the acute and primary care sites was the lack of relationships between the
two settings. In the acute care setting, physicians and nurses stated they did not value
145
communication with primary care providers nor did they value what was considered
“administrative work,” which included planning a patient’s discharge and sending a discharge
summary or letter to the primary care provider. Physicians and nurses working in the community
expressed frustration that the acute care providers seemed to overestimate the information they
were given and the resources that were available. The lack of communication between the
settings was seen to contribute to a sense of exclusion between these two settings, and was
viewed as contributing to inefficient and ineffective care transitions for patients (Hesselink et al.,
2013).
5.7.2 Fostering internal networks
Internal networks are the formal and informal routes that members of an organization
may use to exchange information and communicate about innovations (Lewin, 2011). A
facilitator to adopting evidence-based practice is active encouragement on the part of the
organization to encourage its members to form networks. In a 2013 Canadian study that explored
the features of health service organizations that successfully took on new practices, participants
reported that the organizations created awareness of members with key expertise, and made it
easy for staff to form relationships with librarians, researchers and epidemiologists (Ellen et al.,
2013). In a study that compared the characteristics of high- and low-performing anticoagulation
clinics in Veteran Affairs’ hospitals in the United States, it was found that in the clinics with the
highest level of performance, that health care providers were openly encouraged to discuss and
seek advice about challenging patients (Rose et al., 2012).
The benefits of internal collaboration and multidisciplinary engagement in achieving
project goals were reported in the case studies of a hand hygiene strategy in a California teaching
hospital (Day, 2009) and the introduction of a new ventilation therapy mode into a neonatal
146
intensive care unit (Jackson, Vellucci, Johnson, & Kilbride, 2003). Internal collaboration was
also a key factor in the implementation of clinical practice guidelines in Ontario long-term care
homes (Berta et al., 2013) and in the uptake of multiple practice changes over a 4 year period in
a Canadian pediatric hospital (Kimber et al., 2012). Similarly, the development of CoPs
(communities of practice) within an Australian disability services organization helped members
to connect around practice topics and provided a way to communicate about evidence-based
changes (Plath, 2013).
5.7.3 Standardizing processes
The standardization of documentation and care processes has been done with goals of
promoting consistent practice, increasing organizational efficiency and delivering quality care
(Berta et al., 2010; Rose et al., 2012; Stoeckle-Roberts et al., 2006). The Contingency Model of
Innovation Adoption points out that the standardization and documentation of practices are
structural capabilities that promote the implementation and spread of innovations within
organizations (Berta et al., 2005).
Seven studies of evidence-based practice changes within health service organizations
found that standardized documentation in the form of patient or client assessment forms, flow
sheets, medication charting forms and checklists were associated with the uptake and
implementation of change (Berta et al., 2010; Dufault, 2004; Jackson et al., 2003; Ploeg et al.,
2007; Rose et al., 2012; Stoeckle-Roberts et al., 2006; Taylor, Hepworth, Buerhaus, Dittus, &
Speroff, 2007).
5.7.4 Engaging patients and families
147
Although traditionally this has not been the case, ideally, the adoption of evidence-based
practices takes into consideration the choices and values of patients and families, in other words,
the end users of health care services (Fineout-Overholt et al., 2004). When health care providers
collaborate with patients and families, there is evidence that patient safety, quality of care, and
patient satisfaction increases (Hesselink et al., 2012; Longtin et al., 2010; Oates, Weston, &
Jordan, 2000).
Two participants spoke of value of nurses engaging in bedside reporting with patients.
Nurses engage in reporting when patients’ care is being transferred between nursing staff, such
as at shift handover, or when a patient is being transferred to a different unit in the hospital.
Conventionally, this reporting has taken place in a staff lounge, the nursing station, or another
site away from the patient (Ferguson & Howell, 2015). When nurses engage in reporting at the
patient’s bedside, this has been found to increase patient satisfaction, and patients and families
are able to ask questions and to clarify points of care (Baker, 2010; Evans, Grunawalt, McClish,
Wood, & Friese, 2012; Tobiano, Chaboyer, & McMurray, 2013).
5.7.5 Fostering an inclusive culture
Organizational culture has been defined as the values and norms of an organization that
may be expressed in its behaviours and/or attitudes (Dobbins et al., 2002; Hesselink et al., 2013;
Kimber et al., 2012; Lukas et al., 2007). A 2010 study examined the introduction of evidence-
based clinical practice guidelines into long-term care homes in Ontario. A facilitator to the
adoption of the practice guidelines was having a learning culture in place, and a lack of a
learning culture was found to be a barrier to change (Berta et al., 2010).
148
A 2013 Canadian study interviewed managers, librarians and managers in three different
settings: administrative (provincial health authorities in Ontario and Quebec), acute care and
primary care. It was found that a culture that valued research and learning was fostered by
regular meetings held throughout the organization to present and share research evidence (Ellen
et al., 2013). In order for learning to take place across an organization, it is imperative that
there be an inter-organizational culture that fosters learning and teamwork between units
(Barnsley et al., 1998). In a four year case study of a Canadian pediatric hospital that
implemented multiple nursing best practice guidelines, it was found that an inclusive culture that
involved both direct care providers and management was a facilitator to introducing evidence-
based practices (Kimber et al., 2012).
An exclusive culture, that is, one in which there is a lack of collaboration between units,
can be a barrier to introducing an innovation. The 2004 study that examined the facilitators and
barriers to introducing a screening program to patients attending a medical clinic in a network of
six Veteran Affairs’ hospitals in the United States found that one of the barriers to the success of
this evidence-based practice change was poor communication between services, and in
particular, between the clinic and pharmacy staff (Sharp, Pineros, Hsu, Starks, & Sales, 2004).
5.7.6 Providing leadership
A 2015 study examined the implementation of Project RED in 10 hospitals in the United
States. Strong leadership was found to be associated with successful implementation. The two
hospitals that had the highest level of implementation fidelity had a senior leader on the
implementation team. This was perceived as sending a strong message to staff about the
importance of Project RED. In total, 8 of the sites were able to successfully implement the
practices contained in Project RED. Each of these sites had a team leader who was a middle
149
manager that was well respected and that reported directly to senior management. This was not
the case in the hospitals where implementation was not successful. In one hospital, the manager
did not report to senior management and lacked authority, and in the other hospital, the team
leader was a nurse educator who was not situated in a position to work effectively with hospital
physicians or to coordinate needed IT resources (Mitchell et al., 2016).
The support of leaders and managers has been consistently found to increase the uptake
and implementation of evidence-based practice in nursing care settings. According to the
Contingency Model of Innovation Adoption, the characteristics of clinical leaders may influence
knowledge transfer (Berta et al., 2005). Although this model refers to long-term care settings,
this relationship has been demonstrated in a number of studies in other settings.
A 4 year case study of a Canadian pediatric hospital that implemented multiple nursing
best practice guidelines found that nursing, administrative and managerial staff reported that the
role of leadership was essential to the adoption of the guidelines (Kimber et al., 2012).
In a large survey of clinicians and case managers (n= 303) working in 49 mental health
service organizations in California, it was found that there was a significant relationship between
positive rankings of organizational leadership and positive attitudes towards evidence-based
practice (Aarons, 2006).
In the Massachusetts statewide hospital initiative, it was found that successful
implementation of the practice change within hospitals was correlated with the active
involvement and participation of a senior administrator (Leape et al., 2006). This positive
relationship between leadership and adoption of change was echoed in the survey of pharmacy
directors that examined the relationship between “innovation adoption elements” and the use of
150
evidence-based practices. It was found that the receptivity of leadership roles to change was
significantly associated with the number of innovation adoption elements that were in place
(Okafor & Thomas, 2008).
5.7.7 Evaluating
Once an organization has adopted and implemented an innovation, it needs to evaluate its
use. In this study, the metaroutine of evaluating is synonymous to the metaroutines of reflecting
and updating that Lewin et al (2011) describe. According to Lewin et al (2011), the metaroutines
of reflecting and updating allow an organization to evaluate the utility of an innovation.
Reflecting and updating are necessary precursors to replication or spread, since they allow an
organization to study the relationship between the application of knowledge and its influence on
performance.
Evaluating, or reflecting and updating, needs to include collecting and using feedback
from organizational members. The evaluation of practice changes in a disability services
organization in Australia demonstrates the importance of involving front-line staff in the
evaluation process. Their feedback was essential to discerning contextual factors that influenced
the implementation of changes. In addition, Plath et al (2013) highlight the need to demonstrate
to front-line staff that changes are adapted in response to their feedback. These routines of
evaluating and obtaining feedback from front-line staff not only help to adapt the use of
evidence-based practices, they also help to ensure the use and growth of new knowledge (Lewin
et al., 2011).
5.8 Barriers to the use of health literate discharge practices
151
Barriers to the use of health literate discharge practices were described in the interviews.
For example, although all of the interviews took place in hospitals that used electronic health
records, half of the hospitals relied on the use of facsimile to communicate with community
providers, and on the use of carbon paper for writing patients’ discharge instructions. In other
industries, the use of facsimile and carbon paper are regarded as antiquated modalities. Clearly,
there is a need for hospitals in Ontario to adopt new modes of communication to ensure that
community providers are able to help patients in their transition from the hospital to the
community, and to guarantee that patients and families have a clear record of discharge
instructions.
Another barrier to the use of health literate discharge practices was workload.
Participants who worked in managerial roles felt that staffing was adequate. However, this view
was not shared by participants in non-managerial roles who viewed health literate discharge
practices as being valuable, and who also identified workload as being a barrier to their use.
While the intraprofessional team is involved in planning and carrying out hospital
discharge, it is the nurse who holds the primary responsibility for discharge teaching (Bobay,
Bahr, Weiss, Hughes, & Costa, 2015; McHugh et al., 2013; S. Stevens, 2015). There is evidence
supporting the relationship between nurse staffing in hours per patient day and the quality of
patients’ preparation for discharge and their subsequent hospital readmission and emergency
department use within 30 days of discharge (Nosbusch, Weiss, & Bobay, 2011; Weiss,
Yakusheva, & Bobay, 2011). Nosbusch et al (2011) point out that the role of nursing in hospital
discharge is crucial, however nurses are not given the resources they need, in terms of time and
decreased workload, to educate patients and families prior to discharge. Several participants
echoed this finding. If the uptake of health literate discharge practices is to be successful, it is
152
imperative that nurse to patient staffing ratios allow for nurses to execute them (Bobay et al.,
2015).
In addition, ensuring that staff and managers are able to take advantage of professional
development opportunities is known to increase the adoption and use of evidence-based practices
(Berta et al., 2013; Ellen et al., 2013; Greenhalgh et al., 2004). However, only managers had
ready access to the resources of time and money to attend conferences or to take courses. Front-
line staff were encouraged to attend educational events on their days off, and managers described
only how they were willing to re-arrange schedules for staff. In this era of fiscal constraint, these
are barriers that will require an examination of how current resources are being used.
This study highlights the need to be cognizant of the absorptive capacity metaroutines
and routines that are highlighted in Table 24. Participants described how the routines listed led to
improved discharge processes for patients and families, such as through sharing tools and
resources within the hospital and between organizations. They led to improved communication
within and between teams, as well as improved communication between staff and patients and
families.
5.9 Limitations
A limitation to this study was that two of the small community hospitals had only one
participant. A goal of this study was to interview two health care providers or managers at each
of the hospitals. At one of the small community hospitals, a participant refused to be interviewed
despite agreeing to the interview previously. At the second community hospital, a second
participant was not available on the scheduled date. This limited the amount of data that could be
collected from these two hospitals for this study.
153
Triangulation was not used in this study. The findings from the interviews could be
triangulated with objective measures such as patient satisfaction with discharge processes, as
measured by the Picker© survey that is used by hospitals in Ontario. In addition, questions
related to “therapeutic self care – readiness for discharge” are asked of inpatients in a number of
Ontario hospitals as part of the HOBIC program (Health Outcomes for Better Information and
Care), an initiative of ICES (Institute for Clinical Evaluative Sciences). This data could have
been used for triangulation purposes.
5.10 Future research
In two recent studies of the characteristics of high performing hospitals, performance
monitoring and establishing clear lines of accountability were identified as being key (Luxford,
Safran, & Delbanco, 2011; Taylor, Clay-Williams, Hogden, Braithwaite, & Groene, 2015). In
addition, the 2015 study that examined the implementation of Project RED in hospitals in the
United States found that successful implementation of Project RED was associated with creating
accountability for performance (Mitchell et al., 2016)
Other than identifying accreditation as a driver to monitor hospital care and to introduce
new practices, participants did not describe if and how performance is monitored in the hospital.
Performance measurement, as through the use of an organizational survey, is integral to the
adoption of health literate practices. As organizations, hospitals need to be able to assess current
levels of performance in order to take on new practices and measure changes in performance
(Kripalani et al., 2014).
154
In addition, participants did not speak of accountability. This may be a limitation of the
interview guide that did not examine the use of accountability, however, it may also indicate that
there is a lack of accountability in the hospitals where the interviews took place.
Finally, there is a need to understand how health care organizations sustain such
practices. While it may seem self-evident that the use of the absorptive capacity metaroutines
described in this study would lead to continued growth and change within the organization, there
is a need for longitudinal research to demonstrate this relationship
The model that is presented in Figure 5 includes the absorptive capacity metaroutines that
hospitals in the study used to adopt and implement health literate discharge practices. However,
this model could be extended to the adoption and use of any evidence-based practice. Examining
the use of absorptive capacity metaroutines in health care organizations can be used to gauge
whether they have the capacity to take on and successfully implement evidence-based practices.
Furthermore, examining how different health care organizations use absorptive capacity
metaroutines could be used to understand why health care organizations have different
experiences with the implementation of evidence-based practices.
.
155
Chapter 6:
Relationships between Organizational Characteristics and Use of Health
Literate Discharge Practices in Ontario Hospitals
6.1 Background
To hospitals that have yet to incorporate such practices, health literate discharge
practices represent an innovation – as does any knowledge, including evidence based practices,
that is encountered by an organization for the first time (Damanpour et al., 2009; Panzano &
Roth, 2006). When an organization such as a hospital adopts and implements an innovation, it is
engaging in organizational learning. For implementation of an innovation to succeed, and for
learning to occur that is beneficial or that improves performance, an organization needs to have
resources available to members (Argote, 2012; Berta et al., 2013).
Size, teaching status, and location are characteristics of hospitals that have been
associated with their organizational performance and their uptake and use of innovations
(Damanpour, 1992; Dijkstra et al., 2006; Dupree, Neimeyer, & McHugh, 2014; Greenhalgh et
al., 2004; Heidenreich, Zhao, Hernandez, Yancy, & Fonarow, 2012). A recent systematic review
found that there was a lack of high quality evidence for a relationship between the organizational
characteristics of hospitals and performance (Brand, Barker et al., 2014). In addition, there is a
lack of research on the relationship between hospital characteristics and performance, including
uptake of innovation, in the Canadian context. The purpose of this study was to examine the
relationship between the organizational characteristics of hospitals and one aspect of their
performance; their use of health literate discharge practices as reflected by their score on the
survey developed and administered in earlier chapters.
6.2 Theory and hypotheses
156
6.2.1 Organizational characteristics and hospital readmission
There has been limited study of organizational characteristics associated with hospital
readmission. A recent study of hospitals in the United States examined the relationship between
hospital characteristics including number of hospital beds, financial status (that is, non-profit,
for-profit, or government), and medical school affiliation and found that these three
characteristics accounted for less than 1% of hospital readmissions (Singh, Lin, Kuo, Nattinger,
& Goodwin, 2014). An earlier study examined the relationship between hospital characteristics
and readmission of patients who had an index hospitalization for acute myocardial infarction or
heart failure. Hospital characteristics were defined as number of hospital beds, ownership
(government, non-profit or propriety), teaching status and availability of bypass surgery in the
hospital. This study found that these four characteristics accounted for less than 3% of hospital
readmissions for these patients (Krumholz et al., 2009). Neither of these studies examined
discharge practices and the findings are limited by the focus on a small number of explanatory
variables.
Hospital size has been found to be related to 30-day readmission. Smaller hospitals are
found to have higher readmission rates than larger hospitals in Ontario, however small hospitals
have a higher proportion of medical patients and this may be a reason for this relationship
(Canadian Institute for Health Information, 2012).
6.2.2 Slack resources and organizational size
Slack resources refer to those resources or capacities that are not needed by the
organization to meet the organization’s immediate demands, but can be used to allow the
organization to learn, change and make adaptations to new and existing processes (Cohen et al.,
1972; Cyert & March, 1963; Lawson, 2001). For implementation of an innovation to succeed,
157
the organization needs to have resources or tools available to members of the organization
(Argote, 2012; Berta et al., 2013). While it is difficult to measure slack directly (Nohria &
Gulati, 1996), organizational size may be considered as a proxy measure. The presence of slack
resources, and larger organizational size, have been associated with increased research utilization
and adoption of innovations (Cummings, Estabrooks, Midodzi, Wallin, & Hayduk, 2007;
Damschroder et al., 2009; Greenhalgh et al., 2004; Jiménez-Jiménez & Sanz-Valle, 2011).
Organizational size may be positively correlated with innovation development and use because
larger organizations may have more resources to invest in examining and using new knowledge
(Damschroder et al., 2009; Greenhalgh et al., 2004; Jimenez-Jimenez & Sanz-Valle, 2011).
The positive relationship between organizational size and increased research utilization
and adoption of innovations was found in a 1992 meta-analysis, although the positive effect was
more pronounced in organizations that were for-profit and focused on manufacturing, compared
to organizations that were non-profit and service-oriented. In addition, Damanpour et al (1992)
found that there were both benefits and drawbacks to having a large organizational size. While
large organizations had more resources to invest in innovation and learning, they also had larger
administrations that were less receptive to change. The authors also found that organizational
research typically used number of personnel to measure size, instead of examining volume
measures of size (Damanpour, 1992).
A more recent meta-analysis of 53 studies that built on this 1992 paper found that there
was a clear significant, relationship between increased organizational size and increased
innovation (Camisón-Zornoza, Lapiedra-Alcamí, Segarra-Ciprés, & Boronat-Navarro, 2004).
158
Similarly, a 2006 study of 1200 public organizations in the United States, Damanpour and
Schneider found that there was a positive relationship between organizational size and all stages
of innovation adoption (Damanpour & Schneider, 2006).
However, this positive relationship has not been consistently found in the non-profit
sector and specifically, with hospitals. In a study of the relationship between administrative and
technological innovation with size in 79 non-profit organizations in the United States, size was
not found to be a significant predictor of innovation (Jaskyte, 2013).
Two studies examining the relationship between hospital size and use of best practices
had divergent results. A 2012 study of hospitals in the United States looked at the relationship
between the use of four quality measures for heart failure patients, including the use of patient
discharge instructions. Hospitals with fewer than 200 beds had the lowest levels of performance
on all measures (Heidenreich et al., 2012). In a study that examined the relationship between
performance on quality care measures and hospital characteristics, small hospital size was found
to be associated with higher performance scores (Dupree et al., 2014). Another study of
hospitals in the United States did not provide clarification on this point. This study examined the
relationship between hospital characteristics and performance on clinical care measures, and
found that the hospitals with the highest performance had less than 100 beds, or more than 200
beds (Lehrman et al., 2010).
A 2013 study examined the uptake and usage of evidence-based protocols in long term
care homes in Ontario. Organizational size was associated with approaches to protocol
implementation, but no significant relationship was found between organizational size and
protocol use (Berta et al., 2013).
159
Although there appears to be a clear positive relationship between organizational size and
innovation in the manufacturing and service sectors, the relationship is not as clear for hospitals.
Furthermore, with respect to performance outcomes, there is conflicting evidence of a positive
and negative relationship between organizational size and performance in hospitals. While it was
anticipated that a significant relationship would be found, the nature of this relationship is
unknown.
Hypothesis 1. There is a significant relationship between organizational size and the health
literate survey score.
Although the presence of slack resources has been found to be associated with greater
organizational learning and adoption of innovations, a high level of slack resources has been
associated with poor performance, with respect to learning and innovation. It has been theorized
that an abundant supply of slack resources may indicate waste and inefficiency (Leibenstein,
1969; Williamson, 1964). An inverted U-shaped relationship has been found between the
presence of slack resources and innovation in multinational manufacturing and for-profit service
organizations, suggesting that while slack resources may enhance innovation, extreme levels
may detract from it (Geiger & Cashen, 2002; Nohria & Gulati, 1996). There are no published
studies that this author has found that have looked for such a relationship in non-profit
organizations or hospitals.
Hypothesis 2. The nature of the relationship between organizational size and health literate
survey score is non-monotonic.
6.2.3 Hospital teaching status and performance
160
Similar to the relationship between size and performance, there have been inconsistent
findings when the relationship between performance and teaching status of hospitals is
examined. As noted in the preceding section, teaching status was one of the characteristics
included in the two studies that examined the relationship between hospital characteristics and
readmission. Teaching status was not found to have a strong relationship with the performance
measure of hospital readmission (Krumholz et al., 2009; Singh et al., 2014).
Two studies that examined the relationship between teaching status and performance
found opposing results. The 2012 study that examined the relationship between the use of quality
measures for heart failure patients, found that higher performance was associated with teaching
status (Heidenreich et al., 2012). Conversely, in the study that examined the relationship between
performance on quality care measures and hospital characteristics, non-teaching status was found
to be associated with higher performance scores (Dupree et al., 2014).
A 2006 meta-analysis examined 53 trials that evaluated the influence of organizational
characteristics on the implementation of clinical practice guidelines. All of the trials examined
provider behaviour and patient outcomes as a measure of implementation. The only
organizational characteristic that was found to have an association with implementation was
teaching status. When compared to community hospitals, teaching hospitals had greater
improvements in patient care (Dijkstra et al., 2006).
Being affiliated with a medical school would appear to offer resources to hospitals.
Several studies have found that a relationship with a university is a facilitator to the use of best
practices within health care organizations (Everett & Sitterding, 2011; Olade, 2004; Plath, 2013).
161
Hypothesis 3. Hospitals with teaching status have significantly higher health literate survey
scores than non-teaching hospitals.
6.2.4 Location and organizational performance and uptake of innovation
The geographical location of a hospital may have an influence on its performance
outcomes and its uptake of new practices. Different regions may have different funding sources
as well as different priorities (OECD, 2014). Using hospital readmission as an indicator of
performance, there are significant differences in readmission rates, ranging from 13 to 18%,
among the 14 LHINs in Ontario (Baker, 2011). The different levels of resources between
locations may be contributing to this difference.
Regional differences, and differences between rural and urban settings, have been found
in studies of performance outcomes in hospitals in the United States. In the study that examined
the relationship between surgical performance and hospital characteristics, hospitals that were
located in non-rural locations and in the Northeast and Southern states were found to have higher
performance scores (Dupree et al., 2014). The positive relationship between performance and
non-rural locations was also found in a 2007 study of the relationship between urban and rural
hospitals on quality performance indicators (Lutfiyya et al., 2007).
Conversely, Lehrman et al (2010) found that rural location was positively associated with
performance on clinical care measures. This study also found a regional affect, with location in
the New England or West North Central census areas being associated with the highest levels of
performance (Lehrman et al., 2010).
162
A 2008 study examined the adoption of information technology in Florida hospitals. It
found that geographic location was not associated with adoption of information technology
(Hikmet, Bhattacherjee, Menachemi, Kayhan, & Brooks, 2008).
Hypothesis 4. There is a relationship between the hospital’s geographic location and the health
literate survey score.
6.3 Methods
6.3.1 Sample
The survey measuring use of health literate discharge practices was administered to 143
acute care hospitals in Ontario. Seventy nine hospitals responded, and it is these hospitals that
constitute the sample for this study. The participation rate for hospitals in Ontario was 55%.
6.3.2 Dependent variable: health literate survey score
The dependent variable for this analysis is the health literate survey score generated from
the responses to a 36-item survey measuring use of health literate discharge practices.
Participants were asked to rate the use of 36 practices with a 5-point Likert scale. The survey had
a possible score range of 36 to 180.
As described in Chapter 3, there were multiple responses from 12 hospitals. For each of
these hospitals, the median value for the health literate survey score was used.
6.3.3 Independent variables
6.3.3.1 Organizational size
163
Three variables were considered as measures of organizational size. The first two
variables were measures of the acute and total number of beds for each hospital. This data was
obtained from the Canadian Institute of Health Information’s (CIHI) Canadian Management
Information Systems Database that is available online, and refers to the number of beds staffed
and in operation in the 2013-2014 fiscal year. The acute number of beds includes intensive care
and other acute. “Other acute” includes all acute care beds that are not intensive care beds. The
total number of beds includes intensive care, obstetrics, pediatrics, psychiatric, rehabilitation,
long-term care and other acute (Canadian Institute for Health Information, 2015). The third
variable examined as a measure of organizational size was hospital budget for the 2014-2015
fiscal year. This data was obtained from the Ontario Hospital Association. As will be described
in the results section, the measure of acute care beds was ultimately used.
6.3.3.2 Teaching status
Teaching hospitals are affiliated with medical schools. There are 15 teaching hospitals
and 128 non-teaching hospitals in Ontario. Both large and small community hospitals are
considered non-teaching hospitals (Ministry of Health and Long-Term Care, 2013a, 2013b,
2013c). Dummy coding was used for this variable. Teaching was coded as 0 and non-teaching
was coded as 1.
6.3.3.3 Location
This refers to the geographical location where the hospital is located. This study defined
location with the categorical variable of region and the continuous variable of rurality.
164
There are 5 regions in Ontario, as defined by the Ontario Hospital Association (Ontario
Hospital Association, 2013). These are described in Table 25. Dummy coding was used for this
variable. Region 1, the north region, was used as the reference category.
Table 25
Regions in Ontario
Region Area
1 North (Kenora, Thunder Bay, Sudbury, Parry Sound)
2 East (Ottawa, Kingston, Peterborough, Belleville)
3 Central (Toronto and the GTA)
4 South (Niagara, Kitchener, Oakville, Muskoka)
5 West (Windsor, London, Stratford, Owen Sound)
Region in Ontario may also be defined by the 14 LHINs. The LHINs was not used to
define region in this study, as it would require 13 dummy variables, which would require a
sample size of upwards of 130 hospitals for multiple regression analysis (Field, 2013; Stevens,
2012). See section 6.3.4.1 for a more detailed description of the number of variables that are
recommended for used in multiple regression.
Rurality was also used to define location. As described previously, the rurality score is a
composite value that reflects a community’s location with respect to population density and
travel times for basic and advanced health care services. The rurality index ranges from 0
(highest population density, lowest travel times) to 99 (lowest population density, highest travel
times) (Kralj, 2009)
6.3.4 Analysis
Descriptive statistics were calculated for health literate survey score, number of acute
beds, number of total beds, annual budget, teaching status and rurality.
165
Hierarchical linear regression was used to examine the relationship between
organizational size and health literate survey score, and to determine if there was a significant
non-monotonic relationship between these variables.
Linear regression was used to examine the relationship between teaching status and
health literate survey score, region and health literate survey score, and rurality and health
literate score.
Stepwise backward multiple regression was used to examine the relationship between
organizational size, teaching status and region with the health literate survey score. Stepwise
backward multiple regression was also used to examine the relationship between organizational
size, teaching status and rurality with the health literate survey score. Region and rurality were
examined in separate analyses as they are both measures of location. The advantage of using
stepwise regression is that it obtains the best combination of predictors that have the greatest
variance (Field, 2013, Meyers et al., 2013)
6.3.4.1 Consideration of number of variables in analysis
For multiple regression analysis, it is recommended that there be 10-15 cases per
predictor in the model file (Field, 2013; Stevens, 2012). Other authors are more conservative,
and recommend that the sample size be equivalent to, or greater than, 50 + 8*(number of
independent variables) (Green, 1991; Nelson & Rudestam, 1999). There were 79 hospitals that
responded to the organizational survey. The conservative approach would be to limit the number
of variables to 3 with this sample size (Green, 1991; Nelson & Rudestam, 1999). The approach
in this analysis was to use the recommendations of Field (2013) and Stevens (2012) and limit the
number of variables to 7 or less.
166
6.3.4.2 Consideration of power
To detect a small effect size, assuming a significance criterion of .05 and a power of .8,
with 6 independent variables and using multiple regression, a sample size of 45 would be
required. For a medium effect size with the same criteria, a sample size of 97 would be required.
Finally, for a small effect size with the same criteria, a sample size of 686 would be required
(Cohen, 1992). According to Cohen (1992), a sample of 79 is adequate for this analysis to find a
large effect size.
6.4 Results
6.4.1 Descriptive statistics
Table 26 displays the descriptive statistics for health literate survey score, number of
acute beds, number of total beds, annual budget and rurality.
Table 26
Continuous Variables (n=79)
Variable
(possible range)
Mean Standard
deviation
Range
Health literate survey score
(36-180)
135.92 18.97 78-173
Organizational size
Number of acute beds
2013-14
180.1 206.44 8-770
Number of total beds
2013-14
289.8 328.44 12-1239
Annual budget in dollars
2014-15
252,844,904.5 332,699,726.3 5,885,706 –
1,484,872,537
Rurality
(0-99)
29.58 30.62 0-99
The values for the categorical variables of teaching status and region are found in Table 27.
167
Table 27
Categorical Variables (n=79)
Variable Number of
hospitals
(%)
Mean HL
survey score
(SD)
Teaching
status
Teaching N=15 12 (15.2) 137.5 (4.3)
Non-teaching N=128 67 (84.8) 135.64 (2.41)
Region North 20 (25.3) 146.05 (3.31)
East 14 (17.7) 132.79 (4.05)
Central 11 (13.9) 138.18 (4.55)
South 19 (24.1) 125.16 (5.53)
West 15 (19.0) 137.33 (4.2)
Table 28 shows the characteristics of hospitals by region
Table 28
Characteristics of Hospitals in Each Region
Region Number of
hospitals
Number of
small
community
hospitals*
Number of
acute beds
Mean (SD)
Number of
teaching
hospitals
Rurality
score
Mean (SD)
North 20 16 69.20 (93.77) 2 64.45 (31.63)
East 14 7 147.79
(190.73)
3 23.93 (20.28)
Central 11 1 318.73
(197.34)
4 2.55 (6.58)
South 19 8 265.58
(250.52)
2 15.84 (18.82)
West 15 8 148.20
(196.3)
1 25.6 (19.55)
Total 79 40 180.1
(206.44)
15 29.58 (30.62)
*less than 100 beds
6.4.2 Testing assumptions
168
In order to proceed with regression analysis, several assumptions were tested: normality,
linearity, and homoscedasticity. Outlier analyses were also conducted.
6.4.2.1 Normality
The overall distribution of health literate survey scores is almost normal. The histogram
indicates that the distribution is slightly negatively skewed. See Figure 6 for this histogram. The
Kolmogorov-Smirnov statistic the tests for skewness and kurtosis, and the Q-Q plot indicate
normality (Field, 2013; Meyers, et al., 2013), and can be found in Appendix HH.
Figure 6
Distribution of the Health Literate Survey Score
169
The distribution of health literate survey scores is normal for teaching, non-teaching
hospitals, as well as each of the 5 regions. See Appendix II for the histograms, the Kolmogorov-
Smirnov statistics, the tests for skewness and kurtosis, and the Q-Q plots (Field, 2013; Meyers et
al., 2013).
The distributions of number of acute care beds, total number of beds, budget and rurality
scores are non-normal, and positively skewed. Because the distributions are positively skewed,
they were transformed with a base-10 logarithm to achieve normality (Meyers et al., 2013). This
transformation created distributions that were almost normal, and that demonstrated kurtosis.
For the measure of organizational size, the distribution for the log transformation of the
number of acute beds had the least amount of kurtosis (-1.07). This was less than the log
transformation of the budget (-1.26) and the log transformation of the number of acute beds
(-1.13). Because of this, the log transformation of the number of acute beds was determined to
be the best measure of organizational size.
For the measure of location, a log transformation of rurality was calculated. Log
transformations cannot be done for values of zero, and there were 19 hospitals that had a rurality
of zero. To conduct the log transformation, a constant measure of 1 was added to each value in
the rurality distribution. The log transformation of the rurality score was found to be almost
normal. It was slightly negatively skewed with a skewness of -.48. This level is suitable for
analysis, as several authors recommend that a conservative threshold for skewness lies between
the values of -0.5 and +0.5 (Hair, Black, Babin, & Anderson, 2010; Meyers et al., 2013; Runyon,
Coleman, & Pittenger, 2000).
170
See Appendix JJ for descriptive statistics and histograms for the tests of normality of
acute care beds, total number of beds, budget and rurality, and the base-10 logarithm
transformations of these independent variables.
6.4.2.2 Linearity
Scatterplots were generated to determine if there was a linear relationship between the
independent variable (log transformation of number of acute beds) and the dependent variable
(health literate survey score), and if there was a linear relationship between the independent
variable (log transformation of rurality) and the dependent variable (health literate survey score).
No linear relationships were found. See Appendix KK for the scatterplots.
6.4.2.3 Homoscedasticity
Examination of the residual scatter plot did not support the assumption of
homoscedasticity.
6.4.2.4 Outlier analysis
Outlier analysis, using the three residual statistics: standardized residuals, Cook’s
distance and Leverage, revealed no influential data points or outliers.
6.4.2.5 Multicollinearity
The correlation between the log transformation of the number of acute beds and the
health literate survey score was -.223, p = .02. This indicates that there is a small negative
correlation between the number of acute beds and the health literate survey score. The
correlations between the log transformation of rurality and the health literate survey score was
.13 and this value was non-significant. This indicates that there is a lack of correlation between
171
rurality9 and the health literate survey score. The correlation between the log transformation of
the number of acute beds and the log transformation of rurality was high at -.85.
Multicollinearity would be indicated by a correlation greater than .9 (Field, 2013), and thus, it
was determined that the assumption of no multicollinearity was not violated.
The lack of multicollinearity was confirmed by examining the variance inflation factors.
The factors for all of the models was between 1.005 and 2.025, which is less than the upper limit
of 10 that indicates multicollinearity (Field, 2013). This means that multicollinearity was not a
problem for this analysis.
6.4.3 Regression analysis
Hierarchical linear regression was used to examine the relationship between
organizational size and health literate survey score, and to determine if there was non-monotonic
relationship. The beta values, their standard errors and the standardized betas for the constants
and the independent variables of the two regression models are presented in Table 29. The table
also includes the overall variance explained (R2) and the F value for each model. A significant
negative relationship was found between organizational size and health literate score. That is,
small hospital size was associated with increased health literate score. As Table 29 shows, the
ANOVA indicates a significant relationship for this model, F (1,77) = 4.05, p < .05, calculated η2
= .05. The value of η2 indicates that this is a small effect size.
There was no support for a significant non-monotonic relationship was not found
between organizational size and health literate score. The ANOVA indicates a non-significant
9 From this point, the independent variable of rurality refers to the log transformation of rurality
172
relationship for this model, F (2,76) = 2.65, p = .08. The statistical output is found in Appendix
LL.
Table 29
Summary of Hierarchical Linear Regression Analyses for Relationship for Size as Predictor of
Health Literate Survey Score
Variable Model 1 Model 2
B SE B β B SE B β
Constant 150.89 7.73 178.54 25.96
Acute beds -7.66 3.81 -.22* -38.78 29.16 -1.13
Acute beds
squared
8.05 7.22 .92
R2 .05 .07
F for change
in R2
4.05* 2.65
*p < .05
Linear regression was used to examine the relationship between teaching status and
health literate survey score. See Table 30. The ANOVA indicates a non-significant relationship,
F (1,77) = .1, p = .76. See Appendix MM for the statistical output.
173
Table 30
Relationship between Teaching Status and Health Literate Survey Score
Variable B SE B β
Constant 137.5 5.51
Teaching status -1.86 5.98 -.04
R2 .001
F for change in
R2
.1
Linear regression was used to examine the relationship between region and health literate
survey score. See Table 31. The ANOVA indicates a significant relationship, F (4,74) = 3.51, p
= .01, calculated η2 = .16. The value of η2 indicates that this is a moderate effect size. See
Appendix NN for the statistical output. There is a significant difference between health literate
survey scores in the North and South regions, and in the North and East Regions, with scores in
the North being significantly higher than scores in the South and East.
174
Table 31
Relationship between Region and Health Literate Survey Score
Variable B SE B β
Constant 146.05 3.99
Westa -8.72 6.1 -.18
Southa -20.89 5.72 -.47*
Centrala -7.87 6.7 -.15
Easta -13.26 6.22 -.27**
R2 .16
F for change in
R2
.351**
a Reference group for region is North, * p < .001 ** p < .05
Linear regression was used to examine the relationship between the log transformation of
rurality score and health literate survey score. See Table 32. The ANOVA indicates a non-
significant relationship, F (1,77) = 1.27, p = .26. See Appendix OO for the statistical output.
Table 32
Relationship between Rurality and Health Literate Survey Score
Variable B SE B β
Constant 132.28 3.87
Rurality 3.34 2.96 .13
R2 .02
F for change in
R2
1.27
175
Stepwise backward multiple regression was used to examine the relationship between
organizational size, teaching status, and the log transformation of rurality with the health literate
survey score. The beta values, their standard errors and the standardized betas for the constants
and the independent variables of the six regression models are presented in Table 33. The table
also includes the overall variance explained (R2) and the F value for each model. There was not
a significant relationship between the log transformation of rurality and the health literate survey
score. Model 2 indicates that size accounts for 8% of the variance in the health literate survey
score. As Table 33 shows, the ANOVA indicates a significant relationship for this model, F
(2,76) = 3.47, p < .05, calculated η2 = .08. The value of η2 indicates that this is a small effect size.
See Appendix PP for the statistical output.
Table 33
Summary of Stepwise Backward Regression Analyses for Size and Health Literate Survey Score
Variable Model 1 Model 2
B SE B β B SE B β
Constant 175.51 19.71 168.38 12.95
Acute beds -14.29 7.07 -.42* -11.65 4.46 -.34*
Rurality -2.78 5.77 -.11
Teaching
status
-10.17 7.33 -.19 -11.42 6.83 -.22
R2 .09 .08
F for change
in R2
2.37 3.47a
* p < .05
176
Stepwise backward multiple regression was used to examine the relationship between
organizational size, teaching status and region. The beta values, their standard errors and the
standardized betas for the constants and the independent variables of the six regression models
are presented in Table 34. The table also includes the overall variance explained (R2) and the F
value for each model. See Appendix QQ for the statistical output.
177
Table 34
Summary of Stepwise Backward Regression Analyses for Variables Predicting Health Literate Survey Score
Variable Model 1 Model 2 Model 3 Model 4 Model 5 Model 6
B SE
B
β B SE
B
β B SE
B
β B SE
B
β B SE
B
β B SE
B
β
Constant 162.75 13.28 147.8
8
6.62 146.05 3.99 143.26 3.21 141.33 2.64 139.33 2.33
Acute beds -6.72 5.21 -.2
Teaching -7.02 6.99 -.13 -2.03 5.85 -.039
Westa -6.121 6.42 -.13 -8.65 6.14 -.18 -8.72 6.1 -.18 -5.93 5.63 -.12
Southa -16.62 6.62 -.38* -20.9 5.75 -.47** -20.89 5.78 -.47** -18.1 5.21 .41** -16.17 4.88 -.37** -14.18 4.76 -.32**
Centrala -3,81 7.75 -.07 -8.4 6.92 -.154 -7.87 6.7 -.15
Easta -11.62 6.43 -.24 -13.5 6.29 -.27* -13.26 6.22 -.27* -10.47 5.76 -.21 -8.54 5.47 -.17
R2 .18 .16 .16 .14 .13 .10
F for change
in R2
2.63* 2.8* 4.51* 4.2** 5.74** 8.88**
*p < .05, **p <.001 aReference group for region is North.
178
The results of the regression analysis show 6 models. Hospitals in the North region had
the highest mean score, and hospitals in the South region had the lowest mean score. Each of the
six models demonstrates that this difference is significant. This demonstrates that there is a
significant difference in health literate survey scores between hospitals in the South region and
hospitals in the North region. Hospitals in the South region had significantly lower scores than
hospitals in the North region.
Hospitals in the East region had the second lowest mean on the health literate survey
score. Models 2 and 3 indicate a significant difference in health literate survey scores between
hospitals in the North region and hospitals in the East region. Hospitals in the East region had
significantly lower scores than hospitals in the North region.
The final model indicates that the difference between the North region and the South
region accounts for 10% of the variance in the health literate survey score. As Table 34 shows,
the ANOVA indicates a significant relationship for this model, F (1,77) = 8.88, p < .01,
calculated η2 = .1. The value of η2 indicates that this is a medium effect size.
Post-hoc analyses were done to examine the differences in number of acute beds between
the North and South regions, and the North and East regions. This was examined as there is a
greater number of small community hospitals in the North Region, compared to the South and
East regions. There was a significant difference in number of acute beds between the North and
South regions, t = -3.94 (37), p < .001 (using log transformation of acute beds). There was a
significant difference between the number of acute beds between the North and East regions,
although this relationship was not as strong (as between the North and South regions), t = -2.12
(32), p = .05 (using log transformation of acute beds). See Appendix RR for the statistical output
179
These results led to the question of whether size was a moderator in the relationship
between region and health literate survey score. To test for this relationship, linear regression
was done with Andrew Hayes’ Process Macro: Moderation (Hayes, 2013). There was not a
significant interaction between region and organizational size. As Table 35 shows, the ANOVA
indicates a non-significant relationship for this model, F (9, 69) = 1.62, p = .13. See Appendix
SS for the statistical output.
Table 35
Summary of Regression Analysis for Interaction between Size10 and Region11 as Predictors of
Health Literate Survey Score
Variable Model
B SE B
Constant 143.31 5.0
Size -6.7 7.83
East -10.15 6.48
Central -11.29 9.89
South -15.85 8.08
West -6.35 6.96
Interaction between East and Sizea 15.32 10.97
Interaction between Central and Sizea 19.75 17.67
Interaction between South and Sizea -3.18 14.24
Interaction between West and Sizea .13 13.81
R2 .20
F for change in R2 1.62
aReference group is north region.
As stated earlier, there was a relatively high correlation between number of acute beds
and rurality. The results of the multiple regression led to the question of whether rurality was a
moderator in the relationship between number of acute beds and health literate survey score. To
10 Independent Variable
11 Moderator
180
test for this relationship, linear regression was done with Andrew Hayes’ Process Macro:
Moderation (Hayes, 2013). There was a significant interaction between rurality and
organizational size. As Table 36 shows, the ANOVA indicates a significant relationship for this
model, F (1,75) = 7, p = .01. See Appendix TT for the statistical output.
Table 36
Summary of Regression Analysis for Interaction between Size12 and Rurality13 as Predictors of
Health Literate Survey Score
Variable Model
B SE B
Constant 130.02 3.1
Rurality 1.22 6.04
Size -7.8 8.22
Interaction between Size and Rurality -17.66 6.68*
R2 .13
F for change in R2 3.31**
*p = .01 ** p < .05
6.5 Discussion
This study found a significant negative relationship between organizational size and
health literate survey score, indicating a higher use of health literate discharge practices in
smaller hospitals. As described earlier, the use of health literate discharge practices is an
innovation. Most of the research on organizational size and the use of innovations has taken
place in the manufacturing and service sectors and has shown a positive relationship. This
12 Independent Variable
13 Moderator
181
positive relationship has not been consistently found in the limited research that has examined
the use of innovations in hospitals. The findings of this study are consistent with the work of
Dupree et al (2014) who found that there was a higher level of quality performance in smaller
hospitals.
While a significant relationship was found between organizational size and the health
literate survey score, this relationship was not found to be non-monotonic. Although this finding
is inconsistent with studies of multinational manufacturing and for-profit service organizations
(Nohria & Gulati, 1996; Geiger & Cashen, 2002), Ontario hospitals are operating in a very
different context.
There was no significant relationship found between a hospital’s teaching status and its
health literate survey score. This finding is inconsistent with the findings of Dijkstra et al (2006)
who found a positive relationship in their meta-analysis between hospital teaching status and
uptake of clinical practice guidelines. However, as described, the limited literature that is
available has shown mixed results when examining the influence of teaching status of hospitals
on innovation adoption and performance.
With respect to the relationship between location and health literate survey score, a
significant relationship was found between region and the survey score. In addition, rurality was
found to be a significant moderator in the relationship between organizational size and the health
literate survey score. In other words, there is a significant interaction between size and rurality
in predicting the use of health literate discharge practices.
As illustrated by the findings, hospitals in the North of Ontario had significantly higher
health literate survey scores than hospitals in the South. In addition, hospitals in the North had
182
significantly fewer acute care beds than hospitals in the South. This relationship was also found
between hospitals in the North and hospitals in the East, although it was not as strong as the
relationship between the hospitals in the North and South regions.
This finding is somewhat surprising. As acknowledged by the Ministry of Health and
Long-Term Care, rural and northern communities have decreased access to quality health care.
The Ministry began examining this issue in the 1990s, and has introduced numerous programs
since that time to improve health care access in rural and northern areas (Ministry of Health and
Long-Term Care, 2011). The analysis of the interviews found in Chapter 5 found that the only
participants who described relationships with external hospitals and being members of
communities of practice were working in hospitals that were in rural and/or northern areas of
Ontario. There may be higher levels of external relationships and increased use of social
networks in small, rural hospitals. This is an area for future exploration.
There has been limited investigation of regional differences in the uptake of best
practices. Differences between the regions of Ontario may be due to governance differences
between the LHINs. Each of the 14 LHINs have different levels of performance (Office of the
Auditor General of Ontario, 2015), and there is a need to find out the reasons for these different
levels. Three studies from the United States have found differences between regions of the
country and performance (Dupree et al., 2014; Lutifyya et al., 2007; Lehrman et al., 2010),
however, there has been a lack of investigation into regional differences and performance in
Canada. This study indicates that there is a higher level of use of health literate discharge
practices in the north region of Ontario, which has the lowest number of acute care beds, as well
as the greatest number of rural, small community hospitals.
183
This higher level of performance that was found in the North region is not consistent with
the findings of the 2015 Auditor General’s report, which examined the functioning of LHINs in
Ontario in 2015. They examined the performance of LHINs on 15 indicators, which included
wait times for surgery, unplanned emergency department visits, readmission rates and wait times
for CCAC in-home services. The indicators measured different time periods that ranged from
2007 to 2015. The North East and North West LHINs, which constitute the North region, met
performance targets in only 4 of the 15 areas, which was the lowest level of performance. The
highest level of performance was in the Central LHIN, which is in the South region. This LHIN
met targets in 10 of the 15 areas (Office of the Auditor General, 2015).
The performance targets that were examined are not the same as the indicator used in this
study, that is, the survey score that measured the use of health literate discharge practices. In
addition, the health literate survey score was determined by the participants. In effect, it was a
subjective measure of hospital performance, in contrast to the objective performance indicators
used by the Office of the Auditor General (2015).
There is a need for further study into the differences between the LHINs. While
differences clearly exist, the reasons for the differences are less clear.
6.6 Future Research
There are other organizational characteristics that may have an influence on the health
literate survey score that were not examined in this study.
Hospital discharge is largely carried out by nurses (Bobay et al., 2015). Nursing staff to
patient ratios have been associated with improved patient outcomes (Blegen, Goode, Spetz,
Vaughn, & Park, 2011), increased care quality, increased patient satisfaction (Aiken et al., 2012)
184
and decreased mortality (Shekelle, 2013). Future research could examine the relationship
between nurse to patient ratios and uptake of best practices. The volume of nursing staff within
hospitals could also be used as a measure of organizational size. In their meta-analysis,
Damanpour et al., (1996) noted that the number of personnel has been a common measure of
organizational size in the organizational studies literature. While this has not been the case in the
studies of organizational characteristics of hospitals, it would be interesting to find out if there is
a relationship between the volume of nurses in an organization and the uptake of best practices.
All of the hospitals were unionized. Early research has indicated that when organizations
consider innovations, unions are rarely consulted, and that subsequently, they are not supportive
of them (Fennell, 1984). The more recent study that examined innovation adoption in 1200
municipal governmental bodies in the United States found that the existence of unions was
associated with a lack of support for innovations in the adoption stage, but it did not have a
negative effect on the actual uptake of the innovation (Damanpour & Scheider, 2006). As the
vast majority of nurses working in Ontario hospitals are unionized, it would be interesting to
further explore the relationship between unionization and adoption of health literate discharge
practices.
6.7 Limitations
A limitation of this study is the small sample size. Although significant relationships
were found between region and health literate survey score, and size and health literate survey
score, there may be other significant relationships that would have been found if the sample had
been larger
185
Another limitation was that the health literate survey score is not an objective measure.
Hospitals that participated agreed to participate in the study, and the survey was completed by
members of the hospital organizations.
6.8 Conclusion
These results indicate that there is a significant relationship between hospital size and
location with the use of health literate discharge practices. Smaller hospitals were found to have
increased use of health literate discharge practices compared to larger hospitals, and there was a
significant interaction between size and rurality in predicting the use of health literate discharge
practices.
In addition, the hospitals that indicated they had the highest use of these practices were
located in the North region of Ontario. This indicates that the uptake of health literate discharge
practices was highest in a region with the lowest number of acute care beds, and the highest
number of small rural hospitals. These results were not expected as the adoption and use of
innovations have historically been found to be greater in larger organizations, and in
organizations located in urban regions. There is a need to further investigate these relationships,
in order to determine why the use of health literate discharge practices was higher in small, rural
hospitals, and was higher in the North region.
186
Chapter 7:
Adoption and Use of Health Literate Discharge Practices in Ontario
Hospitals
7.1 Use of health literate discharge practices in Ontario hospitals
The purpose of these studies was to contribute to our understanding of the use of health
literate discharge practices in Ontario hospitals. As previously described, these practices have
been associated with improved patient and hospital outcomes, including decreased readmission
rates. The four studies contained in this thesis demonstrate how these practices can be measured
in Ontario hospitals, and describe how their adoption and use can be facilitated.
The first study determined the essential indicators of health literate discharge practices in
acute care hospitals in Ontario using a Delphi panel. Construct and content validity were
established for the indicators. The panel membership included experts in health literacy and
hospital discharge from across North America, and most of the members were researchers,
providers and managers working in Ontario. This helped to ground the indicators in the context
of Ontario’s health care system.
For example, several participants identified the need to refer patients to a community
pharmacist upon discharge to review medications in adherence to the MedsCheck program which
has been in existence in Ontario since 2008. Through this program, pharmacists conduct a
billable 20-30 minute medication consultation with patients (Ministry of Health and Long-Term
Care, 2008). In other words, providing a patient with a referral to a community pharmacist upon
discharge is a health literate discharge practice specific to the Ontario context.
The second study used the indicators that were developed with the Delphi panel to create
a survey designed to facilitate hospitals’ assessment of their use of each of the indicators. This
187
survey was sent to all adult, acute care hospitals in Ontario, and had a response rate of 55%. The
responses indicate that there is a wide range of use of health literate discharge practices across
Ontario.
Exploratory factor analysis was completed to determine the factor structure of the
construct “use of health literate discharge practices.” Six factors were identified: 1) patient
preparation for follow-up appointments and tests; language assistance; 2) use of a written plan
with the patient and family; 3) coordination of services; contact of informal caregivers; 4)
medication review and patient education; 5) discharge summary and follow up care, and 6)
arrangement of follow up appointments and tests. In addition, reliability of the survey was
established, for all of the 36 items, and for each of these 6 factors.
The surveys were followed by key informant interviews with nursing managers,
educators and front-line staff in ten Ontario hospitals who were participants in the survey. These
interviews led to an understanding of the facilitators and barriers to the use of health literate
discharge practices in Ontario hospitals. Organizational learning theory was used to understand
the absorptive capacity metaroutines that are used in hospitals to adopt and implement new
practices.
The fourth and final study gives insight into the relationship between organizational and
micro-environmental characteristics and use of health literate discharge practices. Most studies
that have explored the relationship between organizational size and use of innovations have been
in the manufacturing and for-profit service sectors and there has been little investigation of this
relationship in health care. The little work that has been done to date in this area has taken place
in the United States, and the results are ambiguous. In this study, it was found that smaller
hospitals reported greater use of health literate discharge practices than their larger counterparts.
188
In addition, rurality was found to be a moderator in the negative relationship between
organizational size and use of health literate discharge practices. This is understandable, as
smaller hospitals, which had higher use of health literate discharge practices, are more likely to
be located in rural locations. However, as will be discussed in a following section, there is a need
to further explore these relationships.
7.2 Conceptual and theoretical implications
The Health Literate Care Model is a relatively new model; to date there has been limited
work to test it. This model has not been applied in the Canadian context. According to this
model, strategies of health literate organizations lead to productive interactions between patients
and the health care team. In this study, the model was adapted to focus on health literate
discharge practices as strategies of health literate organizations in the context of Ontario. While
these strategies have been associated with improved outcomes in hospitals in the United States,
particularly in relationship to readmission rates, there is a need to determine whether the same
association exists in the considerably different Canadian health care context.
Organizational learning theory and the concept of absorptive capacity metaroutines were
used to advance our understanding of how hospitals in Ontario adopt and use health literate
discharge practices. The thematic analysis led to the identification of 11 metaroutines and these
metaroutines were used to develop propositions, which were represented in a new conceptual
framework. This framework illustrates how hospitals in Ontario adopt and use health literate
discharge practices, and it could be used as a guide for future studies that seek to further advance
our understanding of how hospitals adopt and use other evidence-based practices. In addition, it
could be extended to other settings, such as primary and long-term care organizations. Testing of
the propositions comprising the framework stands to provide further insight into how health care
189
organizations use organizational learning metaroutines to increase their adoption of evidence-
based practices.
7.3 Implications for hospital managers and administrators
The survey of health literate discharge practices could be used by managers and
administrators to measure the use of health literate discharge practices in hospitals to obtain a
baseline measure of their use. This would identify areas that need improvement and that could be
addressed through quality improvement plans. Survey results could also be used to develop staff
education sessions, as hospital staff may lack awareness of these practices, and may lack
experience using practices such as teach back.
The key informant interviews identified several areas that need attention on the part of
hospital managers and administrators, to facilitate the uptake of new practices. Communication
between hospital providers was found to be key in planning patients’ discharge from hospital.
Several participants described a lack of communication between physicians, nurses and allied
health team members. On several units, nurses and allied health team members held daily unit
rounds on the inpatient unit to review patient discharge plans. Although it was physicians who
were actually responsible for writing discharge orders, physicians rarely attended these meetings.
This had the potential to lead to patients receiving conflicting information, and being discharged
without plans being completed. This finding highlights a need for improved communication
between physicians, nurses, and members of the allied health team to avoid these problems.
A facilitator to the use of health literate discharge practices was the use of
standardization. In hospitals where health literate discharge practices had been adopted
successfully, participants described the use of standardized checklists, order sets and patient
190
education materials. This standardization helped to ensure that patients received the information
that they needed and that there was communication between patients’ hospital and community
health providers.
Another facilitator to the use of health literate discharge practices for nursing staff and
managers was the ability to take advantage of educational opportunities. This is a known
facilitator to the adoption and use of evidence-based practices in health care (Berta et al., 2013;
Ellen et al., 2013; Greenhalgh et al., 2004). However, as both nursing staff and managers
described, typically only managers, to the exclusion of staff, were able to take advance of
professional development opportunities. There is a need for managers and administrators to find
and allocate time and financial resources for nurses to participate in educational opportunities.
7.4 Implications for policy makers
Ontario hospitals have a high rate of hospital readmission. The survey results show a
wide range of use of health literate discharge practices across Ontario hospitals. This may be a
reflection of the variation of hospital readmission rates across Ontario and is a topic for future
research. As stated earlier, how patients are discharged influences their likelihood of returning to
the hospital within one month of discharge (Vashi et al., 2013). If hospitals in Ontario are able to
increase their uptake of health literate discharge practices, there is the possibility that the high
rate of readmission will decrease.
The survey of health literate discharge practices that was developed is the first tool to
measure the use of these practices in acute care hospitals. As Kripalini et al (2014) notes, in
order to become a health literate organization, it is necessary to establish a baseline, and to assess
the current level of performance. Going forward, the survey could be used in quality
191
improvement efforts in Ontario hospitals, where measuring hospitals’ use of health literate
discharge practices prior to implementing an improvement initiative would identify areas in
which to focus improvement efforts, and would offer a reference point that could be used to
check for improvement upon a post-implementation application of the survey tool.
A number of implications for policy makers were also identified through the analysis of
the key informant interview data. While all of the hospitals used electronic health records, there
was a lack of ability to share electronic records between hospitals and health care providers in
the community, and between hospitals. Only one hospital in the North East LHIN described
being able to share electronic records with 5 other hospitals in their region. This communication
was found to be helpful in sharing important health information about patients, including
previous hospital admissions.
There is a need for policy makers to address the information technology needs of
hospitals. Several hospitals described using carbon paper in their communication with patients
and other health care providers. Carbon paper is notoriously difficult to read. There is a need to
ensure that hospitals, providers and patients have access to electronic resources to share
information.
Policy makers also need to address the issue of health human resources in Ontario. All of
the interviewees referred to the key role that nursing plays in the discharge process, and as
previously described, there is a significant relationship between nurse staffing in hours per
patient day and the quality of patients’ preparation for discharge and their subsequent hospital
readmission and emergency department use within 30 days of discharge (Nosbusch et al., 2011;
Weiss et al., 2011).
192
Several interviewees identified the workload of nurses as a barrier to the use of new
discharge practices, and to educating patients and families about hospital discharge. Using
health literate discharge practices requires increased nursing time. The 2009 study of Project
RED estimated that study nurses spent 90 minutes per study participant and this included
communication with patients and family members throughout the hospitalization (Jack et al.,
2009). Interestingly, a 2003 study that took place in Ontario teaching hospitals found that
patients who were discharged from units that had higher proportions of registered nursing and
registered practical nursing staff had better clinical outcomes, compared to units with lower
proportions (McGillis Hall et al., 2003). There is a need to ensure that there are adequate nursing
staff resources to carry out health literate discharge practices.
7.5 Limitations
A limitation to these studies is that they did not incorporate the experiences of patients
and families. Engaging patients and families in health care has been associated with improved
patient outcomes, as well as improved quality of care and patient safety (Carman et al., 2013;
Koh, Brach, et al., 2013). Project RED was developed with patients, and qualitative work was
done to ensure that the needs of patients and families were being met (Jack et al., 2008).
However, to date, there has been no study of the experiences of patients and families with the
practices contained in Project RED.
A limitation to the exploratory factor analysis that is contained in the second study was
the small sample size. The small sample size limits the ability to generalize the findings of this
study from the sample to all hospitals in Ontario.
193
A limitation of the survey is the subjective nature of the tool. The survey was developed
for use by participants to self-rate their hospitals’ use of health literate discharge practices. It is
possible that the ratings that participants provided do not reflect the reality of their actual use.
This was illustrated during one of the site visits. A large community hospital in the Erie St Clair
LHIN had a health literate survey score of 109, which was the 5th lowest score of the 79 hospitals
in the sample. Two interviews took place with educators at this hospital, and it was noted that
this hospital actually had a high level of use of health literate discharge practices. One of the
interviewees, who had participated in the survey, had recently completed a one-year project
where she was focused on improving the hospital discharge process as her full-time role. This
hospital had developed an electronic template for discharge instructions that nurses were using to
provide discharge instructions to patients and families. It is possible that hospitals that are high
performing may actually be more self-critical, although this was not explored. In summary, the
subjective nature of the survey may limit the ability to make comparisons between hospitals.
7.6 Need for future research
There have been few studies examining the experience of patients and families with
hospital discharge (Coleman, Roman, Hall, & Min, 2015), however the limited literature
indicates that hospital discharge is not performed well. As referred to earlier, a 2012 European
study interviewed patients, families and community health providers about their experience of
hospital discharge to find patients and families are poorly prepared or educated at the time of
discharge regarding their care and follow-up instructions. The education that they are given is
delivered in technical language that is difficult for patients and families to understand.
Furthermore, the education is typically unorganized and is delivered immediately prior to
discharge, in a hurried manner (Hesselink et al., 2012). These findings were reiterated in a more
194
recent qualitative study of patients and family members that took place in the United States
(Coleman & Roman, 2015). In addition, there is a lack of study on the relationship between the
use of health literate discharge practices and patient outcomes such as patient satisfaction and
patients’ ability to self-manage their care. There is a need to explore the experiences of patients
and family members with the use of these practices (Coleman et al., 2015).
A larger study, with a greater number of hospitals, would facilitate further understanding
of the construct “use of health literate discharge practices.” The survey might usefully be
administered to all medical and surgical inpatient units in Ontario, as well as to all units in
Canada. This would afford insights into the current use of health literate discharge practices in
Ontario and Canada, and highlight areas that need improvement. In addition, a larger sample size
would allow for further reliability testing of this survey instrument.
Factor analysis was done with the purpose of understanding the construct “use of health
literate discharge practices.” Another goal of factor analysis is to reduce the size of a data set.
The exploratory factor analysis indicated that there were items that could be removed from the
survey to improve the tool’s parsimony. This was the first time the survey was administered, and
it was not a goal of this study to reduce the survey items. However, reducing the number of
items, and creating a shortened version could help hospitals and policy makers to use the survey
by making it easier to measure current levels of health literate discharge practice use, as well as
changes in the level of use.
Furthermore, the exploratory factor analysis identified 6 factors that defined the construct
“use of health literate discharge practices in Ontario hospitals.” There is a need for future
research to use confirmatory factor analysis, to confirm the relationship between these factors
and this construct (Field, 2013).
195
In the third study, interviews of health care providers and managers were used to find out
about how hospitals in Ontario adopt and use health literate discharge practices. The interview
guide was focused on the four absorptive capacity metaroutines identified in the work of Lewin
et al (2011) of allocating resources, responding to environmental mandates, participative
decision making and scanning. Analysis of the qualitative interview data revealed seven
additional absorptive capacity metaroutines that were integral to the uptake of health literate
discharge practices. There is a need to meet with health care providers and managers to further
explore how the use of these metaroutines can facilitate the adoption and use of health literate
discharge practices
A 2015 study examined the implementation of Project RED in 10 hospitals in the United
States. A key factor of successful implementation was establishing accountability (Mitchell et
al., 2016). Interestingly, accountability was not directly addressed in any of the interviews, nor
did it emerge as a key theme. This may be because of a lack of clear accountability, and
accountability structures, in Ontario hospitals (Kraetschmer et al., 2014; Wachter, 2013). There
is a need to include a focus on accountability in follow-up qualitative work, as it plays a key role
in the uptake of evidence-based practice (Grol & Grimshaw, 2003, Fleiszer et al., 2015).
There has been little study of how health care organizations sustain evidence-based
practices (Innis et al., 2015). While sustainment and sustainability were not within the scopes of
these studies, there is a need for longitudinal research to examine how hospitals sustain their use
of health literate discharge practices.
In addition, a case study approach could be used to focus on how individual hospitals
adopt and use health literate discharge practices. This approach could include interviewing health
care providers, managers, administrative staff and community care providers, as well as patients
196
and families. This would give a more detailed understanding of how hospitals adopt and
implement these practices, and of the experiences of patients and families with these processes.
Organizational characteristics have been found to significantly influence the uptake of
innovation, as well as performance, in other study settings. While the literature in the
manufacturing and for-profit service sectors has shown a positive relationship between
organizational size and the adoption of innovations, the results are not as clear with hospitals.
The results of the little research done to date are mixed. The fourth study found a negative
relationship between hospital size and use of health literate discharge practices. This result
conflicts with some of the studies conducted in other sectors, however it was consistent with the
findings of Dupree et al. (2014) in their study of performance in hospitals in the United States.
The findings from the qualitative data analysis reported and discussed in Chapter 5
corroborate the findings from the regression analysis, where organizational size emerged as an
important factor relating to the use of health literate discharge practices, with smaller hospitals
reporting increased use. Participants from smaller hospitals described external relationships with
managers and providers in other hospitals. Of the ten hospitals, participants in only three
hospitals described being part of a community of practice, where they could meet and share ideas
with managers and health care providers from different hospitals in their region. Each of these
was a small community hospital located in a rural area.
In addition, although a number of interviewees from across the province spoke of the
need for improved IT resources to communicate with other hospitals, there was only one
interviewee, in a small rural hospital in the North, who described such a resource. This senior
manager described how her hospital was connected to five other hospitals through the same
system of electronic health records.
197
These findings highlight that there may be improved information continuity and sharing
of resources between smaller hospitals in rural settings. In addition, there appears to be increased
inter-facility cooperation between small community hospitals. These are areas for future
exploration. This could give insight into how hospitals develop and use relationships with other
hospitals and health care organizations.
7.7 Conclusion
This dissertation had several purposes. I set out to determine the essential indicators of
health literate discharge practices in Ontario hospitals, using the indicators from Project RED,
and a Delphi panel that consisted of health care providers, managers and researchers from across
Ontario, as well as researchers who have expertise in health literacy, hospital readmission and
hospital discharge from Canada and the United States. This set of indicators, contextualized to
Ontario, were then used to measure the use of health literate discharge practices in Ontario
hospitals. I was able to define the construct of “use of health literate discharge practices” through
exploratory factor analysis, and to establish reliability for the survey.
The administration of the survey in Ontario hospitals was followed with key informant
interviews in ten hospitals throughout the province that participated in the survey. Semi-
structured interviews explored the absorptive capacity metaroutines that hospitals in Ontario use
in the adoption and uptake of health literate discharge practices. I used the findings from these
interviews to generate propositions for a new conceptual framework that can be used to guide
future research.
I examined the relationship between organizational characteristics and the use of health
literate discharge practices. Contrary to expectations, a significant negative relationship was
198
found between hospital size and use of health literate discharge practices, with decreased size
being associated with an increased use of health literate discharge practices. In addition, a
significant interaction was found between size and rurality, which is not surprising, as small
hospitals are more likely to be located in rural locations. As described, participants in smaller
hospitals, in rural locations, described increased external relationships with other hospitals
through communities of practice and shared IT resources. This is an area for future research, as
these are strategies that may facilitate the uptake of health literate discharge practices in larger
hospitals.
To date, there are no published studies of how acute care hospitals in Ontario are
managing their discharge processes. The development of the organizational survey tool could be
used by researchers, hospitals and policy makers to measure and monitor the use of health
literate discharge practices in Ontario hospitals. This would help to identify areas of strength as
well as areas that need improvement.
The results of the qualitative study offer insights into how hospital managers and leaders
use absorptive capacity metaroutines to adopt and use health literate discharge practices. There
were varying levels of practice use across hospitals, and there is a need for hospitals in Ontario to
share practices that are working in their organization, so that they can be examined and tailored
for use in other hospitals.
The work done to date in these studies can be used to advance the adoption and use of
health literate discharge practices in acute care hospitals. This has the potential to improve
patient care, enhance patient satisfaction, reduce use of acute care resources, decrease
readmission and lower costs for Ontario’s health care system.
199
References
Aarons, G. A. (2006). Transformational and Transactional Leadership: Association With
Attitudes Toward Evidence-Based Practice. Psychiatric Services, 57(8), 1162-1169.
Accreditation Canada, Canadian Institute for Health Information, & Institute for Safe Medication
Practices Canada. (2012). Medication reconciliation in Canada: Raising the bar –
progress to date and the course ahead. Ottawa, ON: Accreditation Canada.
Adams, C. J., Stephens, K., Whiteman, K., Kersteen, H., & Katruska, J. (2014). Implementation
of the Re-Engineered Discharge (RED) toolkit to decrease all-cause readmission rates at
a rural community hospital. Quality Management in Healthcare, 23(3), 169-177.
Aiken, L. H., Sermeus, W., Van den Heede, K., Sloane, D. M., Busse, R., McKee, M., . . .
Kutney-Lee, A. (2012). Patient safety, satisfaction, and quality of hospital care: cross
sectional surveys of nurses and patients in 12 countries in Europe and the United States.
British Medical Journal, 344, e1717.
Al-Rashed, S. A., Wright, D. J., Roebuck, N., Sunter, W., & Chrystyn, H. (2002). The value of
inpatient pharmaceutical counselling to elderly patients prior to discharge. British
Journal of Clinical Pharmacology, 54(6), 657-664.
Angus, J., Hodnett, E., & O'Brien-Pallas, L. (2003). Implementing evidence-based nursing
practice: a tale of two intrapartum nursing units. Nursing Inquiry, 10(4), 218-228.
Anthony, D., Chetty, V., Kartha, A., McKenna, K., DePaoli, M. R., & Jack, B. (2005). Re-
engineering the hospital discharge: An example of a multifaceted process evaluation. In:
Henriksen K, Battles, JB, Marks ES, et al., (eds.). Advances in patient safety: From
research to implementation. (Volume 2: concepts and methodology). Rockville, MD:
Agency for Healthcare Research and Quality.
Arbaje, A. I., Kansagara, D. L., Salanitro, A. H., Englander, H. L., Kripalani, S., Jencks, S. F., &
Lindquist, L. A. (2014). Regardless of age: Incorporating principles from geriatric
medicine to improve care transitions for patients with complex needs. Journal of General
Internal Medicine, 29(6), 1-8.
Argote, L. (2012). Organizational learning: Creating, retaining and transferring knowledge.
New York: Springer.
Arksey, H., & O'Malley, L. (2005). Scoping studies: Towards a methodological framework.
International Journal of Social Research Methodology, 8(1), 19-32.
Arrindell, W. A., & Van der Ende, J. (1985). An empirical test of the utility of the observations-
to-variables ratio in factor and components analysis. Applied Psychological
Measurement, 9(2), 165-178.
Bailey, S. C., Fang, G., Annis, I. E., O'Conor, R., Paasche-Orlow, M. K., & Wolf, M. S. (2015).
Health literacy and 30-day hospital readmission after acute myocardial infarction. BMJ
Open, 5(6), e006975.
200
Baker, S. J. (2010). Bedside shift report improves patient safety and nurse accountability.
Journal of Emergency Nursing, 36(4), 355-358.
Baker, G. R. (2011). Enhacing the continuum of care: Report of the avoidable hospitalization
advisory panel. . Retrieved from
http://www.health.gov.on.ca/en/common/ministry/publications/reports/baker_2011/baker
_2011.pdf
Baker, D. W., Wolf, M. S., Feinglass, J., Thompson, J. A., Gazmararian, J. A., & Huang, J.
(2007). Health literacy and mortality among elderly persons. Archives of Internal
Medicine, 167(14), 1503-1509.
Barnsley, J., Lemieux-Charles, L., & McKinney, M. M. (1998). Integrating learning into
integrated delivery systems. Health Care Management Review, 23(1), 18-28.
Barr, V. J., Robinson, S., Marin-Link, B., Underhill, L., Dotts, A., Ravensdale, D., & Salivaras,
S. (2003). The expanded Chronic Care Model: An integration of concepts and strategies
from population health promotion and the Chronic Care Model. Healthcare Quarterly,
7(1), 73-82.
Berentson-Shaw, J., & Price, K. (2007). Facilitating effective health promotion practice in a
public health unit: Lessons from the field. Australian & New Zealand Journal of Public
Health, 31(1), 81-86.
Berkman, N. D., Sheridan, S. L., Donahue, K. E., Halpern, D. J., & Crotty, K. (2011). Low
health literacy and health outcomes: An updated systematic review. Annals of Internal
Medicine, 155(2), 97-107.
Berkman, N. D., Sheridan, S. L., Donahue, K. E., Halpern, D. J., Viera, A., Crotty, K., . . .
Harden, E. (2011). Health literacy interventions and outcomes: An updated systematic
review. Evidence Report/Technology Assessment No. 199. (Prepared by RTI
International University of North Carolina Evidence-Based Practice Center under
contract No. 290-2007-10056-I. AHRQ Publication Number 11-E006). Rockville, MD:
Agency for Healthcare Research and Quality.
Bernheim, S. M., Grady, J. N., Lin, Z., Wang, Y., Wang, Y., Savage, S. V., . . . Merrill, A. R.
(2010). National patterns of risk-standardized mortality and readmission for acute
myocardial infarction and heart failure update on publicly reported outcomes measures
based on the 2010 release. Circulation: Cardiovascular Quality and Outcomes, 3(5), 459-
467.
Berta, W., & Baker, R. (2004). Factors that impact the transfer and retention of best practices for
reducing error in hospitals. Health Care Management Review, 29(2), 90-97.
Berta, W., Ginsburg, L., Gilbart, E., Lemieux-Charles, L., & Davis, D. (2013). What, why, and
how care protocols are implemented in Ontario nursing homes. Canadian Journal of
Aging, 32(1), 73-85.
201
Berta, W., Teare, G. F., Gilbart, E., Ginsburg, L. S., Lemieux-Charles, L., Davis, D., & Rappolt,
S. (2005). The contingencies of organizational learning in long-term care: Factors that
affect innovation adoption. Health Care Management Review, 30(4), 282-292.
Berta, W., Teare, G. F., Gilbart, E., Ginsburg, L. S., Lemieux-Charles, L., Davis, D., & Rappolt,
S. (2010). Spanning the know-do gap: Understanding knowledge application and capacity
in long-term care homes. Social Science & Medicine, 70(9), 1326-1334.
Blair, J., Czaja, R. F., & Blair, E. A. (2014). Designing surveys: A guide to decisions and
procedures (Third ed.). Los Angeles: Sage.
Blegen, M. A., Goode, C. J., Spetz, J., Vaughn, T., & Park, S. H. (2011). Nurse staffing effects
on patient outcomes: Safety-net and non-safety-net hospitals. Medical Care, 49(4), 406-
414.
Bobay, K., Bahr, S. J., Weiss, M. E., Hughes, R., & Costa, L. (2015). Models of discharge care
in Magnet® hospitals. Journal of Nursing Administration, 45(10), 485-491.
Boulkedid, R., Abdoul, H., Loustau, M., Sibony, O., & Alberti, C. (2011). Using and reporting
the Delphi method for selecting healthcare quality indicators: a systematic review. PLoS
ONE, 6(6), e20476.
Brach, C., Dreyer, B. P., & Schillinger, D. (2014). Physicians’ roles in creating health literate
organizations: A call to action. Journal of General Internal Medicine, 29(2), 273-275.
Brach, C., Keller, D., Hernandez, L. M., Baur, C., Parker, R., Dreyer, B., . . . Schillinger, D.
(2012). Ten attributes of health literate health care organizations. Washinton DC:
National Academies Press.
Bradley, E. H., Curry, L., Horwitz, L. I., Sipsma, H., Wang, Y., Walsh, M. N., . . . Krumholz, H.
M. (2013). Hospital strategies associated with 30-day readmission rates for patients with
heart failure. Circulation: Cardiovascular Quality & Outcomes, 6(4), 444-450.
Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in
Psychology, 3(2), 77-101.
Brown, J. S., & Duguid, P. (1991). Organizational learning and communities-of-practice:
Toward a unified view of working, learning, and innovation. Organization Science, 2(1),
40-57.
Burns, K. E., Duffett, M., Kho, M. E., Meade, M. O., Adhikari, N. K., Sinuff, T., & Cook, D. J.
(2008). A guide for the design and conduct of self-administered surveys of clinicians. .
Canadian Medical Association Journal, 179, 245-252.
Cadmus, E., Van Wynen, E. A., Chamberlain, B., Steingall, P., Kilgallen, M. E., Holly, C., &
Gallagher-Ford, L. (2008). Nurses' skill level and access to evidence-based practice.
Journal of Nursing Administration, 38(11), 494-503.
202
Calvillo–King, L., Arnold, D., Eubank, K. J., Lo, M., Yunyongying, P., & Halm, E. A. (2013).
Impact of social factors on risk of readmission or mortality in pneumonia and heart
failure: systematic review. Journal of General Internal Medicine, 28(2), 269-282.
Camisón-Zornoza, C., Lapiedra-Alcamí, R., Segarra-Ciprés, M., & Boronat-Navarro, M. (2004).
A meta-analysis of innovation and organizational size. Organization Studies, 25(3), 331-
361.
Canadian Institute for Health Information. (2012). All-cause readmission to acute care and
return to the emergency department. Retrieved from
https://secure.cihi.ca/free_products/Readmission_to_acute_care_en.pdf
Canadian Institute for Health Information. (2015). Quick stats. Retrieved from
https://www.cihi.ca/en/quick-stats?xQSType=Pre-
formatted%252520Table&xTopic=Spending&pageNumber=1&resultCount=10&filterTy
peBy=1&filterTopicBy=14&autorefresh=1
Canadian Institute for Health Information. (2016). Your health system. Retrieved from
http://yourhealthsystem.cihi.ca/hsp/inbrief?lang=en
Carman, K. L., Dardess, P., Maurer, M., Sofaer, S., Adams, K., Bechtel, C., & Sweeney, J.
(2013). Patient and family engagement: A framework for understanding the elements and
developing interventions and policies. Health Affairs, 32(2), 223-231.
Cattell, R. (1978). The scientific use of factor analyses. New York: Plenum Press.
Centers for Medicare & Medicaid Services. (2016). Readmissions reduction program (HRRP).
Retrieved from https://www.cms.gov/medicare/medicare-fee-for-service-
payment/acuteinpatientpps/readmissions-reduction-program.html
Central East Local Health Integration Network. (2014). Central East Home at Last. Retrieved
from http://www.centraleastlhin.on.ca/page.aspx?id=10252.
Chow, S. K. Y., & Wong, F. K. (2010). Health‐related quality of life in patients undergoing
peritoneal dialysis: Effects of a nurse‐led case management programme. Journal of
Advanced Nursing, 66(8), 1780-1792.
Clark, B. R. (2011). Using law to fight a silent epidemic: The role of health literacy in health
care access, quality, and cost. Annals of Health Law, 20, 253-237.
Cloonan, P., Wood, J., & Riley, J. B. (2013). Reducing 30-day readmissions: Health literacy
strategies. Journal of Nursing Administration, 43(7/8), 382-387.
Cohen, J. (1992). A power primer. Psychological Bulletin, 112(1), 155.
Cohen, W. M. & Levinthal, D. A. (1990). Absorptive capacity: A new perspective on learning
and innovation. Administrative Science Quarterly, 35, 128-152.
203
Cohen, W. M., March, J. G., & Olsen, J. P. (1972). A garbage can model of organizational
choice. Administrative Science Quarterly, 17, 1-25.
Coleman, E. & Berenson, R. A. (2004). Lost in transition: Challenges and opportunities for
improving the quality of transitional care. Annals of Internal Medicine, 141(7), 533-536.
Coleman, E., Chugh, A., Williams, M. V., Grigsby, J., Glasheen, J. J., McKenzie, M., & Min, S.-
J. (2013). Understanding and execution of discharge instructions. American Journal of
Medical Quality, 28, 383-391.
Coleman, E., Parry, C., Chalmers, S., & Min, S. (2006). The care transitions intervention:
Results of a randomized controlled trial. Archives of Internal Medicine, 166, 1822 - 1828.
Coleman, E. & Roman, S. P. (2015). Family caregivers' experiences during transitions out of
hospital. Journal for Healthcare Quality, 37(1), 12-21.
Coleman, E, Roman, S. P., Hall, K. A., & Min, S.-j. (2015). Enhancing the Care Transitions
Intervention protocol to better address the needs of family caregivers. Journal for
Healthcare Quality, 37(1), 2-11.
Comrey, A. L., & Lee, H. B. (1992). A first course in factor analysis (2nd. ed.). Hillsdale, NJ:
Erlbaum.
Conway, J. M., & Huffcutt, A. I. (2003). A review and evaluation of exploratory factor analysis
practices in organizational research. Organizational Research Methods, 6(2), 147-168.
Cortina, J. M. (1993). What is coefficient alpha? An examination of theory and applications.
Journal of Applied Psychology, 78(1), 98.
Costantino, M. E., Frey, B., Hall, B., & Painter, P. (2013). The influence of a postdischarge
intervention on reducing hospital readmissions in a Medicare population. Population
Health Management, 16(5), 310-316.
Costello, A. B., & Osborne, J. W. (2005). Best practices in exploratory factor analysis: Four
recommendations for getting the most from your analysis. Practical Assessment,
Research & Evaluation, 10(7), 1-9.
Courtney, M. G. R., & Gordon, M. (2013). Determining the number of factors to retain in EFA:
Using the SPSS R-Menu v2. 0 to make more judicious estimations. Practical Assessment,
Research & Evaluation, 18(8), 1-14.
Creswell, J. W. (2008). Educational research: Planning, conducting and evaluating quantitative
and qualitative research. (Third ed.). Toronto: Pearson.
Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika,
16(3), 297-334.
Cua, Y. M., & Kripalani, S. (2008). Medication use in the transition from hospital to home.
Annals of the Academy of Medicine Singapore, 37(2), 136.
204
Cummings, G. G., Estabrooks, C. A., Midodzi, W. K., Wallin, L., & Hayduk, L. (2007).
Influence of organizational characteristics and context on research utilization. Nursing
Research, 56(4), S24-S39.
Cyert, R. M., & March, J. G. (1963). A behavioral theory of the firm. Englewood Cliffs, NJ:
Prentice-Hall.
Damanpour, F. (1992). Organizational size and innovation. Organization Studies, 13(3), 375-
402.
Damanpour, F., & Schneider, M. (2006). Phases of the adoption of innovation in organizations:
Effects of environment, organization and top managers. British Journal of Management,
17(3), 215-236.
Damanpour, F., Walker, R. M., & Avellaneda, C. N. (2009). Combinative effects of innovation
types and organizational performance: A longitudinal study of service organizations.
Journal of Management Studies, 46(4), 650-675.
Damschroder, L. J., Aron, D. C., Keith, R. E., Kirsh, S. R., Alexander, J. A., & Lowery, J. C.
(2009). Fostering implementation of health services research findings into practice: A
consolidated framework for advancing implementation science. Implementation Science,
4(1), 50.
Davis, D. A., & Taylor-Vaisey, A. (1997). Translating guidelines into practice: A systematic
review of theoretic concepts, practical experience and research evidence in the adoption
of clinical practice guidelines. Canadian Medical Association Journal, 157(4), 408-416.
Day, C. (2009). Engaging the nursing workforce: An evidence-based tool kit. Nursing
Administration Quarterly, 33(3), 238-244.
Deber, R. (2014). Concepts for the policy analyst. In R. Deber & C. Mah (Eds.), Case Studies in
Canadian Health Policy and Management (2nd edition ed., pp. 1-93). Toronto, Canada:
University of Toronto Press.
DeWalt, D. A., Callahan, L., Hawk, V., K, B., A, H., R, R., & Brach, C. (2010). Health literacy
universal precautions toolkit. (Prepared by North Carolina Network Consortium, The
Cecil G. Sheps Center for Health Services Research. The University of North Carolina at
Chapel Hill under Contract No. HHSA290200710014). AHRQ Publication No. 10-0046-
EF). Rockville, MD: Agency for Healthcare Research and Quality.
Dickens, C., Lambert, B. L., Cromwell, T., & Piano, M. R. (2013). Nurse overestimation of
patients' health literacy. Journal of Health Communication, 18(sup1), 62-69.
Dijkstra, R., Wensing, M., Thomas, R., Akkermans, R., Braspenning, J., Grimshaw, J., & Grol,
R. (2006). The relationship between organisational characteristics and the effects of
clinical guidelines on medical performance in hospitals, a meta-analysis. BMC Health
Services Research, 6(1), 53.
205
Dillman, D. A., Smyth, J. D., & Christian, L. M. (2009). Internet, mail and mixed-mode surveys:
The tailored design method. (3rd ed.). Hoboken, NJ: Wiley.
Dobbins, M., Ciliska, D., Cockerill, R., Barnsley, J., & DiCenso, A. (2002). A framework for the
dissemination and utilization of research for health-care policy and practice. Online
Journal of Knowledge Synthesis for Nursing, 9(7).
Doctoroff, L., & McNally, D. (2014). Handoffs in hospital medicine. Hospital Medicine Clinics,
3, e1-e14.
Douglas, S. L., Daly, B. J., Kelley, C. G., O’Toole, E., & Montenegro, H. (2007). Chronically
critically ill patients: Health-related quality of life and resource use after a disease
management intervention. American Journal of Critical Care, 16(5), 447-457.
Dudas, V., Bookwalter, T., Kerr, K. M., & Pantilat, S. Z. (2001). The impact of follow-up
telephone calls to patients after hospitalization. American Journal of Medicine, 111(9),
26-30.
Dufault, M. (2004). Testing a collaborative research utilization model to translate best practices
in pain management. Worldviews on Evidence-Based Nursing, 1 Suppl 1, S26-32.
Retrieved from
Dupree, J. M., Neimeyer, J., & McHugh, M. (2014). An advanced look at surgical performance
under Medicare's Hospital-Inpatient Value-Based Purchasing Program: Who is winning
and who is losing? Journal of the American College of Surgeons, 218(1), 1-7.
Eggink, R., Lenderink, A., Widdershoven, J., & Bernt, P. (2010). The effect of a clinical
pharmacist discharge service on medication discrepancies in patients with heart failure.
Pharmacy World & Science, 32(6), 759-766.
Ellen, M. E., Leon, G., Bouchard, G., Lavis, J. N., Ouimet, M., & Grimshaw, J. M. (2013). What
supports do health system organizations have in place to facilitate evidence-informed
decision-making? A qualitative study. Implementation Science, 8, 84.
Estabrooks, C. A., Squires, J. E., Cummings, G. G., Birdsell, J. M., & Norton, P. G. (2009).
Development and assessment of the Alberta Context Tool. BMC Health Services
Research, 9(1), 234.
Evans, D., Grunawalt, J., McClish, D., Wood, W., & Friese, C. R. (2012). Bedside shift-to-shift
nursing report: Implementation and outcomes. Medsurg Nursing, 21(5), 281.
Everett, L. Q., & Sitterding, M. C. (2011). Transformational leadership required to design and
sustain evidence-based practice: a system exemplar. Western Journal of Nursing
Research, 33(3), 398-426.
Everitt, B. (1975). Multivariate analysis: The need for data, and other problems. The British
Journal of Psychiatry, 126(3), 237-240.
206
Fabrigar, L. R., Wegener, D. T., MacCallum, R. C., & Strahan, E. J. (1999). Evaluating the use
of exploratory factor analysis in psychological research. Psychological Methods, 4(3),
272.
Fallis, B. A., Dhalla, I. A., Klemensberg, J., & Bell, C. M. (2013). Primary medication non-
adherence after discharge from a general internal medicine service. PLoS ONE, 8(5),
e61735.
Fennell, M. L. (1984). Synergy, influence, and information in the adoption of administrative
innovations. Academy of Management Journal, 27(1), 113-129.
Ferguson, T. D., & Howell, T. L. (2015). Bedside reporting: Protocols for improving patient
care. Nursing Clinics of North America, 50(4), 735-747.
Field, A. (2009). Discovering statistics using SPSS (Third ed.). Thousand Oaks, CA: Sage
Publications.
Field, A. (2013). Discovering statistics using IBM SPSS statistics. Thousand Oaks, CA: Sage.
Fineout-Overholt, E., Levin, R. F., & Melnyk, B. M. (2004). Strategies for advancing evidence-
based practice in clinical settings. Journal of the New York State Nurses Association,
35(2), 28-32.
Fink, A., Kosecoff, J., Chassin, M., & Brook, R. H. (1984). Consensus methods: Characteristics
and guidelines for use. American Journal of Public Health, 74(9), 979-983.
Floyd, F. J., & Widaman, K. F. (1995). Factor analysis in the development and refinement of
clinical assessment instruments. Psychological Assessment, 7(3), 286.
Forster, A. J., Clark, H. D., Menard, A., Dupuis, N., Chernish, R., Chandok, N., . . . van
Walraven, C. (2004). Adverse events among medical patients after discharge from
hospital. Canadian Medical Association Journal, 170(3), 345-349.
Forster, A. J., Murff, H., Peterson, J., Gandhi, T., & Bates, D. (2003). The incidence and severity
of adverse events affecting patients after discharge from the hospital. Annals of Internal
Medicine, 138, 161 - 167.
Freudenburg, W. (1988). Perceived risk, real risk: Social science and the art of probabilistic risk
assessment. Science, 242(4875), 44-49.
Gale, B. V., & Schaffer, M. A. (2009). Organizational readiness for evidence-based practice.
Journal of Nursing Administration, 39(2), 91-97.
Gardiner, P., Filippelli, A. C., Sadikova, E., White, L. F., & Jack, B. W. (2015). Medication and
dietary supplement interactions among a low-income, hospitalized patient population
who take cardiac medications. Evidence-Based Complementary and Alternative
Medicine, 2015. Retrieved from http://www.hindawi.com/journals/ecam/2015/429826/
207
Gaskin, C. J., & Happell, B. (2014). On exploratory factor analysis: A review of recent evidence,
an assessment of current practice, and recommendations for future use. International
Journal of Nursing Studies, 51(3), 511-521.
Geiger, S. W., & Cashen, L. H. (2002). A multidimensional examination of slack and its impact
on innovation. Journal of Managerial Issues, 14(1), 68-84.
Gifford, W., Davies, B., Edwards, N., Griffin, P., & Lybanon, V. (2007). Managerial leadership
for nurses' use of research evidence: an integrative review of the literature. Worldviews
on Evidence-Based Nursing, 4(3), 126-145.
Gorsuch, R. L. (1997). Exploratory factor analysis: Its role in item analysis. Journal of
Personality Assessment, 68(3), 532-560.
Government of Canada. (2014). Indigenous and Northern Affairs Canada: Ontario region.
Retrieved from https://www.aadnc-aandc.gc.ca/eng/1100100020284/1100100020288
Graham, R., Mancher, M., Wolman, D. M., Greenfield, S., & Steinberg, E. (2011). Clinical
practice guidelines we can trust. Washington DC: National Academies Press.
Green, S. B. (1991). How many subjects does it take to do a regression analysis. Multivariate
Behavioral Research, 26(3), 499-510.
Greenhalgh, T., Robert, G., Macfarlane, F., Bate, P., & Kyriakidou, O. (2004). Diffusion of
innovations in service organizations: Systematic review and recommendations. Milbank
Quarterly, 82(4), 581-629.
Greenwald, J., Denham, C. R., & Jack, B. W. (2007). The hospital discharge: A review of a high
risk care transition with highlights of a reengineered discharge process. Journal of
Patient Safety, 3(2), 97-106.
Greenwald, J. & Jack, B. W. (2009). Preventing the preventable: Reducing rehospitalizations
through coordinated, patient-centered discharge processes. Professional Case
Management, 14(3), 135-140.
Grimshaw, J., & Eccles, M. P. (2004). Is evidence-based implementation of evidence-based care
possible? Medical Journal of Australia, 180(6 Suppl), S50-51.
Grol, R., Bosch, M. C., Hulscher, M. E., Eccles, M. P., & Wensing, M. (2007). Planning and
studying improvement in patient care: The use of theoretical perspectives. Milbank
Quarterly, 85(1), 93-138.
Grol, R., & Grimshaw, J. (2003). From best evidence to best practice: Effective implementation
of change in patients' care. The Lancet, 362(9391), 1225-1230.
Hair, J. F., Black, W. C., Babin, B. J., & Anderson, R. E. (2010). Multivariate data analysis (7th
ed.). Upper Saddle River, NJ: Prentice Hall.
208
Hansen, L. O., Young, R. S., Hinami, K., Leung, A., & Williams, M. V. (2011). Interventions to
reduce 30-day rehospitalization: A systematic review. Annals of Internal Medicine,
155(8), 520-528.
Harper, W., Cook, S., & Makoul, G. (2007). Teaching medical students about health literacy: 2
Chicago initiatives. American Journal of Health Behavior, 31(Supplement 1), S111-
S114.
Harrison, J. D., Auerbach, A. D., Quinn, K., Kynoch, E., & Mourad, M. (2014). Assessing the
impact of nurse post-discharge telephone calls on 30-day hospital readmission rates.
Journal of General Internal Medicine, 29(11), 1519-1525.
Hasson, F., Keeney, S., & McKenna, H. (2000). Research guidelines for the Delphi survey
technique. Journal of Advanced Nursing, 32(4), 1008-1015.
Hayes, A. F. (2013). Introduction to mediation, moderation and conditional process analysis.
New York: The Guilford Press.
Health Quality Ontario. (2013a). An analysis for improvement, 2013-2014. Retrieved from
http://www.hqontario.ca/portals/0/documents/qi/qip-analysis-hospitals-2013-en.pdf
Health Quality Ontario. (2013b). Health literacy and its effect on health outcomes. Paper
presented at the Health Quality Transformation 2013, Toronto, Canada.
http://www.hqontario/portals/0/Documents/home/hqt2013-session-10-en.pdf
Health Quality Ontario. (2015a). 30-day redmission. Retrieved from
http://www.hqontario.ca/Public-Reporting/Primary-Care/Quality-Indicators/30-day-
readmission
Health Quality Ontario. (2015b). About us. Retrieved from http://www.hqontario.ca/About-
Us/Our-Mandate
Heidenreich, P. A., Zhao, X., Hernandez, A. F., Yancy, C. W., & Fonarow, G. C. (2012). Patient
and hospital characteristics associated with traditional measures of inpatient quality of
care for patients with heart failure. American Heart Journal, 163(2), 239-245.e233.
Hesselink, G., Flink, M., Olsson, M., Barach, P., Dudzik-Urbaniak, E., Orrego, C., . . .
Schoonhoven, L. (2012). Are patients discharged with care? A qualitative study of
perceptions and experiences of patients, family members and care providers. BMJ
Quality & Safety, 21(Suppl 1), i39-i49.
Hesselink, G., Vernooij-Dassen, M., Pijnenborg, L., Barach, P., Gademan, P., Dudzik-Urbaniak,
E., . . . Johnson, J. K. (2013). Organizational culture: an important context for addressing
and improving hospital to community patient discharge. Medical Care, 51(1), 90-98.
Hikmet, N., Bhattacherjee, A., Menachemi, N., Kayhan, V. O., & Brooks, R. G. (2008). The role
of organizational factors in the adoption of healthcare information technology in Florida
hospitals. Health Care Management Science, 11(1), 1-9.
209
Hinkin, T. R. (1998). A brief tutorial on the development of measures for use in survey
questionnaires. Organizational Research Methods, 1(1), 104-121.
Hoomans, T., Severens, J. L., Evers, S. M., & Ament, A. J. (2009). Value for money in changing
clinical practice: Should decisions about guidelines and implementation strategies be
made sequentially or simultaneously? Medical Decision Making, 29(2), 207-216.
Horbar, J. D., Plsek, P. E., & Leahy, K. (2003). NIC/Q 2000: Establishing habits for
improvement in neonatal intensive care units. Pediatrics, 111(4), e397-410.
Howie-Esquivel, J., Carroll, M., Brinker, E., Kao, H., Pantilat, S., Rago, K., & De Marco, T.
(2015). A strategy to reduce heart failure readmissions and inpatient costs. Cardiology
Research, 6(1), 201-208.
Innis, J., & Berta, W. (2016). Routines for change: how managers can use absorptive capacity to
adopt and implement evidence-based practice. Journal of Nursing Management.
doi:10.1111/jonm.12368
Innis, J., Dryden-Palmer, K., Perreira, T., & Berta, W. (2015). How do health care organizations
take on best practices? A scoping literature review. International Journal of Evidence-
Based Healthcare, 13, 254-272.
Institute for Healthcare Improvement. (2014). Evidence-based care bundles. Retrieved from
http://www.ihi.org/Topics/Bundles/Pages/default.aspx
Institute of Medicine. (2009). Measures of health literacy: Workshop summary. Washington DC:
National Academies Press.
Institute of Medicine. (2012). How can health care organizations become more health literate? .
Retrieved from Washington DC.:
Jack, B. W., Chetty, V., Anthony, D., Greenwald, J., Sanchez, G., Johnson, A., . . . Culpepper, L.
(2009). A reengineered hospital discharge program to decrease rehospitalization: a
randomized trial. Annals of Internal Medicine, 150, 178 - 187.
Jack, B. W., Chetty, V. K., Anthony, D., Greenwald, J. L., Sanchez, G. M., Johnson, A. E., . . .
Manasseh, C. (2009). A Reengineered Hospital Discharge Program to Decrease
RehospitalizationA Randomized Trial. Annals of Internal Medicine, 150(3), 178-187.
Jack, B. W., Greenwald, J., Forsythe, S., O'Donnell, J., Johnson, A., Schipelliti, L., . . . Chetty,
V. K. (2008). Developing the tools to administer a comprehensive hospital discharge
program: the ReEngineered Discharge (RED) program. In K. Henriksen, J. B. Battles, &
M. A. Keyes (Eds.), Advances in patient safety: New directions and alternative
approaches. (Vol. 3). Rockville, MD: Agency for Healthcare Resarch and Quality.
Jack, B. W., Paasche-Orlow, M., Mitchell, S. A., Forsythe, S. R., Martin, J., & Brach, C. (2013).
An overview of the Re-Engineered Discharge (RED) toolkit. Rockville, MD: Agnecy for
Healthcare Research and Quality.
210
Jackson, & Gillis, A. (2003). Qualitative research strategies. In W. Jackson (Ed.), Methods:
Doing social research. (pp. 134-189). Toronto: Pearson.
Jackson, Vellucci, J., Johnson, P., & Kilbride, H. W. (2003). Evidence-based approach to change
in clinical practice: Introduction of expanded nasal continuous positive airway pressure
use in an intensive care nursery. Pediatrics, 111(4), e542-547.
Jaskyte, K. (2013). Does size really matter? Organizational size and innovations in nonprofit
organizations. Nonprofit Management and Leadership, 24(2), 229-247.
Jencks, S., Williams, M., & Coleman, E. (2009). Rehospitalizations among patients in the
Medicare fee-for-services program. New England Journal of Medicine, 360, 1418 - 1428.
Jiménez-Jiménez, D., & Sanz-Valle, R. (2011). Innovation, organizational learning, and
performance. Journal of Business Research, 64(4), 408-417.
Jupp, V. (Ed.) (2006). The Sage dictionary of social research methods. Thousand Oaks, CA:
Sage.
Karliner, L. S., Auerbach, A., Nápoles, A., Schillinger, D., Nickleach, D., & Pérez-Stable, E. J.
(2012). Language barriers and understanding of hospital discharge instructions. Medical
Care, 50(4), 283.
Kass, R. A., & Tinsley, H. E. (1979). Factor analysis. Journal of Leisure Research, 11(2), 120-
138.
Keenan, P. S., Normand, S.-L. T., Lin, Z., Drye, E. E., Bhat, K. R., Ross, J. S., . . . Epstein, A. J.
(2008). An administrative claims measure suitable for profiling hospital performance on
the basis of 30-day all-cause readmission rates among patients with heart failure.
Circulation: Cardiovascular Quality and Outcomes, 1(1), 29-37.
Keeney, S., Hasson, F., & McKenna, H. (2006). Consulting the oracle: Ten lessons from using
the Delphi technique in nursing research. Journal of Advanced Nursing, 53(2), 205-212.
Kimber, M., Barwick, M., & Fearing, G. (2012). Becoming an evidence-based service provider:
Staff perceptions and experiences of organizational change. The Journal of Behavioral
Health Services & Research, 39(3), 314-332.
Kline, P. (2014). An easy guide to factor analysis. New York: Routledge.
Kocher, R. P., & Adashi, E. Y. (2011). Hospital readmissions and the affordable care act: Paying
for coordinated quality care. JAMA, 306(16), 1794-1795.
Koelling, T. M., Johnson, M. L., Cody, R. J., & Aaronson, K. D. (2005). Discharge education
improves clinical outcomes in patients with chronic heart failure. Circulation, 111(2),
179-185.
Koh, Baur, C., Brach, C., Harris, L. M., & Rowden, J. N. (2013). Toward a systems approach to
health literacy research. Journal of Health Communication, 18(1), 1-5.
211
Koh, Brach, C., Harris, L. M., & Parchman, M. L. (2013). A proposed ‘Health Literate Care
Model’ would constitute a systems approach to improving patients’ engagement In care.
Health Affairs, 32(2), 357-367. doi:10.1377/hlthaff.2012.1205
Kowalski, C., Lee, S.-Y. D., Schmidt, A., Wesselmann, S., Wirtz, M. A., Pfaff, H., &
Ernstmann, N. (2015). The health literate health care organization 10 item questionnaire
(HLHO-10): Development and validation. BMC Health Services Research, 15(1), 47.
Kraetschmer, N., Jass, J., Woodman, C., Koo, I., Kromm, S., & Deber, R. (2014). Hospitals'
internal accountability. Healthcare Policy, 10(Special issue), 36-44.
Kralj, B. (2009). Measuring rurality – RIO2008_BASIC: Methodology and results. Retrieved
from https://www.oma.org/Resources/Documents/2008RIO-FullTechnicalPaper.pdf
Kripalani, S., Jackson, A. T., Schnipper, J. L., & Coleman, E. A. (2007). Promoting effective
transitions of care at hospital discharge: a review of key issues for hospitalists. Journal of
Hospital Medicine, 2(5), 314-323.
Kripalani, S., Jacobson, T. A., Mugalla, I. C., Cawthon, C. R., Niesner, K. J., & Vaccarino, V.
(2010). Health literacy and the quality of physician‐patient communication during
hospitalization. Journal of Hospital Medicine, 5(5), 269-275.
Kripalani, S., LeFevre, F., Phillips, C. O., Williams, M. V., Basaviah, P., & Baker, D. W. (2007).
Deficits in communication and information transfer between hospital-based and primary
care physicians. JAMA, 297(8), 831-841.
Kripalani, S., Wallston, K. A., Cavanaugh, K., Osborn, C. Y., Mulvaney, S., McDougald Scott,
A., & Rothman, R. (2014). Measures to assess a health-literate organization.
Washington DC: National Academies Press.
Krumholz, H., Merrill, A., Schone, E., Schreiner, G., Chen, J., Bradley, E., . . . Drye, E. (2009).
Patterns of hospital performance in acute myocardial infarction and heart failure 30-day
mortality and readmission. Circulation: Cardiovascular Quality & Outcomes, 2(5), 407-
413.
Lane, N., Pitzul, K., Laporte, A., Nauenberg, E., & Wodchis, W. (2013, May 2013). Cost-
effectiveness of a nurse practitioner-led care transition intervention. Paper presented at
the Canadian Association of Health Services Research and Policy, Vancouver, BC.
Lau, F., Doze, S., Vincent, D., Wilson, D., Noseworthy, T., Hayward, R., & Penn, A. (1999).
Patterns of improvisation for evidence-based practice in clinical settings. Information
Technology & People, 12(3), 287-303.
Lawson, M. B. (2001). In praise of slack: Time is of the essence. Academy of Management
Executive, 15, 125-135.
Leape, L. L., Rogers, G., Hanna, D., Griswold, P., Federico, F., Fenn, C. A., . . . Clarridge, B. R.
(2006). Developing and implementing new safe practices: Voluntary adoption through
statewide collaboratives. Quality & Safety in Health Care, 15(4), 289-295.
212
Lehrman, W., Elliott, M., Goldstein, E., Beckett, M., Klein, D., & Giordano, L. (2010).
Characteristics of hospitals demonstrating superior performance in patient experience and
clinical process measures of care. Medical Care Research and Review, 67(1), 38-55.
Leibenstein, H. (1969). Organizational or frictional equilibria, X-efficiency, and the rate of
innovation. The Quarterly Journal of Economics, 600-623.
Levinson, D. (2008). Adverse events in hospitals: Overview of key issues. Retrieved from
Washington, DC: http://oig.hhs.gov/oei/reports/oei-06-07-00470.pdf
Lewin, A. Y., Massini, S., & Peeters, C. (2011). Microfoundations of internal and external
absorptive capacity routines. Organization Science, 22(1), 81-98.
Lindholm, M., Hargraves, J. L., Ferguson, W. J., & Reed, G. (2012). Professional language
interpretation and inpatient length of stay and readmission rates. Journal of General
Internal Medicine, 27(10), 1294-1299.
Lindquist, L. A., Go, L., Fleisher, J., Jain, N., Friesema, E., & Baker, D. W. (2011). Relationship
of health literacy to intentional and unintentional non-adherence of hospital discharge
medications. Journal of General Internal Medicine, 27(2), 173-178.
Longtin, Y., Sax, H., Leape, L. L., Sheridan, S. E., Donaldson, L., & Pittet, D. (2010). Patient
participation: Current knowledge and applicability to patient safety. Mayo Clinic
Proceedings, 85(1), 53-62.
Lukas, C. V., Holmes, S. K., Cohen, A. B., Restuccia, J., Cramer, I. E., Shwartz, M., & Charns,
M. P. (2007). Transformational change in health care systems: An organizational model.
Health Care Management Review, 32(4), 309-320.
Lutfiyya, M. N., Bhat, D. K., Gandhi, S. R., Nguyen, C., Weidenbacher-Hoper, V. L., & Lipsky,
M. S. (2007). A comparison of quality of care indicators in urban acute care hospitals and
rural critical access hospitals in the United States. International Journal for Quality in
Health Care, 19(3), 141-149.
Luxford, K., Safran, D. G., & Delbanco, T. (2011). Promoting patient-centered care: A
qualitative study of facilitators and barriers in healthcare organizations with a reputation
for improving the patient experience. International Journal for Quality in Health Care,
23(5), 510-515.
MacCallum, R. C., Widaman, K. F., Zhang, S., & Hong, S. (1999). Sample size in factor
analysis. Psychological methods, 4(1), 84.
Makaryus, A. N., & Friedman, E. A. (2005). Patients' understanding of their treatment plans and
diagnosis at discharge. Mayo Clinic Proceedings, 80(8), 991-994.
March, J. G. (1991). Exploration and exploitation in organizational learning. Organization
Science, 2(1), 71-87.
213
Markley, J., Andow, V., Sabharwal, K., Wang, Z., Fennell, E., & Dusek, R. (2013). A project to
reengineer discharges reduces 30-day readmission rates. American Journal of Nursing,
113(7), 55-64.
Marvanova, M., Roumie, C. L., Eden, S. K., Cawthon, C., Schnipper, J. L., & Kripalani, S.
(2011). Health literacy and medication understanding among hospitalized adults. Journal
of Hospital Medicine, 6(9), 488-493.
McGillis Hall, L., Doran, D., Baker, G. R., Pink, G. H., Sidani, S., O’Brien-Pallas, L., & Donner,
G. J. (2003). Nurse staffing models as predictors of patient outcomes. Medical Care,
41(9), 1096-1109.
McHugh, M. D., Kelly, L. A., Smith, H. L., Wu, E. S., Vanak, J. M., & Aiken, L. H. (2013).
Lower mortality in magnet hospitals. Medical Care, 51(5), 382-388.
McNaughton, C. D., Cawthon, C., Kripalani, S., Liu, D., Storrow, A. B., & Roumie, C. L.
(2015). Health literacy and mortality: A cohort study of patients hospitalized for acute
heart failure. Journal of the American Heart Association, 4, e001799.
Meyers, Gaust, G., & Guarino, A. J. (2013). Applied multivariate research: Design and
interpretation. Thousand Oaks, CA: Sage.
Meyers, Salanitro, A., Wallston, K. A., Cawthon, C., Vasilevskis, E. E., Goggins, K. M., . . .
Donato, K. M. (2014). Determinants of health after hospital discharge: Rationale and
design of the Vanderbilt Inpatient Cohort Study (VICS). BMC Health Services Research,
14(1), 10.
Ministry of Health and Long-Term Care. (2006). A guide to better physician documentation:
Physician documentation expert panel. Retrieved from
http://www.health.gov.on.ca/transformation/providers/information/pdf/guide_bpd.pdf
Ministry of Health and Long-Term Care. (2008, January 14, 2014). MedsCheck. Retrieved from
http://www.health.gov.on.ca/en/pro/programs/drugs/medscheck/medscheck_original.aspx
Ministry of Health and Long-Term Care. (2011). Rural and northern health care
framework/plan: Final report. Retrieved from
http://health.gov.on.ca/en/public/programs/ruralnorthern/docs/report_rural_northern_EN.
Ministry of Health and Long-Term Care. (2013a). Classification of hospitals: Group A hospitals.
Retrieved from
http://www.health.gov.on.ca/en/common/system/services/hosp/hospcode.aspx
Ministry of Health and Long-Term Care. (2013b). Classification of hospitals: Group B hospitals.
Retrieved from
http://www.health.gov.on.ca/en/common/system/services/hosp/group_b.aspx
214
Ministry of Health and Long-Term Care. (2013c). Classification of hospitals: Group C hospitals.
Retrieved from
http://www.health.gov.on.ca/en/common/system/services/hosp/group_c.aspx
Ministry of Health and Long-Term Care. (2013d). Excellent care for all act updates - Quality
Committees: Best practice resources to support quality committee roles and
responsibilities. . Retrieved from
http://www.health.gov.on.ca/en/pro/programs/ecfa/legislation/qualitycommittee/bp_resou
rces.aspx
Ministry of Health and Long-Term Care. (2013e). Excellent care for all: Real change in action –
Emergency Department Process Improvement. Retrieved from
http://www.health.gov.on.ca/en/pro/programs/ecfa/quality/research/cst_ed_process.aspx
Ministry of Health and Long-Term Care. (2014a). Excellent care for all: Home first – putting
patients at the centre of their health care. Retrieved from
http://www.health.gov.on.ca/en/pro/programs/ecfa/action/community/com_homefirst.asp
x
Ministry of Health and Long-Term Care. (2014b). Home at Last (HAL): Back to the basics.
Retrieved from
http://www.health.gov.on.ca/en/pro/programs/ecfa/action/acute/pri_hal.aspx
Ministry of Health and Long-Term Care. (2014c). Indicator technical specifications: Quality
Improvement Plan 2015/16. Retrieved from
http://www.health.gov.on.ca/en/pro/programs/ecfa/legislation/qualityimprove/qip_tech.p
df
Ministry of Health and Long-Term Care. (2015a). Health system funding reform: Quality-Based
Procedures. Retrieved from
http://www.health.gov.on.ca/en/pro/programs/ecfa/funding/hs_funding_qbp.aspx
Ministry of Health and Long-Term Care. (2015b). Patients first: Action plan for health care.
Toronto, ON: Ministry of Health and Long-Term Care Retrieved from
http://www.health.gov.on.ca/en/ms/ecfa/healthy_change/docs/rep_patientsfirst.pdf.
Ministry of Health and Long-Term Care. (2015c). Transforming Ontario’s health care system.
Retrieved from
http://www.health.gov.on.ca/en/pro/programs/transformation/community.aspx
Mistiaen, P., & Poot, E. (2006). Telephone follow-up, initiated by a hospital-based health
professional, for postdischarge problems in patients discharged from hospital to home.
Cochrane Database Systematic Review, 4(4).
Mitchell, S. E., Martin, J., Holmes, S., van Deusen, L. C., Cancino, R., Paasche-Orlow, M., . . .
Jack, B. (2016). How hospitals reengineer their discharge processes to reduce
readmissions. Journal for Healthcare Quality, 38(2), 116-126.
215
Mitchell, S. E., Sadikova, E., Jack, B. W., & Paasche-Orlow, M. K. (2012). Health literacy and
30-day postdischarge hospital utilization. Journal of Health Communication, 17(sup3),
325-338.
Moser, D. K., Robinson, S., Biddle, M. J., Pelter, M. M., Nesbitt, T., Southard, J., . . . Dracup, K.
(2015). Health literacy predicts morbidity and mortality in rural patients with heart
failure. Journal of Cardiac Failure, 21, 612-618.
Mudge, A. M., Shakhovskoy, R., & Karrasch, A. (2013). Quality of transitions in older medical
patients with frequent readmissions: Opportunities for improvement. European Journal
of Internal Medicine, 24(8), 779-783.
Mullen, P. M. (2003). Delphi: myths and reality. Journal of Health Organization and
Management, 17(1), 37-52.
Murray, T., Hagey, J., Willms, D., Shillington, R., & Desjardins, R. (2008). Health literacy in
Canada: A healthy understanding Ottawa, ON: Canadian Council on Learning.
Naylor, M., Brooten, D., Jones, R., Lavizzo-Mourey, R., Mezey, M., & Pauly, M. (1994).
Comprehensive discharge planning for the hospitalized elderlya randomized clinical trial.
Annals of Internal Medicine, 120(12), 999-1006.
Naylor, M., Brooten, D. A., Campbell, R. L., Maislin, G., McCauley, K. M., & Schwartz, J. S.
(2004). Transitional care of older adults hospitalized with heart failure: a randomized,
controlled trial. Journal of the American Geriatrics Society, 52(5), 675-684.
Naylor, M., Stephens, C., Bowles, K. H., & Bixby, M. B. (2005). Cognitively impaired older
adults, from hospital to home: An exploratory study of these patients and their caregivers.
American Journal of Nursing, 105(2), 52-61.
Nelson, R. R., & Rudestam, K. E. (1999). Your statistical consultant: Answers to your data
analysis quesitons. London: Sage.
Nielsen-Bohlman, L., Panzer, A. M., & Kindig, D. A. (2004). Health literacy: A prescription to
end confusion. Washington DC: National Academies Press.
Nielsen, G. A., Rutherford, P., & Taylor, J. (2009). How-to-guide: Creating an ideal transition
home. Retrieved from http://ah.cms-
plus.com/files/IHI_How_to_Guide_Creating_an_Ideal_Transition_Home.pdf
Nohria, N., & Gulati, R. (1996). Is slack good or bad for innovation? Academy of Management
Journal, 39(5), 1245-1264.
Nonaka, I. (1994). A dynamic theory of organizational knowledge creation. Organization
Science, 5, 14-37.
Nosbusch, J. M., Weiss, M. E., & Bobay, K. L. (2011). An integrated review of the literature on
challenges confronting the acute care staff nurse in discharge planning. Journal of
Clinical Nursing, 20(5‐6), 754-774.
216
Nunnally, J. (1978). Psychometric methods. New York: McGraw-Hill.
Oates, J., Weston, W. W., & Jordan, J. (2000). The impact of patient-centered care on outcomes.
Family Practice, 49, 796-804.
OECD. (2014). Geographic variations in health care: What do we know and what can be done to
improve health system performance? Retrieved from
http://dx.doi.org/10.1787/978264216594-en
Office of the Auditor General of Ontario. (2015). Annual Report. Retrieved from
http://www.auditor.on.ca/en/reports_en/en15/3.08en15.pdf
Okafor, M. C., & Thomas, J., 3rd. (2008). Presence of innovation adoption-facilitating elements
in hospitals, and relationship to implementation of clinical guidelines. Annals of
Pharmacotherapy, 42(3), 354-360.
Olade, R. A. (2004). Strategic collaborative model for evidence-based nursing practice.
Worldviews on Evidence-Based Nursing, 1(1), 60-68.
Ong, M. S., Magrabi, F., Jones, G., & Coiera, E. (2012). Last orders: Follow-up of tests ordered
on the day of hospital discharge. Archives of Internal Medicine, 172, 1347-1348.
Ontario Hospital Association. (2013). Regional affairs. Retrieved from
https://www.oha.com/AboutUs/RegionalAffairs/Pages/RegionalAffairs.aspx
Paasche-Orlow, Riekert, K. A., Bilderback, A., Chanmugam, A., Hill, P., Rand, C. S., &
Krishnan, J. A. (2005). Tailored education may reduce health literacy disparities in
asthma self-management. American Journal of Respiratory and Critical Care Medicine,
172, 980-986.
Paasche-Orlow, & Wolf, M. S. (2008). Evidence does not support clinical screening of literacy.
Journal of General Internal Medicine, 23(1), 100-102.
Panzano, P., & Roth, D. (2006). The decision to adopt evidence-based and other innovative
mental health practices: Risky business? Psychiatric Services, 57(8), 1153-1161.
Parsons, H. M., Begun, J. W., McGovern, P. M., Tuttle, T. M., Kuntz, K. M., & Virnig, B. A.
(2013). Hospital characteristics associated with maintenance or improvement of
guideline-recommended lymph node evaluation for colon cancer. Medical Care, 51(1),
60-67.
Patton, M. Q. (2002). Qualitative research and evaluation methods. Thousand Oaks, CA: Sage
Publications.
Peeters, C., Massini, S., & Lewin, A. (2014). Sources of variation in the efficiency of adopting
management innovation: The role of absorptive capacity routines, managerial attention
and organizational legitimacy. Organization Studies, 35(9), 1343-1371.
217
Peter, D., Robinson, P., Jordan, M., Lawrence, S., Casey, K., & Salas-Lopez, D. (2015).
Reducing readmissions using teach-back: enhancing patient and family education.
Journal of Nursing Administration, 45(1), 35-42.
Phatak, A., Prusi, R., Ward, B., Hansen, L. O., Williams, M. V., Vetter, E., . . . Postelnick, M.
(2016). Impact of pharmacist involvement in the transitional care of high‐risk patients
through medication reconciliation, medication education, and postdischarge call‐backs
(IPITCH Study). Journal of Hospital Medicine, 11(1), 39-44.
Plath, D. (2013). Organizational processes supporting evidence-based practice. Administration in
Social Work, 37(2), 171-188.
Ploeg, J., Davies, B., Edwards, N., Gifford, W., & Miller, P. E. (2007). Factors influencing best-
practice guideline implementation: lessons learned from administrators, nursing staff, and
project leaders. Worldviews on Evidence-Based Nursing, 4(4), 210-219.
Powell, C. (2003). The Delphi technique: Myths and realities. Journal of Advanced Nursing,
41(4), 376-382.
Registered Nurses’ Association of Ontario. (2015). Best practice spotlight organizations
Retrieved from http://rnao.ca/bpg/bpso
Robert Wood Johnson Foundation. (2013). The revolving door: A report on U.S. hospital
readmissions. Retrieved from
www.rwjf.org/content/dam/farm/reports/reports/2013/rwjf404178
Rootman, I., & Gordon-El-Bihbety, D. (2008). A vision for a health literate Canada. Ottawa,
ON: Canadian Public Health Association.
Rose, A. J., Petrakis, B. A., Callahan, P., Mambourg, S., Patel, D., Hylek, E. M., & Bokhour, B.
G. (2012). Organizational characteristics of high- and low-performing anticoagulation
clinics in the veterans health administration. Health Services Research, 47(4), 1541-1560.
Rowe, G., & Wright, G. (1999). The Delphi technique as a forecasting tool: Issues and analysis.
International Journal of Forecasting, 15(4), 353-375.
Roy, Poon, E. G., Karson, A. S., Ladak-Merchant, Z., Johnson, R. E., Maviglia, S. M., &
Ghandi, T. K. (2005). Patient safety concerns arising from test results that return after
hospital dischage. Annals of Internal Medicine, 143, 121-128.
Ruffolo, M. C., Savas, S. A., Neal, D., Capobianco, J., & Reynolds, K. (2009). The challenges of
implementing an evidence-based practice to meet consumer and family needs in a
managed behavioral health care environment. Social Work in Mental Health, 7(1-3), 30-
41.
Runyon, R. P., Coleman, K. A., & Pittenger, D. J. (2000). Fundamentals of behavioral statistics
(9th ed.). New York, NY: McGraw-Hill.
218
Sandelowski, M. (1995). Qualitative analysis: What it is and how to begin. Research in Nursing
& Health, 18(4), 371-375.
Saufl, N. M., & Fieldus, M. H. (2003). Accreditation: A "voluntary" regulatory requirement.
Journal of PeriAnesthesia Nursing, 18(3), 152-159.
Schein, E. H. (2010). Organizational culture and leadership (4th ed.). Hoboken, NJ: John Wiley
& Sons.
Schillinger, Piette, J., Grumbach, K., Wang, F., Wilson, C., Daher, C., . . . Bindman, A. (2003).
Closing the loop. Physician communication with diabetic patients who have low health
literacy. Archives of Internal Medicine, 163, 83 - 90.
Schnipper, J., Kirwin, J., Cotugno, M., Wahlstrom, S., Brown, B., Tarvin, E., . . . Bates, D.
(2006). Role of pharmacist counseling in preventing adverse drug events after
hospitalization. Archives of Internal Medicine, 166, 565 - 571.
Sharp, N. D., Pineros, S. L., Hsu, C., Starks, H., & Sales, A. E. (2004). A qualitative study to
identify barriers and facilitators to implementation of pilot interventions in the Veterans
Health Administration (VHA) Northwest Network. Worldviews on Evidence-Based
Nursing, 1(2), 129-139.
Shekelle, P. G. (2013). Nurse–patient ratios as a patient safety strategy: A systematic review.
Annals of Internal Medicine, 158(5, Part 2), 404-409.
Shepperd, S., Lannin, N. A., Clemson, L. M., McCluskey, A., Cameron, I. D., & Barras, S. L.
(2013). Discharge planning from hospital to home. Cochrane Database Systematic
Reviews, 1.
Shoeb, M., Merel, S. E., Jackson, M. B., & Anawalt, B. D. (2012). “Can we just stop and talk?”
Patients value verbal communication about discharge care plans. Journal of Hospital
Medicine, 7(6), 504-507.
Shojania, K. G., Duncan, B. W., McDonald, K. M., Wachter, R. M., & Markowitz, A. J. (2001).
Making health care safer: A critical analysis of patient safety practices. Rockville, MD:
Agency for Healthcare Research and Quality
Singh, S., Lin, Y. L., Kuo, Y., Nattinger, A. B., & Goodwin, J. S. (2014). Variation in the risk of
readmission among hospitals: The relative contribution of patient, hospital and inpatient
provider characteristics. Journal of General Internal Medicine, 29, 572-578.
Skulmoski, G. J., Hartman, F. T., & Krahn, J. (2007). The Delphi method for graduate research.
Journal of Information Technology Education, 6, 1.
Smith, M., Saunders, R., Stuckhardt, L., & McGinnis, J. M. (2013). Best care at lower cost: The
path to continuously learning health care in America. Washington DC: National
Academies Press.
219
Soong, C., Daub, S., Lee, J., Majewski, C., Musing, E., Nord, P., . . . Bell, C. M. (2013).
Development of a checklist of safe discharge practices for hospital patients. Journal of
Hospital Medicine, 8(8), 444-449.
St. Michael's Hospital. (2010). Quality healthcare for all. Retrieved from
http://www.stmichaelshospital.com/pdf/corporate/equity_report_2010.pdf
Stephans, R. A. (2004). Systems safety for the 21st century: The updated and revised edition of
system safety 2000. Hoboken, NJ: Wiley.
Stevens, J. P. (2012). Applied multivariate statistics for the social sciences. New York:
Routledge.
Stevens, S. (2015). Preventing 30-day readmissions. Nursing Clinics of North America, 50(1),
123-137.
Stoeckle-Roberts, S., Reeves, M. J., Jacobs, B. S., Maddox, K., Choate, L., Wehner, S., &
Mullard, A. J. (2006). Closing gaps between evidence-based stroke care guidelines and
practices with a collaborative quality improvement project. Joint Commission Journal on
Quality & Patient Safety, 32(9), 517-527.
Sudore, R. L., Yaffe, K., Satterfield, S., Harris, T. B., Mehta, K. M., Simonsick, E. M., . . .
Rubin, S. M. (2006). Limited literacy and mortality in the elderly: The health, aging, and
body composition study. Journal of General Internal Medicine, 21(8), 806-812.
Tabachnick, B. G., & Fidell, L. S. (2007). Using multivariate statistics (5th ed.). Boston:
Pearson.
Taylor, N., Clay-Williams, R., Hogden, E., Braithwaite, J., & Groene, O. (2015). High
performing hospitals: a qualitative systematic review of associated factors and practical
strategies for improvement. BMC health services research, 15(1), 244.
Taylor, C. R., Hepworth, J. T., Buerhaus, P. I., Dittus, R., & Speroff, T. (2007). Effect of crew
resource management on diabetes care and patient outcomes in an inner-city primary care
clinic. Quality & Safety in Health Care, 16(4), 244-247.
Teddlie, C., & Tashakkori, A. (2009). Foundations of mixed methods research. Thousand Oaks,
CA: Sage.
Thomas, D. R. (2006). A general inductive approach for analyzing qualitative evaluation data.
American Journal of Evaluation, 27(2), 237-246.
Titler, M., Kleiber, C., Steelman, V., Goode, C., Rakel, B., Barry-Walker, J., . . . Buckwalter, K.
(1994). Infusing research into practice to promote quality care. Nursing Research, 43(5),
307-313.
Tobiano, G., Chaboyer, W., & McMurray, A. (2013). Family members’ perceptions of the
nursing bedside handover. Journal of Clinical Nursing, 22(1-2), 192-200.
220
Tucker, A. L., Nembhard, I. M., & Edmondson, A. C. (2007). Implementing new practices: An
empirical study of organizational learning in hospital intensive care units. Management
Science, 53(6), 894-907.
U.S. Department of Health and Human Services, O. o. D. P. a. H. P., . (2010). National Action
Plan to Improve Health Literacy. Washington, DC: US Department of Health and Human
Services, Office of Disease Prevention and Health Promotion.
Van Geest, J., & Johnson, T. P. (2011). Surveying nurses: Identifying strategies to improve
participation. Evaluation & the Health Professions, 34, 487-511.
Van Geest, J., Johnson, T. P., & Welch, V. L. (2007). Methodologies for improving response
rates in surveys of physicians a systematic review. Evaluation & the Health Professions,
30(4), 303-321.
van Walraven, C., Bennett, C., Jennings, A., Austin, P. C., & Forster, A. J. (2011). Proportion of
hospital readmissions deemed avoidable: A systematic review. Canadian Medical
Association Journal, 183(7), E391-E402.
Van Walraven, C., Mamdani, M., Fang, J., & Austin, P. C. (2004). Continuity of care and patient
outcomes after hospital discharge. Journal of General Internal Medicine, 19(6), 624-631.
Van Walraven, C., Seth, R., Austin, P. C., & Laupacis, A. (2002). Effect of discharge summary
availability during post-discharge visits on hospital readmission. Journal of General
Internal Medicine, 17(3), 186-192.
van Walraven, C., Taljaard, M., Bell, C. M., Etchells, E., Stiell, I. G., Zarnke, K., & Forster, A. J.
(2010). A prospective cohort study found that provider and information continuity was
low after patient discharge from hospital. Journal of clinical epidemiology, 63(9), 1000-
1010.
Vashi, A. A., Fox, J. P., Carr, B. G., D’Onofrio, G., Pines, J. M., Ross, J. S., & Gross, C. P.
(2013). Use of hospital-based acute care among patients recently discharged from the
hospital: acute care use after recent hospital discharge. JAMA, 309(4), 364-371.
Wachter, R. M. (2013). Personal accountability in healthcare: Searching for the right balance.
BMJ Quality & Safety, 22(2), 176-180.
Wagner, E. H., Austin, B. T., & Von Korff, M. (1996). Organizing care for patients with chronic
illness. The Milbank Quarterly, 74(4), 511-544.
Weiss, M. E., Yakusheva, O., & Bobay, K. L. (2011). Quality and cost analysis of nurse staffing,
discharge preparation, and postdischarge utilization. Health Services Research, 46(5),
1473-1494.
White, M., Garbez, R., Carroll, M., Brinker, E., & Howie-Esquivel, J. (2013). Is “teach-back”
associated with knowledge retention and hospital readmission in hospitalized heart failure
patients? Journal of Cardiovascular Nursing, 28(2), 137-146.
221
Williams, M. V., Davis, T., Parker, R. M., & Weiss, B. D. (2002). The role of health literacy in
patient-physician communication. Family Medicine, 34(5), 383-389.
Williamson, O. E. (1964). The economics of discretionary behavior: Managerial objectives in a
theory of the firm. New Jersey: Prentice-Hall.
Zahra, S. A., & George, G. (2002). Absorptive capacity: A review, reconceptualization, and
extension. Academy of Management Review, 27(2), 185-203.
222
Appendix A: Email for Round 1, Delphi panel
Dear (name of potential participant),
I am contacting you to request your participation in a study that will lead to the
development of a tool to assess how well hospitals discharge patients from the acute care
hospital setting to home. This study is part of my dissertation research at the Institute of Health
Policy, Management and Evaluation at the University of Toronto. This research is being
supervised by Dr. Jan Barnsley and Dr. Whitney Berta.
As you are likely aware, Ontario has a high rate of hospital readmission, particularly for
medical patients, and there is a growing understanding that hospital discharge processes
contribute to this readmission rate. When patients do not understand discharge instructions or
have difficulty following treatment plans, there is an increased chance that they will return to
hospital within 30 days of discharge. At present, there is no organizational tool to assess hospital
discharge practices. Such a tool could be used in quality improvement efforts, and would help
hospital managers and policy makers to monitor hospitals’ performance of discharge.
You have been identified as an expert in (hospital discharge/care transitions/health
literacy) (research/practice), and I would be grateful for your help to develop the
assessment tool.
Please refer to the attached Participant Information Sheet. If you have any questions or
concerns please contact me at jennifer.innis@mail.utoronto.ca.
Thank you for your consideration,
Jennifer Innis
Email: jennifer.innis@mail.utoronto.ca
Phone: (416) 571-3248
223
Appendix B: Participant Information Letter, Round 1
Participant Information Sheet
Health Literate Discharge Practices in Ontario Hospitals
My thanks for your interest in this study. Please read the information provided below carefully.
Background
This study seeks to develop an organizational tool to assess how hospitals discharge patients
from general medicine units to home. As you are likely aware, Ontario has a high rate of hospital
readmission particularly for medical patients, and there is a growing understanding that hospital
discharge processes contribute to this readmission rate. When patients do not understand
discharge instructions or have difficulty following treatment plans, there is an increased chance
that they will return to hospital within 30 days of discharge.
At present, there is no organizational tool to assess hospital discharge practices. Such a tool
could be used in quality improvement efforts, and would help hospitals and policy makers to
monitor hospitals’ performance of discharge. The initial items for this survey will based on the
components of Project RED (Re-Engineered Discharge), a program that has been found to be
significantly associated with decreased hospital readmission.
Methods
A virtual consultation process will be used to examine items that need to be included in this
organizational tool by a diverse group of experts. It is anticipated that this process will take 2-3
rounds. On the first round, you will be asked to rank the importance of the items, and to suggest
additional items that you think are important. Participation will be via electronic mail and you
will be asked to complete an internet based survey. If you prefer, a regular mail option will be
available and you will be mailed a questionnaire with a stamped envelope for reply. After the
results of the first round have been analyzed, you will be sent rankings of the items. Items that
have received poor rankings will not be included. Although suggested additional items will be
included, it is expected that there will be fewer items to rate following the first round. Each
round will take 15-30 minutes to complete, and there will be 2-3 rounds. This Delphi process
will take place from September to December, 2014. The input that you provide will be used to
create an organizational survey tool to measure health literate discharge practices in hospitals.
This tool will allow hospitals and policy makers to measure performance of hospital discharge.
224
Participation in this study is voluntary and there are no risks to you from being involved. Your
identity will be known only to the investigators, not to other participants, and your responses will
be kept strictly anonymous. Any information that could be used to identify you or the
organization you are affiliated with will be removed. You may withdraw from the study at any
time, and if you withdraw, your responses will not be used. If you have any questions about
your rights as a research participant, you may contact the Office of Office of Research Ethics
(ethics.review@utoronto.ca or 416-946-3273).
If you have any questions regarding this research, please contact me at the email address
provided below.
Thank you in advance for your time and cooperation.
Sincerely,
Jennifer Innis, PhD(c)
Institute of Health Policy, Management and Evaluation
Health Sciences Building, Suite 425
155 College Street, Toronto ON M5T 3M6
Email address: jennifer.innis@mail.utoronto.ca
Jan Barnsley, Associate Professor
Institute of Health Policy, Management and Evaluation
Email address: jan.barnsley@utoronto.ca
Whitney Berta, Associate Professor
Institute of Health Policy, Management and Evaluation
Email address: whit.berta@utoronto.ca
Your participation
Given your knowledge and experience with discharge practices, your participation will greatly
enhance expert dialogue and the study’s results. You may also find participating in this study a
unique opportunity to learn about, critique, and shape a new organizational assessment tool.
If you choose to participate, you have 3 options for submitting your questionnaire:
1. Complete on the Web at
http://fluidsurveys.com/surveys/whit-3hx/hospital-discharge/
2. Complete the word document provided and send via email to
jennifer.innis@mail.utoronto.ca
3. Contact jennifer.innis@mail.utoronto.ca for a paper copy of the questionnaire and a
stamped envelope.
Please submit your completed questionnaire by Friday, October 17, 2014.
225
Appendix C: Reminder Notice, Round 1
Dear (participant),
I contacted you earlier this month to request your participation in a study that will lead to the
development of a tool to assess how well hospitals discharge patients from the acute care
hospital setting to home. This study is part of my dissertation research at the Institute of Health
Policy, Management and Evaluation at the University of Toronto. This research is being
supervised by Dr. Jan Barnsley and Dr. Whitney Berta.
If you have already responded, please disregard this email.
If you have not yet responded, please refer to the attached Participant Information Sheet.
The web based link to the survey can be found at:
http://fluidsurveys.com/surveys/whit-3hx/hospital-discharge/
If you have any questions or concerns please contact me at jennifer.innis@mail.utoronto.ca.
Thank you,
Jennifer Innis
226
Appendix D: Email for Round 2, Delphi panel
Dear (participant)
Thank you for participating in the first round of the Delphi study. As you may recall, in Round 1
you were asked to rate a list of 34 indicators for their importance to health literate discharge
practices. All of the indicators received ratings of important or very important.
In response to the feedback received in Round 1, two of the indicators have been reworded, and
two new indicators have been added. In Round 2, I ask that you rate these 4 indicators using a
scale from 1 (least important) to 5 (most important). It will take approximately 10 minutes to
complete this rating process.
You have 3 options for submitting the Round 2 questionnaire:
1. Complete on the Web at
http://fluidsurveys.com/surveys/whit-3hx/health-literate-discharge-practices-round-
2/
2. Complete the word document provided and send via email to
jennifer.innis@mail.utoronto.ca
3. Contact jennifer.innis@mail.utoronto.ca for a paper copy of the questionnaire and a
stamped envelope.
Please submit your completed questionnaire by Friday November 21.
If you have any questions or concerns please contact me at jennifer.innis@mail.utoronto.ca.
Thank you for your participation,
Jennifer Innis
227
Appendix E: Participant Information Letter, Round 2
Participant Information Sheet Health Literate Discharge Practices in Ontario Hospitals
My thanks for your interest in this study. Please read the information provided below carefully.
Background
This study seeks to develop an organizational tool to assess how hospitals discharge patients
from general medicine units to home. As you are likely aware, Ontario has a high rate of hospital
readmission particularly for medical patients, and there is a growing understanding that hospital
discharge processes contribute to this readmission rate. When patients do not understand
discharge instructions or have difficulty following treatment plans, there is an increased chance
that they will return to hospital within 30 days of discharge.
At present, there is no organizational tool to assess hospital discharge practices. Such a tool
could be used in quality improvement efforts, and would help hospitals and policy makers to
monitor hospitals’ performance of discharge. The initial items for this survey will based on the
components of Project RED (Re-Engineered Discharge), a program that has been found to be
significantly associated with decreased hospital readmission.
Methods
A virtual consultation process will be used to examine items that need to be included in this
organizational tool by a diverse group of experts. It is anticipated that this process will take 2-3
rounds. On the first round, you will be asked to rank the importance of the items, and to suggest
additional items that you think are important. Participation will be via electronic mail and you
will be asked to complete an internet based survey. If you prefer, a regular mail option will be
available and you will be mailed a questionnaire with a stamped envelope for reply. After the
results of the first round have been analyzed, you will be sent rankings of the items. Items that
have received poor rankings will not be included. Although suggested additional items will be
included, it is expected that there will be fewer items to rate following the first round. Each
round will take 15-30 minutes to complete, and there will be 2-3 rounds. This Delphi process
will take place from September to December, 2014. The input that you provide will be used to
create an organizational survey tool to measure health literate discharge practices in hospitals.
This tool will allow hospitals and policy makers to measure performance of hospital discharge.
228
Participation in this study is voluntary and there are no risks to you from being involved. Your
identity will be known only to the investigators, not to other participants, and your responses will
be kept strictly anonymous. Any information that could be used to identify you or the
organization you are affiliated with will be removed. You may withdraw from the study at any
time, and if you withdraw, your responses will not be used. If you have any questions about
your rights as a research participant, you may contact the Office of Office of Research Ethics
(ethics.review@utoronto.ca or 416-946-3273).
If you have any questions regarding this research, please contact me at the email address
provided below.
Thank you in advance for your time and cooperation.
Sincerely,
Jennifer Innis, PhD(c)
Institute of Health Policy, Management and Evaluation
Health Sciences Building, Suite 425
155 College Street, Toronto ON M5T 3M6
Email address: jennifer.innis@mail.utoronto.ca
Jan Barnsley, Associate Professor
Institute of Health Policy, Management and Evaluation
Email address: jan.barnsley@utoronto.ca
Whitney Berta, Associate Professor
Institute of Health Policy, Management and Evaluation
Email address: whit.berta@utoronto.ca
Your participation
Given your knowledge and experience with discharge practices, your participation will greatly
enhance expert dialogue and the study’s results. You may also find participating in this study a
unique opportunity to learn about, critique, and shape a new organizational assessment tool.
If you choose to participate, you have 3 options for submitting your questionnaire:
4. Complete on the Web at
http://fluidsurveys.com/surveys/whit-3hx/health-literate-discharge-practices-round-2/
5. Complete the word document provided and send via email to
jennifer.innis@mail.utoronto.ca
6. Contact jennifer.innis@mail.utoronto.ca for a paper copy of the questionnaire and a
stamped envelope.
Please submit your completed questionnaire by Friday, November 14, 2014.
229
Appendix F: Reminder Notice, Round 2
Dear (participant),
I contacted you two weeks ago to request your participation in the second round of the Delphi
study. If you have already responded, please disregard this email. In this second round, I am
asking you to rate the importance of 4 indicators.
You have 3 options for submitting the Round 2 questionnaire:
1. Complete on the Web at http://fluidsurveys.com/surveys/whit-3hx/health-literate-discharge-practices-round-2/
2. Complete the word document provided and send via email to jennifer.innis@mail.utoronto.ca
3. Contact jennifer.innis@mail.utoronto.ca for a paper copy of the questionnaire and a stamped envelope.
If you have any questions or concerns please contact me at jennifer.innis@mail.utoronto.ca.
Thank you,
Jennifer Innis
230
Appendix G: Delphi Panel, Results of Round 1
Statistics
Location Occupation Work location
N Valid 37 37 37
Missing 0 0 0
Location
Frequency Percent Valid Percent Cumulative
Percent
Valid 1 2.7 2.7 2.7
Alabama 1 2.7 2.7 5.4
Massachusetts 2 5.4 5.4 10.8
Ontario 29 78.4 78.4 89.2
United States 3 8.1 8.1 97.3
Wisconsin 1 2.7 2.7 100.0
Total 37 100.0 100.0
Occupation
Frequency Percent Valid Percent Cumulative
Percent
Valid Physician 7 18.9 18.9 18.9
Pharmacist 6 16.2 16.2 35.1
Registered Nurse 5 13.5 13.5 48.6
Nurse practitioner 6 16.2 16.2 64.9
Discharge planner 1 2.7 2.7 67.6
Case manager 2 5.4 5.4 73.0
Hospital administrator 4 10.8 10.8 83.8
Researcher 6 16.2 16.2 100.0
Total 37 100.0 100.0
Work location
Frequency Percent Valid Percent Cumulative
Percent
Valid Hospital 29 78.4 78.4 78.4
CCAC 2 5.4 5.4 83.8
HQO 2 5.4 5.4 89.2
University 4 10.8 10.8 100.0
Total 37 100.0 100.0
231
Language assistance | Language
preference of patient and family is
determined and documented
Language assistance | If
patient and/or family members do not speak English, a
translator is arranged for on
discharge
Language assistance | If
patient and/or family members do not speak English,
written materials are provided in the
preferred language
Post-discharge appointments, tests | Need for primary care and
specialty care follow-up is determined
Mean 4.62 4.38 4.27 4.78
Median 5.00 5.00 4.00 5.00
Range 2 3 3 2
Post-discharge appointments, tests |
If patient does not have a primary care
provider, one is located for patients
Post-discharge appointments, tests
| Need for future tests is determined
Post-discharge appointments, tests | Appointments are made for patient for
follow-up appointments and
testing
Post-discharge appointments,
tests | If appointments
are made, they are made with input from the patient/family regarding the best time/date
Mean 4.68 4.30 4.57 4.11
Median 5.00 4.00 5.00 4.00
Range 2 2 2 4
Post-discharge appointment, tests
continued | If patient requires future
diagnostic testing, patient and/or family member is instructed on any preparation for
testing
Post-discharge appointment, tests
continued | Importance of
clinician appointments and further testing is discussed with patient/family
Post-discharge appointment, tests
continued | Patient/family is
asked about traditional healing
practices, and there is confirmation
made that practices are complementary
with patient’s discharge plan
Post-discharge appointment,
tests continued |
There is confirmation made with
patient/family that they know where to go for
further appointments and tests, and that they have a plan to get to appointments
Mean 4.41 4.46 3.83 4.51
Median 5.00 5.00 4.00 5.00
Range 4 2 3 2
232
Post-discharge appointment, tests
continued | Barriers to keeping appointments
are addressed
Plan for follow-up of results from
diagnostic lab tests or studies that are pending at the time
of discharge | Pending lab and test results are identified with patient/family
Plan for follow-up of results from
diagnostic lab tests or studies that are pending at the time
of discharge | Determination is
made of who will be reviewing the
results, and when and how this
information will be communicated to the patient/family
Organization of post-
discharge outpatient
services and medical
equipment if needed | If
patient requires medical
equipment on discharge, there is a
process for ensuring that the medical
equipment is obtained
Mean 4.49 4.24 4.38 4.54
Median 5.00 4.00 5.00 5.00
Range 2 4 4 2
Organization of post-discharge outpatient services and medical equipment if needed |
Before discharge, patient/family is given
contact information for medical equipment
companies, CCAC (as needed)
Medication review | Medication list is
reviewed with patient/family
Medication review | Medication
reconciliation is done at the time of
discharge
Medication review |
Patient/family member is given an
explanation of what
medications to take and
changes in the medication regimen are emphasized
Mean 4.46 4.92 4.92 4.97
Median 5.00 5.00 5.00 5.00
Range 2 1 2 1
233
Medication review | Each medication’s
purpose, administration and
side effects are reviewed with patient/family
Medication review | Patient/family’s concerns about
medication plan are assessed
Discharge plan is reconciled with use
of guidelines | There is use of
guidelines in the development and planning of the
discharge process for patients and family members
Written discharge plan
| The patient/family is given an
easy-to-understand
written discharge plan that includes medications,
medical equipment,
future appointments,
and future diagnostic
tests to take home
Mean 4.57 4.65 4.22 4.78
Median 5.00 5.00 4.00 5.00
Range 2 2 4 2
Written discharge plan | The written plan is reviewed with the
patient/family
Written discharge plan | Patient/family
is encouraged to ask questions about
the plan
Patient/family education of
diagnosis | Prior to day of discharge,
patient/family is met with to provide
education about patient’s diagnosis and treatment and
to prepare for discharge
Patient/family’s understanding
of the discharge plan is assessed | Patient/family
is asked to explain, in their own words, the details of the
discharge plan
Mean 4.70 4.68 4.57 4.38
Median 5.00 5.00 5.00 4.00
Range 2 2 2 2
Patient/family’s understanding of the
discharge plan is assessed | As needed, family
members and other caregivers who will share in the care-
giving responsibilities are contacted
Patient/family is instructed on what to do if a problem
arises | Patient/family is
made aware of how to contact primary
care provider
Patient/family is instructed on what to do if a problem
arises | Patient/family is
instructed on what constitutes an
emergency and what to do in case
of emergency
Discharge summary is
sent to clinicians
accepting care of the patient |
Within 24 hours of
discharge, a discharge
summary is provided to
primary care provider
Mean 4.24 4.46 4.78 4.73
Median 4.00 5.00 5.00 5.00
Range 3 3 2 2
234
Telephone reinforcement of the
discharge plan is provided |
Patient/family is called within 3 days of discharge to reinforce
the discharge plan and help with
problem-solving
Telephone reinforcement of the
discharge plan is provided | A help
line is staffed where phone calls can be
answered from patients, families
and/or other caregivers with
questions about the at home care plan, hospitalization, and
follow-up plan in order to help
patients transition from hospital care to outpatient care
setting
Mean 4.36 4.11
Median 4.00 4.00
Range 2 2
235
Appendix H: Delphi Panel, Results of Round 2
Statistics
Location Primary
Occupation Work location
N Valid 39 39 39
Missing 0 0 0
Location
Frequency Percent Valid Percent Cumulative
Percent
Valid 1 2.6 2.6 2.6
Canada 1 2.6 2.6 5.1
Florida 1 2.6 2.6 7.7
Maryland 1 2.6 2.6 10.3
Massachusetts 2 5.1 5.1 15.4
Ontario 30 76.9 76.9 92.3
United States 2 5.1 5.1 97.4
Wisconsin 1 2.6 2.6 100.0
Total 39 100.0 100.0
Primary Occupation
Frequency Percent Valid Percent Cumulative
Percent
Valid Physician 6 15.4 15.4 15.4
Pharmacist 6 15.4 15.4 30.8
Registered nurse 6 15.4 15.4 46.2
Nurse practitioner 6 15.4 15.4 61.5
Discharge planner 1 2.6 2.6 64.1
Case manager 3 7.7 7.7 71.8
Hospital administrator 4 10.3 10.3 82.1
Researcher 7 17.9 17.9 100.0
Total 39 100.0 100.0
Work location
Frequency Percent Valid Percent Cumulative
Percent
Valid Hospital 29 74.4 74.4 74.4
CCAC 2 5.1 5.1 79.5
HQO 2 5.1 5.1 84.6
University 6 15.4 15.4 100.0
Total 39 100.0 100.0
236
Medication Review | Patient/family is
referred to community pharmacist within 2 weeks of discharge
for a medication review
Written discharge plan | The
patient/family is given an easy-to-
understand written, prioritized discharge
plan that includes medications,
medical equipment, future
appointments, and future diagnostic
tests to take home
Discharge summary is sent to clinicians accepting care of
the patient | Discharge summary has a standardized
format so that information is easy
to find
Telephone reinforcement
of the discharge plan
is provided | Patient/family
is provided with a phone
number where they can
speak with a hospital staff
member to ask questions
about the at-home care
plan, hospitalization, and follow-up
plan in order to help patients
transition from hospital care to outpatient care setting
Mean 4.18 4.85 4.79 4.31
Median 4.00 5.00 5.00 4.00
Range 3 4 2 3
237
Appendix I: Health Literate Discharge Practices Survey
Health literate discharge practices are discharge practices that meet the health literacy needs of
patients and their families (family as identified by the patient). In other words, these practices
help patients and their families to obtain, understand and use information to make decisions that
help them to maintain and/or promote their health. These include practices that help patients and
families to navigate the health care system.
Please rate the use of the following discharge practices at your hospital, using the 5-point Likert
scale.
Discharge practice 1=
never
2 =
seldom
3 =
undecided
4 =
often
5 =
almost
always
Language Assistance
Language preference of
patient and family is
determined and
documented
If patient and/or family
members do not speak
English, a translator is
arranged for on discharge
If patient and/or family
members do not speak
English, written materials
are provided in the
preferred language
Post-discharge
appointments, tests
Need for primary care
and specialty care
follow-up is determined
If patient does not have a
primary care provider,
one is located for patients
Need for future tests is
determined
Appointments are made
for patient for follow-up
appointments and testing
If appointments are
made, they are made with
input from the
patient/family regarding
the best time/date
238
Discharge practice 1=
never
2 =
seldom
3 =
undecided
4 =
often
5 =
almost
always
If patient requires future
diagnostic testing, patient
and/or family member is
instructed on any
preparation for testing
Importance of clinician
appointments and further
testing is discussed with
patient/family
Patient/family is asked
about traditional healing
practices, and there is
confirmation made that
practices are
complementary with
patient’s discharge plan
There is confirmation
made with patient/family
that they know where to
go for further
appointments and tests,
and that they have a plan
to get to appointments
Barriers to keeping
appointments are
addressed
Plan for follow-up of
results from diagnostic
lab tests or studies that
are pending at the time
of discharge
Pending lab and test
results are identified with
patient/family
Determination is made of
who will be reviewing
the results, and when and
how this information will
be communicated to the
patient/family
239
Discharge practice 1=
never
2 =
seldom
3 =
undecided
4 =
often
5 =
almost
always
Organization of post-
discharge outpatient
services and medical
equipment if needed
If patient requires
medical equipment on
discharge, there is a
process for ensuring that
the medical equipment is
obtained
Before discharge,
patient/family is given
contact information for
medical equipment
companies, CCAC (as
needed)
Medication Review
Medication list is
reviewed with
patient/family
Medication reconciliation
is done at the time of
discharge
Patient/family member is
given an explanation of
what medications to take
and changes in the
medication regimen are
emphasized
Each medication’s
purpose, administration
and side effects are
reviewed with
patient/family
Patient/family’s concerns
about medication plan
are assessed
Patient/family is referred
to community pharmacist
within 2 weeks of
discharge for a
medication review
240
Discharge practice 1=
never
2 =
seldom
3 =
undecided
4 =
often
5 =
almost
always
Discharge plan is
reconciled with use of
guidelines
There is use of guidelines
in the development and
planning of the discharge
process for patients and
family members
Written discharge plan
The patient/family is
given an easy-to-
understand written,
prioritized discharge plan
that includes
medications, medical
equipment, future
appointments, and future
diagnostic tests to take
home
The written plan is
reviewed with the
patient/family
Patient/family is
encouraged to ask
questions about the plan
Patient/family
education of diagnosis
Prior to day of discharge,
patient/family is met with
to provide education
about patient’s diagnosis
and treatment and to
prepare for discharge
241
Discharge practice 1=
never
2 =
seldom
3 =
undecided
4 =
often
5 =
almost
always
Patient/family’s
understanding of the
discharge plan is
assessed
Patient/family is asked to
explain, in their own
words, the details of the
discharge plan
As needed, family
members and other
caregivers who will share
in the care-giving
responsibilities are
contacted
Patient/family is
instructed on what to
do if a problem arises
Patient/family is made
aware of how to contact
primary care provider
Patient/family is
instructed on what
constitutes an emergency
and what to do in case of
emergency
Discharge summary is
sent to clinicians
accepting care of the
patient
Discharge summary has a
standardized format so
that information is easy
to find
Within 24 hours of
discharge, a discharge
summary is provided to
primary care provider
242
Discharge practice 1=
never
2 =
seldom
3 =
undecided
4 =
often
5 =
almost
always
Telephone
reinforcement of the
discharge plan is
provided
Patient/family is called
within 3 days of
discharge to reinforce the
discharge plan and help
with problem-solving
Patient/family is
provided with a phone
number where they can
speak with a hospital
staff member to ask
questions about the at-
home care plan,
hospitalization, and
follow-up plan in order
to help patients transition
from hospital care to
outpatient care setting
243
Appendix J: Teaching Hospitals
Teaching hospitals (N = 15)
Location Hospital Sites (if applicable)
Hamilton Hamilton Health Sciences
Corp
Hamilton General
Juravinski
Kingston Kingston General Hospital
London London Health Sciences
Centre
University
Victoria – Westminster
Ottawa The Ottawa Hospital Civic Campus
General Campus
Ottawa Hopital Montfort
Sudbury Health Services North Laurentian Site
Thunder Bay Thunder Bay Regional Health
Sciences Centre
Toronto Mount Sinai
Toronto St. Michael’s
Toronto Sunnybrook
Toronto University Health Network TGH
TWH
244
Appendix K: Large Community Hospitals
Large community hospitals (N = 48)
Location Hospital Site (if applicable)
Barrie Royal Victoria Hospital
Belleville Quinte Healthcare Corp Belleville General
Brampton William Osler Health Centre Etobicoke General
Brampton Civic Hospital
Brantford Brant Community HC Brantford General
Brockville Brockville General Hospital Brockville General
Burlington Joseph Brant Hospital
Cambridge Cambridge Memorial Hospital
Chatham Public General Hospital
Cobourg Northumberland Hills Hospital
Cornwall Cornwall Community Hospital
Guelph Guelph General Hospital
Kitchener St. Mary’s General Hospital
Kitchener Grand River Hospital Grand River
Lindsay Ross Memorial Hospital
Markham Markham-Stouffville Hospital Markham-Stouffville
Hospital
Mississauga Trillium Health Partners Credit Valley Hospital /
Mississauga Hospital
Newmarket Southlake Regional Health Centre
North Bay North Bay Regional Health Centre
Oakville Halton HC Services Corp Milton District / Oakville
Trafalgar
Orangeville Headwaters Health Care Center Orangeville Dufferin Area
Hospital
Orillia Orillia Soldiers’ Memorial Hospital
Oshawa Lakeridge Health Oshawa General Hospital
Ottawa Queensway-Carleton Hospital
Owen Sound Grey Bruce Health Services Owen Sound
Pembroke Pembroke Regional Hospital
Peterborough Peterborough Regional Health Centre
Richmond Hill MacKenzie Health
Sarnia Bluewater Health Sarnia General Hospital
SSM Sault Area Hospital SAH – General site
Simcoe Norfolk General Hospital
St Catharine’s Niagara Health System Niagara Falls Greater Niagara
St Catharine’s
Welland County General
St. Thomas St. Thomas-Elgin General Hosp.
Stratford Stratford General Hospital
Timmins Timmins & District Gen Hospital
245
Toronto Scarborough Hospital Scarborough General
Scarborough Grace
Toronto St. Joseph’s
Toronto TEGH
Toronto HRRH
Toronto Rouge Valley Health System Ajax & Pickering
Centenary
Toronto North York General Hospital General
Windsor Windsor Regional Hospital Windsor Metropolitan
General
Ouellette Campus
Woodstock Woodstock General Hospital
246
Appendix L: Small Community Hospitals
Small community hospital (N = 80)
Location Hospital Site
Alexandria Glengarry Memorial Hospital
Alliston Stevenson Memorial Hospital
Almonte Almonte General Hospital
Atikokan Atikokan General Hospital
Barry’s Bay St Francis Memorial Hospital
Belleville Quinte Healthcare Corp Picton Prince Edward
County
Trenton Memorial Hospital
Blind River Blind River District Health Centre
Bowmanville Lakeridge Health Bowmanville Site
Port Perry
Campbellford Campbellford Memorial Hospital
Carleton Place Carleton Place & District Memorial
Chapleau Services de sante de Chapleau Health Chapleau General Hospital
Clinton Clinton Public Hospital
Cochrane The Lady Minto Hospital
Collingwood Collingwood General and Marine
Hospital
Deep River Deep River and District Hospital
Dryden Dryden Regional Health Centre
Dunville Haldimand War Memorial Hospital
Elliot Lake St. Joseph’s General Hospital
Englehart Englehart and District Hospital
Espanola Espanola General Hospital
Exeter South Huron Hospital
Fergus Groves Memorial Community Hospital
Fort Frances
Riverside HC Facilities Inc
La Verendrye
Emo site (3 beds)
Rainy River site (3 beds)
Geraldton Geraldton District Hospital
Goderich Alexandra Marine & Gen Hospital
Grimbsy West Lincoln Memorial Hospital
Haldimand West Haldimand General Hospital
Haliburton Haliburton Highlands Health Services
Hanover Hanover and District Hospital
Hawkesbury Hawkesbury & District Gen Hospital
Hearst Hoptial Notre Dame Hospital (Hearst)
Hornepayne Hornepayne Community Hospital
Huntsville Muskoka Algonquin Healthcare Huntsville District
Memorial Hospital
(Huntsville)
247
South Muskoka Memorial
Hospital (Bracebridge)
Ingersoll Alexandra Hospital
Iroquois Falls Anson General Hospital
Kapuskasing Sensenbrenner Hospital
Kemptville Kemptville District Hospital
Kenora Lake-of-the-Woods District Hospital
Kincardine South Bruce Grey Health Centre Durham / Chesley
Walkerton
Kincardine
Kirkland Lake Kirkland Lake and District Hospital
Leamington Leamington District Memorial Hospital
Listowel Listowel Memorial Hospital
Little Current Manitoulin Health Centre Little Current / Mindemoya
Manitouwadge Manitouwadge General Hospital
Marathon Wilson Memorial General Hospital
Matheson Bingham Memorial Hospital
Mattawa Mattawa General Hospital
Midland Georgian Bay General Hospital Midland
Moosonee James Bay General Hospital Fort Albany
Mount Forest North Wellington Health Care Corp. Louise Marshall Hospital
Palmerston & District
Hospital
Napanee Lennox & Addington County Gen
Hospital
New Liskeard Temiskaming Hospital
Nipigon Nipigon District Memorial Hospital
Oakville Halton Healthcare Services Corp Georgetown & District
Parry Sound West Parry Sound Health Centre
Red Lake Red Lake Margaret Cochenour
Memorial Hospital
Renfrew Renfrew Victoria Hospital
Sarnia Bluewater Health Petrolia Charlotte Eleanor
Englehart Hospital
SSM Sault Area Hospital Thessalon site
Seaforth Seaforth Community Hospital
Sioux Lookout Sioux Lookout Meno-Ya-Win Health
Centre
S L Zone Hospital (7th Ave)
Smith Falls Perth & Smith Falls District Hospital
Smooth Rock Falls Smooth Rock Falls Hospital
St Mary’s St Mary’s Memorial Hospital
Strathroy Strathroy Middlesex General Hospital
Sturgeon Falls West Nipissing General Hospital
Terrace Bay McCausland Hospital
Tillsonburg Tillsonburg District Memorial Hospital
Wallaceburg Sydenham District Hospital
248
Wawa Lady Dunn Health Centre
Winchester Winchester District Memorial Hospital
Wingham Wingham & District Hospital
249
Appendix M: Email
Dear (name of potential participant),
I am contacting you to request your participation in a study that is using a survey to
assess how well hospitals discharge patients from the acute care hospital setting to home. This
study is part of my dissertation research at the Institute of Health Policy, Management and
Evaluation at the University of Toronto and it is endorsed by the Ontario Hospital Association.
This research is being supervised by Dr. Jan Barnsley and Dr. Whitney Berta.
As you are likely aware, Ontario has a high rate of hospital readmission, particularly for
medical patients, and there is a growing understanding that hospital discharge processes
contribute to this readmission rate. When patients do not understand discharge instructions or
have difficulty following treatment plans, there is an increased chance that they will return to
hospital within 30 days of discharge. The use of a survey tool to measure the use of hospital
discharge practices could be used in quality improvement efforts, and would help hospital
managers and policy makers to monitor hospitals’ performance of discharge.
You have been identified as a stakeholder in the hospital discharge process, and I
would be grateful for your assistance with this survey. Please complete this survey or share
it with a health care provider on the general medicine patient care unit who has experience
in the hospital discharge process.
Please refer to the attached Participant Information Sheet. If you have any questions or
concerns please contact me at jennifer.innis@mail.utoronto.ca.
Thank you for your consideration,
Jennifer Innis
Email: jennifer.innis@mail.utoronto.ca
Phone: (416) 571-3248
250
Appendix N: Participant Information Letter
Participant Information Sheet
Health Literate Discharge Practices in Ontario Hospitals
My thanks for your interest in this study. Please read the information provided below carefully.
Background
This study seeks to use a survey to assess how hospitals discharge patients from general
medicine inpatient nursing units to home in Ontario. As you are likely aware, Ontario has a high
rate of hospital readmission particularly for medical patients, and there is a growing
understanding that hospital discharge processes contribute to this readmission rate. When
patients do not understand discharge instructions or have difficulty following treatment plans,
there is an increased chance that they will return to hospital within 30 days of discharge.
At present, there is no tool to assess hospital discharge practices. Such a tool could be used in
quality improvement efforts, and would help hospitals and policy makers to monitor hospitals’
performance of discharge. The items on this survey are based on the components of Project RED
(Re-Engineered Discharge), a program that has been found to be significantly associated with
decreased hospital readmission. The items have been reviewed and refined through a Delphi
process with hospital, government, and research-based stakeholders.
Methods
This survey tool is being used to determine the extent of hospital discharge practices that meet
the health literacy needs of patients and families, which are being called “health literate
discharge practices.” This survey has 36 items, and you will be asked to rate how often the
practices are used, in your experience. Participation will be via electronic mail and you will be
asked to complete an internet based survey. If you prefer, a regular mail option will be available
and you will be mailed a questionnaire with a stamped envelope for reply. The survey will take
10-20 minutes to complete.
Participants will be offered mini-summaries of the final study findings. This summary will
contain feedback on the general medicine patient care unit’s use of health literate discharge
practices relative to aggregate levels of use from other hospital sites in Ontario. Your hospital
name will be kept anonymous. Hospitals will only be identified by 3 categories: 1) teaching
hospitals, 2) community hospitals with 100 beds or more; 3) community hospitals with less than
100 beds.
251
Participation in this study is voluntary and there are no risks to you from being involved. Your
identity will be known only to the investigators, not to other participants, and your responses will
be kept strictly anonymous. Any information that could be used to identify you or the
organization you are affiliated with will be removed. You may withdraw from the study at any
time, and if you withdraw, your responses will not be used. Your consent to participate in the
study will be indicated by your choice to respond to the survey. If you have any questions about
your rights as a research participant, you may contact the Office of Office of Research Ethics
(ethics.review@utoronto.ca or 416-946-3273).
Your participation
Given your knowledge and experience with the use of discharge practices, your participation
will greatly enhance the understanding of how hospital discharge practices are used on general
medicine inpatient nursing units in hospitals.
If you choose to participate, you have 3 options for submitting your questionnaire:
7. Complete on the Web at: http://worklife.fluidsurveys.com/surveys/wb/health-
literate-discharge-practices/
8. Complete the word document provided and send via email to
jennifer.innis@mail.utoronto.ca
9. Contact jennifer.innis@mail.utoronto.ca for a paper copy of the questionnaire and
a stamped envelope.
252
If you have any questions regarding this research, please contact me at the email address
provided below.
Thank you in advance for your time and cooperation.
Sincerely,
Jennifer Innis, PhD(c)
Institute of Health Policy, Management and Evaluation
Health Sciences Building, Suite 425
155 College Street, Toronto ON M5T 3M6
Email address: jennifer.innis@mail.utoronto.ca
Jan Barnsley, Associate Professor
Institute of Health Policy, Management and Evaluation
Email address: jan.barnsley@utoronto.ca
Whitney Berta, Associate Professor
Institute of Health Policy, Management and Evaluation
Email address: whit.berta@utoronto.ca
Imtiaz Daniel, Adjunct Lecturer Institute of Health Policy, Management and Evaluation
Senior Consultant, Ontario Hospital Association
Email address: imtiaz.daniel@utoronto.ca
253
Appendix O: Endorsement Letter from the Ontario Hospital Association
March 12, 2015
Jan Barnsley, PhD, Associate Professor
Whitney Berta, PhD, Associate Professor
Institute of Health Policy, Management and Evaluation
University of Toronto
425-155 College St.
Toronto, ON M5T 3M6
Dear Dr. Barnsley and Dr. Berta,
Thank you for contacting us about this research project. Reducing readmission rates within 30
days has been identified as a key initiative in Ontario’s Health System Funding Reform through
the Quality-Based Procedures (QBP) initiative. As you point out, the need to address the health
literacy needs of patients and families during the process of discharge from the hospital to reduce
readmission has been identified and described by the Institute for Healthcare Improvement, the
Institute of Medicine, and the Agency for Healthcare Research and Quality. There is a need for
an organizational survey tool that can be used to measure hospitals’ performance of hospital
discharge and to suggest areas for improvement that might reduce the readmission rates for
specific QBP populations, such as patients with COPD, CHF and stroke.
We would like to endorse, in concept, the organizational survey tool that has been developed,
and adapted for use in the Ontario context, by your doctoral student, Jennifer Innis. It is
understood that Ms Innis is planning to administer the survey to the 151 adult, acute care hospital
sites in Ontario that have a general medicine unit, as it is medical patients that have the highest
readmission rates. Our endorsement may appear on the introductory, opening page of the survey,
and in the initial contact to potential participants.
We welcome the opportunity to be involved in this research project and will be interested in the
findings. It is anticipated that the results of this research will lead to an evaluation tool that can
measure and monitor the performance of hospital discharge in Ontario, as well as direct quality
improvement efforts.
Lou Reidel
Chief System Planning and Performance Officer
Ontario Hospital Association
cc: Imtiaz Daniel, PhD, CPA, CMA, PhD committee member
Jennifer Innis, MA RN
254
Appendix P: Reminder Notice
Dear (participant),
I contacted you earlier this month to request your participation in a study that is using a survey to
assess how well hospitals discharge patients from the acute care hospital setting to home. This
study is part of my dissertation research at the Institute of Health Policy, Management and
Evaluation at the University of Toronto. This research is being supervised by Dr. Jan Barnsley
and Dr. Whitney Berta, and is endorsed by the Ontario Hospital Association.
If you have not yet responded, please refer to the attached Participant Information Sheet.
The web based link to the survey can be found at:
http://worklife.fluidsurveys.com/surveys/wb/health-literate-discharge-practices/. If you have any
questions or concerns please contact me at jennifer.innis@mail.utoronto.ca.
Thank you,
Jennifer Innis
257
Appendix R: Descriptives, Survey Items
Descriptives
Statistic Std. Error
1 - Language assistance | Language preference of patient and family is determined and documented
Mean 3.95 .133
95% Confidence Interval for Mean
Lower Bound 3.68 Upper Bound 4.21
5% Trimmed Mean 4.05 Median 4.00 Variance 1.763 Std. Deviation 1.328 Minimum 1 Maximum 5 Range 4 Interquartile Range 1 Skewness -1.107 .243
Kurtosis -.119 .481
2 - Language assistance | If patient and/or family members do not speak English, a translator is arranged for on discharge
Mean 3.43 .118
95% Confidence Interval for Mean
Lower Bound 3.20 Upper Bound 3.67
5% Trimmed Mean 3.46 Median 4.00 Variance 1.371 Std. Deviation 1.171 Minimum 1 Maximum 5 Range 4 Interquartile Range 2 Skewness -.248 .243
Kurtosis -1.132 .481
3 - Language assistance | If patient and/or family members do not speak English, written materials are provided in the preferred language
Mean 2.67 .126
95% Confidence Interval for Mean
Lower Bound 2.42 Upper Bound 2.92
5% Trimmed Mean 2.63 Median 2.00 Variance 1.571 Std. Deviation 1.254 Minimum 1 Maximum 5 Range 4 Interquartile Range 2 Skewness .405 .243
Kurtosis -1.006 .481
4 - Post-discharge appointments, tests | Need
Mean 4.53 .076
Lower Bound 4.37
258
for primary care and specialty care follow-up is determined
95% Confidence Interval for Mean
Upper Bound 4.68
5% Trimmed Mean 4.64 Median 5.00 Variance .578 Std. Deviation .761 Minimum 2 Maximum 5 Range 3 Interquartile Range 1 Skewness -1.933 .243
Kurtosis 3.852 .481
5 - Post-discharge appointments, tests | If patient does not have a primary care provider, one is located for patients
Mean 3.26 .125
95% Confidence Interval for Mean
Lower Bound 3.01 Upper Bound 3.51
5% Trimmed Mean 3.29 Median 3.00 Variance 1.543 Std. Deviation 1.242 Minimum 1 Maximum 5 Range 4 Interquartile Range 2 Skewness -.255 .243
Kurtosis -1.014 .481
6 - Post-discharge appointments, tests | Need for future tests is determined
Mean 4.43 .071
95% Confidence Interval for Mean
Lower Bound 4.29 Upper Bound 4.57
5% Trimmed Mean 4.52 Median 5.00 Variance .493 Std. Deviation .702 Minimum 2 Maximum 5 Range 3 Interquartile Range 1 Skewness -1.566 .243
Kurtosis 3.476 .481
7 - Post-discharge appointments, tests | Appointments are made for patient for follow-up appointments and testing
Mean 4.16 .104
95% Confidence Interval for Mean
Lower Bound 3.95 Upper Bound 4.37
5% Trimmed Mean 4.28 Median 4.00 Variance 1.076 Std. Deviation 1.037 Minimum 1 Maximum 5
259
Range 4 Interquartile Range 1 Skewness -1.564 .243
Kurtosis 2.154 .481
8 - Post-discharge appointments, tests | If appointments are made, they are made with input from the patient/family regarding the best time/date
Mean 3.01 .128
95% Confidence Interval for Mean
Lower Bound 2.76 Upper Bound 3.26
5% Trimmed Mean 3.01 Median 3.00 Variance 1.622 Std. Deviation 1.274 Minimum 1 Maximum 5 Range 4 Interquartile Range 2 Skewness .071 .243
Kurtosis -1.205 .481
9 - Post-discharge appointments, tests continued | If patient requires future diagnostic testing, patient and/or family member is instructed on any preparation for testing
Mean 3.99 .108
95% Confidence Interval for Mean
Lower Bound 3.78 Upper Bound 4.20
5% Trimmed Mean 4.07 Median 4.00 Variance 1.153 Std. Deviation 1.074 Minimum 1 Maximum 5 Range 4 Interquartile Range 1 Skewness -1.040 .243
Kurtosis .235 .481
10 - Post-discharge appointments, tests continued | Importance of clinician appointments and further testing is discussed with patient/family
Mean 4.32 .081
95% Confidence Interval for Mean
Lower Bound 4.16 Upper Bound 4.48
5% Trimmed Mean 4.43 Median 4.00 Variance .650 Std. Deviation .806 Minimum 1 Maximum 5 Range 4 Interquartile Range 1 Skewness -1.852 .243
Kurtosis 5.168 .481
11 - Post-discharge appointments, tests continued | Patient/family is asked about traditional healing practices, and there
Mean 2.33 .108
95% Confidence Interval for Mean
Lower Bound 2.12 Upper Bound 2.55
5% Trimmed Mean 2.27
260
is confirmation made that practices are complementary with patient’s discharge plan
Median 2.00 Variance 1.163 Std. Deviation 1.079 Minimum 1 Maximum 5 Range 4 Interquartile Range 1 Skewness .692 .243
Kurtosis -.138 .481
12 - Post-discharge appointments, tests continued | There is confirmation made with patient/family that they know where to go for further appointments and tests, and that they have a plan to get to appointments
Mean 3.74 .107
95% Confidence Interval for Mean
Lower Bound 3.52 Upper Bound 3.95
5% Trimmed Mean 3.76 Median 4.00 Variance 1.134 Std. Deviation 1.065 Minimum 2 Maximum 5 Range 3 Interquartile Range 2 Skewness -.540 .243
Kurtosis -.921 .481
13 - Post-discharge appointments, tests continued | Barriers to keeping appointments are addressed
Mean 3.20 .108
95% Confidence Interval for Mean
Lower Bound 2.99 Upper Bound 3.42
5% Trimmed Mean 3.22 Median 3.00 Variance 1.163 Std. Deviation 1.078 Minimum 1 Maximum 5 Range 4 Interquartile Range 2 Skewness -.265 .243
Kurtosis -.914 .481
14 - Plan for follow-up of results from diagnostic lab tests or studies that are pending at the time of discharge | Pending lab and test results are identified with patient/family
Mean 3.78 .107
95% Confidence Interval for Mean
Lower Bound 3.57 Upper Bound 3.99
5% Trimmed Mean 3.83 Median 4.00 Variance 1.134 Std. Deviation 1.065 Minimum 1 Maximum 5 Range 4 Interquartile Range 2 Skewness -.732 .243
261
Kurtosis -.297 .481
15 - Plan for follow-up of results from diagnostic lab tests or studies that are pending at the time of discharge | Determination is made of who will be reviewing the results, and when and how this information will be communicated to the patient/family
Mean 3.53 .112
95% Confidence Interval for Mean
Lower Bound 3.30 Upper Bound 3.75
5% Trimmed Mean 3.57 Median 4.00 Variance 1.252 Std. Deviation 1.119 Minimum 1 Maximum 5 Range 4 Interquartile Range 1 Skewness -.444 .243
Kurtosis -.652 .481
16 - Organization of post-discharge outpatient services and medical equipment if needed | If patient requires medical equipment on discharge, there is a process for ensuring that the medical equipment is obtained
Mean 4.71 .063
95% Confidence Interval for Mean
Lower Bound 4.58 Upper Bound 4.83
5% Trimmed Mean 4.80 Median 5.00 Variance .393 Std. Deviation .627 Minimum 1 Maximum 5 Range 4 Interquartile Range 0 Skewness -2.989 .243
Kurtosis 12.340 .481
17 - Organization of post-discharge outpatient services and medical equipment if needed | Before discharge, patient/family is given contact information for medical equipment companies, CCAC (as needed)
Mean 4.78 .057
95% Confidence Interval for Mean
Lower Bound 4.67 Upper Bound 4.89
5% Trimmed Mean 4.85 Median 5.00 Variance .317 Std. Deviation .563 Minimum 1 Maximum 5 Range 4 Interquartile Range 0 Skewness -3.853 .243
Kurtosis 20.557 .481
18 - Medication review | Medication list is reviewed with patient/family
Mean 4.68 .064
95% Confidence Interval for Mean
Lower Bound 4.55 Upper Bound 4.80
5% Trimmed Mean 4.77 Median 5.00 Variance .405 Std. Deviation .636
262
Minimum 2 Maximum 5 Range 3 Interquartile Range 1 Skewness -2.274 .243
Kurtosis 5.611 .481
19 - Medication review | Medication reconciliation is done at the time of discharge
Mean 4.31 .111
95% Confidence Interval for Mean
Lower Bound 4.09 Upper Bound 4.53
5% Trimmed Mean 4.44 Median 5.00 Variance 1.217 Std. Deviation 1.103 Minimum 1 Maximum 5 Range 4 Interquartile Range 1 Skewness -1.583 .243
Kurtosis 1.485 .481
20 - Medication review | Patient/family member is given an explanation of what medications to take and changes in the medication regimen are emphasized
Mean 4.48 .083
95% Confidence Interval for Mean
Lower Bound 4.32 Upper Bound 4.65
5% Trimmed Mean 4.61 Median 5.00 Variance .681 Std. Deviation .825 Minimum 1 Maximum 5 Range 4 Interquartile Range 1 Skewness -2.008 .243
Kurtosis 4.406 .481
21 - Medication review | Each medication’s purpose, administration and side effects are reviewed with patient/family
Mean 3.90 .107
95% Confidence Interval for Mean
Lower Bound 3.69 Upper Bound 4.11
5% Trimmed Mean 3.95 Median 4.00 Variance 1.133 Std. Deviation 1.064 Minimum 1 Maximum 5 Range 4 Interquartile Range 2 Skewness -.676 .243
Kurtosis -.543 .481
22 - Medication review | Patient/family’s concerns
Mean 4.18 .090
Lower Bound 4.00
263
about medication plan are assessed
95% Confidence Interval for Mean
Upper Bound 4.36
5% Trimmed Mean 4.27 Median 4.00 Variance .803 Std. Deviation .896 Minimum 1 Maximum 5 Range 4 Interquartile Range 1 Skewness -1.324 .243
Kurtosis 1.761 .481
23 - Medication review | Patient/family is referred to community pharmacist within 2 weeks of discharge for a medication review
Mean 2.53 .129
95% Confidence Interval for Mean
Lower Bound 2.27 Upper Bound 2.78
5% Trimmed Mean 2.47 Median 2.00 Variance 1.640 Std. Deviation 1.280 Minimum 1 Maximum 5 Range 4 Interquartile Range 2 Skewness .449 .243
Kurtosis -.768 .481
24 - Discharge plan is reconciled with use of guidelines | There is use of guidelines in the development and planning of the discharge process for patients and family members
Mean 3.66 .122
95% Confidence Interval for Mean
Lower Bound 3.41 Upper Bound 3.90
5% Trimmed Mean 3.73 Median 4.00 Variance 1.473 Std. Deviation 1.214 Minimum 1 Maximum 5 Range 4 Interquartile Range 2 Skewness -.742 .243
Kurtosis -.285 .481
25 - Written discharge plan | The patient/family is given an easy-to-understand written, prioritized discharge plan that includes medications, medical equipment, future appointments, and future diagnostic tests to take home
Mean 4.26 .107
95% Confidence Interval for Mean
Lower Bound 4.05 Upper Bound 4.48
5% Trimmed Mean 4.37 Median 5.00 Variance 1.134 Std. Deviation 1.065 Minimum 1 Maximum 5
264
Range 4 Interquartile Range 1 Skewness -1.426 .243
Kurtosis 1.098 .481
26 - Written discharge plan | The written plan is reviewed with the patient/family
Mean 4.21 .105
95% Confidence Interval for Mean
Lower Bound 4.00 Upper Bound 4.42
5% Trimmed Mean 4.31 Median 5.00 Variance 1.087 Std. Deviation 1.043 Minimum 1 Maximum 5 Range 4 Interquartile Range 1 Skewness -1.321 .243
Kurtosis .981 .481
27 - Written discharge plan | Patient/family is encouraged to ask questions about the plan
Mean 4.37 .088
95% Confidence Interval for Mean
Lower Bound 4.20 Upper Bound 4.55
5% Trimmed Mean 4.48 Median 5.00 Variance .767 Std. Deviation .876 Minimum 1 Maximum 5 Range 4 Interquartile Range 1 Skewness -1.650 .243
Kurtosis 2.718 .481
28 - Patient/family education of diagnosis | Prior to day of discharge, patient/family is met with to provide education about patient’s diagnosis and treatment and to prepare for discharge
Mean 3.62 .111
95% Confidence Interval for Mean
Lower Bound 3.40 Upper Bound 3.84
5% Trimmed Mean 3.67 Median 4.00 Variance 1.219 Std. Deviation 1.104 Minimum 1 Maximum 5 Range 4 Interquartile Range 1 Skewness -.580 .243
Kurtosis -.410 .481
29 - Patient/family’s understanding of the discharge plan is assessed | Patient/family is asked to explain, in their own words,
Mean 2.96 .113
95% Confidence Interval for Mean
Lower Bound 2.74 Upper Bound 3.18
5% Trimmed Mean 2.96
265
the details of the discharge plan
Median 3.00 Variance 1.264 Std. Deviation 1.124 Minimum 1 Maximum 5 Range 4 Interquartile Range 2 Skewness .345 .243
Kurtosis -.904 .481
30 - Patient/family’s understanding of the discharge plan is assessed | As needed, family members and other caregivers who will share in the care-giving responsibilities are contacted
Mean 3.97 .084
95% Confidence Interval for Mean
Lower Bound 3.80 Upper Bound 4.14
5% Trimmed Mean 4.02 Median 4.00 Variance .703 Std. Deviation .839 Minimum 2 Maximum 5 Range 3 Interquartile Range 0 Skewness -.896 .243
Kurtosis .646 .481
31 - Patient/family is instructed on what to do if a problem arises | Patient/family is made aware of how to contact primary care provider
Mean 4.13 .090
95% Confidence Interval for Mean
Lower Bound 3.95 Upper Bound 4.31
5% Trimmed Mean 4.21 Median 4.00 Variance .809 Std. Deviation .900 Minimum 1 Maximum 5 Range 4 Interquartile Range 1 Skewness -1.123 .243
Kurtosis 1.181 .481
32 - Patient/family is instructed on what to do if a problem arises | Patient/family is instructed on what constitutes an emergency and what to do in case of emergency
Mean 3.93 .103
95% Confidence Interval for Mean
Lower Bound 3.73 Upper Bound 4.13
5% Trimmed Mean 3.98 Median 4.00 Variance 1.046 Std. Deviation 1.023 Minimum 2 Maximum 5 Range 3 Interquartile Range 2 Skewness -.615 .243
266
Kurtosis -.734 .481
33 - Discharge summary is sent to clinicians accepting care of the patient | Discharge summary has a standardized format so that information is easy to find
Mean 4.39 .097
95% Confidence Interval for Mean
Lower Bound 4.20 Upper Bound 4.59
5% Trimmed Mean 4.52 Median 5.00 Variance .935 Std. Deviation .967 Minimum 1 Maximum 5 Range 4 Interquartile Range 1 Skewness -1.697 .243
Kurtosis 2.408 .481
34 - Discharge summary is sent to clinicians accepting care of the patient | Within 24 hours of discharge, a discharge summary is provided to primary care provider
Mean 3.60 .129
95% Confidence Interval for Mean
Lower Bound 3.34 Upper Bound 3.85
5% Trimmed Mean 3.66 Median 4.00 Variance 1.651 Std. Deviation 1.285 Minimum 1 Maximum 5 Range 4 Interquartile Range 2 Skewness -.527 .243
Kurtosis -.805 .481
35 - Telephone reinforcement of the discharge plan is provided | Patient/family is called within 3 days of discharge to reinforce the discharge plan and help with problem-solving
Mean 2.21 .138
95% Confidence Interval for Mean
Lower Bound 1.94 Upper Bound 2.49
5% Trimmed Mean 2.12 Median 2.00 Variance 1.883 Std. Deviation 1.372 Minimum 1 Maximum 5 Range 4 Interquartile Range 2 Skewness .841 .243
Kurtosis -.604 .481
36 - Telephone reinforcement of the discharge plan is provided | Patient/family is provided with a phone number where they can speak with a hospital staff member to ask questions about the at-home
Mean 2.54 .152
95% Confidence Interval for Mean
Lower Bound 2.23 Upper Bound 2.84
5% Trimmed Mean 2.48 Median 2.00 Variance 2.292
267
care plan, hospitalization, and follow-up plan in order to
Std. Deviation 1.514 Minimum 1 Maximum 5 Range 4 Interquartile Range 3 Skewness .489 .243
Kurtosis -1.257 .481
268
Appendix S: Tests of Normality of Survey Items
Tests of Normality
Kolmogorov-Smirnova Shapiro-Wilk
Statistic df Sig. Statistic df Sig.
1 - Language assistance | Language preference of patient and family is determined and documented
.283 99 .000 .749 99 .000
2 - Language assistance | If patient and/or family members do not speak English, a translator is arranged for on discharge
.241 99 .000 .875 99 .000
3 - Language assistance | If patient and/or family members do not speak English, written materials are provided in the preferred language
.268 99 .000 .874 99 .000
4 - Post-discharge appointments, tests | Need for primary care and specialty care follow-up is determined
.370 99 .000 .628 99 .000
5 - Post-discharge appointments, tests | If patient does not have a primary care provider, one is located for patients
.219 99 .000 .899 99 .000
6 - Post-discharge appointments, tests | Need for future tests is determined
.305 99 .000 .673 99 .000
7 - Post-discharge appointments, tests | Appointments are made for patient for follow-up appointments and testing
.297 99 .000 .732 99 .000
8 - Post-discharge appointments, tests | If appointments are made, they are made with input from the patient/family regarding the best time/date
.220 99 .000 .892 99 .000
9 - Post-discharge appointments, tests continued | If patient requires future diagnostic testing, patient and/or family member is instructed on any preparation for testing
.292 99 .000 .795 99 .000
269
10 - Post-discharge appointments, tests continued | Importance of clinician appointments and further testing is discussed with patient/family
.273 99 .000 .695 99 .000
11 - Post-discharge appointments, tests continued | Patient/family is asked about traditional healing practices, and there is confirmation made that practices are complementary with patient’s discharge plan
.268 99 .000 .871 99 .000
12 - Post-discharge appointments, tests continued | There is confirmation made with patient/family that they know where to go for further appointments and tests, and that they have a plan to get to appointments
.294 99 .000 .818 99 .000
13 - Post-discharge appointments, tests continued | Barriers to keeping appointments are addressed
.255 99 .000 .881 99 .000
14 - Plan for follow-up of results from diagnostic lab tests or studies that are pending at the time of discharge | Pending lab and test results are identified with patient/family
.290 99 .000 .845 99 .000
15 - Plan for follow-up of results from diagnostic lab tests or studies that are pending at the time of discharge | Determination is made of who will be reviewing the results, and when and how this information will be communicated to the patient/family
.240 99 .000 .890 99 .000
16 - Organization of post-discharge outpatient services and medical equipment if needed | If patient requires medical equipment on discharge, there is a process for ensuring that the medical equipment is obtained
.448 99 .000 .515 99 .000
270
17 - Organization of post-discharge outpatient services and medical equipment if needed | Before discharge, patient/family is given contact information for medical equipment companies, CCAC (as needed)
.472 99 .000 .432 99 .000
18 - Medication review | Medication list is reviewed with patient/family
.442 99 .000 .559 99 .000
19 - Medication review | Medication reconciliation is done at the time of discharge
.370 99 .000 .669 99 .000
20 - Medication review | Patient/family member is given an explanation of what medications to take and changes in the medication regimen are emphasized
.360 99 .000 .647 99 .000
21 - Medication review | Each medication’s purpose, administration and side effects are reviewed with patient/family
.235 99 .000 .844 99 .000
22 - Medication review | Patient/family’s concerns about medication plan are assessed
.288 99 .000 .761 99 .000
23 - Medication review | Patient/family is referred to community pharmacist within 2 weeks of discharge for a medication review
.174 99 .000 .884 99 .000
24 - Discharge plan is reconciled with use of guidelines | There is use of guidelines in the development and planning of the discharge process for patients and family members
.238 99 .000 .863 99 .000
271
25 - Written discharge plan | The patient/family is given an easy-to-understand written, prioritized discharge plan that includes medications, medical equipment, future appointments, and future diagnostic tests to take home
.331 99 .000 .713 99 .000
26 - Written discharge plan | The written plan is reviewed with the patient/family
.300 99 .000 .748 99 .000
27 - Written discharge plan | Patient/family is encouraged to ask questions about the plan
.318 99 .000 .706 99 .000
28 - Patient/family education of diagnosis | Prior to day of discharge, patient/family is met with to provide education about patient’s diagnosis and treatment and to prepare for discharge
.252 99 .000 .881 99 .000
29 - Patient/family’s understanding of the discharge plan is assessed | Patient/family is asked to explain, in their own words, the details of the discharge plan
.238 99 .000 .882 99 .000
30 - Patient/family’s understanding of the discharge plan is assessed | As needed, family members and other caregivers who will share in the care-giving responsibilities are contacted
.333 99 .000 .787 99 .000
31 - Patient/family is instructed on what to do if a problem arises | Patient/family is made aware of how to contact primary care provider
.270 99 .000 .798 99 .000
32 - Patient/family is instructed on what to do if a problem arises | Patient/family is instructed on what constitutes an emergency and what to do in case of emergency
.235 99 .000 .833 99 .000
272
33 - Discharge summary is sent to clinicians accepting care of the patient | Discharge summary has a standardized format so that information is easy to find
.371 99 .000 .674 99 .000
34 - Discharge summary is sent to clinicians accepting care of the patient | Within 24 hours of discharge, a discharge summary is provided to primary care provider
.189 99 .000 .870 99 .000
35 - Telephone reinforcement of the discharge plan is provided | Patient/family is called within 3 days of discharge to reinforce the discharge plan and help with problem-solving
.246 99 .000 .803 99 .000
36 - Telephone reinforcement of the discharge plan is provided | Patient/family is provided with a phone number where they can speak with a hospital staff member to ask questions about the at-home care plan, hospitalization, and follow-up plan
.214 99 .000 .828 99 .000
a. Lilliefors Significance Correction
346
Appendix T: Data Dictionary
Survey items
Language Assistance
1 LApref Language preference of patient and family is determined and documented
2 LAtransl If patient and/or family members do not speak English, a translator is arranged for on discharge
3 LAwrit If patient and/or family members do not speak English, written materials are provided in the preferred
language
Post-discharge appointments, tests
4 PDdeterm Need for primary care and specialty care follow-up is determined
5 PDpcp If patient does not have a primary care provider, one is located for patients
6 PDfuture Need for future tests is determined
7 PDapp Appointments are made for patient for follow-up appointments and testing
8 PDinput If appointments are made, they are made with input from the patient/family regarding the best time/date
9 PDtest If patient requires future diagnostic testing, patient and/or family member is instructed on any preparation
for testing
10 PDimport Importance of clinician appointments and further testing is discussed with patient/family
11 PTtrad Patient/family is asked about traditional healing practices, and there is confirmation made that practices are
complementary with patient’s discharge plan
12 PDconf There is confirmation made with patient/family that they know where to go for further appointments and
tests, and that they have a plan to get to appointments
13 PDbarr Barriers to keeping appointments are addressed
Plan for follow-up of results from diagnostic lab tests or studies that are pending at the time of discharge
14 FUpend Pending lab and test results are identified with patient/family
15 FUwho Determination is made of who will be reviewing the results, and when and how this information will be
communicated to the patient/family
Organization of post-discharge outpatient services and medical equipment if needed
16 ORGequip If patient requires medical equipment on discharge, there is a process for ensuring that the medical
equipment is obtained
17 ORGcont Before discharge, patient/family is given contact information for medical equipment companies, at-home
services (as needed)
347
Medication review
18 MEDlist Medication list is reviewed with patient/family
19 MEDrec Medication reconciliation is done at the time of discharge
20 MEDexp Patient/family member is given an explanation of what medications to take and changes in the medication
regimen are emphasized
21 MEDrev Each medication’s purpose, administration and side effects are reviewed with patient/family
22 MEDcon Patient/family’s concerns about medication plan are assessed
23 MEDpharm Patient/family is referred to community pharmacist within 2 weeks of discharge for a medication review
Discharge plan is reconciled with use of guidelines
24 GL There is use of guidelines in the development and planning of the discharge process for patients and family
members
Written discharge plan
25 WRplan The patient/family is given an easy-to-understand written, prioritized discharge plan that includes
medications, medical equipment, future appointments, and future diagnostic tests to take home
26 WRrev The written plan is reviewed with the patient/family
27 WRask Patient/family is encouraged to ask questions about the plan
Patient/family education of diagnosis
28 EDUC Prior to day of discharge, patient/family is met with to provide education about patient’s diagnosis and
treatment and to prepare for discharge
Patient/family’s understanding of the discharge plan is assessed
29 UNdetail Patient/family is asked to explain, in their own words, the details of the discharge plan
30 UNfam As needed, family members and other caregivers who will share in the care-giving responsibilities are
contacted
Patient/family is instructed on what to do if a problem arises
31 PROBpcp Patient/family is made aware of how to contact primary care provider
32 PROBer Patient/family is instructed on what constitutes an emergency and what to do in case of emergency
Discharge summary is sent to clinicians accepting care of the patient
33 SUMformat Discharge summary has a standardized format so that information is easy to find
34 SUMpcp Within 24 hours of discharge, a discharge summary is provided to primary care provider
Telephone reinforcement of the discharge plan is provided
35 TRthree Patient/family is called within 3 days of discharge to reinforce the discharge plan and help with problem-
solving
348
36 TRphone Patient/family is provided with a phone number where they can speak with a hospital staff member to ask
questions about the at-home care plan, hospitalization, and follow-up plan in order to help patients transition
from hospital care to outpatient care setting
349
Appendix U: Correlations between Survey Items
LApref LAtransl LAwrit PDdeterm PDpcp PDfuture PDapp PDinput PDtest PDimport PPDtrad
Lapref Correlation Coefficient
1.000 .333** .340** .230* .284** .109 .223* .217* .263** .283** .397**
Sig. (2-tailed)
.001 .001 .022 .004 .285 .026 .031 .009 .005 .000
N 99 99 99 99 99 99 99 99 99 99 99
LAtransl Correlation Coefficient
.333** 1.000 .252* .152 .238* .130 .273** .238* .387** .292** .263**
Sig. (2-tailed)
.001 .012 .132 .018 .200 .006 .018 .000 .003 .009
N 99 99 99 99 99 99 99 99 99 99 99
LAwrit Correlation Coefficient
.340** .252* 1.000 .152 .240* .018 .140 .153 .124 .047 .233*
Sig. (2-tailed)
.001 .012 .132 .017 .859 .166 .129 .221 .645 .020
N 99 99 99 99 99 99 99 99 99 99 99
PDdeterm Correlation Coefficient
.230* .152 .152 1.000 .293** .675** .436** .305** .389** .366** .405**
Sig. (2-tailed)
.022 .132 .132 .003 .000 .000 .002 .000 .000 .000
N 99 99 99 99 99 99 99 99 99 99 99
PDpcp Correlation Coefficient
.284** .238* .240* .293** 1.000 .298** .395** .409** .166 .273** .380**
350
Sig. (2-tailed)
.004 .018 .017 .003 .003 .000 .000 .100 .006 .000
N 99 99 99 99 99 99 99 99 99 99 99
PDfuture Correlation Coefficient
.109 .130 .018 .675** .298** 1.000 .494** .406** .379** .348** .262**
Sig. (2-tailed)
.285 .200 .859 .000 .003 .000 .000 .000 .000 .009
N 99 99 99 99 99 99 99 99 99 99 99
LApref LAtransl LAwrit PDdeterm PDpcp PDfuture PDapp PDinput PDtest PDimport PDtrad
PDapp Correlation Coefficient
.223* .273** .140 .436** .395** .494** 1.000 .487** .431** .431** .327**
Sig. (2-tailed)
.026 .006 .166 .000 .000 .000 .000 .000 .000 .001
N 99 99 99 99 99 99 99 99 99 99 99
PDinput Correlation Coefficient
.217* .238* .153 .305** .409** .406** .487** 1.000 .440** .329** .315**
Sig. (2-tailed)
.031 .018 .129 .002 .000 .000 .000 .000 .001 .002
N 99 99 99 99 99 99 99 99 99 99 99
PDtest Correlation Coefficient
.263** .387** .124 .389** .166 .379** .431** .440** 1.000 .423** .330**
351
Sig. (2-tailed)
.009 .000 .221 .000 .100 .000 .000 .000 .000 .001
N 99 99 99 99 99 99 99 99 99 99 99
PDimport Correlation Coefficient
.283** .292** .047 .366** .273** .348** .431** .329** .423** 1.000 .365**
Sig. (2-tailed)
.005 .003 .645 .000 .006 .000 .000 .001 .000 .000
N 99 99 99 99 99 99 99 99 99 99 99
PTtrad Correlation Coefficient
.397** .263** .233* .405** .380** .262** .327** .315** .330** .365** 1.000
Sig. (2-tailed)
.000 .009 .020 .000 .000 .009 .001 .002 .001 .000
N 99 99 99 99 99 99 99 99 99 99 99
PDconf Correlation Coefficient
.289** .339** .165 .363** .237* .289** .446** .424** .640** .460** .415**
Sig. (2-tailed)
.004 .001 .104 .000 .018 .004 .000 .000 .000 .000 .000
N 99 99 99 99 99 99 99 99 99 99 99
PDbarr Correlation Coefficient
.273** .317** .203* .390** .386** .261** .420** .404** .593** .451** .480**
Sig. (2-tailed)
.006 .001 .044 .000 .000 .009 .000 .000 .000 .000 .000
N 99 99 99 99 99 99 99 99 99 99 99
LApref LAtransl LAwrit PDdeterm PDpcp PDfuture PDapp PDinput PDtest PDimport PDtrad
352
FUpend Correlation Coefficient
.405** .233* .168 .419** .336** .271** .341** .426** .428** .399** .374**
Sig. (2-tailed)
.000 .021 .096 .000 .001 .007 .001 .000 .000 .000 .000
N 99 99 99 99 99 99 99 99 99 99 99
FUwho Correlation Coefficient
.432** .330** .270** .374** .256* .254* .366** .438** .466** .463** .383**
Sig. (2-tailed)
.000 .001 .007 .000 .011 .011 .000 .000 .000 .000 .000
N 99 99 99 99 99 99 99 99 99 99 99
ORGequip Correlation Coefficient
.237* .181 .081 .315** .187 .265** .341** .291** .392** .361** .142
Sig. (2-tailed)
.018 .074 .427 .002 .064 .008 .001 .003 .000 .000 .161
N 99 99 99 99 99 99 99 99 99 99 99
ORGcont Correlation Coefficient
.134 .186 .040 .309** .254* .246* .339** .196 .302** .398** .090
Sig. (2-tailed)
.186 .065 .698 .002 .011 .014 .001 .052 .002 .000 .377
N 99 99 99 99 99 99 99 99 99 99 99
MEDlist Correlation Coefficient
.139 .129 .080 .460** .231* .301** .345** .350** .345** .554** .339**
Sig. (2-tailed)
.170 .202 .431 .000 .022 .002 .000 .000 .000 .000 .001
N 99 99 99 99 99 99 99 99 99 99 99
353
MEDrec Correlation Coefficient
.069 .095 .030 .301** .176 .296** .222* .133 .244* .362** .216*
Sig. (2-tailed)
.496 .351 .770 .002 .082 .003 .027 .191 .015 .000 .032
N 99 99 99 99 99 99 99 99 99 99 99
MEDexp Correlation Coefficient
.136 .162 .065 .371** .208* .282** .268** .129 .302** .495** .363**
Sig. (2-tailed)
.179 .108 .521 .000 .038 .005 .007 .204 .002 .000 .000
N
99
99 99 99 99 99 99 99 99 99 99
LApref LAtransl LAwrit PDdeterm PDpcp PDfuture PDapp PDinput PDtest PDimport
PDtrad
MEDrev Correlation Coefficient
.195 .172 .203* .323** .341** .280** .167 .082 .157 .368** .295**
Sig. (2-tailed)
.053 .088 .044 .001 .001 .005 .098 .421 .120 .000 .003
N 99 99 99 99 99 99 99 99 99 99 99
MEDcon Correlation Coefficient
.255* .194 .263** .359** .314** .286** .274** .162 .290** .321** .324**
354
Sig. (2-tailed)
.011 .055 .009 .000 .002 .004 .006 .108 .004 .001 .001
N 99 99 99 99 99 99 99 99 99 99 99
MEDpharm Correlation Coefficient
.278** .199* .261** .177 .402** .073 .195 .223* .007 .245* .293**
Sig. (2-tailed)
.005 .049 .009 .079 .000 .473 .054 .027 .949 .014 .003
N 99 99 99 99 99 99 99 99 99 99 99
GL Correlation Coefficient
.110 .184 .085 .279** .172 .246* .216* .153 .229* .243* .322**
Sig. (2-tailed)
.280 .068 .403 .005 .088 .014 .032 .131 .023 .015 .001
N 99 99 99 99 99 99 99 99 99 99 99
WRplan Correlation Coefficient
-.038 .158 -.020 .183 .084 .185 .269** -.049 .198* .299** .074
Sig. (2-tailed)
.708 .119 .846 .070 .411 .067 .007 .629 .050 .003 .469
N 99 99 99 99 99 99 99 99 99 99 99
WRrev Correlation Coefficient
-.005 .223* .037 .200* .151 .166 .274** -.025 .172 .397** .122
Sig. (2-tailed)
.962 .026 .716 .047 .135 .100 .006 .803 .088 .000 .228
N 99 99 99 99 99 99 99 99 99 99 99
WRask Correlation Coefficient
.072 .185 .042 .333** .135 .298** .317** .066 .331** .419** .282**
355
Sig. (2-tailed)
.477 .066 .681 .001 .183 .003 .001 .515 .001 .000 .005
N
99 99 99 99 99 99 99 99 99 99 99
LApref LAtransl LAwrit PDdeterm PDpcp PDfuture PDapp PDinput PDtest PDimport PDtrad
EDUC Correlation Coefficient
.279** .144 .225* .413** .371** .310** .199* .175 .282** .460** .428**
Sig. (2-tailed)
.005 .156 .025 .000 .000 .002 .048 .083 .005 .000 .000
N 99 99 99 99 99 99 99 99 99 99 99
UNdetail Correlation Coefficient
.216* .317** .228* .220* .292** .157 .175 .169 .362** .409** .357**
Sig. (2-tailed)
.032 .001 .023 .028 .003 .120 .083 .095 .000 .000 .000
N 99 99 99 99 99 99 99 99 99 99 99
UNfam Correlation Coefficient
.171 .135 .061 .337** .183 .285** .239* .163 .321** .251* .117
Sig. (2-tailed)
.090 .182 .551 .001 .070 .004 .017 .107 .001 .012 .250
N 99 99 99 99 99 99 99 99 99 99 99
PROBpcp Correlation Coefficient
.276** .224* .224* .373** .286** .268** .280** .157 .395** .494** .285**
Sig. (2-tailed)
.006 .025 .026 .000 .004 .007 .005 .120 .000 .000 .004
356
N 99 99 99 99 99 99 99 99 99 99 99
PROBer Correlation Coefficient
.293** .295** .282** .290** .255* .278** .217* .138 .312** .380** .391**
Sig. (2-tailed)
.003 .003 .005 .004 .011 .005 .031 .174 .002 .000 .000
N 99 99 99 99 99 99 99 99 99 99 99
SUMformat Correlation Coefficient
.169 .096 -.040 .186 .058 .300** .240* .106 .188 .311** .025
Sig. (2-tailed)
.094 .343 .697 .065 .567 .003 .017 .295 .062 .002 .808
N 99 99 99 99 99 99 99 99 99 99 99
SUMpcp Correlation Coefficient
.117 .044 -.076 .059 .129 .122 .209* -.009 .117 .210* .016
Sig. (2-tailed)
.249 .669 .453 .562 .204 .227 .038 .927 .250 .037 .873
N
99 99 99 99 99 99 99 99 99 99 99
LApref LAtransl LAwrit PDdeterm PDpcp PDfuture PDapp PDinput PDtest PDimport PDtrad
TRthree Correlation Coefficient
.332** -.021 .046 .239* .174 .189 .097 .167 .243* .155 .283**
Sig. (2-tailed)
.001 .833 .650 .017 .086 .061 .338 .098 .015 .126 .005
N 99 99 99 99 99 99 99 99 99 99 99
357
TRphone Correlation Coefficient
.344** .027 .144 .215* .187 .199* .214* .150 .249* .241* .374**
Sig. (2-tailed)
.000 .792 .156 .033 .065 .049 .033 .137 .013 .016 .000
N 99 99 99 99 99 99 99 99 99 99 99
*p < .05 **p < .01
358
PDconf PDbarr FUpend FUwho ORGequip ORGcont MEDlist MEDrec MEDexp MEDrev MEDcon
Lapref .289** .273** .405** .432** .237* .134 .139 .069 .136 .195 .255*
Sig. (2-tailed)
.004 .006 .000 .000 .018 .186 .170 .496 .179 .053 .011
N 99 99 99 99 99 99 99 99 99 99 99
LAtransl Correlation Coefficient
.339** .317** .233* .330** .181 .186 .129 .095 .162 .172 .194
Sig. (2-tailed)
.001 .001 .021 .001 .074 .065 .202 .351 .108 .088 .055
N 99 99 99 99 99 99 99 99 99 99 99
LAwrit Correlation Coefficient
.165 .203* .168 .270** .081 .040 .080 .030 .065 .203* .263**
Sig. (2-tailed)
.104 .044 .096 .007 .427 .698 .431 .770 .521 .044 .009
N 99 99 99 99 99 99 99 99 99 99 99
PDdeterm Correlation Coefficient
.363** .390** .419** .374** .315** .309** .460** .301** .371** .323** .359**
Sig. (2-tailed)
.000 .000 .000 .000 .002 .002 .000 .002 .000 .001 .000
N 99 99 99 99 99 99 99 99 99 99 99
PDpcp Correlation Coefficient
.237* .386** .336** .256* .187 .254* .231* .176 .208* .341** .314**
Sig. (2-tailed)
.018 .000 .001 .011 .064 .011 .022 .082 .038 .001 .002
N 99 99 99 99 99 99 99 99 99 99 99
359
PDfuture Correlation Coefficient
.289** .261** .271** .254* .265** .246* .301** .296** .282** .280** .286**
Sig. (2-tailed)
.004 .009 .007 .011 .008 .014 .002 .003 .005 .005 .004
N 99 99 99 99 99 99 99 99 99 99 99
PDapp Correlation Coefficient
.446** .420** .341** .366** .341** .339** .345** .222* .268** .167 .274**
Sig. (2-tailed)
.000 .000 .001 .000 .001 .001 .000 .027 .007 .098 .006
N
99 99 99 99 99 99 99 99 99 99 99
PDconf PDbarr FUpend FUwho ORGequip ORGcont MEDlist MEDrec MEDexp MEDrev MEDcon
PDinput Correlation Coefficient
.424** .404** .426** .438** .291** .196 .350** .133 .129 .082 .162
Sig. (2-tailed)
.000 .000 .000 .000 .003 .052 .000 .191 .204 .421 .108
N 99 99 99 99 99 99 99 99 99 99 99
PDtest Correlation Coefficient
.640** .593** .428** .466** .392** .302** .345** .244* .302** .157 .290**
Sig. (2-tailed)
.000 .000 .000 .000 .000 .002 .000 .015 .002 .120 .004
N 99 99 99 99 99 99 99 99 99 99 99
PDimport Correlation Coefficient
.460** .451** .399** .463** .361** .398** .554** .362** .495** .368** .321**
360
Sig. (2-tailed)
.000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .001
N 99 99 99 99 99 99 99 99 99 99 99
PTtrad Correlation Coefficient
.415** .480** .374** .383** .142 .090 .339** .216* .363** .295** .324**
Sig. (2-tailed)
.000 .000 .000 .000 .161 .377 .001 .032 .000 .003 .001
N 99 99 99 99 99 99 99 99 99 99 99
PDconf Correlation Coefficient
1.000 .750** .434** .496** .367** .288** .413** .233* .397** .385** .392**
Sig. (2-tailed)
.000 .000 .000 .000 .004 .000 .020 .000 .000 .000
N 99 99 99 99 99 99 99 99 99 99 99
PDbarr Correlation Coefficient
.750** 1.000 .420** .455** .369** .319** .434** .222* .391** .360** .368**
Sig. (2-tailed)
.000 .000 .000 .000 .001 .000 .027 .000 .000 .000
N 99 99 99 99 99 99 99 99 99 99 99
FUpend Correlation Coefficient
.434** .420** 1.000 .730** .310** .211* .349** .194 .278** .304** .292**
Sig. (2-tailed)
.000 .000 .000 .002 .036 .000 .055 .005 .002 .003
N
99 99 99 99 99 99 99 99 99 99 99
PDconf PDbarr FUpend FUwho ORGequip ORGcont MEDlist MEDrec MEDexp MEDrev MEDcon
361
FUwho Correlation Coefficient
.496** .455** .730** 1.000 .382** .238* .349** .143 .323** .323** .369**
Sig. (2-tailed)
.000 .000 .000 .000 .017 .000 .159 .001 .001 .000
N 99 99 99 99 99 99 99 99 99 99 99
ORGequip Correlation Coefficient
.367** .369** .310** .382** 1.000 .667** .297** .137 .280** .240* .372**
Sig. (2-tailed)
.000 .000 .002 .000 .000 .003 .175 .005 .017 .000
N 99 99 99 99 99 99 99 99 99 99 99
ORGcont Correlation Coefficient
.288** .319** .211* .238* .667** 1.000 .407** .207* .367** .236* .233*
Sig. (2-tailed)
.004 .001 .036 .017 .000 .000 .040 .000 .019 .021
N 99 99 99 99 99 99 99 99 99 99 99
MEDlist Correlation Coefficient
.413** .434** .349** .349** .297** .407** 1.000 .598** .686** .539** .539**
Sig. (2-tailed)
.000 .000 .000 .000 .003 .000 .000 .000 .000 .000
N 99 99 99 99 99 99 99 99 99 99 99
MEDrec Correlation Coefficient
.233* .222* .194 .143 .137 .207* .598** 1.000 .632** .565** .586**
Sig. (2-tailed)
.020 .027 .055 .159 .175 .040 .000 .000 .000 .000
N 99 99 99 99 99 99 99 99 99 99 99
362
MEDexp Correlation Coefficient
.397** .391** .278** .323** .280** .367** .686** .632** 1.000 .711** .659**
Sig. (2-tailed)
.000 .000 .005 .001 .005 .000 .000 .000 .000 .000
N 99 99 99 99 99 99 99 99 99 99 99
MEDrev Correlation Coefficient
.385** .360** .304** .323** .240* .236* .539** .565** .711** 1.000 .768**
Sig. (2-tailed)
.000 .000 .002 .001 .017 .019 .000 .000 .000 .000
N
99 99 99 99 99 99 99 99 99 99 99
PDconf PDbarr FUpend FUwho ORGequip ORGcont MEDlist MEDrec MEDexp MEDrev MEDcon
MEDcon Correlation Coefficient
.392** .368** .292** .369** .372** .233* .539** .586** .659** .768** 1.000
Sig. (2-tailed)
.000 .000 .003 .000 .000 .021 .000 .000 .000 .000
N 99 99 99 99 99 99 99 99 99 99 99
MEDpharm Correlation Coefficient
.205* .316** .177 .197 .170 .158 .266** .209* .278** .427** .340**
Sig. (2-tailed)
.042 .001 .079 .051 .092 .119 .008 .038 .005 .000 .001
N 99 99 99 99 99 99 99 99 99 99 99
GL Correlation Coefficient
.444** .365** .253* .188 .191 .146 .367** .297** .434** .400** .432**
363
Sig. (2-tailed)
.000 .000 .011 .062 .058 .149 .000 .003 .000 .000 .000
N 99 99 99 99 99 99 99 99 99 99 99
WRplan Correlation Coefficient
.246* .129 .175 .238* .200* .287** .353** .377** .477** .322** .383**
Sig. (2-tailed)
.014 .204 .083 .018 .047 .004 .000 .000 .000 .001 .000
N 99 99 99 99 99 99 99 99 99 99 99
WRrev Correlation Coefficient
.267** .117 .250* .315** .212* .310** .365** .365** .435** .387** .411**
Sig. (2-tailed)
.008 .249 .013 .002 .035 .002 .000 .000 .000 .000 .000
N 99 99 99 99 99 99 99 99 99 99 99
WRask Correlation Coefficient
.361** .257* .327** .388** .301** .277** .441** .428** .532** .456** .607**
Sig. (2-tailed)
.000 .010 .001 .000 .002 .006 .000 .000 .000 .000 .000
N 99 99 99 99 99 99 99 99 99 99 99
EDUC Correlation Coefficient
.466** .476** .418** .460** .384** .366** .436** .355** .511** .515** .463**
Sig. (2-tailed)
.000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000
N
99 99 99 99 99 99 99 99 99 99 99
PDconf PDbarr FUpend FUwho ORGequip ORGcont MEDlist MEDrec MEDexp MEDrev MEDcon
364
UNdetail Correlation Coefficient
.415** .423** .317** .334** .277** .217* .347** .366** .387** .388** .476**
Sig. (2-tailed)
.000 .000 .001 .001 .005 .031 .000 .000 .000 .000 .000
N 99 99 99 99 99 99 99 99 99 99 99
UNfam Correlation Coefficient
.341** .228* .335** .375** .343** .355** .247* .167 .162 .152 .193
Sig. (2-tailed)
.001 .023 .001 .000 .001 .000 .014 .098 .108 .133 .056
N 99 99 99 99 99 99 99 99 99 99 99
PROBpcp Correlation Coefficient
.466** .357** .394** .408** .407** .392** .408** .345** .498** .418** .435**
Sig. (2-tailed)
.000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000
N 99 99 99 99 99 99 99 99 99 99 99
PROBer Correlation Coefficient
.345** .328** .437** .500** .332** .228* .312** .288** .393** .469** .518**
Sig. (2-tailed)
.000 .001 .000 .000 .001 .023 .002 .004 .000 .000 .000
N 99 99 99 99 99 99 99 99 99 99 99
SUMformat Correlation Coefficient
.173 .060 .251* .289** .131 .142 .269** .295** .395** .343** .312**
Sig. (2-tailed)
.087 .553 .012 .004 .198 .160 .007 .003 .000 .001 .002
N 99 99 99 99 99 99 99 99 99 99 99
365
SUMpcp Correlation Coefficient
.228* .163 .310** .292** .147 .205* .121 .197 .290** .303** .182
Sig. (2-tailed)
.023 .106 .002 .003 .146 .041 .233 .050 .004 .002 .072
N 99 99 99 99 99 99 99 99 99 99 99
TRthree Correlation Coefficient
.265** .235* .273** .232* .081 .079 .054 .092 .121 .083 .140
Sig. (2-tailed)
.008 .019 .006 .021 .423 .435 .596 .368 .235 .416 .168
N
99 99 99 99 99 99 99 99 99 99 99
PDconf PDbarr FUpend FUwho ORGequip ORGcont MEDlist MEDrec MEDexp MEDrev MEDcon
TRphone Correlation Coefficient
.326** .288** .358** .282** .156 .205* .174 .223* .315** .305** .301**
Sig. (2-tailed)
.001 .004 .000 .005 .123 .042 .085 .026 .001 .002 .002
N 99 99 99 99 99 99 99 99 99 99 99
*p < .05 **p < .01
366
MEDpharm GL WRplan WRrev WRask EDUC UNdetail UNfam PROBpcp PROBer
Lapref Correlation
Coefficient
.278** .110 -.038 -.005 .072 .279** .216* .171 .276** .293**
Sig. (2-tailed)
.005 .280 .708 .962 .477 .005 .032 .090 .006 .003
N 99 99 99 99 99 99 99 99 99 99
LAtransl Correlation Coefficient
.199* .184 .158 .223* .185 .144 .317** .135 .224* .295**
Sig. (2-tailed)
.049 .068 .119 .026 .066 .156 .001 .182 .025 .003
N 99 99 99 99 99 99 99 99 99 99
LAwrit Correlation Coefficient
.261** .085 -.020 .037 .042 .225* .228* .061 .224* .282**
Sig. (2-tailed)
.009 .403 .846 .716 .681 .025 .023 .551 .026 .005
N 99 99 99 99 99 99 99 99 99 99
PDdeterm Correlation Coefficient
.177 .279** .183 .200* .333** .413** .220* .337** .373** .290**
Sig. (2-tailed)
.079 .005 .070 .047 .001 .000 .028 .001 .000 .004
N 99 99 99 99 99 99 99 99 99 99
PDpcp Correlation Coefficient
.402** .172 .084 .151 .135 .371** .292** .183 .286** .255*
367
Sig. (2-tailed)
.000 .088 .411 .135 .183 .000 .003 .070 .004 .011
N 99 99 99 99 99 99 99 99 99 99
PDfuture Correlation Coefficient
.073 .246* .185 .166 .298** .310** .157 .285** .268** .278**
Sig. (2-tailed)
.473 .014 .067 .100 .003 .002 .120 .004 .007 .005
N 99 99 99 99 99 99 99 99 99 99
PDapp Correlation Coefficient
.195 .216* .269** .274** .317** .199* .175 .239* .280** .217*
Sig. (2-tailed)
.054 .032 .007 .006 .001 .048 .083 .017 .005 .031
N 99 99 99 99 99 99 99 99 99 99
MEDpharm GL WRplan WRrev WRask EDUC UNdetail UNfam PROBpcp PROBer
PDinput Correlation Coefficient
.223* .153 -.049 -.025 .066 .175 .169 .163 .157 .138
Sig. (2-tailed)
.027 .131 .629 .803 .515 .083 .095 .107 .120 .174
N 99 99 99 99 99 99 99 99 99 99
PDtest Correlation Coefficient
.007 .229* .198* .172 .331** .282** .362** .321** .395** .312**
Sig. (2-tailed)
.949 .023 .050 .088 .001 .005 .000 .001 .000 .002
368
N 99 99 99 99 99 99 99 99 99 99
PDimport Correlation Coefficient
.245* .243* .299** .397** .419** .460** .409** .251* .494** .380**
Sig. (2-tailed)
.014 .015 .003 .000 .000 .000 .000 .012 .000 .000
N 99 99 99 99 99 99 99 99 99 99
PTtrad Correlation Coefficient
.293** .322** .074 .122 .282** .428** .357** .117 .285** .391**
Sig. (2-tailed)
.003 .001 .469 .228 .005 .000 .000 .250 .004 .000
N 99 99 99 99 99 99 99 99 99 99
PDconf Correlation Coefficient
.205* .444** .246* .267** .361** .466** .415** .341** .466** .345**
Sig. (2-tailed)
.042 .000 .014 .008 .000 .000 .000 .001 .000 .000
N 99 99 99 99 99 99 99 99 99 99
PDbarr Correlation Coefficient
.316** .365** .129 .117 .257* .476** .423** .228* .357** .328**
Sig. (2-tailed)
.001 .000 .204 .249 .010 .000 .000 .023 .000 .001
N 99 99 99 99 99 99 99 99 99 99
FUpend Correlation Coefficient
.177 .253* .175 .250* .327** .418** .317** .335** .394** .437**
369
Sig. (2-tailed)
.079 .011 .083 .013 .001 .000 .001 .001 .000 .000
N 99 99 99 99 99 99 99 99 99 99
MEDpharm GL WRplan WRrev WRask EDUC UNdetail UNfam PROBpcp PROBer
FUwho Correlation Coefficient
.197 .188 .238* .315** .388** .460** .334** .375** .408** .500**
Sig. (2-tailed)
.051 .062 .018 .002 .000 .000 .001 .000 .000 .000
N 99 99 99 99 99 99 99 99 99 99
ORGequip Correlation Coefficient
.170 .191 .200* .212* .301** .384** .277** .343** .407** .332**
Sig. (2-tailed)
.092 .058 .047 .035 .002 .000 .005 .001 .000 .001
N 99 99 99 99 99 99 99 99 99 99
ORGcont Correlation Coefficient
.158 .146 .287** .310** .277** .366** .217* .355** .392** .228*
Sig. (2-tailed)
.119 .149 .004 .002 .006 .000 .031 .000 .000 .023
N 99 99 99 99 99 99 99 99 99 99
MEDlist Correlation Coefficient
.266** .367** .353** .365** .441** .436** .347** .247* .408** .312**
Sig. (2-tailed)
.008 .000 .000 .000 .000 .000 .000 .014 .000 .002
370
N 99 99 99 99 99 99 99 99 99 99
MEDrec Correlation Coefficient
.209* .297** .377** .365** .428** .355** .366** .167 .345** .288**
Sig. (2-tailed)
.038 .003 .000 .000 .000 .000 .000 .098 .000 .004
N 99 99 99 99 99 99 99 99 99 99
MEDexp Correlation Coefficient
.278** .434** .477** .435** .532** .511** .387** .162 .498** .393**
Sig. (2-tailed)
.005 .000 .000 .000 .000 .000 .000 .108 .000 .000
N 99 99 99 99 99 99 99 99 99 99
MEDrev Correlation Coefficient
.427** .400** .322** .387** .456** .515** .388** .152 .418** .469**
Sig. (2-tailed)
.000 .000 .001 .000 .000 .000 .000 .133 .000 .000
N 99 99 99 99 99 99 99 99 99 99
MEDpharm GL WRplan WRrev WRask EDUC UNdetail UNfam PROBpcp PROBer
MEDcon Correlation Coefficient
.340** .432** .383** .411** .607** .463** .476** .193 .435** .518**
Sig. (2-tailed)
.001 .000 .000 .000 .000 .000 .000 .056 .000 .000
N 99 99 99 99 99 99 99 99 99 99
371
MEDpharm Correlation Coefficient
1.000 .208* .035 .095 .036 .410** .285** -.009 .236* .350**
Sig. (2-tailed)
.039 .729 .351 .720 .000 .004 .928 .018 .000
N 99 99 99 99 99 99 99 99 99 99
GL Correlation Coefficient
.208* ### .307** .291** .359** .363** .320** .259** .362** .464**
Sig. (2-tailed)
.039 .002 .003 .000 .000 .001 .010 .000 .000
N 99 99 99 99 99 99 99 99 99 99
WRplan Correlation Coefficient
.035 .307** 1.000 .844** .658** .246* .237* .314** .376** .345**
Sig. (2-tailed)
.729 .002 .000 .000 .014 .018 .002 .000 .000
N 99 99 99 99 99 99 99 99 99 99
WRrev Correlation Coefficient
.095 .291** .844** 1.000 .724** .219* .282** .368** .352** .400**
Sig. (2-tailed)
.351 .003 .000 .000 .030 .005 .000 .000 .000
N 99 99 99 99 99 99 99 99 99 99
WRask Correlation Coefficient
.036 .359** .658** .724** 1.000 .369** .388** .337** .440** .486**
Sig. (2-tailed)
.720 .000 .000 .000 .000 .000 .001 .000 .000
372
N 99 99 99 99 99 99 99 99 99 99
EDUC Correlation Coefficient
.410** .363** .246* .219* .369** 1.000 .561** .253* .603** .504**
Sig. (2-tailed)
.000 .000 .014 .030 .000 .000 .012 .000 .000
N 99 99 99 99 99 99 99 99 99 99
MEDpharm GL WRplan WRrev WRask EDUC UNdetail UNfam PROBpcp PROBer MEDpharm GL
UNdetail Correlation Coefficient
.285** .320** .237* .282** .388** .561** 1.000 .299** .401** .516**
Sig. (2-tailed)
.004 .001 .018 .005 .000 .000 .003 .000 .000
N 99 99 99 99 99 99 99 99 99 99
UNfam Correlation Coefficient
-.009 .259** .314** .368** .337** .253* .299** 1.000 .359** .223*
Sig. (2-tailed)
.928 .010 .002 .000 .001 .012 .003 .000 .026
N 99 99 99 99 99 99 99 99 99 99
PROBpcp Correlation Coefficient
.236* .362** .376** .352** .440** .603** .401** .359** 1.000 .603**
Sig. (2-tailed)
.018 .000 .000 .000 .000 .000 .000 .000 .000
N 99 99 99 99 99 99 99 99 99 99
373
PROBer Correlation Coefficient
.350** .464** .345** .400** .486** .504** .516** .223* .603** 1.000
Sig. (2-tailed)
.000 .000 .000 .000 .000 .000 .000 .026 .000
N 99 99 99 99 99 99 99 99 99 99
SUMformat Correlation Coefficient
.119 .334** .208* .223* .310** .225* .111 .341** .342** .223*
Sig. (2-tailed)
.240 .001 .038 .026 .002 .025 .272 .001 .001 .027
N 99 99 99 99 99 99 99 99 99 99
SUMpcp Correlation Coefficient
.137 .154 .308** .284** .259** .140 .120 .209* .261** .244*
Sig. (2-tailed)
.178 .128 .002 .004 .010 .167 .236 .038 .009 .015
N 99 99 99 99 99 99 99 99 99 99
TRthree Correlation Coefficient
.089 .312** .088 .044 .199* .282** .329** .262** .219* .258**
Sig. (2-tailed)
.380 .002 .387 .664 .049 .005 .001 .009 .030 .010
N 99 99 99 99 99 99 99 99 99 99
MEDpharm GL WRplan WRrev WRask EDUC UNdetail UNfam PROBpcp PROBer
TRphone Correlation Coefficient
.265** .329** .231* .235* .356** .440** .430** .154 .369** .421**
374
Sig. (2-tailed)
.008 .001 .021 .019 .000 .000 .000 .129 .000 .000
N 99 99 99 99 99 99 99 99 99 99
*p < .05 **p < .01
375
SUMformat SUMpcp TRthree TRphone
Lapref .169 .117 .332** .344**
Sig. (2-tailed)
.094 .249 .001 .000
N 99 99 99 99
LAtransl Correlation Coefficient
.096 .044 -.021 .027
Sig. (2-tailed)
.343 .669 .833 .792
N 99 99 99 99
LAwrit Correlation Coefficient
-.040 -.076 .046 .144
Sig. (2-tailed)
.697 .453 .650 .156
N 99 99 99 99
PDdeterm Correlation Coefficient
.186 .059 .239* .215*
Sig. (2-tailed)
.065 .562 .017 .033
N 99 99 99 99
PDpcp Correlation Coefficient
.058 .129 .174 .187
376
Sig. (2-tailed)
.567 .204 .086 .065
N 99 99 99 99
PDfuture Correlation Coefficient
.300** .122 .189 .199*
Sig. (2-tailed)
.003 .227 .061 .049
N 99 99 99 99
PDapp Correlation Coefficient
.240* .209* .097 .214*
Sig. (2-tailed)
.017 .038 .338 .033
N 99 99 99 99
SUMformat SUMpcp TRthree TRphone
PDinput Correlation Coefficient
.106 -.009 .167 .150
Sig. (2-tailed)
.295 .927 .098 .137
N 99 99 99 99
PDtest Correlation Coefficient
.188 .117 .243* .249*
Sig. (2-tailed)
.062 .250 .015 .013
N 99 99 99 99
377
PDimport Correlation Coefficient
.311** .210* .155 .241*
Sig. (2-tailed)
.002 .037 .126 .016
N 99 99 99 99
PTtrad Correlation Coefficient
.025 .016 .283** .374**
Sig. (2-tailed)
.808 .873 .005 .000
N 99 99 99 99
PDconf Correlation Coefficient
.173 .228* .265** .326**
Sig. (2-tailed)
.087 .023 .008 .001
N 99 99 99 99
PDbarr Correlation Coefficient
.060 .163 .235* .288**
Sig. (2-tailed)
.553 .106 .019 .004
N 99 99 99 99
FUpend Correlation Coefficient
.251* .310** .273** .358**
Sig. (2-tailed)
.012 .002 .006 .000
378
N 99 99 99 99
SUMformat SUMpcp TRthree TRphone
FUwho Correlation Coefficient
.289** .292** .232* .282**
Sig. (2-tailed)
.004 .003 .021 .005
N 99 99 99 99
ORGequip Correlation Coefficient
.131 .147 .081 .156
Sig. (2-tailed)
.198 .146 .423 .123
N 99 99 99 99
ORGcont Correlation Coefficient
.142 .205* .079 .205*
Sig. (2-tailed)
.160 .041 .435 .042
N 99 99 99 99
MEDlist Correlation Coefficient
.269** .121 .054 .174
Sig. (2-tailed)
.007 .233 .596 .085
N 99 99 99 99
MEDrec Correlation Coefficient
.295** .197 .092 .223*
379
Sig. (2-tailed)
.003 .050 .368 .026
N 99 99 99 99
MEDexp Correlation Coefficient
.395** .290** .121 .315**
Sig. (2-tailed)
.000 .004 .235 .001
N 99 99 99 99
MEDrev Correlation Coefficient
.343** .303** .083 .305**
Sig. (2-tailed)
.001 .002 .416 .002
N 99 99 99 99
SUMformat SUMpcp TRthree TRphone
MEDcon Correlation Coefficient
.312** .182 .140 .301**
Sig. (2-tailed)
.002 .072 .168 .002
N 99 99 99 99
MEDpharm Correlation Coefficient
.119 .137 .089 .265**
Sig. (2-tailed)
.240 .178 .380 .008
N 99 99 99 99
380
GL Correlation Coefficient
.334** .154 .312** .329**
Sig. (2-tailed)
.001 .128 .002 .001
N 99 99 99 99
WRplan Correlation Coefficient
.208* .308** .088 .231*
Sig. (2-tailed)
.038 .002 .387 .021
N 99 99 99 99
WRrev Correlation Coefficient
.223* .284** .044 .235*
Sig. (2-tailed)
.026 .004 .664 .019
N 99 99 99 99
WRask Correlation Coefficient
.310** .259** .199* .356**
Sig. (2-tailed)
.002 .010 .049 .000
N 99 99 99 99
EDUC Correlation Coefficient
.225* .140 .282** .440**
Sig. (2-tailed)
.025 .167 .005 .000
381
N 99 99 99 99
SUMformat SUMpcp TRthree TRphone
UNdetail Correlation Coefficient
.111 .120 .329** .430**
Sig. (2-tailed)
.272 .236 .001 .000
N 99 99 99 99
UNfam Correlation Coefficient
.341** .209* .262** .154
Sig. (2-tailed)
.001 .038 .009 .129
N 99 99 99 99
PROBpcp Correlation Coefficient
.342** .261** .219* .369**
Sig. (2-tailed)
.001 .009 .030 .000
N 99 99 99 99
PROBer Correlation Coefficient
.223* .244* .258** .421**
Sig. (2-tailed)
.027 .015 .010 .000
N 99 99 99 99
SUMformat Correlation Coefficient
1.000 .544** .202* .176
382
Sig. (2-tailed)
.000 .045 .081
N 99 99 99 99
SUMpcp Correlation Coefficient
.544** 1.000 .183 .340**
Sig. (2-tailed)
.000 .070 .001
N 99 99 99 99
TRthree Correlation Coefficient
.202* .183 1.000 .626**
Sig. (2-tailed)
.045 .070 .000
N 99 99 99 99
SUMformat SUMpcp TRthree TRphone
TRphone Correlation Coefficient
.176 .340** .626** 1.000
Sig. (2-tailed)
.081 .001 .000
N 99 99 99 99
*p < .05 ** p < .01
383
Appendix V: Factor Correlation Matrix for 9 Factor Solution
Factor 1 2 3 4 5 6 7 8 9
1 1.000 -.345 -.344 .223 .105 .202 .274 .251 .217
2 -.345 1.000 .268 -.218 -.118 -.120 -.192 -.176 .005
3 -.344 .268 1.000 -.348 -.201 -.316 -.198 -.320 -.119
4 .223 -.218 -.348 1.000 .070 .266 .233 .393 .258
5 .105 -.118 -.201 .070 1.000 .098 .141 .114 -.058
6 .202 -.120 -.316 .266 .098 1.000 .166 .203 .081
7 .274 -.192 -.198 .233 .141 .166 1.000 .345 .156
8 .251 -.176 -.320 .393 .114 .203 .345 1.000 .303
9 .217 .005 -.119 .258 -.058 .081 .156 .303 1.000
385
Appendix X: Summary of Items and Factor Loadings for Direct Oblimin Nine Factor Solution
Items with factor loadings greater than 0.3 are highlighted.
Survey items
Factor Loading Communality
1 2 3 4 5 6 7 8 9
1 Language preference of
patient and family is
determined and documented
-.040 .210 -.119 .086 .118 -.101 .145 .201 .351 .361
2 If patient and/or family
members do not speak
English, a translator is
arranged for on discharge
-.082 -.193 -.009 .420 .053 -.056 -.216 0.54 .361 .419
3 If patient and/or family
members do not speak
English, written materials are
provided in the preferred
language
.019 .031 -.056 .061 -.121 -.041 -.044 0.259 .307 .248
4 Need for primary care and
specialty care follow-up is
determined
.248 -.106 -.205 -.054 -.111 .541 .020 .232 -.109 .704
5 If patient does not have a
primary care provider, one is
located for patients
.005 -.051 -.121 -.055 -.071 .335 .122 .008 .053 .488
6 Need for future tests is
determined
.066 -.042 .035 -.042 .049 .665 .075 .140 -.044 .537
7 Appointments are made for
patient for follow-up
appointments and testing
-.019 -.181 -.127 .208 .087 .617 .017 -.141 .125 .661
8 If appointments are made,
they are made with input
-.061 .170 -.046 .296 -.018 .514 .002 .064 .148 .517
386
from the patient/family
regarding the best time/date
9 If patient requires future
diagnostic testing, patient
and/or family member is
.instructed on any
preparation for testing
-.017 .018 -.091 .822 .041 .056 .001 .053 -.177 .750
10 Importance of clinician
appointments and further
testing is discussed with
patient/family
.262 -.047 -.321 .134 -.002 .161 .051 .068 .090 .509
11 Patient/family is asked about
traditional healing practices,
and there is confirmation
made that practices are
complementary with
patient’s discharge plan
.155 -.052 .112 .245 -.186 .152 .212 .176 .179 .440
12 There is confirmation made
with patient/family that they
know where to go for further
appointments and tests, and
that they have a plan to get to
appointments
.147 -.029 -.011 .699 .013 .075 .142 .041 -.005 .709
13 Barriers to keeping
appointments are addressed
.171 .108 -.086 .609 -.086 .085 .166 .026 .103 .676
14 Pending lab and test results
are identified with
patient/family
-.021 -012 -.004 .046 .144 .183 .071 .627 .017 .599
15 Determination is made of
who will be reviewing the
results, and when and how
this information will be
-.071 -.065 -.007 .148 .178 .137 -.088 .792 .021 .844
387
communicated to the
patient/family
16 If patient requires medical
equipment on discharge,
there is a process for
ensuring that the medical
equipment is obtained
.031 .025 -.758 .075 .013 -.008 -.057 .071 -.022 .647
17 Before discharge,
patient/family is given
contact information for
medical equipment
companies, at-home services
(as needed)
.007 -.035 -1.004 -.046 .034 .029 -.025 -.199 .098 .962
18 Medication list is reviewed
with patient/family
.604 -.053 -.210 .067 -.070 .304 -.089 .085 -.117 .743
19 Medication reconciliation is
done at the time of discharge
.619 -.176 .055 .037 .039 .138 -.012 -.090 .000 .507
20 Patient/family member is
given an explanation of what
medications to take and
changes in the medication
regimen are emphasized
.764 -.084 -.176 .098 .166 -.008 -.045 -.098 .014 .823
21 Each medication’s purpose,
administration and side
effects are reviewed with
patient/family
.718 -.077 -.019 -.052 .111 -.013 -.015 .141 .182 .744
22 Patient/family’s concerns
about medication plan are
assessed
.679 -.036 -.145 .102 .083 -.065 -.034 .138 .066 .730
23 Patient/family is referred to
community pharmacist
within 2 weeks of discharge
for a medication review
.282 .072 -.005 -.087 .040 .017 .104 .031 .535 .456
388
24 There is use of guidelines in
the development and
planning of the discharge
process for patients and
family members
.403 -.108 .014 .154 .045 -.039 .269 -.072 .013 .387
25 The patient/family is given
an easy-to-understand
written, prioritized discharge
plan that includes
medications, medical
equipment, future
appointments, and future
diagnostic tests to take home
.092 -.891 -.007 -.014 -.009 .034 .016 -.067 -.020 .851
26 The written plan is reviewed
with the patient/family
.000 -.979 -.031 -.051 .004 .035 -.042 0.16 .042 .958
27 Patient/family is encouraged
to ask questions about the
plan
.114 -.611 -.095 .053 .009 .014 .121 .139 -.127 .644
28 Prior to day of discharge,
patient/family is met with to
provide education about
patient’s diagnosis and
treatment and to prepare for
discharge
.295 .023 -.208 .001 -.227 -.061 .341 .333 .047 .625
29 Patient/family is asked to
explain, in their own words,
the details of the discharge
plan
.109 -.165 -.105 .234 -.134 -.224 .357 .166 .134 .537
30 As needed, family members
and other caregivers who will
share in the care-giving
responsibilities are contacted
-.155 -.199 -.314 .047 .142 .054 .138 .167 .168 .348
389
31 Patient/family is made aware
of how to contact primary
care provider
.220 -.199 -.349 .087 .022 -.055 .142 .197 -.038 .569
32 Patient/family is instructed
on what constitutes an
emergency and what to do in
case of emergency
.163 -.276 .083 .066 .008 -.103 .185 .395 .151 .534
33 Discharge summary has a
standardized format so that
information is easy to find
.300 .128 -.094 .034 .696 .057 .003 .091 -.074 .702
34 Within 24 hours of
discharge, a discharge
summary is provided to
primary care provider
.020 -.077 -.006 -.026 .657 -.030 .137 .074 .087 .528
35 Patient/family is called
within 3 days of discharge to
reinforce the discharge plan
and help with problem-
solving
-.130 .026 .004 .080 .080 .084 .733 -.027 .071 .550
36 Patient/family is provided
with a phone number where
they can speak with a
hospital staff member to ask
questions about the at-home
care plan, hospitalization,
and follow-up plan in order
to help patients transition
from hospital care to
outpatient care setting
.015 -.089 .014 -.039 .111 .019 .747 .002 .136 .670
390
Appendix Y: Summary of Items and Factor Loadings Greater than .3, for Nine Factor Solution
Survey items Factor Loading Communality
1 2 3 4 5 6 7 8 9
20 Patient/family member is
given an explanation of what
medications to take and
changes in the medication
regimen are emphasized
.764 .823
21 Each medication’s purpose,
administration and side
effects are reviewed with
patient/family
.718 .744
22 Patient/family’s concerns
about medication plan are
assessed
.679 .730
19 Medication reconciliation is
done at the time of discharge
.619 .507
18 Medication list is reviewed
with patient/family
.604 .304* .743
24 There is use of guidelines in
the development and
planning of the discharge
process for patients and
family members
.403 .387
26 The written plan is reviewed
with the patient/family
-.979 .958
25 The patient/family is given
an easy-to-understand
written, prioritized discharge
plan that includes
-.891 .851
391
medications, medical
equipment, future
appointments, and future
diagnostic tests to take home
27 Patient/family is encouraged
to ask questions about the
plan
-.611 .644
17 Before discharge,
patient/family is given
contact information for
medical equipment
companies, at-home services
(as needed)
-1.004 .962
16 If patient requires medical
equipment on discharge,
there is a process for
ensuring that the medical
equipment is obtained
-.758 .647
31 Patient/family is made aware
of how to contact primary
care provider
-.349 .569
10 Importance of clinician
appointments and further
testing is discussed with
patient/family
-.321 .509
30 As needed, family members
and other caregivers who will
share in the care-giving
responsibilities are contacted
-.314 .348
9 If patient requires future
diagnostic testing, patient
and/or family member is
.822 .750
392
.instructed on any
preparation for testing
12 There is confirmation made
with patient/family that they
know where to go for further
appointments and tests, and
that they have a plan to get to
appointments
.699 .709
13 Barriers to keeping
appointments are addressed
.609 .676
2 If patient and/or family
members do not speak
English, a translator is
arranged for on discharge
.420 .361 .419
11 Patient/family is asked about
traditional healing practices,
and there is confirmation
made that practices are
complementary with
patient’s discharge plan
.245a .440
33 Discharge summary has a
standardized format so that
information is easy to find
.696 .702
34 Within 24 hours of
discharge, a discharge
summary is provided to
primary care provider
.657 .528
6 Need for future tests is
determined
.665 .537
7 Appointments are made for
patient for follow-up
appointments and testing
.617 .661
393
4 Need for primary care and
specialty care follow-up is
determined
.541 .704
8 If appointments are made,
they are made with input
from the patient/family
regarding the best time/date
.514 .517
36 Patient/family is provided
with a phone number where
they can speak with a
hospital staff member to ask
questions about the at-home
care plan, hospitalization,
and follow-up plan in order
to help patients transition
from hospital care to
outpatient care setting
.747 .670
35 Patient/family is called
within 3 days of discharge to
reinforce the discharge plan
and help with problem-
solving
.733 .550
29 Patient/family is asked to
explain, in their own words,
the details of the discharge
plan
.357 .537
28 Prior to day of discharge,
patient/family is met with to
provide education about
patient’s diagnosis and
treatment and to prepare for
discharge
.341 .333 .625
394
15 Determination is made of
who will be reviewing the
results, and when and how
this information will be
communicated to the
patient/family
.792 .844
14 Pending lab and test results
are identified with
patient/family
.627 .599
32 Patient/family is instructed
on what constitutes an
emergency and what to do in
case of emergency
.395 .534
23 Patient/family is referred to
community pharmacist
within 2 weeks of discharge
for a medication review
.535 .456
5 If patient does not have a
primary care provider, one is
located for patients
.335 .488
1 Language preference of
patient and family is
determined and documented
.351 .361
3 If patient and/or family
members do not speak
English, written materials are
provided in the preferred
language
.307 .248
aItem 11 did not have a factor loading greater than .3. The highest factor loading it had was .245.
* Items marked with an asterisk were not used. They were assigned to the factor with which they fit best conceptually
395
Appendix Z: Factor Correlation Matrix for 5 Factor Solution
Factor 1 2 3 4 5
1 1.000 -.224 .357 -.399 .245
2 -.224 1.000 -.085 .283 -.237
3 .357 -.085 1.000 -.283 .138
4 -.399 .283 -.283 1.000 -.202
5 .245 -.237 .138 -.202 1.000
396
Appendix AA: Summary of Items and Factor Loadings for Direct Oblimin Five Factor Solution
Items with factor loadings greater than 0.3 are highlighted.
Survey items
Factor Loading Communality
1 2 3 4 5
1 Language preference of patient and family is
determined and documented
.173 .280 .737 -.092 .156 .319
2 If patient and/or family members do not speak
English, a translator is arranged for on discharge
.277 -.085 .235 -.017 -.152 .193
3 If patient and/or family members do not speak
English, written materials are provided in the
preferred language
.127 .102 .320 -.123 -.130 .186
4 Need for primary care and specialty care follow-up
is determined
.577 -.148 -.073 -.228 .046 .554
5 If patient does not have a primary care provider,
one is located for patients
.333 .074 .225 -.191 -.079 .296
6 Need for future tests is determined .511 -.032 -.045 -.027 .104 .304
7 Appointments are made for patient for follow-up
appointments and testing
.695 -.134 -.106 -.026 .014 .520
8 If appointments are made, they are made with input
from the patient/family regarding the best time/date
.715 .202 .069 .065 -.065 .485
9 If patient requires future diagnostic testing, patient
and/or family member is .instructed on any
preparation for testing
.604 -.096 .189 .121 .007 .460
10 Importance of clinician appointments and further
testing is discussed with patient/family
.431 -.070 .047 -.339 .073 .509
11 Patient/family is asked about traditional healing
practices, and there is confirmation made that
.248 -.059 .465 -.142 -.170 .437
397
practices are complementary with patient’s
discharge plan
12 There is confirmation made with patient/family that
they know where to go for further appointments and
tests, and that they have a plan to get to
appointments
.498 -.127 .317 -.045 -.004 .543
13 Barriers to keeping appointments are addressed .507 .034 .365 -.135 -.088 .586
14 Pending lab and test results are identified with
patient/family
.341 .004 .369 .035 .272 .469
15 Determination is made of who will be reviewing the
results, and when and how this information will be
communicated to the patient/family
.420 -.033 .389 .065 .243 .553
16 If patient requires medical equipment on discharge,
there is a process for ensuring that the medical
equipment is obtained
.505 -.036 -.149 -.189 .190 .425
17 Before discharge, patient/family is given contact
information for medical equipment companies, at-
home services (as needed)
.500 -.074 -.274 -.271 .193 .495
18 Medication list is reviewed with patient/family .424 -.135 -.167 -.572 -.011 .702
19 Medication reconciliation is done at the time of
discharge
.033 -.212 -.062 -.589 -.020 .456
20 Patient/family member is given an explanation of
what medications to take and changes in the
medication regimen are emphasized
.075 -.141 -.112 -.796 .135 .799
21 Each medication’s purpose, administration and side
effects are reviewed with patient/family
-.065 -.064 .142 -.786 .095 .734
22 Patient/family’s concerns about medication plan are
assessed
.076 -.078 .101 -.716 .100 .710
23 Patient/family is referred to community pharmacist
within 2 weeks of discharge for a medication
review
-.034 .188 .314 -.451 -.018 .341
398
24 There is use of guidelines in the development and
planning of the discharge process for patients and
family members
-.028 -.174 .219 -.379 .094 .333
25 The patient/family is given an easy-to-understand
written, prioritized discharge plan that includes
medications, medical equipment, future
appointments, and future diagnostic tests to take
home
-.018 -.890 .014 -.123 -.068 .837
26 The written plan is reviewed with the patient/family .014 -.908 .041 -.075 -.059 .860
27 Patient/family is encouraged to ask questions about
the plan
.098 -.673 .138 -.099 .096 .651
28 Prior to day of discharge, patient/family is met with
to provide education about patient’s diagnosis and
treatment and to prepare for discharge
.113 -.058 .438 -.327 .052 .500
29 Patient/family is asked to explain, in their own
words, the details of the discharge plan
.025 -.215 .557 -.153 .010 .486
30 As needed, family members and other caregivers
who will share in the care-giving responsibilities
are contacted
.300 -.253 .019 .152 .338 .343
31 Patient/family is made aware of how to contact
primary care provider
.234 -.264 .162 -.273 .202 .525
32 Patient/family is instructed on what constitutes an
emergency and what to do in case of emergency
-.041 -.263 .545 -.186 .085 .520
33 Discharge summary has a standardized format so
that information is easy to find
.103 .124 -.123 -.278 .687 .611
34 Within 24 hours of discharge, a discharge summary
is provided to primary care provider
-.056 -.016 .086 -.069 .620 .417
35 Patient/family is called within 3 days of discharge
to reinforce the discharge plan and help with
problem-solving
.048 -.055 .401 .138 .311 .292
36 Patient/family is provided with a phone number
where they can speak with a hospital staff member
to ask questions about the at-home care plan,
-.081 -.112 .516 -.074 .316 .448
399
hospitalization, and follow-up plan in order to help
patients transition from hospital care to outpatient
care setting
400
Appendix BB: Summary of Items and Factor Loadings Greater than .3, for Five Factor Solution
Survey items Factor Loading Communality
1 2 3 4 5
8 If appointments are made, they are made with input
from the patient/family regarding the best time/date
.715 .485
7 Appointments are made for patient for follow-up
appointments and testing
.695 .520
9 If patient requires future diagnostic testing, patient
and/or family member is .instructed on any preparation
for testing
.604 .460
4 Need for primary care and specialty care follow-up is
determined
.577 .554
6 Need for future tests is determined .511 .304
13 Barriers to keeping appointments are addressed .507 .365* .586
16 If patient requires medical equipment on discharge,
there is a process for ensuring that the medical
equipment is obtained
.505 .425
17 Before discharge, patient/family is given contact
information for medical equipment companies, at-
home services (as needed)
.500 .495
12 There is confirmation made with patient/family that
they know where to go for further appointments and
tests, and that they have a plan to get to appointments
.498 .317* .543
10 Importance of clinician appointments and further
testing is discussed with patient/family
.431 -.339* .509
15 Determination is made of who will be reviewing the
results, and when and how this information will be
communicated to the patient/family
.420 .389* .553
401
5 If patient does not have a primary care provider, one is
located for patients
.333 .296
2 If patient and/or family members do not speak English,
a translator is arranged for on discharge
.277* .235a .193
26 The written plan is reviewed with the patient/family -.908 .860
25 The patient/family is given an easy-to-understand
written, prioritized discharge plan that includes
medications, medical equipment, future appointments,
and future diagnostic tests to take home
-.890 .837
27 Patient/family is encouraged to ask questions about the
plan
-.673 .651
29 Patient/family is asked to explain, in their own words,
the details of the discharge plan
.557 .486
32 Patient/family is instructed on what constitutes an
emergency and what to do in case of emergency
.545 .520
36 Patient/family is provided with a phone number where
they can speak with a hospital staff member to ask
questions about the at-home care plan, hospitalization,
and follow-up plan in order to help patients transition
from hospital care to outpatient care setting
.516 .316* .448
11 Patient/family is asked about traditional healing
practices, and there is confirmation made that practices
are complementary with patient’s discharge plan
.465 .437
28 Prior to day of discharge, patient/family is met with to
provide education about patient’s diagnosis and
treatment and to prepare for discharge
.438 -.327* .500
35 Patient/family is called within 3 days of discharge to
reinforce the discharge plan and help with problem-
solving
.401 . .311* .292
1 Language preference of patient and family is
determined and documented
.373 .319
14 Pending lab and test results are identified with
patient/family
.341* .369 .469
402
3 If patient and/or family members do not speak English,
written materials are provided in the preferred
language
.320 .186
20 Patient/family member is given an explanation of what
medications to take and changes in the medication
regimen are emphasized
-.796 .799
21 Each medication’s purpose, administration and side
effects are reviewed with patient/family
-.786 .734
22 Patient/family’s concerns about medication plan are
assessed
-.716 .710
19 Medication reconciliation is done at the time of
discharge
-.589 .456
18 Medication list is reviewed with patient/family .424* -.572 .702
23 Patient/family is referred to community pharmacist
within 2 weeks of discharge for a medication review
.314* -.451 .341
24 There is use of guidelines in the development and
planning of the discharge process for patients and
family members
-.379 .333
31b Patient/family is made aware of how to contact
primary care provider
.234a -.264* .273* .525
33 Discharge summary has a standardized format so that
information is easy to find
.687 .611
34 Within 24 hours of discharge, a discharge summary is
provided to primary care provider
.620 .417
30 As needed, family members and other caregivers who
will share in the care-giving responsibilities are
contacted
.300 .338* .343
*These are factor loadings of items that are cross-loaded. Items marked with an asterisk were not used. They were assigned to the
factor with which they fit best conceptually. aItem 2 did not have a factor loading greater than .3. It was included in this analysis. Item 2 fit best with factor 3: patient/family
education and language assistance. bItem 31 did not have a factor loading greater than .3. It was included in this analysis. Item 31 fit best with factor 1: Follow-up
appointments and tests, and coordination of services
403
Appendix CC: Seven Factor Solution
Summary of Items and Factor Loadings Greater Than .3, for 7 Factor Solution
Survey items Factor Loading Communality
1 2 3 4 5 6 7
9 If patient requires future diagnostic testing, patient
and/or family member is .instructed on any preparation
for testing
.890 .766
12 There is confirmation made with patient/family that
they know where to go for further appointments and
tests, and that they have a plan to get to appointments
.745 .702
13 Barriers to keeping appointments are addressed .646 .658
2 If patient and/or family members do not speak English,
a translator is arranged for on discharge
.423 .240
15 Determination is made of who will be reviewing the
results, and when and how this information will be
communicated to the patient/family
.291 .545
26 The written plan is reviewed with the patient/family -.908 .867
25 The patient/family is given an easy-to-understand
written, prioritized discharge plan that includes
medications, medical equipment, future appointments,
and future diagnostic tests to take home
-.875
.836
27 Patient/family is encouraged to ask questions about the
plan
-.667 .649
17 Before discharge, patient/family is given contact
information for medical equipment companies, at-
home services (as needed)
-.859 .796
16 If patient requires medical equipment on discharge,
there is a process for ensuring that the medical
equipment is obtained
-.775 .684
31 Patient/family is made aware of how to contact
primary care provider
-.335 .561
404
30 As needed, family members and other caregivers who
will share in the care-giving responsibilities are
contacted
-.316 .357
10 Importance of clinician appointments and further
testing is discussed with patient/family
-.302 .510
20 Patient/family member is given an explanation of what
medications to take and changes in the medication
regimen are emphasized
-.776 .834
21 Each medication’s purpose, administration and side
effects are reviewed with patient/family
-.697 .729
22 Patient/family’s concerns about medication plan are
assessed
-.639 .722
19 Medication reconciliation is done at the time of
discharge
-.618 .513
18 Medication list is reviewed with patient/family -.528 .335 .703
24 There is use of guidelines in the development and
planning of the discharge process for patients and
family members
-.382 .352
33 Discharge summary has a standardized format so that
information is easy to find
-.330 .691 .686
34 Within 24 hours of discharge, a discharge summary is
provided to primary care provider
.623 .420
36 Patient/family is provided with a phone number where
they can speak with a hospital staff member to ask
questions about the at-home care plan, hospitalization,
and follow-up plan in order to help patients transition
from hospital care to outpatient care setting
.408 .408 .492
35 Patient/family is called within 3 days of discharge to
reinforce the discharge plan and help with problem-
solving
.397 .323
14 Pending lab and test results are identified with
patient/family
.303 .476
6 Need for future tests is determined .742 .557
405
7 Appointments are made for patient for follow-up
appointments and testing
.620 .585
4 Need for primary care and specialty care follow-up is
determined
.604 .655
8 If appointments are made, they are made with input
from the patient/family regarding the best time/date
.300 .556 .515
28 Prior to day of discharge, patient/family is met with to
provide education about patient’s diagnosis and
treatment and to prepare for discharge
.505 .534
23 Patient/family is referred to community pharmacist
within 2 weeks of discharge for a medication review
-.325 .473 .373
29 Patient/family is asked to explain, in their own words,
the details of the discharge plan
.438 .519
1 Language preference of patient and family is
determined and documented
.421 .342
32 Patient/family is instructed on what constitutes an
emergency and what to do in case of emergency
-.316 .418 .516
5 If patient does not have a primary care provider, one is
located for patients
.379 .403 .396
3 If patient and/or family members do not speak English,
written materials are provided in the preferred
language
.401 .220
11 Patient/family is asked about traditional healing
practices, and there is confirmation made that practices
are complementary with patient’s discharge plan
.354 .446
406
Appendix DD: Eight Factor Solution
Summary of Items and Factor Loadings Greater Than .3, for 8 Factor Solution
Survey items Factor Loading Communality
1 2 3 4 5 6 7 8
20 Patient/family member is given an explanation of
what medications to take and changes in the
medication regimen are emphasized
.747 .826
21 Each medication’s purpose, administration and side
effects are reviewed with patient/family
.708 .747
22 Patient/family’s concerns about medication plan are
assessed
.634 .724
19 Medication reconciliation is done at the time of
discharge
.605 .510
18 Medication list is reviewed with patient/family .506 .341 .699
24 There is use of guidelines in the development and
planning of the discharge process for patients and
family members
.401 .388
26 The written plan is reviewed with the patient/family -.948 .907
25 The patient/family is given an easy-to-understand
written, prioritized discharge plan that includes
medications, medical equipment, future
appointments, and future diagnostic tests to take
home
-.878 .832
27 Patient/family is encouraged to ask questions about
the plan
-.657 .646
17 Before discharge, patient/family is given contact
information for medical equipment companies, at-
home services (as needed)
-.985 .907
407
16 If patient requires medical equipment on discharge,
there is a process for ensuring that the medical
equipment is obtained
-.776 .658
31 Patient/family is made aware of how to contact
primary care provider
-353 .559
10 Importance of clinician appointments and further
testing is discussed with patient/family
-.331 .511
30 As needed, family members and other caregivers
who will share in the care-giving responsibilities
are contacted
-.305 .348
9 If patient requires future diagnostic testing, patient
and/or family member is .instructed on any
preparation for testing
.843 .750
12 There is confirmation made with patient/family that
they know where to go for further appointments and
tests, and that they have a plan to get to
appointments
.725 .718
13 Barriers to keeping appointments are addressed .626 .684
2 If patient and/or family members do not speak
English, a translator is arranged for on discharge
.379 .301
11 Patient/family is asked about traditional healing
practices, and there is confirmation made that
practices are complementary with patient’s
discharge plan
.254 .253 .443
33 Discharge summary has a standardized format so
that information is easy to find
.764 .728
34 Within 24 hours of discharge, a discharge summary
is provided to primary care provider
.594 .422
6 Need for future tests is determined .727 .550
7 Appointments are made for patient for follow-up
appointments and testing
.608 .586
4 Need for primary care and specialty care follow-up
is determined
.593 .654
408
8 If appointments are made, they are made with input
from the patient/family regarding the best time/date
.548 .512
5 If patient does not have a primary care provider,
one is located for patients
.356 .312 .391
36 Patient/family is provided with a phone number
where they can speak with a hospital staff member
to ask questions about the at-home care plan,
hospitalization, and follow-up plan in order to help
patients transition from hospital care to outpatient
care setting
.747 .656
35 Patient/family is called within 3 days of discharge
to reinforce the discharge plan and help with
problem-solving
.745 .570
29 Patient/family is asked to explain, in their own
words, the details of the discharge plan
.378 .537
28 Prior to day of discharge, patient/family is met with
to provide education about patient’s diagnosis and
treatment and to prepare for discharge
.345 .546
15 Determination is made of who will be reviewing the
results, and when and how this information will be
communicated to the patient/family
.380 .576 .774
3 If patient and/or family members do not speak
English, written materials are provided in the
preferred language
.455 .254
14 Pending lab and test results are identified with
patient/family
.315 .451 .561
32 Patient/family is instructed on what constitutes an
emergency and what to do in case of emergency
-.326 .422 .538
1 Language preference of patient and family is
determined and documented
.399 .344
23 Patient/family is referred to community pharmacist
within 2 weeks of discharge for a medication
review
.370 .375* .375
409
Appendix EE: Email Communication for Interview
Dear (participant),
Earlier this spring, you participated in a survey to assess how well hospitals discharge patients
from the acute care hospital setting to home. This study is part of my dissertation research at the
Institute of Health Policy, Management and Evaluation at the University of Toronto and it is
endorsed by the Ontario Hospital Association. This research is being supervised by Dr. Jan
Barnsley and Dr. Whitney Berta.
You indicated that you would be willing to participate in a follow-up interview, and provided
your contact information. I am contacting you to request your participation in an interview to
find out about facilitators and barriers to the use of health literate discharge practices. I would
like to make an appointment to interview you in-person.
If you are agreeable to this, could you please suggest a second health care provider or manager at
your hospital who may be willing to be interviewed?
Thank you,
Jennifer
Jennifer Innis
Email: jennifer.innis@mail.utoronto.ca
Phone: (416) 571-3248
410
Appendix FF: Informed Consent
Project Title Uptake of health literate discharge practices in Ontario acute care hospitals
Principal Investigator Jennifer Innis, doctoral candidate, University of Toronto
Co-Investigators Dr. Jan Barnsley, University of Toronto
Dr. Whitney Berta, University of Toronto
Dr. Imtiaz Daniel, University of Toronto
Informed Consent Form for Interview
Before agreeing to participate in this research study, it is important that you read and
understand this research consent form. This form provides all the information we think you will
need to know in order to decide whether you wish to participate in the study. If you have any
questions after you read through this form, ask the investigative team. You should not sign this
form until you are sure you understand the information. All research is voluntary and you may
choose to withdraw at any point without consequence.
Purpose of the Research This study is part of a larger study that is examining the use of health literate discharge
practices in Ontario hospitals. The aim of this study is to discover the facilitators and barriers
within hospitals to the uptake of discharge practices that meet the health literacy needs of
patients.
Description of the Research This qualitative study will interview health care providers and managers in Ontario acute
care hospitals who are involved with, or have working knowledge of, the discharge practice on a
general medicine inpatient care unit. Study participants will be asked to participate in a semi-
structured interview session (approximately 30 minutes). In addition, participants will be asked
to provide documents that are used at their hospital in the discharge process. This will include,
but is not limited to, patient discharge summary templates, guidelines, checklists, policies, and
patient education materials.
Potential Harms (Injury, Discomfort or Inconvenience) There are no known harms associated with participation in this study.
Potential Benefits You will not benefit directly from participating in this study. Findings from the study
will be instrumental in evaluating the use of health literate discharge practices in Ontario
hospitals.
Treatment Options Not applicable.
411
Confidentiality and Privacy Confidentiality will be respected and no information that discloses the identity of the
subject will be released or published without consent. All identifying information will be
destroyed. This means that no information will be released or printed that would disclose
personal identity. Data will be aggregated (with no personal identifiers for the analysis).
Consent forms of participants will be stored separately from the data files (transcripts and
analytical worksheets). Only the investigators will have access to the consent forms which will
be kept in a locked cabinet for a period of up to three years. The dialogue from the interview
sessions will be audiotaped and transcribed by the investigator. The hard copies of data
(transcripts and analytical worksheets/notes) and informed consent forms will be shredded 3
years after study completion. Only the study investigators will have access to the data prior to
destruction.
Publication of Results Aggregate results may be shared through oral presentations (conferences) and written
documents (academic and professional journal publications). No identifying information will be
included in any of the publications or presentations associated with this study.
Reimbursement There is no reimbursement for your participation in the study.
Compensation for Injury There is no risk for injury associated with participation in this study.
Participation and Withdrawal Participation in research is voluntary. If you choose not to participate there will be no
consequences. If you choose to participate in this study you can withdraw from the study at any
time without any consequences. Research findings from the study will be made available to the
participants upon completion of the research.
Sponsorship The sponsor of this research is the Institute of Health Policy, Management and Evaluation
at the University of Toronto.
Conflict of Interest The research team members have no conflict of interest to declare.
Research Ethics Board Contact If you have any questions as a research participant you may contact the Office of Office
of Research Ethics (ethics.review@utoronto.ca or 416-946-3273).
412
Uptake of health literate discharge practices in Ontario acute care hospitals
Participant Consent Form
I, __________________________________ , understand that I am being asked to take part in a
study of health care providers and managers in Ontario acute care hospitals who are involved
with, or have working knowledge of, the discharge practices on a general medicine inpatient care
unit. This study is examining the use of health literate discharge practices in Ontario acute care
hospitals and facilitators and barriers to the use of these practices. Study participants will be
asked to participate in a semi-structured interview session (approximately 30 minutes).
I understand that it is entirely my choice to participate in this study or not; that taking part is
voluntary. I am aware that the interview will take about 30 minutes. I understand that the
interview will be audiotaped for the purpose of data analysis and that no one outside of the
research team will have access to the recordings. In addition, I understand that I will be asked to
provide documents that are used in the discharge process. This may include, but is not limited to,
patient discharge summary templates, guidelines, checklists, policies, and patient education
materials. All information I provide will remain confidential and my name will not be identified
in any report of the study results. The audio recordings will be stored on a secure, password-
protected file on a computer. Only members of the research team will have access to them.
Following completion of the study, the audio recordings will be deleted.
I have been informed that this is a voluntary study and that I have the right to not participate, and
to withdraw from the study at any time. The potential risks, harms, and discomforts have been
explained to me and I also understand the benefits (if any) of participating in the research study.
I understand that I have not waived my legal rights nor released the investigators, sponsor or
involved institution from their legal and professional duties. I know that I may ask now, or in the
future, any questions I have about the study or the research procedures. I have been given
sufficient time to read and understand the above information. By signing this form, I agree that:
1. You have explained this study to me. You have answered all my questions.
2. You have explained the possible harms and benefits (if any) of this study.
3. I know what I could do instead of taking part in this study. I understand that I have the
right not to take part in the study and the right to stop at any time.
4. I am free now, and in the future, to ask questions about the study and the audio recording.
5. I understand that no information about who I am will be given to anyone or be published
without first asking my permission.
6. I agree, or consent, to take part in this study.
7. I agree to be audio recorded during this study. These recordings will be used to explore
the use of health literate discharge practices in Ontario acute care hospitals and the
facilitators and barriers to the use of these practices. These interviews will be recorded as
part of the data analysis.
413
_____________________________ _________________________________
Printed name of participant Participants’ signature and date
_____________________________ _________________________________
Printed name of person who explained consent Signature and date
_____________________________ _________________________________
Printed witness’s name Witness’s signature and date
If you have questions as a research subject you may contact the Office of Office of Research
Ethics (ethics.review@utoronto.ca or 416-946-3273).
If you have any questions or comments about the study, please contact the investigator at the
address given below.
Jennifer Innis, PhD(c)
University of Toronto
Institute of Health Policy, Management & Evaluation
University of Toronto
425 – 155 College St.
Toronto ON M5T 3M6
Phone: 416-571-3248
Email: jennifer.innis@mail.utoronto.ca
414
Appendix GG: Interview Questions
1. What is your role at the hospital? How long have you been working at the hospital? How
long have you been working in this role?
Questions related to discharge practices:
2. During a hospitalization, when are patients and families given information about their
discharge?
3. What kinds of interactions do you have with patients and families at the time of
discharge? (How do you interact with patients and families? What topics do you discuss
with them?)
4. In preparing patients and families for discharge, can you describe the ways in which you
help them understand information?
5. Who gives information to the patient and family at the time of discharge? Can you
describe their roles?
6. What resources does the hospital provide to patients and families in preparing for
discharge? (Example: medication pamphlets, information about follow-up appointments,
home care information)
Organizational Facilitators and Barriers to Health Literate Discharge Practices
7. Does your nursing unit or hospital provide resources that help you to communicate with
patients and families? Are these resources helpful? Are there workshops on
communication techniques that you can attend?
8. In establishing discharge practices, what barriers have you – or has your unit -
encountered?
9. What has been supportive/helpful in establishing discharge practices on your unit (or in
you hospital)?
10. Is there any kind of discharge checklist being used? (There may be systems supports in
place that support the process like a checklist)
11. Is there communication with the patient’s primary care provider at the time of discharge?
If yes, how does this communication occur?
Responding to Environmental Mandates
12. Since you have been working at this hospital, have there been any changes made in the
way patients are discharged? Can you describe these changes? How did these changes
occur?
13. What leads your hospital, or ward, to make changes in patient care/education? What are
the forces outside the hospital that have led to changes? (e.g. accreditation, funding
opportunities, government initiatives, educational partnerships, professional associations)
415
Scanning
14. What opportunities do you have for professional development? Does the hospital support,
or not support, you in finding out about innovations, new practices? (e.g. attending
conferences, membership and engagement with external organizations)
15. Are there opportunities for you to take education days, to attend conferences, to take
courses? Does your hospital provide funds for you to take part in educational activities?
(such as education days, conferences, courses) Where does this funding come from? (e.g.
outside funder such as corporate sponsor)
16. What relationships do you have with members of other health care organizations (e.g.
hospital, community agencies, professional associations such as RNAO)? Does your
hospital encourage these relationships? (e.g. through hospital partnerships, educational
events)
Participative decision making
17. How are decisions made at your hospital/your unit to introduce new ways of delivering
care? (i.e. are there unit based councils where decisions are made to trial and use
innovations? Do hospital-wide and/or program level committees make decisions? And/or
do decisions come from senior-level committees?)
18. When decisions are made about new ways of delivering care, are you asked for input?
Allocating resources
19. When innovations or new practices are adopted by your hospital, how are they funded?
20. Is there space you can use on the unit (other than the hallway, patient’s room) to discuss
patient discharge plans and to share knowledge about best practices?*
21. If there is space, how often do you use it?*
22. When a patient’s discharge is being planned, are staff available to talk to patients? Are
families consistently involved in the discussions?*
23. Is there enough staff available to give patients and families discharge information, and to
answer all of their questions and concerns?*
*Based on questions present in the Albert Context Tool (Estabrooks, Squires, Cummings,
Birdsell, & Norton, 2009).
416
Appendix HH: Tests of Normality
Case Processing Summary
Cases
Valid Missing Total
N Percent N Percent N Percent
HL score 79 100.0% 0 0.0% 79 100.0%
Descriptives
Statistic Std. Error
HL score Mean 135.92 2.134
95% Confidence Interval for Mean
Lower Bound 131.68 Upper Bound 140.17
5% Trimmed Mean 136.37 Median 138.00 Variance 359.661 Std. Deviation 18.965 Minimum 78 Maximum 173 Range 95 Interquartile Range 27 Skewness -.326 .271
Kurtosis .247 .535
Tests of Normality
Kolmogorov-Smirnova Shapiro-Wilk
Statistic df Sig. Statistic df Sig.
HL score .075 79 .200* .985 79 .484
*. This is a lower bound of the true significance. a. Lilliefors Significance Correction
417
Appendix II: Distribution of Health Literate Survey Scores for Categorical Independent Variables
Descriptives
Region Statistic Std. Error
HL score North Mean 146.05 3.314
95% Confidence Interval for Mean
Lower Bound 139.11 Upper Bound 152.99
5% Trimmed Mean 145.94 Median 147.00 Variance 219.629 Std. Deviation 14.820 Minimum 121 Maximum 173 Range 52 Interquartile Range 25 Skewness .058 .512
Kurtosis -.750 .992
East Mean 132.79 4.051
95% Confidence Interval for Mean
Lower Bound 124.03 Upper Bound 141.54
5% Trimmed Mean 132.76 Median 136.50 Variance 229.720 Std. Deviation 15.157 Minimum 111 Maximum 155 Range 44 Interquartile Range 26 Skewness -.040 .597
Kurtosis -1.612 1.154
Central Mean 138.18 4.548
95% Confidence Interval for Mean
Lower Bound 128.05 Upper Bound 148.32
5% Trimmed Mean 138.20 Median 142.00 Variance 227.564 Std. Deviation 15.085 Minimum 112 Maximum 164 Range 52 Interquartile Range 22 Skewness -.122 .661
418
Kurtosis -.303 1.279
South Mean 125.16 5.526
95% Confidence Interval for Mean
Lower Bound 113.55 Upper Bound 136.77
5% Trimmed Mean 125.23 Median 124.00 Variance 580.140 Std. Deviation 24.086 Minimum 78 Maximum 171 Range 93 Interquartile Range 30 Skewness .147 .524
Kurtosis .000 1.014
West Mean 137.33 4.197
95% Confidence Interval for Mean
Lower Bound 128.33 Upper Bound 146.34
5% Trimmed Mean 137.04 Median 138.00 Variance 264.238 Std. Deviation 16.255 Minimum 109 Maximum 171 Range 62 Interquartile Range 15 Skewness .203 .580
Kurtosis .474 1.121
Tests of Normality
Region
Kolmogorov-Smirnova Shapiro-Wilk
Statistic df Sig. Statistic df Sig.
HL score North .095 20 .200* .974 20 .831
East .190 14 .182 .907 14 .142
Central .145 11 .200* .982 11 .976
South .137 19 .200* .970 19 .772
West .187 15 .167 .959 15 .678
*. This is a lower bound of the true significance. a. Lilliefors Significance Correction
428
Descriptives
Teaching status Statistic Std. Error
HL score Teaching Mean 137.50 4.307
95% Confidence Interval for Mean
Lower Bound 128.02 Upper Bound 146.98
5% Trimmed Mean 137.44 Median 141.00 Variance 222.636 Std. Deviation 14.921 Minimum 112 Maximum 164 Range 52 Interquartile Range 20 Skewness -.222 .637
Kurtosis -.124 1.232
Non-teaching Mean 135.64 2.405
Lower Bound 130.84
429
95% Confidence Interval for Mean
Upper Bound 140.44
5% Trimmed Mean 136.17 Median 138.00 Variance 387.415 Std. Deviation 19.683 Minimum 78 Maximum 173 Range 95 Interquartile Range 27 Skewness -.312 .293
Kurtosis .180 .578
Tests of Normality
Teaching status
Kolmogorov-Smirnova Shapiro-Wilk
Statistic df Sig. Statistic df Sig.
HL score Teaching .207 12 .166 .944 12 .550
Non-teaching .055 67 .200* .985 67 .588
*. This is a lower bound of the true significance. a. Lilliefors Significance Correction
434
Appendix JJ: Tests of Normality for Continuous Independent Variables and Base-10 Logarithm Transformations of
Independent Variables
Acute beds, log transformation of acute beds
Descriptives
Statistic Std. Error
Acute beds Mean 180.10 23.227
95% Confidence Interval for Mean
Lower Bound 133.86 Upper Bound 226.34
5% Trimmed Mean 157.78 Median 90.00 Variance 42618.861 Std. Deviation 206.443 Minimum 8 Maximum 770 Range 762 Interquartile Range 213 Skewness 1.645 .271
Kurtosis 2.009 .535
log10AcuteBeds Mean 1.9543 .06225
95% Confidence Interval for Mean
Lower Bound 1.8303 Upper Bound 2.0782
5% Trimmed Mean 1.9566 Median 1.9542 Variance .306 Std. Deviation .55332 Minimum .90 Maximum 2.89 Range 1.98 Interquartile Range .96 Skewness -.079 .271
Kurtosis -1.068 .535
Tests of Normality
Kolmogorov-Smirnova Shapiro-Wilk
Statistic df Sig. Statistic df Sig.
Acute beds .202 79 .000 .768 79 .000 log10AcuteBeds .088 79 .200* .962 79 .018
*. This is a lower bound of the true significance. a. Lilliefors Significance Correction
439
Total number of beds, log transformation of total number of beds
Descriptives
Statistic Std. Error
Number of beds Mean 289.80 36.952
95% Confidence Interval for Mean
Lower Bound 216.23 Upper Bound 363.36
5% Trimmed Mean 253.35 Median 145.00 Variance 107872.369 Std. Deviation 328.439 Minimum 12 Maximum 1239 Range 1227 Interquartile Range 365 Skewness 1.581 .271
Kurtosis 1.839 .535
log10CIHIBeds Mean 2.1536 .06353
95% Confidence Interval for Mean
Lower Bound 2.0272 Upper Bound 2.2801
5% Trimmed Mean 2.1565 Median 2.1614 Variance .319 Std. Deviation .56469 Minimum 1.08 Maximum 3.09 Range 2.01 Interquartile Range .94 Skewness -.096 .271
Kurtosis -1.133 .535
Tests of Normality
Kolmogorov-Smirnova Shapiro-Wilk
Statistic df Sig. Statistic df Sig.
Number of beds .199 79 .000 .781 79 .000 log10CIHIBeds .113 79 .014 .956 79 .008
a. Lilliefors Significance Correction
444
Budget 2014-2015, log transformation of budget 2014-2015
Descriptives
Statistic Std. Error
Budget 2014-2015 Mean 252844904.52 37431643.669
95% Confidence Interval for Mean
Lower Bound 178324226.44 Upper Bound 327365582.60
5% Trimmed Mean 214200074.58 Median 107427797.00 Variance 1106891078735
34864.000
Std. Deviation 332699726.290 Minimum 5885706 Maximum 1484872537 Range 1478986831 Interquartile Range 327860843 Skewness 1.847 .271
Kurtosis 2.933 .535
log10Budget Mean 7.9708 .07721
95% Confidence Interval for Mean
Lower Bound 7.8170 Upper Bound 8.1245
5% Trimmed Mean 7.9717 Median 8.0311 Variance .471 Std. Deviation .68626 Minimum 6.77 Maximum 9.17 Range 2.40 Interquartile Range 1.18 Skewness -.069 .271
Kurtosis -1.258 .535
Tests of Normality
Kolmogorov-Smirnova Shapiro-Wilk
Statistic df Sig. Statistic df Sig.
Budget 2014-2015 .229 79 .000 .736 79 .000 log10Budget .114 79 .013 .946 79 .002
a. Lilliefors Significance Correction
449
Rurality, log transformation of rurality
Descriptives
Statistic Std. Error
Ruralityplus1 Mean 30.58 3.445
95% Confidence Interval for Mean
Lower Bound 23.72 Upper Bound 37.44
5% Trimmed Mean 28.52 Median 23.00 Variance 937.682 Std. Deviation 30.622 Minimum 1 Maximum 100 Range 99 Interquartile Range 52 Skewness .828 .271
Kurtosis -.513 .535
logRuralityplus1 Mean 1.0904 .08148
95% Confidence Interval for Mean
Lower Bound .9282 Upper Bound 1.2526
5% Trimmed Mean 1.1011 Median 1.3617 Variance .524 Std. Deviation .72417 Minimum .00 Maximum 2.00 Range 2.00 Interquartile Range 1.26 Skewness -.478 .271
Kurtosis -1.289 .535
Tests of Normality
Kolmogorov-Smirnova Shapiro-Wilk
Statistic df Sig. Statistic df Sig.
Ruralityplus1 .167 79 .000 .860 79 .000 logRuralityplus1 .174 79 .000 .858 79 .000
a. Lilliefors Significance Correction
454
Appendix KK: Scatterplots of Continuous Independent Variables
Log transformation of number of acute beds and the health literate survey score
456
Appendix LL: Regression Analysis for Size and Size-Squared
Descriptive Statistics
Mean Std. Deviation N
HL score 135.92 18.965 79 Size 1.9543 .55332 79 Square_size 4.1214 2.15882 79
Correlations
HL score Size Square_size
Pearson Correlation HL score 1.000 -.223 -.205
Size -.223 1.000 .991
Square_size -.205 .991 1.000
Sig. (1-tailed) HL score . .024 .035
Size .024 . .000
Square_size .035 .000 .
N HL score 79 79 79
Size 79 79 79
Square_size 79 79 79
Model Summaryc
Model R R Square Adjusted R Square
Std. Error of the Estimate
Change Statistics
Durbin-Watson R Square Change F Change df1 df2
Sig. F Change
1 .223a .050 .038 18.605 .050 4.048 1 77 .048 2 .255b .065 .041 18.575 .015 1.244 1 76 .268 1.720
a. Predictors: (Constant), Size b. Predictors: (Constant), Size, Square_size c. Dependent Variable: HL score
ANOVAa
Model Sum of Squares df Mean Square F Sig.
1 Regression 1401.088 1 1401.088 4.048 .048b
Residual 26652.457 77 346.136 Total 28053.544 78
2 Regression 1830.436 2 915.218 2.652 .077c
Residual 26223.108 76 345.041
457
Total 28053.544 78 a. Dependent Variable: HL score b. Predictors: (Constant), Size c. Predictors: (Constant), Size, Square_size
Model
Unstandardized Coefficients Standardized Coefficients
t Sig.
Collinearity Statistics
B Std. Error Beta Tolerance VIF
1 (Constant) 150.893 7.729 19.523 .000 Size -7.660 3.807 -.223 -2.012 .048 1.000 1.000
2 (Constant) 178.540 25.958 6.878 .000 Size -38.788 28.163 -1.132 -1.377 .172 .018 54.896
Square_size 8.052 7.218 .917 1.116 .268 .018 54.896
458
Appendix MM: Relationship between Teaching Status and Health Literate Survey Score
Model Summaryb
Model R R Square Adjusted R
Square Std. Error of the
Estimate
Change Statistics
Durbin-Watson R Square Change F Change df1 df2 Sig. F Change
1 .035a .001 -.012 19.076 .001 .097 1 77 .757 1.924
a. Predictors: (Constant), Teaching status b. Dependent Variable: HL score
ANOVAa
Model Sum of Squares df Mean Square F Sig.
1 Regression 35.141 1 35.141 .097 .757b
Residual 28018.403 77 363.875 Total 28053.544 78
a. Dependent Variable: HL score b. Predictors: (Constant), Teaching status
Coefficientsa
Model
Unstandardized Coefficients Standardized Coefficients
t Sig.
Correlations
B Std. Error Beta Zero-order Partial Part
1 (Constant) 137.500 5.507 24.970 .000 Teaching status -1.858 5.979 -.035 -.311 .757 -.035 -.035 -.035
a. Dependent Variable: HL score
459
Appendix NN: Relationship between Region and Health Literate Survey Score
Descriptive Statistics
Mean Std. Deviation N
HL score 135.92 18.965 79 Difference between Region 5 and Region 1
.1899 .39471 79
Difference between Region 4 and Region 1
.2405 .43012 79
Difference between Region3 and Region 1
.1392 .34841 79
Difference between Region2 and Region 1
.1772 .38429 79
Model Summaryb
Model R R Square Adjusted R
Square Std. Error of the
Estimate
Change Statistics
Durbin-Watson R Square Change F Change df1 df2 Sig. F Change
1 .399a .160 .114 17.850 .160 3.513 4 74 .011 1.819
a. Predictors: (Constant), Difference between Region2 and Region 1, Difference between Region3 and Region 1, Difference between Region 5 and Region 1, Difference between Region 4 and Region 1 b. Dependent Variable: HL score
ANOVAa
Model Sum of Squares df Mean Square F Sig.
1 Regression 4476.741 4 1119.185 3.513 .011b
Residual 23576.803 74 318.605 Total 28053.544 78
a. Dependent Variable: HL score b. Predictors: (Constant), Difference between Region2 and Region 1, Difference between Region3 and Region 1, Difference between Region 5 and Region 1, Difference between Region 4 and Region 1
Coefficientsa
460
Model
Unstandardized Coefficients Standardized Coefficients
t Sig.
Correlations
B Std. Error Beta Zero-order Partial Part
1 (Constant) 146.050 3.991 36.592 .000 Difference between Region 5 and Region 1
-8.717 6.097 -.181 -1.430 .157 .036 -.164 -.152
Difference between Region 4 and Region 1
-20.892 5.718 -.474 -3.654 .000 -.322 -.391 -.389
Difference between Region3 and Region 1
-7.868 6.700 -.145 -1.174 .244 .048 -.135 -.125
Difference between Region2 and Region 1
-13.264 6.220 -.269 -2.133 .036 -.077 -.241 -.227
a. Dependent Variable: HL score
461
Appendix OO: Relationship between Rurality and Health Literate Survey Score
Descriptive Statistics
Mean Std. Deviation N
HL score 135.92 18.965 79 logRuralityplus1 1.0904 .72417 79
Correlations
HL score logRuralityplus1
Pearson Correlation HL score 1.000 .127
logRuralityplus1 .127 1.000
Sig. (1-tailed) HL score . .131
logRuralityplus1 .131 .
N HL score 79 79
logRuralityplus1 79 79
Model Summaryb
Model R R Square Adjusted R
Square Std. Error of the
Estimate
Change Statistics
Durbin-Watson R Square Change F Change df1 df2 Sig. F Change
1 .127a .016 .003 18.932 .016 1.272 1 77 .263 1.856
a. Predictors: (Constant), logRuralityplus1 b. Dependent Variable: HL score
ANOVAa
Model Sum of Squares df Mean Square F Sig.
1 Regression 455.761 1 455.761 1.272 .263b
Residual 27597.784 77 358.413 Total 28053.544 78
a. Dependent Variable: HL score b. Predictors: (Constant), logRuralityplus1
462
Coefficientsa
Model
Unstandardized Coefficients
Standardized Coefficients
t Sig.
95.0% Confidence Interval for B Correlations Collinearity Statistics
B Std. Error Beta Lower Bound Upper Bound
Zero-order Partial Part Tolerance VIF
1 (Constant) 132.284 3.867 34.207 .000 124.584 139.985 logRuralityplus1 3.338 2.960 .127 1.128 .263 -2.556 9.232 .127 .127 .127 1.000 1.000
a. Dependent Variable: HL score
463
Appendix PP: Relationship between Organizational Size, Teaching Status and Rurality with Health Literate Survey Score
Correlations
HL score log10AcuteBeds logRuralityplus1 Teaching status
Pearson Correlation HL score 1.000 -.223 .127 -.035
log10AcuteBeds -.223 1.000 -.845 -.536
logRuralityplus1 .127 -.845 1.000 .612
Teaching status -.035 -.536 .612 1.000
Sig. (1-tailed) HL score . .024 .131 .378
log10AcuteBeds .024 . .000 .000
logRuralityplus1 .131 .000 . .000
Teaching status .378 .000 .000 .
N HL score 79 79 79 79
log10AcuteBeds 79 79 79 79
logRuralityplus1 79 79 79 79
Teaching status 79 79 79 79
Model Summaryc
Model R R Square Adjusted R
Square Std. Error of the
Estimate
Change Statistics
Durbin-Watson R Square Change F Change df1 df2 Sig. F Change
1 .294a .087 .050 18.485 .087 2.367 3 75 .078 2 .289b .084 .060 18.391 -.003 .232 1 75 .631 1.694
a. Predictors: (Constant), Teaching status, log10AcuteBeds, logRuralityplus1 b. Predictors: (Constant), Teaching status, log10AcuteBeds c. Dependent Variable: HL score
464
ANOVAa
Model Sum of Squares df Mean Square F Sig.
1 Regression 2426.754 3 808.918 2.367 .078b
Residual 25626.790 75 341.691 Total 28053.544 78
2 Regression 2347.427 2 1173.714 3.470 .036c
Residual 25706.117 76 338.238 Total 28053.544 78
a. Dependent Variable: HL score b. Predictors: (Constant), Teaching status, log10AcuteBeds, logRuralityplus1 c. Predictors: (Constant), Teaching status, log10AcuteBeds
Residuals Statisticsa
Minimum Maximum Mean Std. Deviation N
Predicted Value 123.58 146.44 135.92 5.486 79 Std. Predicted Value -2.251 1.917 .000 1.000 79 Standard Error of Predicted Value
2.247 5.614 3.423 1.067 79
Adjusted Predicted Value 121.17 146.61 135.93 5.568 79 Residual -48.566 35.820 .000 18.154 79 Std. Residual -2.641 1.948 .000 .987 79 Stud. Residual -2.710 1.968 .000 1.007 79 Deleted Residual -51.151 36.697 -.001 18.894 79 Stud. Deleted Residual -2.833 2.007 -.001 1.020 79 Mahal. Distance .177 6.281 1.975 1.903 79 Cook's Distance .000 .130 .014 .023 79 Centered Leverage Value .002 .081 .025 .024 79
a. Dependent Variable: HL score
465
Appendix QQ: Relationship between Organizational Size, Teaching Status and Region with Health Literate Survey Score
Descriptive Statistics
Mean Std. Deviation N
HL score 135.92 18.965 79 log10AcuteBeds 1.9543 .55332 79 Teaching status .85 .361 79 Difference between Region 5 and Region 1
.1899 .39471 79
Difference between Region 4 and Region 1
.2405 .43012 79
Difference between Region3 and Region 1
.1392 .34841 79
Difference between Region2 and Region 1
.1772 .38429 79
Model Summaryh
Model R R Square Adjusted R
Square Std. Error of the
Estimate
Change Statistics
Durbin-Watson R Square Change F Change df1 df2 Sig. F Change
1 .424a .180 .099 17.999 .180 2.227 7 71 .042 2 .413b .170 .101 17.980 -.010 .845 1 71 .361 3 .413c .170 .113 17.856 .000 .000 1 72 .986 4 .399d .160 .114 17.850 -.011 .943 1 73 .335 5 .379e .144 .110 17.895 -.016 1.379 1 74 .244 6 .362f .131 .108 17.907 -.013 1.108 1 75 .296 7 .322g .103 .092 18.074 -.028 2.441 1 76 .122 1.960
a. Predictors: (Constant), Difference between Region2 and Region 1, log10AcuteBeds, Difference between Region 5 and Region 1, Difference between large community and small community, Difference between Region3 and Region 1, Difference between Region 4 and Region 1, Difference between teaching and small community b. Predictors: (Constant), Difference between Region2 and Region 1, Difference between Region 5 and Region 1, Difference between large community and small community, Difference between Region3 and Region 1, Difference between Region 4 and Region 1, Difference between teaching and small community c. Predictors: (Constant), Difference between Region2 and Region 1, Difference between Region 5 and Region 1, Difference between large community and small community, Difference between Region3 and Region 1, Difference between Region 4 and Region 1 d. Predictors: (Constant), Difference between Region2 and Region 1, Difference between Region 5 and Region 1, Difference between Region3 and Region 1, Difference between Region 4 and Region 1 e. Predictors: (Constant), Difference between Region2 and Region 1, Difference between Region 5 and Region 1, Difference between Region 4 and Region 1
466
f. Predictors: (Constant), Difference between Region2 and Region 1, Difference between Region 4 and Region 1 g. Predictors: (Constant), Difference between Region 4 and Region 1 h. Dependent Variable: HL score
ANOVAa
Model Sum of Squares df Mean Square F Sig.
1 Regression 5051.247 7 721.607 2.227 .042b
Residual 23002.297 71 323.976 Total 28053.544 78
2 Regression 4777.529 6 796.255 2.463 .032c
Residual 23276.015 72 323.278 Total 28053.544 78
3 Regression 4777.431 5 955.486 2.997 .016d
Residual 23276.113 73 318.851 Total 28053.544 78
4 Regression 4476.741 4 1119.185 3.513 .011e
Residual 23576.803 74 318.605 Total 28053.544 78
5 Regression 4037.392 3 1345.797 4.203 .008f
Residual 24016.152 75 320.215 Total 28053.544 78
6 Regression 3682.552 2 1841.276 5.742 .005g
Residual 24370.992 76 320.671 Total 28053.544 78
7 Regression 2899.685 1 2899.685 8.876 .004h
Residual 25153.860 77 326.674 Total 28053.544 78
a. Dependent Variable: HL score b. Predictors: (Constant), Difference between Region2 and Region 1, log10AcuteBeds, Difference between Region 5 and Region 1, Difference between large community and small community, Difference between Region3 and Region 1, Difference between Region 4 and Region 1, Difference between teaching and small community
467
c. Predictors: (Constant), Difference between Region2 and Region 1, Difference between Region 5 and Region 1, Difference between large community and small community, Difference between Region3 and Region 1, Difference between Region 4 and Region 1, Difference between teaching and small community d. Predictors: (Constant), Difference between Region2 and Region 1, Difference between Region 5 and Region 1, Difference between large community and small community, Difference between Region3 and Region 1, Difference between Region 4 and Region 1 e. Predictors: (Constant), Difference between Region2 and Region 1, Difference between Region 5 and Region 1, Difference between Region3 and Region 1, Difference between Region 4 and Region 1 f. Predictors: (Constant), Difference between Region2 and Region 1, Difference between Region 5 and Region 1, Difference between Region 4 and Region 1 g. Predictors: (Constant), Difference between Region2 and Region 1, Difference between Region 4 and Region 1 h. Predictors: (Constant), Difference between Region 4 and Region 1
Coefficientsa
Model
Unstandardized Coefficients
Standardized Coefficients
t Sig.
95.0% Confidence Interval for B Correlations Collinearity Statistics
B Std. Error Beta
Lower Bound
Upper Bound
Zero-order Partial Part Tolerance VIF
1 (Constant) 155.135 10.261 15.119 .000 134.675 175.594 log10AcuteBeds -6.241 6.790 -.182 -.919 .361 -19.779 7.297 -.223 -.108 -.099 .294 3.398
Difference between teaching and small community
6.329 9.441 .121 .670 .505 -12.496 25.155 .035 .079 .072 .357 2.800
Difference between large community and small community
-.697 6.348 -.018 -.110 .913 -13.354 11.960 -.193 -.013 -.012 .452 2.210
Difference between Region 5 and Region 1
-6.101 6.462 -.127 -.944 .348 -18.987 6.784 .036 -.111 -.101 .638 1.566
Difference between Region 4 and Region 1
-16.661 6.675 -.378 -2.496 .015 -29.971 -3.352 -.322 -.284 -.268 .504 1.985
Difference between Region3 and Region 1
-3.734 7.834 -.069 -.477 .635 -19.356 11.887 .048 -.056 -.051 .557 1.794
Difference between Region2 and Region 1
-11.581 6.484 -.235 -1.786 .078 -24.510 1.349 -.077 -.207 -.192 .669 1.495
468
2 (Constant) 146.502 4.128 35.494 .000 138.273 154.730 Difference between teaching and small community
-.110 6.322 -.002 -.017 .986 -12.714 12.493 .035 -.002 -.002 .795 1.258
Difference between large community and small community
-4.406 4.895 -.111 -.900 .371 -14.164 5.353 -.193 -.105 -.097 .759 1.317
Difference between Region 5 and Region 1
-7.399 6.300 -.154 -1.174 .244 -19.957 5.159 .036 -.137 -.126 .670 1.492
Difference between Region 4 and Region 1
-19.245 6.047 -.436 -3.182 .002 -31.301 -7.190 -.322 -.351 -.342 .613 1.632
Difference between Region3 and Region 1
-5.877 7.472 -.108 -.787 .434 -20.771 9.018 .048 -.092 -.084 .612 1.635
Difference between Region2 and Region 1
-12.434 6.411 -.252 -1.939 .056 -25.213 .346 -.077 -.223 -.208 .683 1.464
3 (Constant) 146.487 4.018 36.457 .000 138.479 154.495 Difference between large community and small community
-4.373 4.503 -.110 -.971 .335 -13.349 4.602 -.193 -.113 -.104 .885 1.130
Difference between Region 5 and Region 1
-7.405 6.247 -.154 -1.185 .240 -19.855 5.046 .036 -.137 -.126 .672 1.487
Difference between Region 4 and Region 1
-19.258 5.963 -.437 -3.230 .002 -31.142 -7.374 -.322 -.354 -.344 .621 1.609
Difference between Region3 and Region 1
-5.920 6.997 -.109 -.846 .400 -19.864 8.024 .048 -.099 -.090 .688 1.454
Difference between Region2 and Region 1
-12.452 6.278 -.252 -1.983 .051 -24.965 .061 -.077 -.226 -.211 .702 1.424
4 (Constant) 146.050 3.991 36.592 .000 138.097 154.003 Difference between Region 5 and Region 1
-8.717 6.097 -.181 -1.430 .157 -20.865 3.431 .036 -.164 -.152 .705 1.418
Difference between Region 4 and Region 1
-20.892 5.718 -.474 -3.654 .000 -32.286 -9.498 -.322 -.391 -.389 .675 1.481
469
Difference between Region3 and Region 1
-7.868 6.700 -.145 -1.174 .244 -21.219 5.483 .048 -.135 -.125 .750 1.334
Difference between Region2 and Region 1
-13.264 6.220 -.269 -2.133 .036 -25.658 -.871 -.077 -.241 -.227 .715 1.399
5 (Constant) 143.258 3.214 44.574 .000 136.856 149.661 Difference between Region 5 and Region 1
-5.925 5.628 -.123 -1.053 .296 -17.137 5.287 .036 -.121 -.112 .832 1.202
Difference between Region 4 and Region 1
-18.100 5.214 -.411 -3.472 .001 -28.486 -7.714 -.322 -.372 -.371 .816 1.225
Difference between Region2 and Region 1
-10.472 5.762 -.212 -1.817 .073 -21.951 1.006 -.077 -.205 -.194 .837 1.194
6 (Constant) 141.326 2.640 53.527 .000 136.068 146.585 Difference between Region 4 and Region 1
-16.168 4.883 -.367 -3.311 .001 -25.895 -6.442 -.322 -.355 -.354 .932 1.073
Difference between Region2 and Region 1
-8.540 5.466 -.173 -1.562 .122 -19.427 2.346 -.077 -.176 -.167 .932 1.073
7 (Constant) 139.333 2.333 59.714 .000 134.687 143.980 Difference between Region 4 and Region 1
-14.175 4.758 -.322 -2.979 .004 -23.650 -4.701 -.322 -.322 -.322 1.000 1.000
a. Dependent Variable: HL score
470
Residuals Statisticsa
Minimum Maximum Mean Std. Deviation N
Predicted Value 125.16 139.33 135.92 6.097 79 Std. Predicted Value -1.766 .559 .000 1.000 79 Standard Error of Predicted Value
2.333 4.146 2.769 .780 79
Adjusted Predicted Value 122.61 139.85 135.92 6.135 79 Residual -47.158 45.842 .000 17.958 79 Std. Residual -2.609 2.536 .000 .994 79 Stud. Residual -2.681 2.606 .000 1.010 79 Deleted Residual -49.778 48.389 .000 18.553 79 Stud. Deleted Residual -2.797 2.711 .001 1.025 79 Mahal. Distance .313 3.118 .987 1.207 79 Cook's Distance .000 .200 .017 .037 79 Centered Leverage Value .004 .040 .013 .015 79
a. Dependent Variable: HL score
471
Appendix RR: Post Hoc Analyses to Examine Difference in Number of Acute Beds Between Regions
Group Statistics
Region N Mean Std. Deviation Std. Error Mean
log10AcuteBeds North 20 1.5459 .49570 .11084
South 19 2.1875 .52118 .11957
Independent Samples Test
Levene's Test for Equality of Variances t-test for Equality of Means
F Sig. t df Sig. (2-tailed) Mean
Difference Std. Error Difference
95% Confidence Interval of the Difference
Lower Upper
log10AcuteBeds Equal variances assumed
.020 .889 -3.940 37 .000 -.64157 .16283 -.97148 -.31165
Equal variances not assumed
-3.935 36.614 .000 -.64157 .16304 -.97204 -.31110
Group Statistics
Region N Mean Std. Deviation Std. Error Mean
log10AcuteBeds North 20 1.5459 .49570 .11084
East 14 1.9109 .49207 .13151
472
Independent Samples Test
Levene's Test for Equality of Variances t-test for Equality of Means
F Sig. t df Sig. (2-tailed) Mean
Difference Std. Error Difference
95% Confidence Interval of the Difference
Lower Upper
log10AcuteBeds Equal variances assumed
.011 .916 -2.119 32 .042 -.36494 .17222 -.71574 -.01413
Equal variances not assumed
-2.122 28.269 .043 -.36494 .17199 -.71710 -.01278
473
Appendix SS: Regression Analysis to Examine Interaction between Region and Size as Predictors of Health Literate Survey
Score
************** PROCESS Procedure for SPSS Release 2.15 *******************
Written by Andrew F. Hayes, Ph.D. www.afhayes.com
Documentation available in Hayes (2013). www.guilford.com/p/hayes3
**************************************************************************
Model = 1
Y = HLscore
X = Region
M = Size
Sample size
79
Coding of categorical X variable for analysis:
Region D1 D2 D3 D4
1.00 .00 .00 .00 .00
2.00 1.00 .00 .00 .00
3.00 .00 1.00 .00 .00
4.00 .00 .00 1.00 .00
5.00 .00 .00 .00 1.00
**************************************************************************
Outcome: HLscore
Model Summary
R R-sq MSE F df1 df2 p
.4502 .2027 324.1711 1.6151 9.0000 69.0000 .1281
Model
coeff se t p LLCI ULCI
constant 143.3130 5.0010 28.6571 .0000 133.3363 153.2896
Size -6.7034 7.8364 -.8554 .3953 -22.3366 8.9298
D1 -10.1534 6.4766 -1.5677 .1215 -23.0739 2.7670
D2 -11.2858 9.8927 -1.1408 .2579 -31.0212 8.4496
D3 -15.8486 8.0807 -1.9613 .0539 -31.9693 .2721
D4 -6.3514 6.9585 -.9128 .3646 -20.2332 7.5305
int_1 15.3219 10.9729 1.3963 .1671 -6.5685 37.2123
int_2 19.7503 17.6699 1.1177 .2676 -15.5004 55.0009
int_3 -3.1846 14.2436 -.2236 .8237 -31.5998 25.2307
int_4 .1252 13.8094 .0091 .9928 -27.4239 27.6743
Product terms key:
int_1 : D1 X Size
int_2 : D2 X Size
int_3 : D3 X Size
int_4 : D4 X Size
474
Appendix TT: Regression Analysis to Examine Interaction between Rurality and Size as Predictors of Health Literate Survey
Score
************** PROCESS Procedure for SPSS Release 2.15 *******************
Written by Andrew F. Hayes, Ph.D. www.afhayes.com
Documentation available in Hayes (2013). www.guilford.com/p/hayes3
**************************************************************************
Model = 1
Y = HLscore
X = logABeds
M = logRone
Sample size
79
**************************************************************************
Outcome: HLscore
Model Summary
R R-sq MSE F df1 df2 p
.3579 .1281 326.1423 3.3112 3.0000 75.0000 .0245
Model
coeff se t p LLCI ULCI
constant 130.0213 3.1038 41.8909 .0000 123.8381 136.2044
logRone 1.2174 6.0355 .2017 .8407 -10.8060 13.2408
logABeds -7.7952 8.2192 -.9484 .3460 -24.1688 8.5783
int_1 -17.6642 6.6759 -2.6460 .0099 -30.9633 -4.3651
Product terms key:
int_1 logABeds X logRone
top related