Understanding the Impact of the Rural Health Network Development and Rural Health Care Services Outreach Grant Programs: A Sustainability Assessment of.

Understanding the Impact of the Rural Health Network Development and

Rural Health Care Services Outreach Grant Programs:

A Sustainability Assessment of Former Grantees

Community Health System Development TeamGeorgia Health Policy Center

August 2, 2010

Today’s Objectives

• Present the goals of the Sustainability Assessment of former ORHP grantees

• Share findings and insights

Sustainability Assessment

• Purposes: – Describe the extent to which ORHP-funded

community-based demonstration programs have been sustained post-funding

– Identify recurrent patterns that influence likelihood of sustainability

Sustainability Assessment

• Population studied:– Rural Health Care Services Outreach and

Rural Health Network Development grantees funded in 2000, 2002 and 2004.

• Telephone interviews completed with 102 of 129 grantees

Sustainability Assessment

• Analysis Approach– Study designed with rural communities in

mind– Findings applicable to a wide audience:

rural communities and funders– Qualitative and quantitative analyses

conducted

Sustainability Assessment

• Summary of conclusion:– No discrete set of best practices. – Each situation has unique context.

Transferability of practices may be limited.– There are dynamics, though, that impact

sustainability.

Findings and Insights

1. Broadening the Concept of Sustainability

2. Selected Quantitative Findings

3. Dynamics that Influence Sustainability

Framing the Sustainability Assessment

• Broadening the concept of sustainability– More than just whether or not the grant-funded

activities and collaboratives have been sustained - such an assessment understates grantees’ long-term impact and misses the opportunity to improve outcomes

– We must look in a more comprehensive way • What has been sustained…for how long?• How it has been sustained?• Why it has been sustained (or not)?

What has been sustained?

What has been sustained?

And:

Community and Population Benefits Sustained

• Culture changes– Relationships, foundation for working

together, trust, resolved conflict• Practice standards• Policy• Public awareness and perceptions• Capacity

Assets Purchased or Created

• Equipment

• Curricula

• HIT capacity

How Have They Been Sustained?Means of Sustaining Activities

• Absorbed by partners• Other grants• In-kind• Third party reimbursement• Client fees• Taken over by newly-formed FQHC

How Have They Been Sustained?Means of Sustaining Coalitions

• Member in-kind contributions/ Volunteerism

• Grants

• Service revenue

• Member financial contributions/ dues

Stories Provide Windows of Understanding

• 102 grantees interviewed

• Each told a story– Insights– Examples– Reality

Grantee Stories

1. Read the story

2. Reflect individually– What was sustained? Why?

3. Discuss in small group

Grantees InterviewedTotal

(n) OutreachNetwork

Development

Grantees to be interviewed

129 100 29

Interviews Completed

79% (102) 78% (78) 83% (24)

Contact made, Confirmed not sustained

3% (4) 3% (3) 3% (1)

Declined participation

6% (6) 6% (6) 0

Not able to locate/ Unresponsive

17% (17) 13% (13) 14% (4)

What was Sustained:Status of Outreach Consortia*

Consortium Currently Sustained Outreach Grantees

Yes, formally 34% (26)

Yes, informally 12% (9)

No 55% (42)

*Based on 77 of the 78 Outreach grantee programs reviewed. One grantee was unable to confirm the current status of the consortium.

What was Sustained:Status of Networks*

Network Sustained Currently Network Development grantees

Yes, formally 61% (14)

Yes, informally 4% (1)

No 35% (8)

*This table presents information from 23 of 24 Network Development grantees interviewed. One informant was unable to confirm the current status of the Network.

The Means of Sustaining Outreach Consortia

Means of Sustaining the Consortium* Outreach grantees that have sustained consortia, formally or

informally (n=35)

Member In-kind contributions/ Volunteerism 71% (25)

Grants 46% (16)

Service revenue 23% (8)

Member financial contributions/dues 11% (4)

Fundraising and Donations 6% (2)

*Grantees may have reported multiple means of sustaining their consortia

The Means of Sustaining Networks

Means of Sustaining the Network* Network Development grantees that sustained Network post grant period,

formally or informally (n=15)

Grants 67% (10)

Member In-kind contributions/ Volunteerism

53% (8)

Service revenue53% (8)

Member financial contributions/dues27% (4)

*Grantees may have reported multiple means of sustaining their Networks

What was Sustained:Activities and Services

All ORHP-funded activities sustained

for any period of time post-funding

Some ORHP-funded activities sustained

for any period of time

None of the ORHP-funded activities following end of

grant period

Network Development

Grantees

63% (15) 25% (6) 13% (3)

Outreach Grantees 60% (47) 36% (28) 4% (3)

Total 61% (62) 33% (34) 6% (6)

What was Sustained:Activities and Services

All grantees All ORHP-funded activities sustained

currently

Some ORHP-funded activities sustained

currently

None of the ORHP-funded activities

sustained currently

2000 Cohort (n=51) 47% (24) 37% (19) 16% (8)

2002 Cohort (n=33) 42% (14) 39% (13) 18% (6)

2004 Cohort (n=18) 61% (11) 33% (6) 6% (1)

Total (n=102) 48% (49) 37% (38) 15% (15)

.

Means of Sustaining Activities and ServicesPercentage of Grantees

Sustaining Activity by mean* (n)N=96

Percentage of ActivitiesSustained by Mean (n)

N=214

Absorbed by partner 57% (55) 42% (89)

Other Grants 46% (44) 31% (67)

In Kind/Volunteerism 46% (44) 31% (67)

Third Party Reimbursement 39% (37) 27% (57)

Client Fees 27% (26) 15% (33)

Government/Public Subsidy 23% (22) 20% (42)

Earned Income 14% (13) 9% (19)

Taken Over by Newly Formed FQHC

11% (11) 10% (21)

Member Fees/Dues 5% (5) 6% (13)

Equipment/Resources still available in the community

4% (4) 2% (5)

* Grantees reported multiple means of sustaining each activity

Insights from the Sustainability Assessment

• Sustainability requires a broader definition– What gets sustained?– How is it sustained?

• For this assessment to be applicable to other rural communities, it must also explain the dynamics that influence sustainability– Why (or not) is it sustained?

Dynamics Influencing Grantee Sustainability

• No set of best practices or criteria that determine level of sustainability success– Context specific– Inter-relatedness of factors

• There are, however, dynamics that seem to influence sustainability– Useful in predicting success– Can impact some factors to favorably, or unfavorably,

change dynamics

Influencing Dynamics

WHO Dynamic

• Primarily about leadership– Ability to engage the “right” people

• Strategic• Influential

– Create shared ownership and commitment– Resolve conflict

WHO Dynamic

Favorable Characteristics Unfavorable Characteristics

Engaged leaders with strategic mindset

Partners sense struggle for control

Widespread sense of trust among partners

Lack of consistent presence by partners

Key decision makers and influencers are involved or easily accessed

Conflict left unresolved

Partners share passion and purpose for the program

Belief that each partner has own agenda first

Vision for program is shared Partners are informed about activities not engaged in leading and planning

Examples

The right people at the table

• A network in the Northeast writes participation in Network into job descriptions of CEOs

• A consortia in the Midwest lost the decision-makers and lost its way.

WHAT Dynamic

• Related to the substance of the intervention– Designed program demonstrates:

• Relevance• Practicality• Value

– Program is aligned to community context

WHAT Dynamic

Favorable Characteristics Unfavorable Characteristics

Solution based on clear assessment of community needs

Solution only addresses symptoms and not root causes

Partners have leverage to address the problem long-term

Solution approach does not match partner and provider culture

Stakeholders benefit directly (Network)

Legal barriers – real or perceived – halt progress

Program or service has a value that others would be willing to reimburse and/or purchase

Stakeholders and potential partners do not perceive recognizable value of solution

The solution evolves and adapts as the environment changes and knowledge is acquired

Solution attempts to solve too many issues with too many small, unrelated efforts

Examples

Aligning approach to context/root cause of issue:

• Training program for EMT Medical Directors in the Mountain West

• Providing primary care to poor, uninsured through a cost sharing model in the Pacific Northwest

WHY Dynamic

• About the motivation for partners to work together– Well-defined vision for working together– Shared vision and understanding of roles– Use HRSA grants as a means to an end

versus being an end unto itself

WHY Dynamic

Favorable Characteristics Unfavorable Characteristics

There is a clear focus to the vision Sense of fear and desperation among partners

Efforts are aligned and prioritized to vision and goals

Many partners believe that others are just “looking for money”

Partners have formed a strong sense of group identity

Partners believe they’ve come together just to deliver on grant

Focused both on system change and meeting partner member needs

All goals and plans are short-term without alignment to a larger vision

Clear understanding of need based on evidence

Partners can not clearly explain why they are at the table

Examples

Shared vision and understanding of project goals

• Network in New England coordinates case management, provider training and a regional needs assessment

• School-based referral network in a Plains state

HOW Dynamic

• Relates to the way programs or networks are operationalized– Effective strategies employed– Adequate capacity built– Impact and value is documented and

communicated

HOW Dynamic

Favorable Characteristics Unfavorable Characteristics

Capable and skilled staff are employed to deliver solution

Sustainability is a topic of discussion late in the grant period

Impact and value is documented and communicated

Organization lacks staff capacity to fully implement solution

Partner assets are utilized to integrate program components into existing infrastructure

Spend resources on unreasonable and unwarranted overhead expenses

Focused both on system change and meeting partner member needs

Value of program is communicated only to partners

Clear understanding of need based on evidence

Coalition fails to begin with the end in mind

Examples

How a plan is put into action - Efficiency and effectiveness:

• Establishing a clinic in the fire department in the Northwest: developing new infrastructure vs. building on existing assets

• Hiring new staff vs. building the capacity of existing staff

WHERE and WHEN Dynamic

• Factors typically beyond the control of grantees– The legal, social, cultural, and economic

environment at the time of program implementation

– Coincidental trends at the time of program delivery

– Makes each situation unique and prevents application of universal best practices

WHERE and WHEN Dynamics

Where Factors When Factors

State policies and mandates Economic pressures

Geography of program location Receipt of multiple grants

Availability of technology Length of planning

Reimbursement climate History of working together & turnover of key staff

Organizational culture – university, FQHC, PH

Idea surfaces before its time – acceptance, technology, etc.

Hostile or competitive environment Ability to collect timely results to demonstrate impact

Examples

Timing and context

• Establishing a program to certify medical interpreters in a hospital in the Southeast

• Building a mental health tele-network in the Mid Atlantic

Review• Communities that receive grant funding have

sustained a broad range of community impacts– From short term outputs to long term community

benefits like changes in policy or practice • Most communities report sustaining activities

through diversified funding streams – Absorption, In-kind, Other grants, Reimbursement

• Sustainability cannot be attributed to any one factor or type of intervention.– The “who, what, why, where, when and how”

factors combine in infinite ways leading to a broad range of outcomes at the community level

Questions?

Thank youFor more information, contact:

Amanda Phillips [email protected]

404-413-0293