Module 3 Evaluation of Partnerships NBCCEDP Enhancing Program Performance through Evaluation.

Module 3

Evaluation of Partnerships

NBCCEDPEnhancing Program Performance

through Evaluation

Module 3: Evaluation of Partnerships

Welcome to Module 3: Evaluation of Partnerships. This module consists of the following sections:

Choose from the menu above to go to a specific section or choose the next button below to go view the objectives, overview, and organization of this module.

Objectives

Role of Partnerships

CDC Framework for Program Evaluation

Your Evaluation Plan for Partnerships

References

Module 3 Objectives

At the conclusion of Module 3, you should be able to:

• Describe why evaluating partnerships is important

• Identify two evaluation questions related to partnerships

• Describe outcomes related to partnerships• Conduct an evaluation of your partnership

activities

Module 3 Overview

This module will show you how to evaluate partnerships and coalitions that are integral to your program’s success. It will lead you through the steps in evaluating partnership activities using the CDC Framework for Program Evaluation.

The module will present an evaluation case study of a partnership through each step of the CDC Framework for Program Evaluation. You will have the opportunity to produce an evaluation workplan for evaluating partnership activities at the end of the module.

Organization of Module 3

In this module, you will cover these sections:

1. Description of partnerships and evaluation

2. Presentation of the CDC Framework for Program Evaluation and a partnership case study

3. Development of your own evaluation plan

Who are partners?Partners are usually groups of people who share common goals.

They may participate in the following ways:• Network and exchange information

• Plan and coordinate screening events and programs

• Generate resources for partnership activities

• Provide technical assistance and training on the issue of cancer screening

• Advocate for individual and system supports for screening and treatment of breast and cervical cancer

Role of Partnerships

Partnership activities are important to the BCCEDP because of their ability to expand the reach, capacity, and resources of the program.

Partners can help by:• coordinating program activities,

such as free screening days,• overcoming obstacles for

recruiting priority populations,• expanding the program’s

visibility, and• promoting the delivery,

improvement, and evaluation of breast and cervical cancer screening services

If you would like more information about Partnerships, you can review the Partnership Chapter of the Program Guidance Manual.

Click here to link to the Partnership Chapter.* PDF | WORD

*This chapter was made available in April 2007. Please check with your program consultant regarding potential updates to the chapter.

More Information on Partnerships

Where to Begin with PartnershipsPartnerships should be initiated to

meet specific strategic needs. Some questions a program might ask to assess its needs are:

• Where is the program having difficulties reaching its goals and objectives?

• What could be done more effectively with strategic partners?

• How might strategic partnerships assist in program efforts?

• To what extent do the perceived benefits of partnerships outweigh the perceived costs?

Assessing Potential Partnerships

Finding the right partner is critical. Some things to consider include:

• What potential partners have been identified?

• What is the expected outcome of the partnership?

• Has the program sought out new and non-traditional partners?

• Which partners could help the program better achieve its goals and objectives?

• Does the program have a history of good relations with these potential partners?

• What specific resources will these potential partners contribute to the outcomes or products expected from the partnership?

• What might be some potential drawbacks in partnering with a particular organization?

Role of Evaluation

Programs must evaluate their partnerships in order to ensure that they are meeting the group’s goals and objectives that led to the formation of the partnership.

Assessment of the partnership component may encompass three major areas depending on the reason for establishing the partnership:

• Expanding and maximizing resources

• Collaborating on program activities and goals

• Overcoming obstacles to the conduct of component areas such as recruitment, quality assurance and professional development activities

Evaluation and PartnershipsEvaluating partnerships and coalitions is important because it can ask questions about three areas of a partnership.

Capacity•Ensure participation from all relevant constituents•Improve decision-making ability

Operations•Improve communications•Enhance member involvement

Effectiveness•Meet major goals and activities•Measure the level of satisfaction with the program•Assess whether the partnership is effective

Evaluating Partnerships

Evaluating partnership should be guided by specific and measurable objectives that reflect a program’s interests and priorities. Refer back to goals and objectives set by the partners to develop measurable outcome measures.

For example, for the major area of Expanding and maximizing resources, one outcome measure might be:By the end of the second quarter, two partners will assist in the development of an evaluation plan for the recruitment component of the program.

For Overcoming obstacles for recruiting priority populations, one outcome measure might be:By the end of the fiscal year, the number of women recruited from the African-American population will increase from 50 to 150 through the assistance of local beauty salons and churches.

Referring back to the partnership section of your workplan will be essential for guiding any evaluation process.

Partnership OutcomesThe following are a sample of partnership outcomes that can help program staff create measurable objectives and target evaluation activities.

Process-Centered Outcomes

Use of partnerships to recruit and retain providers.

Use of partnerships to promote professional development activities.

Service Delivery

Use of partnerships to promote and facilitate breast and cervical cancer screening.

Effectiveness of partnerships to conduct program activities over the life of the program.

Early Detection

Sustainability and effectiveness of partnerships to address breast and cervical cancer early detection issues.

Engagement of community partners in activities that promote breast and cervical cancer screening services for NBCCEDP priority populations.

Why is Evaluation Important?Program Reflection

Now, think about your program.

Which organizations do you partner with to help implement your program objectives?

Are there other groups or agencies that may be helpful to reaching priority populations?

How might you evaluate your joint activities?

Where to Begin with Evaluating Partnerships?

You may want to evaluate your partnership based on the stage of development for your partnership. These stages include formation, maintenance, and institutionalization.

You may also want to ask questions about your partnership. Areas of partnerships that are typically evaluated include:•Capacity: skills, expertise, and diversity of the membership

•Operations (Processes and Structure): participation, communications, procedures, and implementation activities

•Effectiveness: individual (screening) and community level changes (community education, training or policy changes within provider networks to promote screening)Source: Granner & Sharpe, 2004

CDC Framework for Evaluation

In the next section, we will review the CDC Framework for Program Evaluation. It is a process that you can use to conduct an evaluation project.

Click on CDC Framework for Program Evaluation if you want to learn more about the Framework.

CDC Framework for Evaluation and Case Study IntroductionAs we apply the CDC Framework for Program Evaluation to the program component of partnership, you will be presented with a case study for each step.

The case study will focus on evaluating diversity of partner organizations. You will see this icon when we present the example.

CDC Framework for Program EvaluationNow let’s begin working through the CDC Framework for Program Evaluation to see how we would evaluate partnerships. Let’s quickly review the six steps in the evaluation process. We will go through each of these in our example of evaluating a partnership activity.

The six steps in the Evaluation for Program Framework are:

1. Engage stakeholders2. Describe the program3. Focus the evaluation design4. Gather credible evidence5. Justify conclusions6. Ensure use and share lessons learned

CDC Framework for Program Evaluation Step 1: Engaging Stakeholders

Initially, it is important to include program stakeholders in the evaluation process. Consider who may be helpful in evaluating partnerships and who would benefit most from the evaluation findings.

Possible stakeholders for partnerships are: • Program director• Recruitment coordinator• Professional education coordinator• Partnership coordinator• Program partners• Local health department• Local American Cancer Society• Advocacy groups• Members of a near by comprehensive

cancer consortium• Program clients

Partnership Case Study

This is the case study that we will use as we walk through the CDC Framework for Program Evaluation.

Your program has been operating for five years now. The management, recruitment, tracking and follow-up, as well as case management program components are operating smoothly and effectively based on your previous program evaluation and progress reports.

However, there is still work to be done in meeting the goal to reach priority populations in your community. You decide it is time to evaluate your partnership, the Partnership for Women’s Health. There may be opportunities to strengthen the existing partnership or develop a new relationship in an effort to reach more women in these priority populations. Therefore, an evaluation plan is set in motion!

Engage StakeholdersCase Study

You work with staff to help you develop a list of stakeholders to include in the evaluation process.

The list of stakeholders may include:• your program director who can help provide insight into potential

partners and provide historical background on existing partnerships;• your professional education coordinator who performs a lot of outreach

activities and helps manage various partnerships;• your recruitment coordinator;• the local American Cancer Society unit office;• the cancer control coordinator at the local health department; • the community faith-based coalition that reaches African-American

women;• the local Asian/Pacific Islander business coalition; and• the local Hispanic Health Alliance.

CDC Framework for Program Evaluation Step 2: Describe the Program

Once you have the appropriate stakeholders at the table, you need to make sure that they all have the same knowledge and information about the program so they view it from a shared point of view. You will need to describe your partnership and coalition activities and their intended purposes clearly.

Important aspects of the description for the partnership component of are: 1) need for the partnerships and coalitions,2) resources needed for maintaining the partnership and resources

leveraged by the partnership,3) partnership and coalition activities, 4) expected outcomes of partnerships and coalitions, and5) how partnership activities fit into your program

CDC Framework for Program Evaluation Step 2: Describe the Program

Statement of Need

Expectations Activities Resources

What we need to know to describe the rationale for the partnership

What the results will be (partnership goals)

What we need to do to change participation in screening

What resources do we need for our partnership efforts

Who is not screened?

To what extent are providers trained and promote screening?

How is it changing?

What are our expectations for the partnership?

What are immediate and intermediate consequences of partnership efforts?

What are our specific partnership activities?

Determine:

•time

•talent

•equipment

•information

•money

•other assets

Describe the Program Case StudyReferring back to the partnership’s statement of work, you begin reviewing the measurable objectives and goals that the group formed together.

The overall partnership goal is:

By creating the Partnership for Women’s Health, we will be able to increase efforts to promote preventive health through breast and cervical cancer screenings among our community’s most underserved women.

Some established partnership objectives include:

• By the end of the first year, each partner will help develop recruitment activities for specific populations.

• By the end of the first year, two partners will assist the program recruitment coordinator in establishing a mobile mammography event.

• By the end of the second year, the number of African-American women will increase from 50 to 100.

• By the end of the second year, the number of Latina women will increase from 100 to 300.

• By the end of the third year, the number of Asian/Pacific Islander women will increase from 30 to 75.

CDC Framework for Program Evaluation Step 3: Focus the Evaluation Design

Working together with your stakeholders, you will next need to clarify the primary purpose and uses for the evaluation and identify the most appropriate questions to ask.

Consider asking your stakeholders and staff what they want to know about the Partnership for Women’s Health such as:

• Extent of partnership activities• Extent to which partnership processes are

working• Measurable partnership objectives • Reach to program participants or potential

participants• Impact of partnership activities on program goals

and objectives • External factors that influence the partnership

activities (e.g. funding, demographics of your community, policies)

Priority Areas to Consider for Evaluation

Evaluation efforts need to be focused to be most effective due to constraints of resources and time.

First, examine your partnership section of the workplan in its entirety to help you choose the most pressing activities to focus your efforts. It is not efficient nor realistic to evaluate every aspect of partnership at one time.

As you review your partnership in the larger context, consider the following:

• Do partners provide information and insights which can lead to program improvement?

• Should relations between individual partners be continued?

• How satisfied are partners with the program?

• Are more women from priority populations being reached as a result of the partnership? What members are assisting in these efforts?

• Are partners working collaboratively to assist in program activities?

What are evaluation questions for partnerships?

Here are some example of general evaluation questions:

Type of Activity Question

Capacity What sectors of the community does the partnership represent?

Do the members represent the desired diversity and breadth of the partnership?

Processes and structure Are the communications of the partnership effective?

What factors contribute to member involvement in the partnership?

Effectiveness To what extent is the partnership working with organizations to recruit priority populations?

To what extent does the partnership help expand your professional development activities?

Types of Evaluation

Type Questions

Process evaluation Capacity:

Are the right partners on board?

Do we have the capacities to carryout our goals? (resources)

Processes/Structure:

How is the partnership operating?

What are milestones and critical events (activities)?

What has been the participation in partnership events?

What are the reactions to the activities?

There are three types of evaluation. You can also form evaluationquestions based on process, impact (short-term results) and outcomes (long-termresults) evaluation.

Types of Evaluation

Type Questions

Impact evaluation

(intermediate results)

Is the partnership meeting its objectives?

What has happened or changes as a result of the partnership? (learning or attitude change )

Have the activities lead to women being screened? (behavior or organizational change)

Outcome evaluation

(long-term results)

Effectiveness:

To what extent has the partnership changed recruitment, professional development or quality assurance activities (action)?

How has the partnership impacted cancer incidence, morbidity and mortality?

Focus the EvaluationCase Study

Now you need to develop your evaluation questions. It will be simpler to focus on one or two questions to begin with. Do you know where you would begin? Refer back to your measurable objectives to identify the most appropriate question to ask regarding the Partnership for Women’s Health.

By working with your stakeholders and staff, you realize it might be appropriate to evaluate whether you have all the right partners at the table to help reach your priority populations. In other words, you ask the question:

Now that you have your evaluation question, you can concentrate on what type of data to collect and how you will obtain them to be able to answer this question.

Does the Partnership for Women’s Health members represent all relevant constituencies or stakeholders to reach our priority populations?

CDC Framework for Program Evaluation Step 4: Gather Credible Evidence

A combination of quantitative and qualitative data can provide the most accurate view of the partnership activities of your program.

For qualitative data collection, you may find it useful to:•review partnership meeting notes to determine whether partners are providing information and insights to help improve the program•review partnership activity logs for the types of activities and changes that have resulted in the community (click here to see an example of a log), •interview partners on how to maximize the capacity or increase the efficiency of operations of the partnership, or•conduct interviews or focus groups with partners and program staff to assess whether people are working collaboratively to assist in program activities.


Some examples of quantitative data collection to evaluate partnerships include:•reviewing meeting logs to track partner’s participation, •counting milestones or critical events related to the partnership, •conducting a survey to measure areas related to the capacity and operations of the partnership, or •reviewing enrollment, screening, and diagnostic service forms and vital records to assess whether your partnership activities are reaching priority populations for screening.

You may also want to review progress reports to assess if the number of women screened has increased, remained the same, or declined since joint partnership activities were initiated.

Examples of Evaluation Data Collection for Partnership

Quantitative Data

Enrollment, screening, and diagnosis service formsTracking of media clipsLogs and sign-in sheets from partnership meetings and/or activities Medical records of clients recruited through partnership activitiesPartner surveyProgress reports

Qualitative Data

Exit interviews with participants

Partner and/or staff interviews

Partner meeting notes

Observations of partner activities including partner involvement

Site visit/observation of activities sponsored by partners

Administrative reports

The following examples are types of quantitative and qualitative data collection methods. Using a variety of different strategies can provide a comprehensive view of your partnership efforts.


Data are specific, measurable characteristics that represent an achievement of the outcome (e.g., number of organizations representing each priority population in your partnership). The most common sources of information for evaluating partnership are:

Remember in this step you will want to review what methods you have in place to collect information about partnership activities.

Is there a process to collect “detailed” data for each partnership activity or do you need to develop a system to collect the data (e.g., progress reports, meeting sign-in sheets, survey, etc.)? Click here to see an example of an Partnership Progress Report that you can use to document major activities.

•members of the partnerships•meeting minutes or notes•partnership activity logs•women served by the program•program screening records


Many instruments and surveys exist for evaluating different aspects of partnerships. You may want review the questions and decide if you want use or adapt previously existing instruments.

Here are examples of some instruments:•Coalition self-assessment survey

•Partnership self-assessment tool

•A practical approach to evaluating coalitions

See Granner and Sharpe (2004) for many more resources of articles on the evaluation of partnerships.

http://partnershiptool.net/


Let’s think some more about data of interest for partnership activities. What would be some possible data that can help answer different partnership evaluation questions? Review the table for some suggestions of data to collect.

Data Data source

Partner diversity to reflect demographics of priority populations

Partnership rosters over the course of the group’s existence

Partner satisfaction Partner survey

Partner contribution Meeting minutes and sign-in sheets; interview with key leaders of partnerships; annual accomplishment document

Partnership milestones Progress reports; meeting minutes

What are data to address your evaluation questions?Here are some examples of data for partnership:

Type of Activity

Question Indicator

Capacity What sectors of the community does the partnership represent?

Do the members represent the desired diversity and breadth of the partnership?

Partnership Roster and Minutes

Processes and structure

Are the communications of the partnership effective?

What factors contribute to member involvement in the partnership?

Frequency of communications

Quality of communications

Factors described by partners

Effectiveness To what extent is the partnership working with organizations to recruit priority populations?

List of partnership activities with these populations

Women reached by partnership activities

Gather Credible EvidenceCase Study

Returning to your evaluation question, you want to determine if the Partnership for Women’s Health members provide a comprehensive representation of people and agencies who can reach the priority populations in your community. You first develop a list of data of interest, type of organizations in the partnership and current membership, and sources from where you can gather this information.

Here is a table of potential data to collect and sources for those data.

Data Source

Category of partners Meeting minutes; Partnership roster

Members in each category Partnership roster; sign-in sheets


Some suggested category of partners suggested by the NBCCEDP partnership chapter are:

• National partners• Local health agencies (e.g., providers, public health clinics, public health

department. American Cancer Society)• Service agencies that serve women (e.g., beauty salons, shopping centers)• Community organizations that may serve or have women participants (e.g.,

civic organizations, faith institutions)• Local organizations that serve populations who may not have been screened

for breast and cervical cancer

Click here to see examples of partners for your program.

After seeking out data sources, you begin reviewing the partnership meeting notes and sign-in sheets (quantitative data). You also decide to set up interviews and solicit feedback from current members regarding potential partners to invite to join the Partnership for Women’s Health (qualitative data).

You talk to your program staff who you have asked to be a part of this evaluation and discuss if there are other community groups that should be approached to join the partnership.


CDC Framework for Program Evaluation Step 5: Justify Conclusions

Some ways you can analyze and judge partnership data are to:

• Compare how many women from priority populations participated in the program since the beginning of when partnership activities were implemented,

• Benchmark against your workplan or your performance in previous years in areas such as the growth of your partnership membership, activities, and measures of success,

• Assess your results in meeting partnership goals and objectives, or

• Summarize current partner satisfaction and suggestions for additional partners.

After you have gathered your data, it will be necessary to analyze and interpret your findings. Processing the data will help you examine the findings and determine what they actually say about the program.

CDC Framework for Program Evaluation Analyzing ResultsEvaluation of partnership may result in data analysis that judges the quality and effectiveness of the partnership in qualitative and quantitative ways.

For questions that ask about features of the partnership such as membership, you may have to describe the number of members and then write information about the sectors and resources that they bring to the table. For other questions, you may have very detailed summaries of the effectiveness of the partnerships through the documentation of co-sponsored recruitment events or training for providers through meeting notes.

It is important to realize that in reviewing the data, you may be making subjective judgments about the partnership through summaries of milestones or successes of the collaboration through member interviews. You will also have quantitative aspects of the partnership (e.g., size of partnership, satisfied members, reach to priority populations) to document results.

Justify ConclusionsCase Study

Now that you have your data and it is summarized, you can begin to think about opportunities to strengthen the Partnership for Women’s Health. You need to assess your program’s list of partners against the types of partners suggested by literature and NBCCEDP. The key is to see in what areas your program may have missing partners.

Evaluation findings allow you to conclude that you have solid representation from the American Cancer Society, the local Hispanic Health Alliance, and the faith-based coalition. All organizations represent and reach Latina and African-American women. Through the interview data, you discover that although the American Cancer Society can reach Asian/Pacific Islander women at health fairs and community events, it does not have the funds to do concerted outreach by producing Chinese (both Cantonese & Mandarin) and Vietnamese brochures.

The Asian/Pacific Islander Chamber of Commerce was upfront with you during your interview and acknowledge that they are stretched thin due to staff shortages and competing priorities.

Justify ConclusionsCase Study

What conclusions can you make from these findings? Can you identify opportunities to strengthen your partnership?

You conclude that it would be helpful to invite additional community organizations to join the Partnership for Women’s Health who can reach the Asian/Pacific Islander population. You refer back to interview data that captured partner suggestions for other groups to involve in the process.

Want to learn more tactics for developing and strengthening partnerships? Click here to learn how one real state program reactivated and rebuilt a coalition.

Field Example

One BCCEDP program reactivated a coalition by sending a letter to all previous members. Many responded that they were interested in continued participation. For those who were not interested or did not have the time, a letter was sent to let them know that if their interest or time commitments changed, they would be welcomed by the Coalition.

The small core group met and had original members provide the history and continuity to support the new members.

CDC Framework for Program Evaluation Step 6: Ensure Use and Share Lessons Learned

Remember to involve your program staff and stakeholders in analyzing and interpreting the data. They will help you form conclusions and brainstorm possible recommendations to improve and strengthen the partnership outcomes.

Then, consider how best to use the data. Consider discussing the results in program meetings or upcoming partnership meetings.

CDC Framework for Program Evaluation Step 6: Ensure Use and Share Lessons Learned

Early on in evaluation planning, a strategy should be developed for disseminating evaluation findings to users of the evaluation data. Consider not only staff and stakeholders, but also the public, if it is appropriate.

Share evaluation results about the partnership with partners. It is helpful for all of them to reflect on the data and brainstorm solutions.

Don’t forget to document your progress in the area of partnership and coalitions in your annual progress report and to your program consultant.

Ensure Use and Share Lessons Learned Case Study

Reviewing your findings, you decide to examine whether there are other organizations who have access to the Asian/Pacific Islander community to ask to join the Partnership for Women’s Health. You also conclude that you will ask staff to look into applying for a grant to help support producing the screening brochure in both Chinese and Vietnamese.

You review the list of national BCCDP partners and decide to contact the Asian and Pacific Islander American Health Forum to see if they have a local affiliate or know of similar groups in your community.

You also consider revisiting activities with the Asian/Pacific Islander Chamber of Commerce. It may be helpful to reassess their workload and set objectives that are realistic to accomplish between the partnership and their staff. Remember, small successes are just as good!

Ensure Use and Share LessonsLearned Case Study

Now that you have finalized your evaluation report and developed recommendations together with your partners, how are you going to disseminate this information? Before sharing the findings with the general public, you determine that a briefing with staff and partners at a partnership or coalition meeting is the most logic first step in your plan to share how partnership activities are going and make sure everyone has the same information.

In addition, you may build into your partnership agenda to have a presentation of accomplishments and evaluation results from partnership activities. Once you are able to distribute the report to them, you encourage stakeholders and the partners to share the findings with their staff and constituents. You inform them of presentations you have organized with the local health department and program providers so as to avoid doubling efforts to reach community members.

Ensure Use and Share LessonsLearned Case Study

You realize it might be a good idea to ask one or two partner representatives to join you at the presentations. You also suggest the possibility of organizing various town hall meetings with the priority populations and even with the local editorial board of the local newspaper. You may be able to demonstrate through press coverage the positive impact the Partnership for Women’s Health is having on your community.

Hopefully now you see how easy it is to evaluate an aspect of a partnership. It requires some planning but by including your stakeholders and staff, it should move right along!

Evaluation WorkplanYou will be given an opportunity to apply elements of your own partnership component to an evaluation workplan. The evaluation workplan starts with the evaluation questions and takes you through how you will analyze the data. It also gets you to consider how to use the data and potential users of the data. To develop your evaluation plan in a Word document, click here.

Evaluation Question

Uses of Evaluation Findings Data Collection Processes

Data Analysis

Questions that ask about aspects of the program that you want to examine. You can begin with evaluation questions related directly to your objectives and/or measures of success described in the workplan.

You may also want to consider other bigger picture questions of interest to your program that go beyond the workplan.

Users. A subset of stakeholders can clarify intended uses, help prioritize questions and methods, and prevent the evaluation from becoming misguided. The users are the participants and will use the evaluation findings.

Uses. To be useful to the program, evaluation results must be applied. The results could, for example, identify an ineffective activity, suggest a way to adjust resources, document a positive outcome or mobilize support for the program.

The data collection processes events or things that will need to be in place to answer the evaluation question.

Data analysis that includes reviewing data or comparing data to set targets or measures of success.

Partnership Evaluation WorkplanCase Study

Here is a sample of the evaluation workplan completed based on our partnership evaluation case study.

Evaluation Question Uses of Evaluation Findings Data Collection Processes

Data Analysis

Does the Partnership for Women’s Health members represent all relevant constituencies or stakeholders?

Users.

Program director, partnership coordinatorPartners

Uses.

Program staff. Ensure that all potential partners will help increase goal’s of the program

Partners. Discern their role in collaboration with the program and suggest other potential partners.

CDC. Learn what partners help program achieve its goals

Review partnership roster, sign-in sheets and meeting minutes for category of partners and membership represented in each category.

Interview staff and partners for ideas about other partners to reach women for screening.

Compare categories of partners suggested in the literature and partnership chapter to your list of current partners.

Compile list of suggested partners for program staff discussion.

My Evaluation Plan for Partnership

Now it’s your turn to develop an evaluation plan for a partnership activity. We will work through each step of the CDC Framework for Program Evaluation. By the end of the exercise, your responses will be displayed in

an evaluation plan.

Step 1: Engage StakeholdersLet’s start with identifying your stakeholders.

Who might benefit from evaluating your partnership or coalition? How would they use the evaluation findings?

Type potential stakeholders and how they might use the partnership evaluation results.

Stakeholder Uses

Type stakeholder 1 here Type Use 1 here



Hint

Step 1: Engage Stakeholders

Stakeholder Uses

Partnership coordinator Assess how partnership is functioning and effects of the partnership

Program partners Learn how they are doing as part of the partnership and how they play a role in the screening and service delivery for BCCEDP women

State health department Find out if state contacts have suggestions for other possible organizational contacts to consider joining the partnership.

Local ethnic group associations (e.g., Hispanic coalition, Chinese association)

Assess their role in building awareness of breast and cervical cancer among priority populations and increasing recruitment of women into the program.

These are people to consider as stakeholders and how they might use the evaluation data.

Role of StakeholdersEarlier you mentioned these stakeholders and how they might use the results learned from the evaluation.

Again, remember that it might be necessary to think about stakeholders that appear less obvious that can help your partnership efforts. Community organizations, local businesses, and potential partners are examples of stakeholders who can share an interest in your partnership’s success.

Stakeholder Uses




Step 2: Describe the Program

Now you have some stakeholders ready to work with you. We now need to think about which partnership activity to focus your efforts on. Refer back to your workplan. Can you think of one or two partnership activities or concerns that could be evaluated?

Type 1 or 2 partnership activities in the space provided below:

Type Partnership Activity 1 here

Hint

Type Partnership Activity 2 here

Step 2: Describe the Program

It might be helpful to begin thinking about the partnership activities over the past year.

What have you been able to do through the partnership in terms of reaching women from priority populations. What were key activities in this area? Have you reached your goals?

What are critical events mentioned in your partnership log of activities that could be evaluated?

Step 3: Focus the Evaluation Question

Remember, partnership evaluation should be guided by specific and measurable objectives that reflect your partnership’s priorities. Look to the outcomes you developed in your workplan that reflect the partnership activity you are interested in evaluating.

Type in your evaluation question(s) that you are interested in here:

Type evaluation question(s) here

Hint

Step 3: Focus the Evaluation Question

Here are some questions to consider:

• How representative are your partners of the communities you want to reach?

• What are the key milestones of your partnership in the past fiscal year? What are the effects of one or two of those milestones?

• To what extent are your partners satisfied with the group’s scope of activity in the past year?

• To what extent were women from priority populations recruited based

on the partnership activities in the past year?

Step 4: Gather Credible EvidenceYour evaluation question was:

Now that you have your evaluation question formed to help guide the rest of the process, you can begin to collect your data. Remember, to think about potential data of interest. These are what you want to measure from a partnership activity (e.g. number of partners; number of partners contributing over the year; key milestones of the partnership).

What data sources will you need? Can your stakeholders assist you in efforts to collect data? How will you access the data?

Type in data for the evaluation and their source below:

Data indicator Data source

Type data indicator 1 here Type data source 1 here

Type data indicator 2 here Type data source 2 hereHint

Step 4: Gather Credible Evidence

You can work with recruitment coordinator and case management staff to review in-service records and screening service forms to determine how many women from priority populations were screened and determine if you can link it with activities conducted in collaboration with some partners.

Review interview and survey data from partners to determine their satisfaction with the partnership. Again, you want to measure how satisfied are partners with the achievements of the partnership or how satisfied they are with communications.

Data Indicator Data Source

Number of women screened from priority populations

Screening intake forms, patient records

Data Indicator Data Source

Satisfaction with partner communication

Partner survey

Step 5: Justify Conclusions

You are almost done. Now that you have your findings, you can contemplate how you will interpret them. Then, you should take actions based on what you found. How will you analyze the results?

Type below how you would analyze the results:

Type how you would analyze here

Hint

Step 5: Justify ConclusionsHere are some ways you can analyze partnership data.

Describe answers to evaluation questions:What are the range of response and descriptive statistics (i.e., number and frequencies)? What was the general feeling about the partnership? What level of satisfaction was found: high, medium, or low?

Compare findings to your partnership objective(s):Link your findings back to your program’s partnership objectives. For example, did linkage with the faith-based coalition result in the recruitment of more African-American women over 50 for screening?

Compare findings to what the members of the partnerships expected:What were the expectations of the partnership members for reaching individual goals? Was the goal meet? Why or why not?

What do stakeholders, including yourself, think about the results:Is there any reason based on your findings, to change any partner activities or add new ones? Did partners provide additional input in how to continually improve the partnership? Did you collect suggestions for potential partners to consider inviting to the table?

Step 6: Ensure use and Share Lessons Learned

Now that you have developed your conclusions and recommendations based on your findings, we need to get the word out! Who will you share your findings and recommendations with? How will you disseminate the evaluation results? Think about how you will share and use the data internal to the program first.

Type in the table below who you will share your findings with and how you will present it:

Hint

Audience Method of Sharing Data

Type Audience 1 here Type Method of Sharing 1 here



Step 6: Ensure use and Share Lessons Learned

Here are some ways to share your partnership evaluation results.

•Prepare a report summary for program staff, stakeholders, and partners. •Create a PowerPoint presentation for partners to use if they decide to disseminate the information, too. •Present at breast and cervical cancer conferences and other public health venues to share partnership success strategies.

Don’t forget that getting the word out to the public is just as important to build program awareness and acknowledge the efforts taking place to protect the health of women in your community. Work with your communication staff to disseminate findings to the local media through a news release or consider holding a town hall meeting or community forum with a priority population.

Your Partnership Evaluation PlanNow let’s take a look at your evaluation plan based on your efforts in this exercise. It is designed to give you a model to begin evaluation activities within your partnership program component.

Evaluation Question Uses of Evaluation Findings Data Collection Processes

Data Analysis

Type evaluation question(s) here

Type stakeholder 1 here

Type Use 1 here


Type Use 2 here


Type Use 3 here

Type data indicator 1 hereType data source 1 here

Type data indicator 2 hereType data source 2 here

Type how you would analyze data here

References

Centers for Disease Control and Prevention. (2005). Partnership analysis and enhancement toolkit. http://www.cdc.gov/cancer/ncccp/cccpdf/Guidance-Toolkit.pdf

Butterfoss, F.D., Goodman, R.M., & Wandersman, A. (1993). Community coalitions for prevention and health promotion. Health Education Research, 8(3), 315-330.

Granner, M.L., & Sharpe, P. A. Sharpe. (2004). Evaluating community coalition characteristics and functioning: A summary of measurement tools. Health Education Research, 19(5), 514 - 532.

Himmelman, A.T. (1996). Collaboration for a change: Definitions, models, roles and a guide to collaborative processes. New York State Department of Health.

Kegler, M.C. (1998). A multiple case study of implementation in 10 local Project ASSIST coalitions in North Carolina. Health Education Research, 13(2), 225-238.

Mattessich, P.W. & Monsey, B.R. (1992). Collaboration: What makes it work. Saint Paul, MN: Amherst H. Wilder Foundation.

Partnership Resources

Partnership Assessment Tools

National Breast and Cervical Cancer Early Detection Program Guidance Manual Partnership Chapter WORD | PDF

Wolff, T. A practical approach to evaluating coalitions.

http://www.tomwolff.com/resources/backer.pdf

Evaluation ToolsTaylor-Powell, E., Rossing, B., & Geran, J. (1998). Evaluating collaboratives: Reaching the potential. Madison, WI: University of Wisconsin Extension.

http://learningstore.uwex.edu/Evaluating-Collaboratives-Reaching-the-Potential-P1032C238.aspx

http://www.tomwolff.com/resources/backer.pdf



Module 3 Evaluation of Partnerships NBCCEDP Enhancing Program Performance through Evaluation.

Documents

program activities

evaluation of breast

evaluation questions

evaluation planwho

evaluation workplan

program consultant

program efforts

partnership chapter