Values-Based Evaluation What, How, Why presented by Barbara Kahan & Debbie Bang at Best Practice for Better Health 6th IUPHE European Conference on the.

Values-Based Evaluation

What, How, Why

presented by Barbara Kahan & Debbie Bang at

Best Practice for Better Health

6th IUPHE European Conference on the Effectiveness and Quality on Health PromotionStockholm, June 2005

Presentation Objectives

To introduce the Interactive Domain Model

To understand ~>• What is the role of values in evaluation• How to evaluate consistency between

values and practice• Why there is a relationship between

values-based evaluation and program effectiveness

Two meanings for values-based evaluation

• using values to shape how we conduct our evaluations

• using evaluation to judge how well our values are integrated into practice

The basis for our approach to values-based evaluation

• the Interactive Domain Model (IDM), a comprehensive and systematic approach to best practices

Internal & external environments(social, political & economic structures &systems; psychological and physical conditions)

The Interactive Domain Model of Best Practices in Health Promotion & Public HealthKahan & Goodstadt 2005 (revised from 2001 version)

Underpinnings

theories /beliefs

evidence

values /goals / ethics

vision

analysis:health

Understanding of the Environment

analysis:organization

Practiceprocesses & activities directed to:

organizationalissues

research &evaluation

health-relatedissues

Effectiveness in health promotion

Effective health promotion practice occurs when there is consistency between our processes and activities and health promotion:

• values, goals and ethics• theories and beliefs• evidence• understanding of the environment

List values which reflect health promotion philosophy.

Initiating Values-Based Evaluation

• In own setting review ~>• Mission / values statements for

organization• Goals for the program to be

evaluated• Strategic direction documents for

organization• Impact on community/population

How values shape evaluation

• the next slide shows how two different sets of values result in two different evaluation approaches

Values-Based Evaluation Worksheet description of project: decreasing rates of diabetes

values: health experiential knowledge process context community

values: health expert knowledge cost saving the quantifiable the individual

priority evaluation questions has health increased has community capacity increased how well do project processes support activities

have costs been reduced information sources physicians project managers staff carrying out project activities members of the priority population data collection methods in-depth interviews focus groups surveys

How to use evaluation to judge consistency between values & practice

• develop a values logic model– identify guidelines for implementing

values into practice– identify and define values– develop objectives for integrating

values into each of the three practice areas

– develop indicators which will show whether objectives are being met

objectives

IDM BEST PRACTICES LOGIC MODEL: reflection in practice of underpinnings & understanding of the environment(B. Kahan & M. Goodstadt April 2005)

guiding principles for implementing sub-domains in practice

values, goals,ethics

indicators & measures

addressorganizational

issues

research andevaluation

addresshealth issues

Planning and evaluation worksheet to build consistency between practice and other domains Address organizational issues Address health/ social issues Research and evaluation values/ goals/ ethics

objectives

indicators & measurement methods

objectives

indicators & measurement methods

objectives

indicators & measurement methods

guideline: review consistency between practice & values regularly

evaluate three programs annually for values-practice consistency re. organizational issues

generate report of findings

evaluate three programs annually for values-practice consistency re. health/ social issues

generate report of findings

evaluate three programs annually for values-practice consistency re. research/ evaluation

generate report of findings

value: inclusivity

all key stakeholders are involved in deciding the organization’s mission, vision, goals, direction, etc.

key stakeholders are represented in the governance structure

meeting minutes indicate discussion and involvement of all key stakeholders

all key stakeholders are involved in decision making about program strategies

formation of a group representing all key stakeholders

meeting minutes indicate discussion and involvement of all key stakeholders

all key stakeholders are involved in defining and conducting research & evaluation and reviewing results

key stakeholders are represented on the research & evaluation committee

meeting minutes indicate discussion and involvement of all key stakeholders

Values & effective practice• effective practice involves more

than meeting program outcomes• effective practice includes the

translation of values into concrete reality

• evaluation of effective practice requires looking at consistency between practice and: values, theories, evidence, understanding of the environment

Relationship between values-based evaluation and program effectiveness Evaluation findings Values Theories Evidence Understanding

of the environment

community dental program to improve dental outcomes for children core value is equity

program was based on a literature review

dental outcomes have improved for participating children

children from across the tracks have never attended

x x

an advisory group is set up to plan an elementary school breakfast program core value is empowerment

program was based on a literature review

children attending are paying more attention in morning classes

parents of the children were not invited to participate

x x

Resources

See <www.idmbestpractices> for:• IDM (Interactive Domain Model)

Resources • Monthly Features • Annotated General Resources

Contact Information

Barbara KahanCentre for Health Promotion(University of Toronto)[email protected]

Debbie BangSt. Joseph’s Healthcare, [email protected]