YOU ARE DOWNLOADING DOCUMENT

Please tick the box to continue:

Transcript
Page 1: Evaluating Physical Activity Intervention Programs

Evaluating Physical Activity Intervention Programs

Thomas Schmid, PhDPhysical Activity and Health Branch

CDCAtlanta, Georgia, USA

Page 2: Evaluating Physical Activity Intervention Programs

Evaluation Definitions and Principles

CDC Physical Activity Evaluation Handbook – 6 Steps

Evaluation issues

Overview

Page 3: Evaluating Physical Activity Intervention Programs

The systematic examination and assessment of features of a program and its effects. Evaluation is designed to produce information that can be used by those who have an interest in the improvement or effectiveness of an initiative or program.

What is Evaluation?

Page 4: Evaluating Physical Activity Intervention Programs

To assess the effects and value of a program◦for accountability◦for progress ◦to identify best practice◦as a planning tool◦for research

Why Evaluate?

Page 5: Evaluating Physical Activity Intervention Programs

the program target groups and needs realistic achievable objectives is the program ready for evaluation?

Match Evaluation of Health Promotion Programs to:

Page 6: Evaluating Physical Activity Intervention Programs

Identify needs

State goals and measurable objectives

Assess progress towards objectives

Process of conducting evaluation

Page 7: Evaluating Physical Activity Intervention Programs

EfficacyIntervention tested under ideal conditions with optimal study designEffectivenesField based intervention testing whether the intervention works as intended for a defined population in the real worldEfficiencyResults in comparison to the efforts required to obtain them – often cost or cost effectiveness

Evaluation Terminology

Page 8: Evaluating Physical Activity Intervention Programs

EVALUATION

IMPACT

OUTCOME

Measures activities, program quality, reach

Measures long term effects (goals)

PROCESS

Formative Measures short term objectives

Develop program & resources

Page 9: Evaluating Physical Activity Intervention Programs

What is likely to work Best practices in health promotion Idea or concept development Developing and pilot testing materials Focus groups

Formative Evaluation

Page 10: Evaluating Physical Activity Intervention Programs

Formative Evaluation Focus groups and

interviews with partners about their needs

Reviewing best practices from other programs such as Agita Sao Paulo

Pilot testing in one company

Page 11: Evaluating Physical Activity Intervention Programs

Process evaluation◦Program implemented as intended ?◦ Quality of program components◦Attendance rates ◦Audience awareness of message◦ Appropriate message learned◦Participant satisfaction

Page 12: Evaluating Physical Activity Intervention Programs

Process Evaluation Companies enrolled Workshops held Promoters trained Companies

participating in community events

Page 13: Evaluating Physical Activity Intervention Programs

Impact & Outcome Evaluation◦ Short and long term effects of program◦ Hierarchy of effects : what level of effects

indicates program success◦ Assessing the impact upon those who attend or

participate◦ Community effects

Page 14: Evaluating Physical Activity Intervention Programs

Impact & Outcome Evaluation

Physical activity level of employees

PA level in community

Disease rates or costs

Page 15: Evaluating Physical Activity Intervention Programs

Program Evaluation FrameworkPhysical Activity Evaluation Handbook

Page 16: Evaluating Physical Activity Intervention Programs

Program Evaluation FrameworkPhysical Activity Evaluation Handbook

1. Engage Stakeholders1. Engage Stakeholders

Include people with a vested interest in the program: implementers, partners, participants, possible opponents and decision makers.

2. Clearly Describe the ProgramThe problem being addressed, descriptions of the planning, implementation and maintenance

phases of the program, and finally, defining and communicating how all phases work together (eg. the "logic model").

Page 17: Evaluating Physical Activity Intervention Programs

Program Evaluation FrameworkPhysical Activity Evaluation Handbook

3. Clearly Focus3. Clearly Focus

Asking a clear, straightforward question and ensuring linkage to the problem being addressed.  Includes clarifying process and/or outcome evaluations of interest

4. Develop and Implement Data CollectionDesign of data collection efforts, selection of

appropriate control groups, designation of appropriate baseline values and decisions about how much time should be built in to adequately measure "change”

Page 18: Evaluating Physical Activity Intervention Programs

Program Evaluation FrameworkPhysical Activity Evaluation Handbook

5. Develop Conclusions5. Develop Conclusions

Data analyses and interpretation.

6. CommunicatePromotes the importance of tangible products from the evaluation, dissemination of those products, and follow-up (as appropriate).

Page 19: Evaluating Physical Activity Intervention Programs

www.iuhpe.org

Page 20: Evaluating Physical Activity Intervention Programs

20

Page 21: Evaluating Physical Activity Intervention Programs

All physical activity interventions should be evaluated. Evaluation begins with program planning

There are many good programs, but very few are well evaluated

Involve stakeholders Community – University partnerships can be

very useful

Observations on Evaluation

Page 22: Evaluating Physical Activity Intervention Programs

     Process as well as outcome evaluation Quantitative and qualitative Tailor evaluation to stakeholders and situation Evaluation is often undervalued compared with “hard science” Adequate resources should be devoted to evaluation. 10% is recommended

Observations on Evaluation

Page 23: Evaluating Physical Activity Intervention Programs

     Evaluation results should be used to

adjust programs and inform policy Share results with stakeholders Academic and practical training are

needed The CDC Physical Activity Evaluation

Handbook provides a good framework

Observations on Evaluation

Page 24: Evaluating Physical Activity Intervention Programs

Gracias/Thank you


Related Documents