National Child Nutrition Program Cluster Evaluation: Preliminary Findings Evaluation team Colin Bell Jan Garrard Dionne Holland Boyd Swinburn.

National Child Nutrition Program Cluster Evaluation: Preliminary Findings

Evaluation team

Colin Bell

Jan Garrard

Dionne Holland

Boyd Swinburn

Deakin University 2003

Background: National Child Nutrition Program

• A community grants program established in 1999 by Commonwealth Department of Health and Ageing

• Aimed at improving nutrition among children aged 0 to 12 years and pregnant women

• Focus on priority communities:• Rural and remote communities• Aboriginal and Torres Strait Islander communities• Lower socio-economic communities

• 110 community-based projects funded nationally • 18 projects in Victoria - $2,190,300 over three year

period

Deakin University 2003

Evaluation: Background

• Evaluation not built into overall program planning • No specific evaluation budget• Importance of evaluation recognised by Victorian

office of Dept of Health & Ageing• 9 projects committed $3,000 each to fund a

‘cluster’ evaluation• Additional $60,000 from Dept of Health & Ageing• Deakin University contracted to conduct the

evaluation

Deakin University 2003

Evaluation: Preview

• A ‘retrofitted’ evaluation strategy• Optimum balance of common measures while

acknowledging the uniqueness of projects, populations and settings

• Evidence base for effective community-based child nutrition programs is poor – good practice often not documented, disseminated and acknowledged

• A practical, feasible evaluation strategy designed to meet the needs of funding body, project management agencies and practitioners

Deakin University 2003

Evaluation: Objectives

• To assess intended and unintended impacts of individual projects

• To gain an understanding of key implementation issues impacting on project success

• To draw upon collective learnings from the 9 projects to guide future efforts in community-based child nutrition

Deakin University 2003

Evaluation: Design & Methods

• Multiple case study design (9 ‘cases’)• Process measures (ie. what have they done?) and • Impact measures (ie. what have they achieved?) • At individual project level - evaluation team assisted

each project to conduct process and impact evaluation• Across projects, evaluation team conducted a total of

53 key informant interviews – combination of core questions and project-specific questions

• Enabled ‘detection’ (but not quantitative measurement) of impacts – useful for unintended impacts

• Provided data on important how? and why? questions

Deakin University 2003

Evaluation: Lessons Learned

• Build evaluation planning into project planning• Provide adequate resources for evaluation

(approx 10-15% of project budget)• Recognise that responsive, needs-based,

individually tailored community-based projects will rarely meet the requirements for rigorous quantitative assessment

• Such projects rarely find their way into conventional evidence-based reviews.

Deakin University 2003

Lessons Learned (cont)

• Focus instead on what can be achieved (measured impacts) in what settings with what population groups under what conditions – recognise this will vary in different priority settings

• Distil common achievements and learnings if possible

• Provide a mechanism for sharing experiences via a ‘community-based projects’ evidence-base.

Deakin University 2003

Cluster Evaluation Projects

• ‘High Five’ School project – Warrnambool• ‘Eat Well, Grow Well’ – Hamilton, Southern

Grampians Shire• H2O for Moonee Valley Kids – North Melbourne• ‘Breast is Best’ – Brimbank, West Melbourne• ‘Healthy Eating for Country Kids’ – Swan Hill• ‘Who’s Eating Gilbert Grape and Vicki Vegetable’ –

Murray Valley Aboriginal Cooperative, Robinvale• ‘A healthy family is built on a good diet of nutritious

food’ - Robinvale• ‘Child nutrition in the West’ – West Melbourne• ‘Food Facts for preschoolers’ - KPV

Deakin University 2003

Aim: To increase the fruit and vegetable intake in order to improve the overall nutrition of prep to year 4 children in the South West of Victoria

Children with fruit in lunchbox

0%

20%

40%

60%

80%

100%

School1 School2 School3 AVERAGE

PRE

POST 1

POST 2

• ↑largest in yr 2&3

• Also ↑ in serves

• Vegetable intake was very low (~5%) & only increased at one of the schools

• Supporting curriculum changes and parent involvement

South West Health Care

Deakin University 2003

Aim:To increase the proportion and duration of breastfed babies in the City of Brimbank, particularly in low SES and CALD communities

• Distributed BF promotion kits to GPs & MCHNs

• Prenatal BF promotion checklist adopted by Sunshine Hospital

• Breastfeeding guide for new mothers (Vietnamese)

• Specific support and promotion of BF in Vietnamese community

• 66 BF-friendly venues• Adoption of Lactation policy by

ISIS

Change in Brimbank BF rates 00/01 to 01/02

0

1

2

3

4

5

Discharge 2 weeks 3 months 6 months

Per

cent

Cha

nge

01/02 75% 69% 51% 34%

ISIS Primary Care

Deakin University 2003

Murray Valley Aboriginal Co-Op

• Murray Valley River Quest

• PAN messages at ‘stops’ on route

• Reached 516 children

• Teachers said it was easy to implement and raised awareness

• In 2003 hours have ↑

Murray Valley Aboriginal Cooperative

Hours walked as a class on MRQ 2002

0

5

10

15

20

25

30

35

School1 School2 School3

Hou

rs w

alke

d Average

Prep

Grade 1/2

Grade 3/4

Grade 5/6

48 towns53 towns100 towns

Aim: Improve nutritional knowledge and healthy eating patterns of pregnant women, young mothers/caregivers of children aged 0-12 years and children within local Aboriginal and Torres Strait Islanders

Deakin University 2003

Western District Health Service

Increases in knowledge after education sessions

0 20 40 60 80 100

Toddler SM tour

Toddler food talk

Intro solids

Healthy eating & BF

Healthy eating & preg

Percent

Pre

Post

• Education sessions effective in improving knowledge of participants

• Supermarket tours and visits to new mums also beneficial

• ↑awareness of conditions which may require dietetic referral among WDHS staff

Eat Well – Grow Well

Aim: To improve the nutritional health of children aged 0-5 years in the Southern Grampians Shire

Deakin University 2003

Swan Hill District Hospital

• ↑access to nutrition education for pregnant/new mums

• Good readership of newsletter and newspaper column

• These were helpful for making healthy food choices

Evaluation of newsletter & newspaper column (n=125)

0 20 40 60 80 100

Agree/SA news helps with healthy food choices

Occ/Always read newspaper column

Agree/SA tips help with healthy food choices

Agree/SA tips useful

Read tips in newsletter

Percent

Aim: To improve the eating habits of all children and pregnant women in Swan Hill and surrounding district, providing GPs with nutrition information kit

Deakin University 2003

Doutta Galla Community Health Service

• ‘H2O for Moonee Valley Kids’• Drink water rather than

sweet drinks

• ‘Food and Families’• 4 week course on

nutrition information and parents skills

• 10-14 Somali women

• Breastfeeding, bottle, solids, vitamin D

Sweet Drink Consumption

0

20

40

60

80

100

Moonee Valley Colac

Aim: To improve the healthy food intake of infants and children aged 0-12 years with Kensington, Flemington, North Melbourne and Ascot Vale

Deakin University 2003

Western Region Health Centre

Fruit in Lunchboxes

0

20

40

60

80

100

Prep Grade 1/2 Grade 3/4 Grade 5/6

Mar-01 Nov-02 Mar-03

• ‘Eating altogether-5 times better’• Fruit, vegetable & water

promotion at 2 primary schools

• Bilingual child education program• In 1 year, 700 people from LD

communities participated in 75 nutrition education sessions

Western Region Health Centre

Aim: Increase nutritional knowledge and capacity of pregnant women, parents and children ages 0-12 with CALD and recently arrived communities, to enhance healthy eating patterns

Deakin University 2003

Robinvale District Health Service

At the MCCD I Learnt How To ...

0

20

40

60

SA S Neither

A/DA

DA SD NR

Cook healthy food Give healthy food to family

Make favorite food healthier Make food from different cultures

• Parent cooking classes• Multi-cultural food day

• Attended by 120-180 people

• Nutrition education sessions

• Large Polynesian population

• Breakfast program at schools

• Linked with MVAC

Aim: Improve nutritional knowledge and healthy eating patterns of parents and children in Robinvale

Deakin University 2003

Kindergarten Parents Victoria

• Staff training sessions• 9 locations throughout Victoria with over 170 participants• ↑ knowledge of Aust. Dietary Guidelines for children, role of nutrition

policy, value of partnerships b/w parents and early childhood staff

• Parent workshops• At sessional preschools, long day care, other services• ↑ understanding of childhood nutrition

• Resources• Staff training sessions, ‘parent workshop’ package, Food Safety

Information Sheet, nutrition built into existing KPV resources (eg Preschool Matters’

Aim: To increase the capacity of early childhood services to deliver consistent, relevant and current nutrition information to preschool children and their parents

Deakin University 2003

Cluster Evaluation

• 53 semi-structured key informant interviews completed• 11 project officer interviews• 39 key informant interviews (single and focus groups)• 2 Steering Committee member interviews

• Combination of core and project specific questions

• Presenting preliminary findings today• Report available early 2004

Deakin University 2003

Reported Impacts

Increased collaborative links

Human services NGOs Govt Depts

• Community

Health Centres

• PCPs

• Royal Children’s

Hospital

• Breast Feeding

Association

• Ecumenical

Migration Centre

• Dept. of Health

and Ageing

• LGAs

• City West Water

Deakin University 2003

Reported Impacts (cont)

Diffusion of nutrition messages• Reached target groups

• Children, parents, carers, pre-schools, kindergartens, primary schools

• And beyond…..• Program participants ‘spread the message’

• Fathers read brochures on breastfeeding aimed at mothers

• Program participants ‘modelled’ healthy eating behaviours

Deakin University 2003

Reported Impacts (cont)

Spin-offs• Children

• Increased concentration in class• Likely reduction in dental carries

• Families• Budget savings (eg. not purchasing soft drink)

• Bilingual workers• Future work• Empowerment

Deakin University 2003

Capacity Building Framework

Organisational Development

Workforce development

Resource allocation

Partnerships

Leadership

Context

Build Capacity

• Infrastructure

• Program sustainability

• Problem solving

Source: Hawe 1999

Deakin University 2003

Organisational Development

• Management support• Projects more effective when management

understood the complexity of community work and supported it

• Recognition of work• Projects better supported within the organisation

when encouraged to showcase their achievements• Additional support from co-workers

• Policy development

Deakin University 2003

Workforce Development

• Project officer training and previous experience in community work

• Needed a sound knowledge of community work and how it differs from clinical work

• Generally, new graduates who lacked field experience struggled unless provided with additional support within the workplace

• Projects easier to implement when working in a team environment

• Important to share learnings with co-workers to gain support and increase organisational capacity to carry out community work

Deakin University 2003

Resource Allocation

• Control of project resources (incl budget)

• Advice on how to get the ‘best bang for your buck’• Project officers need to be aware existing resources

• ‘Cost-effectiveness’ of different strategies

• Continual planning towards obtaining additional resources, especially into the future

Deakin University 2003

Partnerships

• Can be extremely effective• Working towards a common goal, but from varied view

points• Provided additional ‘sponsorship’• Often opened up opportunities for shared access to experts

and resources

• Enhanced opportunities for sustainability as multiple organisations/individuals were able to share the role lessening the burden (time and money) on one person

• Often enabled strategic responses across multiple settings

Deakin University 2003

Leadership

• Essential

• Creative problem solving skills• Needs to be able to take on the view points and the

needs of the local community and develop and re-develop strategies where needed

• Continuity• Leadership role best held by the same person

throughout the project

• Empowerment of others

Deakin University 2003

Context

• Physical, economic, political, organisational and cultural environments of the project• Currently high profile of nutrition• Organisations with community-based and

population focus

Deakin University 2003

What does the literature say about success factors for community-based interventions?

• Limited literature around community-based child nutrition programs

• Community-based smoking programs have been at it a lot longer!

• Systematic review of 32 studies covering 30 years of community-based interventions to reduce smoking (Secker-Walker et al. 2003)

Deakin University 2003

Lessons for child nutrition promotion(Adapted from Secker-Walker et al. 2003)

• Child nutrition needs to be recognised by the community as an important issue

• Community organisation, assessment of community capacity, and the identification of individuals and organisations interested in supporting nutrition interventions are important during project development.

• Community members and staff need skills in working with diverse groups and in health education

• Coalitions need several months to form, and a year or more to become effective change agents in their community

Deakin University 2003

• Interventions to improve child nutrition need to continue for several years

• Resources for promoting good nutrition and support for healthy eating should be readily available throughout the community

• It is important to involve, train, and provide referral services and resources for health professionals

• The use of mass media (print, radio, television) is especially useful for modelling behaviour change, and for changing community norms about nutrition

Lessons for child nutrition promotion (cont)

Deakin University 2003

Conclusion

• Projects have had some good impacts at the local level

• Contributed to the knowledge base about effective implementation

• Important to disseminate these findings