SLIDE FOR INFO- not part of presentation

1003 Evaluation—Measuring the Impact of Food Safety and Nutrition Policy Plaza Building, Concourse Level, Plaza Court 3 FSN Section Track SessionThis session takes stock of what we “know” about what works in influencing food safety and nutrition and provides a forum to share information on what is being done on evaluation of new and existing policy initiatives. Participants from federal and state governments will share their experiences in program evaluation and discuss common themes in defining objectives and measuring outcomes. The role of economics in the process will be highlighted in the presentations. Organizer: Victoria Salin, Texas A&M UniversityModerator: Victoria Salin, Texas A&M UniversityPresentations:GAO’s Approach to Evaluating Federal Programs in Food Safety Lisa Shames, Alfredo Gomez and, David Bennett, U.S. Government Accountability Office Evaluation of School Nutrition Policy Phillip Gleason, Mathematica Policy Research Using a Balanced Scorecard to Evaluate a State Food Quality and Safety Program Timothy Herrman, Office of the Texas State Chemist; Peggy Wantwadi, Texas A&M University Program Evaluation Components of the United Kingdom Food Standards Agency’s 5-Year Strategic Plan Derrick Jones, U.K. Food Standards Agency

SLIDE FOR INFO- not part of presentation

Program Evaluation Components of the United Kingdom Food Standards Agency's 5-Year Strategic PlanAbstract: The Food Standards Agency is an independent UK government department set up by an Act of Parliament in 2000 to protect the public's health and consumer interests in relation to food. In 2006 the Agency introduced its first annual review of its Strategic Plan 2005-10.  That review measured and reported progress against strategic targets to ensure that the strategic plan is an evolving, dynamic document suitable for the Agency’s multiple purposes.  A new Strategic Plan for 2010-2015 has now been published and sets out a clear direction towards a vision of 'Safe food and healthy eating for all.'  The updated plan does not signal a change in direction for the Agency. The key aims of food safety, eating for health and choice and the supporting strategies for delivery, remain the same.  However, the approaches to measure impact are changing.  This presentation will report on the innovations in evaluation that are under way to judge the impact of the new strategic plan.

SLIDE FOR INFO- not part of presentation

Programme Evaluation Components of the United Kingdom Food Standards Agency's 5-Year Strategic Plan

Derrick JonesDerrick JonesHead of Analysis and Research Division

Food Standards Agency

AAEA Annual

Meeting

FSN Evaluation

Track Session

25 July 2010

This session takes stock of what we “know” about what works in influencing food safety and nutrition and provides a forum to share information on what is being done on evaluation of new and existing policy initiatives. Participants from federal and state governments will share their experiences in program evaluation and discuss common themes in defining objectives and measuring outcomes. The role of economics in the process will be highlighted in the presentations.

(From Concurrent Session planner, highlights added)

Evaluation—Measuring the Impact of Food Safety and Nutrition Policy

Overview• UK Food Standards Agency- Role and remit • Development of FSA’s Strategic plan 2010-15

– Food chain analysis

• Evaluation work to support strategic plan delivery (establishing what we know, what is being done, objectives, outcomes)

• Some emerging lessons / issues• NOTE – Change of UK Government May 2010 –

change in “home” of nutrition policy and some labelling

About the Food Standards Agency• Independent Government department set up in 2000

to protect consumers’ interests in relation to food• A UK wide remit covering both Food Safety and

Nutritional Health • Vision:

– “Safe food and healthy eating for all”• Three core values:

– Putting the consumer first– Openness and independence– Being science- and evidence-based

Improve food safety and the balanceof people’s diets

3. Food products and catering meals are healthier

3. Food products and catering meals are healthier

2. Imported food is safe to eat2. Imported food is safe to eat

4. Consumers understand about safe food and healthy eating, and have the information they need to make informed choices

4. Consumers understand about safe food and healthy eating, and have the information they need to make informed choices

1. Food produced or sold in the UK is safe to eat1. Food produced or sold in the UK is safe to eat

Food Safety-Estimates of the main public health risks

  Death pa. Number Cases pa. Number of incidents pa.

(2007)

FSA Consumer tracker -

% concerned Dec 2008

Foodborne Disease (FBD) (microbiological)

4501 950,0001 270 53%

Allergens/ intolerance ~5-10 4,8002 90 22%

Chemical contamination

Not Known Not Known 660 32%3

TSEs-vCJD 54 0 new cases4 10 24%

Radiological contamination

less than 105 less than 355 40 15%6

1 England and Wales for 2006, Health Protection Agency2 Estimated new cases with peanut allergy for 2005 taken from ‘Primary care epidemiology of allergic disorders’ applied to UK population (http://www.qresearch.org/Public_Documents/HSCIC%20allergies%20report%20from%20QRESEARCH%20.pdf)3 Figure is for consumer concern about pesticides, as a proxy4 2007 figures taken from ‘Incidence of variant Creutzfeldt-Jakob disease diagnoses and deaths in the UK January 1994 – December 2007’ (http://www.cjd.ed.ac.uk/cjdq56.pdf)5 FSA estimates for UK population based on data on exposure from radiological discharges into the environment 6 June 2006 figure for consumer concern about irradiated food, used as a proxy. Irradiated food is no longer tracked in the consumer survey.

Summary: Key Pathogens 2007

• Campylobacter is clearly the highest priority pathogen in terms of public health impacts• Salmonellas and Listeria monocytogenes are also significant• Contribution of VTEC O157 to deaths and costs are low by comparison• Food related Norovirus cases are estimated to have trebled between 2003 and 2007 however,

there are large uncertainties around the role of food, which make this trend far from certain – new IID study will provide a clearer picture.

Pathogens 

Total Cost DeathHospitalisati

on CasesKey Sources of risk from

the UK food chain

% Rank % Rank % Rank % Rank

Campylobacter spp

33% 1 18% 3 83% 1 36% 1Poultry meat & environmental contamination by farm animals

Salmonellas non-typhoidal

15% 2 21% 2 6% 2 3% 5Varied - but eggs important

sources

VTEC O157 4% 5 5% 5 2% 5 0% 10Beef, lamb & environmental

contamination by farm animals

Listeria monocytogenes

12% 3 37% 1 2% 3 0% 11Ready to eat foods

(environment), vulnerable consumers

Norovirus6% 4 7% 4 1% 8 19% 2 Molluscs, food handlers

Total (Based on 2007

HPA Data)

£1,520m 

443 

18,906 

925,766 

Evaluation

• “Learning from what we do”, evaluation covers processes and outcomes.

• Need to consider evaluation from the outset • Part of “ROAMEF” cycle, also fits with Impact

Assessment (Post Implementation Review- i.e. did the policy achieve its aims?)

• Evaluation key to delivery of Strategic Plan

Forms of Evaluation

Evaluation Programme• Body of work to support delivery of Strategic

plan objectives• Not all parts of Strategic Plan are at same

stage• Cannot evaluate everything to same degree• What is appropriate? Where should we focus?• Evaluation of individual projects and impact of

overall programmes of activity

Evaluation Programme work• Looking at Strategy and components • Role of Evaluation Advisory Studies- looking at

specific policy areas to help clarify issues and how best to evaluate.

• Recent studies in both nutrition(front of pack labelling, calorie labelling) and safety (national Scores on the Doors scheme)

• FSA currently developing its Foodborne Disease Strategy-involving many parts.

Evaluation Issues • Importance of clear objectives and outcomes in

determining appropriate evaluation • Policy environment- legal framework (EU law),

voluntary action, Better regulation agenda• Multiple policy interventions (defining

counterfactual?)• Importance of understanding behaviour and

incentives, needs multi-disciplinary approach. • Focus – what can be evaluated?• -How best to go about it?

Wider lessons-Evaluation themes

• Needs to be seen as part of policy cycle• Think about evaluation at outset• clear about objectives• Issues of counterfactual in a changing, multi-

policy environment • Appropriate / proportionate• Share findings

Further info…

• [email protected]• FSA website: www.food.gov.uk