Seventh Framework Programme Grant Agreement No. 207727 Risk Based Inspections in organic farming Raffaele Zanoli Università Politecnica delle Marche, IT.

Seventh Framework Programme

Grant Agreement No. 207727

Risk Based Inspections in organic farming

Raffaele Zanoli

Università Politecnica delle Marche, IT

Improving the Organic Certification SystemWorkshop in Brussels, October 14, 2011

A working definition of a RBI

The goal of Risk Based Inspections (RBIs) is to develop a cost-effective inspection and maintenance program that provides assurance of acceptable integrity and reliability of a control system

A risk based approach to inspection planning is used to:

Ensure risk is reduced as low as reasonably practicable

Optimize the inspection schedule

Focus inspection effort onto the most critical areas

Identify and use the most appropriate methods of inspection

Modelling RBI systems: Objectives

1. Assessment and description of the current inspection practices in terms of risk and efficiency

2. Define a probabilistic model to increase the efficiency of the system based on probability theory

3. Optimisation of enforcement measures to reduce the occurrence of objectionable organic production

Modelling RBI systems: Data required

In order to predict the risk of non-complianceAt farmer/operator levelDepending on crop type, farm type, geographic location, operators characteristics, etc.

We need data on:detected non-compliances; structural, financial and managerial information at operator level

Modelling RBI systems: Data available

Collected during CERTCOST EU project

Data from 6 different European CBs (from CH, CZ, DE, DK, IT, UK)

Three years covered (2007-2009)

We used standard data that is routinely recorded by inspection bodies

Available data do not match the requirements

Databases mainly contain structural dataCBs collect NC data with non-homogenised textual descriptions: hard to rank NC severitySanction data are more standardised, but:

they are only a proxy of NCno common definition of sanctions across CBs / countriesno clear relationship between NC and sanctions (with some exceptions); no information available about why an operator receives a sanction (e.g. use of pesticides in wheat production, use of unauthorised feed for livestock, etc.)

Homogenisation of sanctions across CBs and countries

IT, CZ (and UK) CBs use a similar 4 sanction category (UK: NC)classification

Further aggregation in terms of slight and severe sanction categories

IT, CZ, UK straightforward interpretation; DE, DK, CH: input from CBs to correctly classify sanctions

Distribution of farms, by sanction category, country, and year

Modelling RBI systems: Analytical tools

Potential risk factors

46 hypothesis concerning factors affecting the probability for an operator to get a sanction has been generated with collaboration from partnersThe hypothesis refers to the following aspects:

general risk, structural / managerial for farms, structural/managerial for processors, specific crop, livestock and product variables, control related issues

Some of the hypothesis cannot be tested for all countries/years due to missing data (eg processor turnover, risk class)

Factors increasing/decreasing risk

Factors increasing/decreasing risk

Factors increasing/decreasing riskFew risk factors found relevant for all countries: Past behaviour, Farm Size, Bovine livestock

History dependence: operators who are not compliant tend to continue to be so

if one operator has been non compliant the previous year is more likely to be non compliant in the next year

If one operator has committed minor irregularities is more likely to be found to have committed major infringements

No overall risk pattern for crop types, though country specific risks

For livestock, bovines and pigs entail higher risk

In countries where (slight) non compliances are more numerous (DK, UK, partly CH) there might be a higher farms homogeneity, hence lower discrimination effects of explanatory variables

Personal, farmer-specific variables are probably crucial in explaining risk but we have VERY limited data on these

General conclusions

We can say with some confidence which factors contribute to risk, but we cannot rule out those who don’tAs a consequence, we cannot define low risk operator types To implement more efficient Risk Based Inspection procedures CBs would need better or different datasets

RBI based on past experience can limit predictable risk, but cannot avoid potential ‘catastrophic’ eventsuncertainty is an essential factor that should inform inspection procedures (black swans): think what can impact (the sector, the consumer, the CB, etc.) most, even if the risk (probability) of occurrence is low (but maybe the cost of detection is also low)

Some statements to open discussion

Harmonised RBI is fundamental to guarantee integrity, improve efficiency and reduce the cost of inspection: a growing body of small “organic” farmers and growers are refusing certification and inspection schemes and selling on alternative short supply-chains – this creates further confusion among consumers

Without clear and uniform criteria for classifying non-compliances as irregularities or infringement AND without better data and better information systems, no RBI system can work on a global scale

Without global trust on certification and inspection procedures no global organic trade can survive

Seventh Framework Programme

Grant Agreement No. 207727

Grazie!Thank you!

[email protected]

Limitations of the study

Data issues:

Data suffer from censoring (i.e. missing data): we only have information on NCs that were detected by the CBs, but we have no idea how many and what kind of NCs have NOT been detected

Inspection data contain varying quality/quantity of management & structural data, but little/no personal information on operators

All operators should be inspected at least once per year (legal requirement), but the share of subsequent inspections (either unannounced or follow ups) varies across countries and CBs

Data are little/no harmonised both within a country and across various countries

Limitations of the study (2)

Epistemological/methodological issues:

What is the data generating process (DGP)? Since CBs are actually using some form of internal RBI protocol to inform timing of compulsory announced inspections as well as follow-ups and unannounced inspections, the risk factors that we have observed may simply depend on their inspection planning and NOT actual risk (confirmation bias)

Due to limited amount of severe NCs and related sanctions in the database, the reliability of the analysis of factors influencing severe risks is limited by statistical reasons