Metrics in the CGIAR – reviewing Panel Report and next steps - Jeffrey Sayer

Definitions• Data are observations, such as weight, height, plot size. • Metrics are computed by aggregating and combining

raw data, for example, yield, or height for age. They often represent the values on which indicators are built.

• Indicators are summary measures that reflect system properties. Examples include infant mortality rates and the prevalence of acute malnutrition or changes in these values.

Methods• Diverse panel – 2 face to face meetings• Questionnaire – workshop• Review of CRPs• Contact persons in CRPs• Review of what others are doing• Study of some examples of metrics use

Concerns• CGIAR has made valuable contributions – Gene

banks – commodity crops – IFPRI models• Leading specialists on DMI• But overall patchy – no strategy, poor archiving

and curation – low accessibility – lack of skills• Proliferation of recent initiatives – some naïve

Purpose – accountability vs learning Research resource - understanding changes Data management – ground rules in the open access policy Impact assessment Global assessment and monitoring – SDGs etc. Alignment – with others Resources for basic metrics – measuring a few things well Rationalising CGIAR efforts Key scientific issues

ISSUES

Action for CO• Normative role for Consortium• Peer review of data – quality control• Ontology – Consistency in use of terms – i.e.

“sentinel site”• Community of practice• Fund allocation

ISPC Commentary• Provision of comprehensive, accessible high-quality data

and metrics on agricultural systems should a major public goods product of the CGIAR;

• Need for both a learning focus and accountability in a data, metrics and indicators system.

• Indicator targets for CRPs must use consistent vocabulary to maximise comparability beyond Open Access Policy

• ISPC advises caution against establishing a centralized system

• special attention to aggregation/disaggregation of data collected at different spatial and temporal scales, to allow analysis of trade-offs and interactions across system components

• More is not better – get the basics right