Workshop on Priority Setting

Workshop on Priority Workshop on Priority SettingSetting

February 11, 2009February 11, 2009

Albania-Hawaii Higher Education and Economic Development Project

Overview of the workshopOverview of the workshop

Workshop GoalWorkshop Goal

To present a systematic approach for To present a systematic approach for research program prioritization and research program prioritization and strategic action that respond to:strategic action that respond to:

profitability for the agricultural sector profitability for the agricultural sector clients' needs for new technologies and clients' needs for new technologies and

informationinformation improves effectiveness of research impactimproves effectiveness of research impact

Major ObjectivesMajor Objectives

1.1. Provide a method for Provide a method for research/program prioritizationresearch/program prioritization

2.2. Suggest a strategy for Suggest a strategy for implementation of research implementation of research priorities through self-initiative and priorities through self-initiative and partnershipspartnerships

Expected Outputs/OutcomesExpected Outputs/Outcomes

1.1. Improved skills in priority settingImproved skills in priority setting

2. Strategic plan for research support 2. Strategic plan for research support and activities and activities

3. Improved commitment to work as a 3. Improved commitment to work as a team in research priority setting team in research priority setting and research supportand research support

Standardize our Vocabulary

General Agreement on Six TermsGeneral Agreement on Six Terms

1.1. planning by objectivesplanning by objectives

2.2. programprogram

3.3. constraintconstraint

4.4. research projectresearch project

5.5. research activityresearch activity

6.6. accompanying measuresaccompanying measures

1. Planning by Objectives1. Planning by ObjectivesA SYSTEMATIC APPROACH TO PLAN A SYSTEMATIC APPROACH TO PLAN

ACTIVITIESACTIVITIES– Determines which research Determines which research objectivesobjectives

are necessary to achieve the are necessary to achieve the goalgoal identified identified

– Determine each Determine each programprogram that that contributes to achieve the objectives contributes to achieve the objectives that contributes to the overall goalthat contributes to the overall goal

– And determine research And determine research activitiesactivities of a of a projectproject within a program. within a program.

2. Research Program2. Research Program

Focuses on the implementation Focuses on the implementation of of new new technologies/constraintstechnologies/constraints, and , and on obtaining on obtaining informationinformation in in view of optimum and sustainable view of optimum and sustainable exploitation of the potentialities exploitation of the potentialities in a in a domaindomain (commodity, (commodity, region)region)

Focuses of a Research ProgramFocuses of a Research Program

Responds to the users' needs of Responds to the users' needs of research results on:research results on:– Commodity (or a group of Commodity (or a group of

commodities)commodities)– agroecological zoneagroecological zone– production systemproduction system– production factorproduction factor– market intelligence informationmarket intelligence information

3. Constraints3. Constraints

Situations or factors that hinder Situations or factors that hinder development of the domain's development of the domain's potentialpotential

Two Types of ConstraintsTwo Types of Constraints

can most certainly be solved by can most certainly be solved by research results (yield, )research results (yield, )

linked to environmental or economic linked to environmental or economic factors, or can be solved by factors, or can be solved by interventions in other areas of interventions in other areas of development (policy on import tax, development (policy on import tax, level of interest rate)level of interest rate)

Examples of ConstraintsExamples of Constraintstechnicaltechnical - low productivity of hybrid - low productivity of hybrid

varietiesvarieties

- absence of optimal crop - absence of optimal crop rotation patternrotation pattern

physical physical environmentenvironment

- erratic rainfall- erratic rainfall

- low soil depth- low soil depth

socioeconomicsocioeconomic - general political environment- general political environment

- high-input costs- high-input costs

institutionalinstitutional - inadequate credit system- inadequate credit system

- absence of effective - absence of effective integration with University integration with University researchersresearchers

- insufficient transfer of - insufficient transfer of technology systemtechnology system

A Program Consists of ProjectsA Program Consists of Projects

Project consists of ActivitiesProject consists of Activities

A Project Allows for:A Project Allows for:

6. Accompanying Measures6. Accompanying Measures

Actions recommended to Actions recommended to decision makers, to enhance decision makers, to enhance

the adoption of new the adoption of new technology/policies technology/policies

generated by the projects of generated by the projects of the research programthe research program

Examples of Accompanying Examples of Accompanying MeasuresMeasures

Extension campaignExtension campaign Market structures for inputsMarket structures for inputs Form Professional organizationsForm Professional organizations

- producers- producers- nursery owners- nursery owners- traders- traders- extension with University - extension with University researchersresearchers

Infrastructure improvementsInfrastructure improvements

TerminologyTerminologyResearch Research program (2)program (2)

Research project Research project (4)(4)

Research activity Research activity (5) (5)

Research focuses on the Research focuses on the users' needs of results in users' needs of results in a well-defined domain a well-defined domain i.e. commodity , regioni.e. commodity , region

Set of activities required Set of activities required to overcome a given to overcome a given constraint constraint

Experiment or studyExperiment or study

Agricultural Research PlanAgricultural Research Plan

5 Programs5 Programs

* 3 (projects/programs)* 3 (projects/programs)

15 Projects15 Projects

* 3 (activities/projects)* 3 (activities/projects)

45 Activities45 Activities

Workshop Workshop ObjectiveObjective::

How to define which How to define which programs and programs and which activities to which activities to undertake?undertake?

Project activities complement each other:

their combined results lead to the achievement of the

project objective

Example of Fier Region Research PlanExample of Fier Region Research PlanAgricultural Agricultural research plan of research plan of

Is made up of 5 Is made up of 5 programs . . . . . .programs . . . . . .

is made up of 3 is made up of 3 projects . . . . .projects . . . . .

is made up ofis made up of

2 activities 2 activities each . . . . . . each . . . . . .

FierFier

Prioritized program list: Prioritized program list: Tomatoes,Tomatoes, Olives etc. Olives etc.

--disease control disease control -value-added-value-added-price stability-price stability

-Spider mitesSpider mites-NematodesNematodes-Supply Chain AnalysisSupply Chain Analysis-Export PotentialsExport Potentials-Price AnalysisPrice Analysis-Farmers Collaboration Farmers Collaboration

Framework for developing a Framework for developing a cohesive plan and strategy cohesive plan and strategy for implementing prioritized for implementing prioritized research needsresearch needs

PRISM (Framework)PRISM (Framework)

PPathwayathway

forfor

RResearchesearch

IImplementationmplementation

SStrategytrategy

MModelodel

Pathway for Research Implementation Strategy Model (PRISM) – 6 steps strategy

Step 1 : Develop a list of Prioritized Research Programs

Step 2: Identify Top Constraints of Prioritized Research Programs (project activities)

Step 3. Identify Potential Collaborators, their Roles and begin Networking

Step 4. Seek out Potential Sources of Funding for Research Activities

Step 5. Conduct Research Activities

Step 6. Evaluation and Monitoring Research Activities

BREAKBREAK

ReviewAgricultural

Sector

Analyze Constraints

DetermineProgram Objectives

Prioritize Programs

Step 1: Develop a list of Prioritized Research Programs for Agricultural Sector

EvaluateResearch

Determine Objectives and Programs to Determine Objectives and Programs to meet the Goals and then Prioritized the meet the Goals and then Prioritized the Programs.Programs.

So what is the Research Goal? –profitable So what is the Research Goal? –profitable agricultural sectoragricultural sector

So what are the Objectives or Criteria to So what are the Objectives or Criteria to attain the Goal – efficiency, enhance attain the Goal – efficiency, enhance exports, high net profits etc.exports, high net profits etc.

So what are the Programs – Tomatoes, So what are the Programs – Tomatoes, Olives, Potatoes, Maize etc..Olives, Potatoes, Maize etc..

Priority Setting: Approaches and Priority Setting: Approaches and Methodologies (to come up with Methodologies (to come up with

prioritized programs –PRISM prioritized programs –PRISM Step 1Step 1

1.1. Explain why priority-setting Explain why priority-setting methods are necessarymethods are necessary

2.2. Understand that a formal Understand that a formal priority-setting process does priority-setting process does not substitute personal not substitute personal judgment, knowledge, and judgment, knowledge, and experienceexperience

3.3. List priority-setting methodsList priority-setting methods

Principles of Priority SettingPrinciples of Priority Setting

Definitions of Priority SettingDefinitions of Priority Setting

A logical, consistent, and formal A logical, consistent, and formal approach to identify most approach to identify most important possible activitiesimportant possible activities

A process of arriving at the best A process of arriving at the best possible set of research possible set of research activitiesactivities

Why Set Priorities?Why Set Priorities?

Make most effective use of resourcesMake most effective use of resources Help in planningHelp in planning Help to reach consensus on objectivesHelp to reach consensus on objectives Renovate and modernize researchRenovate and modernize research Provide guidance to managementProvide guidance to management Increase credibilityIncrease credibility

Why Use Structured Why Use Structured Methods?Methods?

Why Use Structured Why Use Structured Methods?Methods?

To organize and use available dataTo organize and use available data

To identify and openly resolve To identify and openly resolve

conflicts of interestconflicts of interest

To repeat analysis and arrive at same To repeat analysis and arrive at same

conclusionsconclusions

Characteristics of Structured Characteristics of Structured MethodsMethods

Do not substitute for the Do not substitute for the

judgment, experience, and judgment, experience, and

knowledge of researchersknowledge of researchers

Make systematic use of Make systematic use of

experienceexperience

Depend on the quality of inputsDepend on the quality of inputs

Priority-Setting MethodsPriority-Setting MethodsSingle-criterion Single-criterion methodsmethods

ChecklistChecklist

Congruency Congruency analysisanalysis

Cost-benefit Cost-benefit (economic surplus (economic surplus model)model)

Multiple-criteria Multiple-criteria methodsmethods

Scoring methodsScoring methods

Multiple objective Multiple objective programmingprogramming

Analytical Hierarchy Analytical Hierarchy Process (AHP)Process (AHP)

ChecklistChecklist

SimpleSimple

CheapCheap

Based on an existing situationBased on an existing situation

Doesn't require a profound Doesn't require a profound

knowledge of agricultureknowledge of agriculture

Congruence AnalysisCongruence Analysis

- Single-criterion method- Single-criterion method

- Allows budget allocation- Allows budget allocation

- Simple, cheap, transparent- Simple, cheap, transparent

- Poor theoretical logic — doubtful - Poor theoretical logic — doubtful resultsresults

Cost-Benefit AnalysisCost-Benefit Analysis

- Complicated- Complicated

- Uses ratios for prioritizing- Uses ratios for prioritizing

- Relates to efficiency and - Relates to efficiency and

sustainabilitysustainability

- Equity not consideredEquity not considered

- Needs data and takes timeNeeds data and takes time

SCORINGSCORING

Multi-criteriaMulti-criteria SimpleSimple Easy to understandEasy to understand Collective decision makingCollective decision making Must get all stakeholders togetherMust get all stakeholders together Exercise gets complicated if there are Exercise gets complicated if there are

many objectives and many alternativesmany objectives and many alternatives

Analytic Hierarchy Process (AHP)Analytic Hierarchy Process (AHP)

Handle multi-criteria decision makingHandle multi-criteria decision making Provide feedback right awayProvide feedback right away Offers a sound technique for eliciting and Offers a sound technique for eliciting and

quantifying decision makers’ preferencesquantifying decision makers’ preferences Decision maker can focus on smaller sets Decision maker can focus on smaller sets

of problemsof problems Deals with both qualitative and Deals with both qualitative and

quantitative judgments based on quantitative judgments based on experience, intuition and hard factsexperience, intuition and hard facts

Good method if you have incomplete Good method if you have incomplete informationinformation

Analytic Hierarchy Process (AHP)Analytic Hierarchy Process (AHP)

AHP is a decision-support tool to tackle AHP is a decision-support tool to tackle complex multi-criteria problems, such as complex multi-criteria problems, such as the situation in this study of selecting a the situation in this study of selecting a smaller list of priority research projects.smaller list of priority research projects.

The steps of AHP are as follows:The steps of AHP are as follows: Develop a hierarchically structure of the decision problem Develop a hierarchically structure of the decision problem

(See Figure)(See Figure) A set of major objectives (criteria) that are deemed relevant A set of major objectives (criteria) that are deemed relevant

to achieve the goal (first level of hierarchy) needs to be to achieve the goal (first level of hierarchy) needs to be established as the second level of the hierarchyestablished as the second level of the hierarchy

Then the third level or bottom level consists of the different Then the third level or bottom level consists of the different alternative programs for research.alternative programs for research.

GoalIncrease Income

Criterion 1 Criterion 2 Criterion 3 Criterion 4 Criterion 5

Program 1 Program 2 Program 3

A basic hierarchyA basic hierarchy

Choice of the Tool Depends Choice of the Tool Depends onon

Accuracy of available dataAccuracy of available data Time for decision makingTime for decision making Resources availableResources available Degree of participation requiredDegree of participation required TransparencyTransparency SimplicitySimplicity Theoretical logicTheoretical logic

It is important to involve It is important to involve representatives of key representatives of key stakeholder groups in stakeholder groups in program planning and program planning and

prioritizingprioritizing

Advantages ofAdvantages of Group Process Group Process

Comprehensive, unbiased Comprehensive, unbiased analysisanalysis

Builds consensusBuilds consensus Fosters supportFosters support Increases chances of Increases chances of

implementationimplementation

Stakeholders and ClientsStakeholders and Clients

in Program Planningin Program Planning

Principal Clients of ResearchPrincipal Clients of Research

Producers and agribusinessesProducers and agribusinesses

ConsumersConsumers

ScientistsScientists

PolicymakersPolicymakers

AHP was the choice from priority AHP was the choice from priority setting of research needs.setting of research needs.

Procedure for AHP ParticipantsProcedure for AHP Participants

1.1. Weigh the criteria givenWeigh the criteria given

2.2. Pairwise compares 2 programs Pairwise compares 2 programs against each criteria by picking the against each criteria by picking the one that is most important to one that is most important to fulfilling the criterion and then by fulfilling the criterion and then by how muchhow much

AHP -CriteriaAHP -Criteria

Increase EfficiencyIncrease Efficiency Natural Resource SustainabilityNatural Resource Sustainability MarketabilityMarketability Promote ExportsPromote Exports Increase Net ProfitsIncrease Net Profits

Alternative ProgramsAlternative Programs

OlivesOlives TomatoesTomatoes MaizeMaize WheatWheat SheepSheep PotatoesPotatoes

Data Collection and AdministrationData Collection and Administration

To get a representative sample To get a representative sample for the priority setting exercisefor the priority setting exercise

FarmersFarmersAgribusinessesAgribusinessesExtension staffExtension staffResearchersResearchers

Results of Criteria WeightsResults of Criteria WeightsCriterionCriterion OverallOverall FarmerFarmer ProfessionalProfessional

EfficiencyEfficiency

Promote ExportsPromote Exports

Natural Resources Natural Resources SustainabilitySustainability

High Net ProfitsHigh Net Profits

MarketabilityMarketability

Coherence TestCoherence Test

Represents the last step in Represents the last step in setting priorities among setting priorities among research programsresearch programs

Consists of examining the Consists of examining the ranking of programs (based on ranking of programs (based on their score) for the purpose of their score) for the purpose of verifying the logic and coherenceverifying the logic and coherence

The Goal of the Coherence The Goal of the Coherence TestTest

To point out differences between rank To point out differences between rank order of programs resulting from order of programs resulting from application of the priority-setting application of the priority-setting formula and rank order established formula and rank order established by intuition of the participantsby intuition of the participants

To guide the discussion on the To guide the discussion on the differences and eventually correct the differences and eventually correct the values given to the criteria for those values given to the criteria for those programs whose scores are programs whose scores are questionedquestioned

Benefits of the Coherence Benefits of the Coherence TestTest

Refines the priority-setting Refines the priority-setting resultsresults

Smooths insufficiencies of the Smooths insufficiencies of the used method and the used method and the imperfection of its applicationimperfection of its application

Often provokes a lively Often provokes a lively discussiondiscussion

New Ranking as a Result of New Ranking as a Result of the Coherence Testthe Coherence Test

Choice of Priority Program Choice of Priority Program then activitiesthen activities

Determine the most urgent Determine the most urgent research projects for research projects for implementationimplementation

Done in accordance with the new Done in accordance with the new ranking resulting from the ranking resulting from the coherence testcoherence test

Must be realistic: taking into Must be realistic: taking into account the available resourcesaccount the available resources

Pathway for Research Implementation Strategy Model (PRISM) – 6 steps strategy

Step 1 : Develop a list of Prioritized Research Programs

Step 2: Identify Top Constraints of Prioritized Research Programs Using Value-Chain Approach

Step 3. Identify Potential Collaborators, their Roles and begin Networking

Step 4. Seek out Potential Sources of Funding for Research Activities

Step 5. Conduct Research Activities

Step 6. Evaluation and Monitoring Research Activities

Step 2: Identify Top Constraints of Prioritized Research Programs Using Value-Chain Approach

Step 3. Identify Potential Research Collaborators, their roles and begin networking

Step 4. Seek out potential sources of funding for research activities

Programs Potential Research Collaborators

Potential Sources of Funding (to be sourced out continuously)

Production Processing

Marketing

Step 5. Conduct research activities

Step 6. Evaluation and Monitoring Research Activities

Collaboration requires MOUCollaboration requires MOU

Develop MOUs on who is going to do Develop MOUs on who is going to do what, when, expected outputs and what, when, expected outputs and associated budget for the associated budget for the responsibilitiesresponsibilities

Develop mutual trust, common Develop mutual trust, common mission, abide by deadlines, commit mission, abide by deadlines, commit to confidentiality and intellectual to confidentiality and intellectual property rights and frequent property rights and frequent communication on progresscommunication on progress

Step 6Step 6 The activities have to be monitored in The activities have to be monitored in

relation to plans, results and changing relation to plans, results and changing circumstances. circumstances.

Periodic reviews are necessary for Periodic reviews are necessary for successful outcomes of the research successful outcomes of the research endeavorsendeavors

Make necessary adjustments to Make necessary adjustments to unanticipated circumstancesunanticipated circumstances

The reviews are useful management tools The reviews are useful management tools to run an efficient program and be to run an efficient program and be accountable stakeholders and donors. accountable stakeholders and donors.

The management cycle integrating The management cycle integrating M&EM&E

PRISM•Prioritized program•Identify constraints•Find collaborator

•Find funding•Develop Research

Activities•M&E- identify indicators

Implementation•Carry out research

•Adjust research

Stop

Dissemination

Redesign

ContinueResearch

REVIEW

Results of Priority Program Sent Results of Priority Program Sent to Youto You

List the top three agricultural List the top three agricultural economics researchable contraints economics researchable contraints i.e. identify markets, branding i.e. identify markets, branding effectiveness, value-added etc. for effectiveness, value-added etc. for the top 2 programsthe top 2 programs

QuestionnaireQuestionnaireIdeas for action itemsIdeas for action items

Date : Date : Workshop title: Workshop title: Workshop on Research Workshop on Research

Priority SettingPriority Setting Venue:Venue: Name: _______________________________________Name: _______________________________________ Ideas I would like to try when I return to work at Ideas I would like to try when I return to work at

my research institute, based on what I have my research institute, based on what I have learned in this training workshoplearned in this training workshop

__________________________________________ __________________________________________ __________________________________________

AcknowledgementAcknowledgement

USAID supportUSAID support AUT/FEA supportAUT/FEA support Dean Bahri Dean Bahri Dr. SkreliDr. Skreli Ana ManeAna Mane MOAMOA Dr. TatijanaDr. Tatijana

FeedbackFeedback

Questions?Questions? Workshop feedbackWorkshop feedback

Thank youThank you