Ex-ante Evaluation /Prioritization MulticriteriaApproaches ... · Ex-ante Evaluation /Prioritization MulticriteriaApproaches -and the context - ... Attribution of a relative value

Ex-anteEvaluation/ PrioritizationMulticriteria Approaches

- andthecontext-

SergioSalles-Filho&AdrianaBin04.07.2017

Outline

1. FundamentalconceptstoRDIprioritization

2. Howtoselectthemostsuitableapproachesunderdifferentconditionsofboundeduncertainty/partialknowledge?

3. Fourcases

4. Agenda

1.FundamentalconceptstoRDIprioritization

RDISpecificities

Thingsthatneverhappenedbefore

Thingsthatdependonknowledgetobedeveloped

Thingswedonotknowwhethertheywillwork

Fundamentalconcepts

UncertaintyRisk

Ambiguity

Rationality

Expectation

Intuition

Twocomplementarydefinitions

• “…Itisaworldofchangeinwhichwelive,andaworldofuncertainty.• (…)• Theessenceofthesituationisactionaccordingtoopinion,ofgreaterorlessfoundationandvalue,neitherentireignorancenorcompleteandperfectinformation,butpartialknowledge.“

Risk,Uncertainty,andProfitFrankKnight,1921

Twocomplementarydefinitions

• “Inapredestinateworld,decisionwouldbeillusory• Inaworldofperfectforesight,empty• Inaworldwithoutnaturalorder,powerless• (…)• Sincedecisioninthissenseexcludesbothperfectforesightandanarchyinnatureitmustbedefinedaschoiceinfaceofboundeduncertainty”

DecisionOrderandTimeinHumanAffairsGeorgeShackle(1969)

PartialKnowledge

BoundedUncertainty

Ex-anteEvaluation

1+2: Thestartingpoint

Howprioritizeunderpartialignoranceandboundeduncertainty?

Data&Opinion

Data&Opinion’

Opinion

Opinion’

Data

Data’

Opinion&Data

Opinion&Data’

BasedontheStaceyDiagram(RalphStaceyprof.Hertfordshire)

ProfessorRobertGeyerofLancasterUniversity

StaceyDiagram

2.Howtoselectthemostsuitableapproachesunderdifferentconditionsofboundeduncertainty/partialknowledge?Ageneralproposition

Foresig

htto

ols

DataandTextmining

SurveysandDelphi

Scenarios

Projections

Panels

Roadmapping

Horizonscanning

… MeansofP

rioritiza

tion Mathematicalprogramming

Multicriteria MethodsMCDA

Descriptive/multivariateStatistics

Simplescoring

Potentialsurprise

BayesianStatistics

RealOptions

Geneticalgorithms

Artificialintelligence

...

ForesightToolsandMeansofPrioritization

Severalclassificationsofmethods

MedidasdeBenefício

métodoscomparativos

métodosdepontuação

modeloseconômicos

técnicasdedecisãoemgrupo

ProgramaçãoMatemática

TeoriadosJogos

ModelosdeSimulação

ModelosHeurísticos

EmulaçãoCognitiva

Pontuação

ModelosEconômicos

MétodosInterativos

AnálisedeDecisão

ProgramaçãoMatemática

InteligênciaArtificial

OtimizaçãodePortfolio

AvaliaçãopelosPares

MedidasdeBenefício

ProgramaçãoMatemática

EmulaçãoCognitiva

ModelosHeurísticosedeSimulação

OpçõesReais

Modelosadhoc

MétodosEconômicos

ProgramaçãoMatemática

AnáliseDecisória

MétodosInterativos

Pontuação

ModelosEstratégicos

Verbano eNosella (2010)Iamratanakul etal.(2008)Henriksen eTraynor (1999)

Heidenberger eStummer (1999)

MathematicalProgrammingOptimization

EconomicModelsUtitlity,B/C,RealOptions

MachineLearningModeling AI

ADHOCModelsExperts,Citzen

MCDA- MulticriteriaDecisionAnalysis

MCDA

OperationsResearch

Statistics

MachineLearning

Psychology(BehavioralEconomics)

SocialChoiceTheory

WhatisMCDA?

SlidekindlyprovidedbyProf.LeonardoTomazeli FCA/UNICAMP

MCDA:aninterdisciplinaryfield

SlidekindlyprovidedbyProf.LeonardoTomazeli FCA/UNICAMP

AGGREGATION• MultiAtribute Utility(MAUT)• MultiAtribute ValueTheory(MAVT)

OUTRANKING• Electre• Promethee

MCDA:maincategories

Analytichierarchyprocess(AHP)

Analyticnetworkprocess(ANP)

ELECTRE(Outranking)

Goalprogramming

Innerproductofvectors

UTA,UTAII,UTADIS

NonstructuralFuzzyDecisionSupportSyst.

PAPRIKA

PROMETHEE(Outranking)

Superiorityandinferiorityrankingmet.

TOPSIS

Valueengineering

FuzzyVIKORmethod

Weightedproductmodel

Weightedsummodel

Multi-attributevaluetheory(MAVT)

Multi-attributeutilitytheory(MAUT)

SlidekindlyprovidedbyProf.Anibal Azevedo FCA/UNICAMP

MCDA:maincategories

Goal

Attributionofarelativevalueorofacomparisonstructure

Evaluationofthealternativestotheselectedcriteria

Decisionaiding

DM

Agent1 Agent2 ... Agentk

Alternatives

Criteria

Informationprocessing

MCDAmethod

Problemmodeling

Decisionmakingauthority

Data

SlideofProf.LeonardoTomazeli FCA/UNICAMP– slightlymodified

TheprocessofMCDA

1. Data

1. Agentsinvolved

1. Goals

2. Criteria

3. Alternatives

Subjective

SlideofProf.LeonardoTomazeli FCA/UNICAMP– slightlymodified

ObjectiveandSubjectiveElements

WhyoutrankingMCDAinRDIex-anteevaluation?

• Theyaremoresuitabletodealwithboundeduncertainty• Theyarenotbasedinone-off(best)choice• Theybuildakindoffuzzyrankingcomparingallcriteriaagainsteachother• Theydealequallywithobjectiveandsubjectiveinformation (DATAANDOPINION)• Theyaresuitabletocombinewithanyotherprioritizingmethod

MCDAuses

2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017ModelosdeSimulação 1 1

AprendizadodeMáquina 2 1 1 2 1

MétodosEconômicos 1 1 1 3 1 1

ModelosAd-Hoc 2 1 1 2 2 1 2 1 2 1

ProgramaçãoMatemática 1 1 2 1 2 2 3 2 4 4 2 1 2 2

MétodosMulticritério 1 1 1 2 2 2 1 6 5 8 6 6 4 3 6 5 2

0

2

4

6

8

10

12

14

WoK:2000– 2017à 109papers(R&Dprojectselection+differentmethods)

MathematicalProgramming

Economicmodels

Machinelearning

Adhocmodels

MCDA

Simulation

Massaguer,P.(2017,forthcoming)

MCDAcombinationsShowValues>=1and<=28

AHPAnalyticNetwork

Delphi DEADynamic

programmingLinearprogram

Integerprogramming

Fuzzysets

Goalprogramming

NPVGametheory

Decisiontree

Expertsystems

MonteCarlo

Peerreview

RateofReturn

ScoringDiscountedcash

AHP 28 1 1 2 1AnalyticNetwork 1 24 4 4 1Delphi 1 4 11 1DEA 10 1Dynamicprogramming 8 1Linearprogramming 8Integerprogramming 2 1 7Fuzzysets 5Goalprogramming 4 1 4NPV 1 4 1 2Gametheory 3Decisiontree 1 1 2Expertsystems 2MonteCarlo 1 2Peerreview 2RateofReturn 2 2Scoring 2Discountedcashflow 1

Massaguer,P.(2017,forthcoming)

PrioritysettinganddecisionmakinginRDIusingtheStaceyDiagram (Slightlymodified…)

Agreem

ent

Assertivenessofinformation+ -

-Data&Opinion

Data&Opinion’

Opinion

Opinion’

Data

Data’

Opinion&Data

Opinion&Data’

Math.Programming

EconomicModels

Adhoc models

MCDAMachinelearnning

Machinelearnning

PrioritysettinganddecisionmakinginRDIusingtheStaceyDiagram (Slightlymodified…)

Adhoc models

3.Fourcases

Methodstoemphasis

convergence

Methodsforboth

CalculationMethodstoemphasisinformation

Agreem

ent

Assertivenessofinformation+ -

-Aproposalbasedonthediagram

4 cases

Materialsforpackaging

perceptionofwell-beingin10yearstime

projectstoaccomplishwithan

enforcement

technologiesina

“promising”areaof

knowledge

Agreem

ent

+

-

Assertivenessofinformation+ -

Case1:selectingwell-beingconceptsforcosmeticsindustry

Theobjectiveofthestudy:• Identifyrelevantcomponentstocharacterizeandtomeasureindividualwell-being

TheChallenge:• highlysubjective,non-structured,andvariableinformationtobegatheredandselectedaccordingtoseveral

criteria• Horizonof10years• Lowlevelofagreementaboutwhatpromoteswell-being• Lowlevelofassertivenessofinformation

TheApproach:• Firstlist:literaturereview+Interviews• Internalvalidation• Backgrounddocument• Multi-ExpertPanel:collectivescores• Simpleranking(importancexcontext)

Case1:selectingwell-beingconceptsforcosmeticsindustry

Findings:• Mostadequateconceptstobeexploitedindifferentsituations:

3rdBest:joy–optimism- discomfort(reduction)

2ndBest:emotionalstability- anxiety/distress(reduction)- love

Bestranked:happiness- self-esteem- physicalhealth

OpinionBased

• Literaturereview• Imaginationanddiversity• Heterogeneousopinions• Discussions

Case%1%

future%of%thermoplas0c%

resins%in%packaging%

percep0on%of%well7being%for%

new%cosme0cs%company%

projects%in%order%to%

accomplish%with%an%

enforcement%

technologies%in%a%

“promising”%area%of%

knowledge%

Agreem

ent%

Asser0veness%of%informa0on%+% 7%

7%

Case2:Technologiesforapromisingareaintheelectricitysector:Grids

Theobjectiveofthestudy:• IdentifytechnologiestocomposefutureR&Dprojectportfoliowithfocusongrid/smartgrid

TheChallenge:

• Identifytrends,strongandweaksignals,andopportunitiestodevelop“Grid”technologies• HighlevelofagreementabouttheimportanceofGrid

• Lowlevelofassertivenessofinformation

TheApproach:• Firstlist:Dataandtextmining

• Internalvalidation

• Backgrounddocument

• ExpertPanel• ELECTRE(severalscenarios)

• Classificationof“robust”technologies

• Internalvalidation

••Smart meters (AMI)••Meterdatamanagementsystems(MDMS)integratedwithinformationsystemsoperating(AMI)••Artificialintelligence systems(AdaptiveProtection,Control Technologiesand DynamicReconfiguration)••Virtualandaugmentedrealityplatformsforsimulation(TrainingMethods)

Robustselection

Searchforexpertopinionsandnewdata

• Asmuchdataaspossible• Searchforweaksignals• Expertopinion• Popperianfalseability(KarlPopper)

Case%2%

future%of%thermoplas0c%

resins%in%packaging%

percep0on%of%well7being%for%

new%cosme0cs%company%

projects%in%order%to%

accomplish%with%an%

enforcement%

technologies%in%a%

“promising”%area%of%

knowledge%Ag

reem

ent%

Asser0veness%of%informa0on%+% 7%

7%

Case3:Selectingprojectstoaccomplishwithregulatoryframework

Theobjectiveofthestudy:• SelectthemostsuitableprojectproposalstoaccomplishwiththeBrazilianregulatoryframeworkinthe

electricalsectorwhichobligesfirmstoinvest1%ortheirrevenuesinR&Dperyear(underthethreatofbeingfinnedbytheregulatoryAgency)

TheChallenge:• Selectingtheproposalsthatfulfilltheregulatoryrequirementsandreducetheregulatoryrisk• Highlevelofagreementaboutthealternatives• Highlevelofassertivenessofinformation

TheApproach:• Firstlist:Proposalsreceivedbythecompany• Optimization+multicriteria method(knapsack+Promethee)+MonteCarlo• ClassificationoftheBestsolutionbasedonoptimizationandoutranking

5 10 15 20 25 300

10

20

30

40

50

60

Total number of projects to be selected

Mean

cons

umpti

on (%

)

KNAPSACK-OUTRANKINGPROMETHEE IIPROMETHEE V

MaximizationApproaches

• Optimization(knapsack)problem• +Multicriteria• +MonteCarlo• Etc.

Case%3%

future%of%thermoplas0c%

resins%in%packaging%

percep0on%of%well7being%for%

new%cosme0cs%company%

projects%in%order%to%

accomplish%with%an%

enforcement%

technologies%in%a%

“promising”%area%of%

knowledge%Ag

reem

ent%

Asser0veness%of%informa0on%+% 7%

7%

Case4:Applicationsofthermoplasticsinpackaging

Theobjectiveofthestudy:• Identifynewapplicationsofdifferenttypesofresinstowardspackaging

TheChallenge:• Identifynewpossibilitiesofusingthermoplasticresinstodevelopnew(ortoreplaceexisting)packaging

materials• Lowlevelofagreementaboutthefuture(environmentalandconsumptiontrends)• Highlevelofassertivenessofinformation(knownmaterialproperties)

TheApproach:• Firstlist:marketandtechnicalavailabledata• Backgrounddocument• ExpertPanel• Multiplecorrespondenceanalysis• Selectionofnewdevelopmentsandpotentialreplacements

Buildingconvergence

• Monitoring(dataandtextmining)• Trendsanddriversconstantlyadjusted• Searchforconvergence

Case%4%

future%of%thermoplas0c%

resins%in%packaging%

percep0on%of%well7being%for%

new%cosme0cs%company%

projects%in%order%to%

accomplish%with%an%

enforcement%

technologies%in%a%

“promising”%area%of%

knowledge%

Agreem

ent%

Asser0veness%of%informa0on%+% 7%

7%

Agreem

ent

Assertivenessofinformation+ -

-Packaging

“convergence”Well-beingKnowledge+convergence

RegulatoryconstraintsOptimization

Gridknowledge

4cases

4.Agenda

Agendaonmeansofcalculation

• Workmoreonpossibilitiesthaninprobabilities• Shackleanapproach• Horizonscanningandmonitoring

• Workonmethodsofconstantrecalculation• Bayesianapproaches

• Workonmethodsrelatedtoevolution• Geneticalgorithmsapproaches

• WorkonArtificialIntelligenceapproaches• BigDataDrivenapproach– Watsonandbeyond

Thankyouadriana.bin@fca.unicamp.brsallesfi@ige.unicamp.br