Four different views of a policy model an analysis and some suggestions Bruce Edmonds Centre for Policy Modelling Manchester Metropolitan University
Oct 31, 2014
Four different views of a policy modelan analysis and some suggestions
Bruce EdmondsCentre for Policy Modelling
Manchester Metropolitan University
Two Worlds
Four different views of a policy model: an analysis and some suggestions, Bruce Edmonds, Policy Workshop@ECCS, Vienna, September 2011, slide 2
Research• Ultimate Goal is
Agreement with Observed (Truth)
• Modeller also has an idea of what the model is and how it works
Policy• Ultimate Goal is in
Final Outcomes (Usefulness)
• Decisions justified by a communicable causal story
Policy Model• Labels/Documentation may be
different from all of the above!
Modeller Policy Advisor
Four Meanings (of the PM)
Research World
1. The researcher’s idea/intention for the PM
2. The fit of the PM with the evidence/data
The ideavalidation relation extensively discussed within research world
Policy World
3. The usefulness of the PM for decisions
4. The communicable story of the PM
The goalinterpretation relation extensively discussed within policy world
Four different views of a policy model: an analysis and some suggestions, Bruce Edmonds, Policy Workshop@ECCS, Vienna, September 2011, slide 3
Exemplars of the Divide between Researchers and Policy Makers/Advisors
• Policy makers try to get researchers to predict the unpredictable
• Researchers can’t get the input (time, money, data) they need from policy makers
• Policy advisors want support for a policy (their best guess that it’s good) and researchers reply with caveats and complications
• Policy Makers (over) simplify the results• Researchers make the model so complex
nobody but they can understand itFour different views of a policy model: an analysis and some suggestions, Bruce Edmonds, Policy Workshop@ECCS, Vienna, September 2011, slide 4
Two Models of Truth
Realism• Truth through relationship
of output to observations• Feedback through
correspondence (RMSE error, stylized facts, knowledge about processes)
• ForecastObserve loop• Often associated with hard
sciences, numbers and analytic models
Instrumentalism• Truth through trying
something and assessing how well it works
• Feedback through success (cost, votes, pleasure, pain)
• ActionAssess loop• Often associated with the
humanities, narratives, politics, perspective and aggregate statistics
Four different views of a policy model: an analysis and some suggestions, Bruce Edmonds, Policy Workshop@ECCS, Vienna, September 2011, slide 5
The Anti-Anthropocentric Assumption
• That the universe is not arranged for our benefit (as academics trying to understand it)
• That assumptions as these are likely to be wrong:– Our planet is the centre of the universe– Planetary orbits are circles– Risky events follow a normal distribution
• But here… that the social phenomena we study happen to be such that numerical and other easy ‘surface’ methods of understanding will be sufficient
• In other words, we can not side-step the difficulties with some clever proxies/mathematics/statistics/etc.
• Simplicity (what is easier) is not any guide to truth
Four different views of a policy model: an analysis and some suggestions, Bruce Edmonds, Policy Workshop@ECCS, Vienna, September 2011, slide 6
Consequences of the AAA
• There is no reason to suppose that a model that is adequate to modelling policy issues will be simple enough for us to understand– (e.g. that it will be analytically solvable)
We may well have to abandon hope of generic models and settle for context-specific approaches
We will need to use a combination of different approaches
We will need to model our models etc. to have a change of understanding them
Four different views of a policy model: an analysis and some suggestions, Bruce Edmonds, Policy Workshop@ECCS, Vienna, September 2011, slide 7
Some modelling tensions
precision (model not vague)
generalityof scope
Lack of error (accuracy of results)
realism(reflects knowledge of processes)
Economic Models
Scenarios
Agent-basedmodels
Stats/regressionmodels
Reality?
Wanted forpolicy decisions
Four different views of a policy model: an analysis and some suggestions, Bruce Edmonds, Policy Workshop@ECCS, Vienna, September 2011, slide 8
But this is the same as what confronts any organism/robot…• An adaptive entity faces an environment
which is overwhelmingly complex, uncertain, unknown etc.
• Thus this is a dilemma that AI/Robotics also confronts… whether to make robots:– Intelligent, learning knowledge about its world
and reasoning (i.e. a realist approach)– Or “hard-wired” with some imperfect but
effective tricks to enable it to survive and do the essential (i.e. an instrumental approach)
Four different views of a policy model: an analysis and some suggestions, Bruce Edmonds, Policy Workshop@ECCS, Vienna, September 2011, slide 9
AI/ML/Robotics/Adaptive Behaviour etc.
• They are deeply split about the best approach, with competing traditions that do communicate
• Started with reasoning and problem solving, assuming correct knowledge was at hand (a strictly realist limited-context approach)…
• …but in more challenging domains, quick and dirty instrumentalist approaches (following Brooks) seemed to do better but these had limitations and were specific to a certain goal
• They have not resolved this issue…• …but at least the competing approaches are recognised
and much discussed.• So here are some outcomes from that discussion…
Four different views of a policy model: an analysis and some suggestions, Bruce Edmonds, Policy Workshop@ECCS, Vienna, September 2011, slide 10
1: the Symbol Grounding Problem
• Harnad (1990) Physica D• Formal representation can not gain meaning from
internal relations with other part of the formal representation
• In order to attach internal symbols to their meaning one needs a repeated and frequent interaction in context– By repeatedly changing a model with respect to the
available evidence – By repeatedly using the model for decision making and
seeing the results– Via frequent participatory methods, involving policy
makers/advisors/stakeholders in the modellingFour different views of a policy model: an analysis and some suggestions, Bruce Edmonds, Policy Workshop@ECCS, Vienna, September 2011, slide 11
2: Starting “small”
• Elman (1993) Cognition• Start with easy and specific situations and
learn about them• Does not matter if these are instrumental
and largely wrong, but the incremental building up to more useful domains essential
• Only generalising to harder and more general situations slowly
• In other words, a commitment to a continual and lengthy modelling process
Four different views of a policy model: an analysis and some suggestions, Bruce Edmonds, Policy Workshop@ECCS, Vienna, September 2011, slide 12
3: Multi-Level Modelling
• Brooks (1991) Artificial Intelligence• Has fast local and specific reactive circuits for
control of short-term behaviour• With layers that coordinate these lower level
control strategies at progressively higher levels• Lower levels continue what they are doing
feeding back to higher levels• Higher levels occasionally adjust or interrupt
lower level units• Resulted in robots that could navigate rough and
unknown domains like insectsFour different views of a policy model: an analysis and some suggestions, Bruce Edmonds, Policy Workshop@ECCS, Vienna, September 2011, slide 13
A Layered approach to modelling
The phenomena of
concern
The phenomena of
concern
SimulationSimulationSimulation
Social Network Models
Social Network Measures
Four different views of a policy model: an analysis and some suggestions, Bruce Edmonds, Policy Workshop@ECCS, Vienna, September 2011, slide 14
A tighter loop involving stakeholders...• Stakeholders are involved in parts of the
“modelling” loop: criticising model, providing data, specifying model, determining goal etc.
• Involvement comes from: relevance to their goals, having some effect/control, quickly seeing the results, feeling involved, not onerous, being situated in their lives
• This inevitably means a loss of control by modellers! This is unavoidably political.
• A radical move: giving this power more directly to people rather than their representatives
Four different views of a policy model: an analysis and some suggestions, Bruce Edmonds, Policy Workshop@ECCS, Vienna, September 2011, slide 15
Two Worlds –
Empirical• Ultimate Goal is
Agreement with Observed (Truth)
• Modeller also has an idea of what the model is and how it works
Instrumental• Ultimate Goal is in
Final Outcomes (Usefulness)
• Decisions justified by a communicable causal story
Model• Labels/Documentation may be
different from all of the above!
Tighter loop =
participatory modelling
Four different views of a policy model: an analysis and some suggestions, Bruce Edmonds, Policy Workshop@ECCS, Vienna, September 2011, slide 16
Top-level Policy Advisors, Academics, Official Stats.
Crowd sourcing/input, individuals
Regional-level Policy Makers, Stakeholders
A Multi-Level Vision of PM
Top-level PM
Topic Specific PM
Region Specific PM
Citizen-level PM
Citizen-level PM
Citizen-level PM
Citizen-level PM
Top-level Policy Makers
Local Stakeholders, activists
Feedback on Use
ContinualDocking
ContinualDocking
ContinualDocking
Empirical Input
Pressure groups, Qual. Data, Academics
Four different views of a policy model: an analysis and some suggestions, Bruce Edmonds, Policy Workshop@ECCS, Vienna, September 2011, slide 17
Consequences of Multi-Level PM
• Different models being “pulled” in different ways by different groups, inputs and needs
• Continual re-modelling to keep models ‘docked’ with each other and to incorporate new observed processes (maybe with a distributed ‘wiki’-like structure)
• A lot of work by stakeholders as well as researchers
• A lot of data of ALL levels and kinds: textual, anecdotal, network, aggregate statistical, mass data, time-series etc.
Four different views of a policy model: an analysis and some suggestions, Bruce Edmonds, Policy Workshop@ECCS, Vienna, September 2011, slide 18
Main Conclusions
• The Policy/Research divide is deep, not just a matter of inclination, stubbornness or training, but rooted in their different goals
• Lessons from the AI/Robotics world imply:– Need for tight and sustained iteration of
modelling processes– Start with easier problems, work up later– Multi-level approach, with different levels of
model, generalising only when possible and necessary
Four different views of a policy model: an analysis and some suggestions, Bruce Edmonds, Policy Workshop@ECCS, Vienna, September 2011, slide 19
Auxiliary Conclusions
• A need for techniques (human and computer aided) for “translating” qualitative evidence from conversations and texts to the agent rules in simulations
• The need for an ethnographic observation and study of researchers and policy people interacting
• Possibly writing “manuals” for policy advisors “Researchers: how to deal with them, their advantages and bugs”
• A similar manual for researchers about policy advisors/makers!
Four different views of a policy model: an analysis and some suggestions, Bruce Edmonds, Policy Workshop@ECCS, Vienna, September 2011, slide 20
The End
Bruce Edmonds
http://bruce.edmonds.name
Centre for Policy Modelling
http://cfpm.org
Slides uploaded to http://slideshare.com