Master Économie et Affaires Internationales Master Économie et Affaires Internationales Paris Dauphine -October 2007 Dr. Ramón Mahía Professor of Applied Economics Department www.uam.es/ramon.mahia 20 KEYS TO BETTER SIMULATING
Master Économie et Affaires InternationalesMaster Économie et Affaires InternationalesParis Dauphine -October 2007
Dr. Ramón MahíaProfessor of Applied Economics Department
www.uam.es/ramon.mahia
20 KEYS TO BETTER
SIMULATING
20 KEYS TO BETTER
SIMULATING STRUCTURE OF DOCUMENT AND EXPOSITION
Not a technical document
Our experience in credible simulation for
complex systems in the real world
Text focused on know-how to do useful real
simulations rather than technical skills
Simulation as a way of using econometrics
in a useful way with analytical restrictions
20 KEYS TO BETTER
SIMULATING STRUCTURE OF DOCUMENT AND EXPOSITION
I. What does Simulation mean?
Concept
Simulation Vs other topics
Some Simulation “last-names”
II. Basic elements of a Simulation Model
III. 20 Keys for an Efficient Simulation
20 KEYS TO BETTER
SIMULATING I. WHAT DOES SIMULATION MEAN?
“To represent something, feigning or
imitating what it is not”
Simplified representation of a real complex
system useful for:
Understanding the working of a real system
Experimenting with, for evaluating different
strategies to be developed on it.
20 KEYS TO BETTER
SIMULATING I. WHAT DOES SIMULATION MEAN?
20 KEYS TO BETTER
SIMULATING I. WHAT DOES SIMULATION MEAN?
Real System:Real System:
A lot of elements A lot of elements inter-relatedinter-related
Simulated System:Simulated System:
Few elements and Few elements and selected relationsselected relations
20 KEYS TO BETTER
SIMULATING I. WHAT DOES SIMULATION MEAN?
Arrival of powerful computers and flexible programming systems = generalized use of simulation
Modeler
Use of complex techniques
User
Use of simple interfaces
Simulation with a model is a wide spread exercise:
•Macro economic relations: trade, labor, supply – demand,•Financial markets: treasury ship, stock exchange, commodity prices…•Technology innovation processes •Business strategy games•Business organization games…
20 KEYS TO BETTER
SIMULATING I. WHAT DOES SIMULATION MEAN?
20 KEYS TO BETTER
SIMULATING I. WHAT DOES SIMULATION MEAN?
FINANCE:FINANCE: Wall Street Raider Wall Street Raider
20 KEYS TO BETTER
SIMULATING I. WHAT DOES SIMULATION MEAN?
MARKETING:MARKETING: MarktStrat MarktStrat
20 KEYS TO BETTER
SIMULATING I. WHAT DOES SIMULATION MEAN?
BUSSINES ORGANISATION:BUSSINES ORGANISATION: EIS Game EIS Game
20 KEYS TO BETTER
SIMULATING I. WHAT DOES SIMULATION MEAN?
MACRO-POLICY:MACRO-POLICY: National Budget Simulation National Budget Simulation
20 KEYS TO BETTER
SIMULATING I. WHAT DOES SIMULATION MEAN?
Structural Analysis Forecasting
Simulation
Borders between Simulation, Forecast and
Structural Analysis are diffuse.
But the analytical approach, technical resources,
and ways of use permit us to distinguishes it.
20 KEYS TO BETTER
SIMULATING I. WHAT DOES SIMULATION MEAN?
Simulations Vs. Optimization
Optimization systems concentrates mainly on
reaching a well predefined objective given a set of
restrictions.
Simulation is an open strategy that use the links
between inputs and outputs without setting a priori
what must be considered an optimum solution.
That’s why we usually say that simulation models are
not “resolved”, but “runned”.
20 KEYS TO BETTER
SIMULATING I. WHAT DOES SIMULATION MEAN?
Simulations Vs. Optimization: Design car routes to
pick up employees to the factory from distant locations
OPTIMISATION SIMULATION
Objective function:
• Minimize time
Restrictions: • 3 cars• 1 hour to finish• 13 passengers
Inputs:• Cars•time to finish• # of passengers
Ouputs (results): • Route design
Results: • Route design
20 KEYS TO BETTER
SIMULATING I. WHAT DOES SIMULATION MEAN?
Deterministic (MKT Mix effects evaluation)
Deterministic Inputs (controlled values):• Advertising effort• Price policy• Distribution policy • Sales Force
Random (Agricultural Crop Yield)
Random Inputs (not controlled values):• Climate conditions• Plagues
20 KEYS TO BETTER
SIMULATING I. WHAT DOES SIMULATION MEAN?
Deterministic + Random (MKT Mix evaluation)
Deterministic Inputs (controlled values):• Advertising effort• Price policy• Distribution policy • Sales Force
Inputs to be randomly modeled (forecasted)• MKT Mix of existing or new competitors• Economic conditions of country• Market Demand (2nd stage input)
Reg
ress
ion
an
alys
is
20 KEYS TO BETTER
SIMULATING I. WHAT DOES SIMULATION MEAN?
Static Vs. Dynamic: Does “passing of time” result in a key
variable (even an input) for the simulation system?
Example: Time as a basic input of a simulation system (two
real examples):
Economic impact of an immigration flow will not only depend on the
amount of immigration, but in the speed of it.
Effects on prices or trade flows of a tariff removal between two
areas, will drastically change if you consider a gradual removal or
not.
20 KEYS TO BETTER
SIMULATING I. WHAT DOES SIMULATION MEAN?
Time as a basic input…..(example from Femise)
The inputs and outputs of the model will be
proportional distributed in a basis of a 7 years
scenarioThe inputs and outputs of the model will be
aggregated in a single year
20 KEYS TO BETTER
SIMULATING II. BASIC ELEMENTS OF A SIMULATION MODEL?
(i) System to be analyzed.
The collection of elements and its interactions which is
trying to be analysed by means of the simulation.
It is critical to identify the system (or sub-system) of
interest and concentrate the effort on a suitable
dimension, but…..
….without missing the links with the rest of the systems
or other collateral sub-systems
20 KEYS TO BETTER
SIMULATING II. BASIC ELEMENTS OF A SIMULATION MODEL
(i) System to be analyzed
TRADE BARRIES
PRODUCTION STRUCTURES
MACRO ECONOMIC ENVIROMENT
TRADE SUPPORT
TRADE FLOWS
POLITICAL ISSUES
20 KEYS TO BETTER
SIMULATING II. BASIC ELEMENTS OF A SIMULATION MODEL
(ii) Analytical – Mathematical Model (an example for
international trade market equilibrium):
J
j
I
iijij
J
j
I
iijij
I
ijiii
J
jjjjj
j
XXTTXTdZWZu
dQYQhAD
MAX
1 11 11
1
),(
),(1
1
Ad-Quantum Tariff Matrix
Ad-Valorem Tariff Matrix
Import Inverse
Function
Export Inverse
Function
Existing Quota
Regimes
I
ijiii
J
jjjjj dZWZudQYQh
11
),(),(
•Equilibrium reached making equal the inverse functions
of imports and exports revenues
20 KEYS TO BETTER
SIMULATING II. BASIC ELEMENTS OF A SIMULATION MODEL
(iii) Inputs, Outputs, (coherent design of inputs
and outputs)
Tariffs to be removed
Span of time
Productivities
Fiscal pressure
Exchange rate
Saving rate
……
Changes in trade flows
Changes in trade prices
Changes in Va by sector
Changes in employment
Changes in fiscal revenues
……
20 KEYS TO BETTER
SIMULATING II. BASIC ELEMENTS OF A SIMULATION MODEL
(v) Interface:
20 KEYS TO BETTER
SIMULATING III. 20 KEYS
Results analysis and validation of the simulation process
Try to understand the problem
Information recollection and treatment
Mathematical formulation of the model
Computer interface design
Validation of simulation model
Design of Simulation scenarios
Is there a simulation problem?
Try another approach
NO
YES
Does it work?
YES
NO
Prepare an offer to the client
If you have convinced…
Calibrate
Results analysis and validation of the simulation process
Try to understand the problem
Information recollection and treatment
Mathematical formulation of the model
Computer interface design
Validation of simulation model
Design of Simulation scenarios
Is there a simulation problem?
Try another approach
NO
YES
Does it work?
YES
NO
Prepare an offer to the client
If you have convinced…
Calibrate
20 KEYS TO BETTER
SIMULATING III. 20 KEYS
1.- 1.- Be sure that you a have a Be sure that you a have a simulation problemsimulation problem
Have to deal with a multivariate problem Can clearly identify inputs and outputs Input variables can vary in a wide range of values Output variables clearly respond to changes in inputs There is not a single scenario to be established
20 KEYS TO BETTER
SIMULATING III. 20 KEYS
2.- 2.- Offer feasibility of its Offer feasibility of its prospects and involve the end prospects and involve the end
users in the whole proposalusers in the whole proposal
Don’t make the mistake of offer maximum proposals and bear in mind its realism.
The final user usually prefers work proposals in which they feel an active part from the beginning
20 KEYS TO BETTER
SIMULATING III. 20 KEYS
3.- Program enough time to study Program enough time to study deeply the system to be deeply the system to be
analysedanalysed
The largest part of the technical decisions regarding the estimation, calibrating, scenario and interface design are conditioned by the comprehension of the elements and interrelations of the system to be analysed.
20 KEYS TO BETTER
SIMULATING III. 20 KEYS
4.- Integrate in your team Integrate in your team theoretical experts been familiar theoretical experts been familiar
with the systemwith the system “Research needs more heads than hands”. Save time, which could be used to improve the
technical issues for the simulation model Help even for taking technical decisions Establish a theoretical rigour to the whole of the
analysis Give reality to the simulation mechanism
20 KEYS TO BETTER
SIMULATING III. 20 KEYS
5.- Prioritise the wishes of users in all the stages of the
construction of the simulation model and take their advices
Nobody is interested in a technically refined tool that does not serve their interests
The users only want the model to be adjusted to their demands, nor the other way round.
20 KEYS TO BETTER
SIMULATING III. 20 KEYS
6.- 6.- Study “in deep” the work Study “in deep” the work carried out by otherscarried out by others
Originality must never be an aim in itself Explore previous main sources of data - limitations of
the exercise - different techniques available
20 KEYS TO BETTER
SIMULATING III. 20 KEYS
7.- 7.- Dedicate time to analyze all of Dedicate time to analyze all of the available datathe available data
"Measure twice, and cut once". Use homogeneous data Choose carefully the samples Assess the data provided by the end user Be extremely scrupulous in the handling of data Agree with the user that data to be used responds
faithfully to the reality perceived by him.
20 KEYS TO BETTER
SIMULATING III. 20 KEYS
8.- 8.- Explore the analytical - Explore the analytical - mathematical techniques that mathematical techniques that
best adapt to the systembest adapt to the system Its adaptation towards capturing the specific
phenomena observed in the specific system its feasibility in calculating its flexibility the quantity of theoretical hypothesis required its robustness towards eventual changes its simplicity the available resources
20 KEYS TO BETTER
SIMULATING III. 20 KEYS
9.- 9.- Try to adapt the analytical Try to adapt the analytical technique to the problem and technique to the problem and
not the other way roundnot the other way round
Guide the technical procedures by the suitability of the real characteristics observed in the system.
The technique used is only valid if it works, independently of the “objective” scientific considerations
20 KEYS TO BETTER
SIMULATING III. 20 KEYS
10.- 10.- Do not complicate the Do not complicate the technical models if it does not technical models if it does not
lead to clear benefitslead to clear benefits
"If your intention is to discover the truth, do it with simplicity and leave the elegance for the tailors."
20 KEYS TO BETTER
SIMULATING III. 20 KEYS
11.- 11.- Take care about the forecast Take care about the forecast power of the modelpower of the model
Try to anticipate the needs with regard to the prediction at the time of choosing the variables
Evaluate with the focus on “cross validation” its vulnerability to eventual prediction errors in the key variables
Avoid using single results as regards to the prediction; use always intervals of variation or alternative values with probabilities of occurrence.
20 KEYS TO BETTER
SIMULATING III. 20 KEYS
12.- I12.- If a prediction exercise is f a prediction exercise is necessary try to involve the necessary try to involve the user criteria in the interfaceuser criteria in the interface
"If you have to forecast, forecast often."
20 KEYS TO BETTER
SIMULATING III. 20 KEYS
13.- 13.- If inferential statistics are If inferential statistics are used, check the sensitivity of used, check the sensitivity of the system to changes in the the system to changes in the
estimationsestimations When the analytical procedure means the use of
statistical inference, the system can depend more or less critically on these estimations
Check the sensitivity of the results of the system against variations in the estimated coeficients
Check the robustness of the estimations with question such as changes in the data sample
20 KEYS TO BETTER
SIMULATING III. 20 KEYS
14.- 14.- Do not underestimate the Do not underestimate the political or qualitative aspects political or qualitative aspects of which experts advise youof which experts advise you
The systems are not capable of being modelled using purely quantitative structures.
It is not a question of choosing between a quantitative or qualitative approximation, rather than knowing how to combine both.
20 KEYS TO BETTER
SIMULATING III. 20 KEYS
15.- 15.- Let simplicity guide the Let simplicity guide the design of the interface in all of design of the interface in all of
its scoreits score
"The majority of the fundamental ideas in science are essentially simple and, as a general rule, they can be expressed in language understandable to everyone."
20 KEYS TO BETTER
SIMULATING III. 20 KEYS
16.- 16.- Call for software professionals into the design of
the interface
20 KEYS TO BETTER
SIMULATING III. 20 KEYS
17.- 17.- Establish different levels of Establish different levels of user for the handling of the user for the handling of the
interfaceinterface
Directors, politicians, media technicians, technical experts, etc.
“There is no inept user, only badly designed systems”.
20 KEYS TO BETTER
SIMULATING III. 20 KEYS
18.- 18.- Instruct the users on the Instruct the users on the correct use of the systemcorrect use of the system
20 KEYS TO BETTER
SIMULATING III. 20 KEYS
19.- 19.- Limit the use of the model to Limit the use of the model to real scenariosreal scenarios
The greatest part of the simulation systems can be labelled as “rubbish in / rubbish out”.
Design an interface which stops, or at least warns the user, of possible errors in the design of scenarios.
20 KEYS TO BETTER
SIMULATING III. 20 KEYS
20.- 20.- Ensure the perfect display of Ensure the perfect display of the resultsthe results
Pretest: Before giving the final ok to designs in this area, carry out several tests among your work colleagues to make sure that the results are understandable and do not oppose the proposals suggested to improve it.