Modeling and Simulation for National Security Decision Making or “A Plea for a V&V and UQ Community of Practice” Laura A. McNamara, PhD Timothy G. Trucano,

Modeling and Simulation for National Security Decision Making

or “A Plea for a V&V and UQ Community of Practice”

Laura A. McNamara, PhDTimothy G. Trucano, PhD

Sandia National LaboratoriesAlbuquerque, NM

Sandia Review and Approval2005-7679 C 

Part One

Computational Social Simulation,

or“Scaling to China”

DECISION MAKING; or,

Who cares about Balinese Water Temples?

• Lots of computational social science takes place outside of decision context– Anasazi Longhouse

– Balinese Water Temples (Lansing)

Computational Science

Why Models are Fraught with Conflict

CUD STUDY:Mod-Sim, Forecasting, and Decision Making

Decision Making Under Uncertainty

EconomicForecasts

IntelligenceAnalysis

Weather/Climate

Forecasts

StockpileStewardship

Factors:

• Information

• Analytic processes and tools

• Uncertainty and incomplete knowledge

• Risk and confidence

• Decision impact

• Social issues (roles, bureaucracy/org)

Factors:

• Information

• Analytic processes and tools

• Uncertainty and incomplete knowledge

• Risk and confidence

• Decision impact

• Social issues (roles, bureaucracy/org)

Black ClassifiedSocial science dominated

Physical science dominated

Modeling and Simulation in Intelligence Analysis

A Case Study

PART ONE

Models in Physics

Model vs. Simulation

• MODEL represents one or more concepts and the relationships among them.

• SIMULATION reveals what emerges as the relationship is invoked over time.

As A increases, B decreasesAs A increases, B decreases

TIME

INCOME

From Physics Knowledge to Computational Model

PHYSICS KNOWLEDGE

Mathematical Representation(e.g., PDE, ODE)

REAL WORLD EVENT‘Solutions’

ALGORITHM C++, FORTRAN

MACHINE‘CODE’

‘OUTPUT’Graph,

3D Simulation

CODES IN X DIVISION

A Changing Role for Physics and Engineering Codes

• 1945-1992: Computational models for “numerical experiments”• NW Testing program ends, 1992• Science Based Stockpile Stewardship, 1995 • Accelerated Strategic Computing Initiative

– “Virtual test environment” for nuclear physics and engineering problems

• Computational models for – knowledge integration?– Diagnostics– prediction– certification?

• The deterrent will be effected/affected by decisions we make with these models

From Design and Test toVerify, Validate, and Quantify Uncertainty

• VERIFICATION: is the model solving the equations correctly and accurately?

• VALIDATION: does the model output compare well with experimental data?

• UNCERTAINTY QUANTIFICATION: What are the primary sources of uncertainty; are they reducible; and how do these dictate investment in research priorities?

From Physics Knowledge to Computational Model

PHYSICS KNOWLEDGE

Mathematical Representation(e.g., PDE, ODE)

REAL WORLD EVENT‘Solutions’

ALGORITHM C++, FORTRAN

MACHINE‘CODE’

‘OUTPUT’Graph,

3D Simulation

PART TWO

Complexity and Agent Based Modeling

Complexity, complex systems, complex adaptive systems…

• “Something almost entirely random, with practically no regularities, would have effective complexity near zero. So would something completely regular, such as a but string consisting entirely of zeroes. Effectively complexity can be high only in a region intermediate between total order and complete disorder.” *

• Complicated systems aren’t necessarily complex!• A complex system: nonlinear, feedback, iterative

nondeterministic, resists decomposition, emergent, self organizing….

*From Gell-Mann, Murray. 1995. ‘What is Complexity?” in Complexity. New York: Wiley and Sons.

Consider the Boid

Maintain separation

Maintain alignment

Maintain cohesion

BOIDS Film

PART THREE

Agent Based Modeling

And Anthropology

• PRIMATOLOGY: troop behaviors• ARCHAEOLOGY: hunter-gatherer foraging; settlement

patterns in SW Anasazi society; the emergence of states from chiefdoms or tribal societies

• CULTURAL ANTHROPOLOGY: Marriage and stratification in Tonga; Balinese water temples as an emergent system for efficient agriculture

• MORE GENERAL PROBLEMS: Why cooperation or competition emerge as behavioral preferences in societies; one-on-one or spontaneous violence vs. organized warfare as a function of environmental characteristics, population size, other variables

The Anasazi Longhouse Simulation:Why was the valley abandoned?

• Households harvest, consume their nutritional requirements of grain.

• Households store any remaining grain for later consumption.

• Households evaluate availability of grain for following year, based on current harvest and grain stores.

• If the amount is insufficient, the household moves to a new viable location to farm

Dean, Gumerman, Epstein and Axtell, 1998. ‘Understanding Anasazi culture change through agent-based modeling’. Santa Fe Institute Working Papers ref: 98-10-094.

“The most humanistic of the sciences, the most scientific of the humanities…”

• ETHNOGRAPHY– Verstehen– Experiential– Idiographic– “Thick Description”– Narrative– Qualitative– Postmodern– Holistic– Meaning

• ABM– Logical positivism– Distanced– Nomothetic– Abstraction– Simplification– Rules, equations– Modern – Extractive– Observables

Why Computational Social Modeling is Really Hard (1)

“If X occurs, then Y is likely.”

P(y,x) > .5

Part Four

Agent Based Models in

National Security Decision Making

EPISIMS: Day 2

EPISIMS: Day 24

EPISIMS: Day 34

Part Five

The Pen and the Sword

Anthropology and National Security:An “allergic response?”

• The sordid history of anthropology• Applied vs. Pure Research

– Applied research is always done with an end in mind, not with an open mind

• First Amendment Rights– McCarthyism: anthropologists being watched, monitored

for leftist behavior– Hoover’s FBI: academics in general, and anthropologists

in particular, monitored in their classrooms• Moral epistemology of anthropology: work to preserve

cultural integrity, diversity• Protection of human subjects

Cold War Anthropology:The Time of Troubles

• WWII: ‘National Character’ studies – Ruth Benedict, The Chrysanthemum and the Sword

• 1963-1965: Project Camelot– SORO/American University project– Develop models of insurgency/ counter insurgency– Chilean fiasco– Was the work any ‘good’?

• National Student Mobilization Committee– Photocopies of anthropologists’ notes from meetings

with intelligence, military officials– Ethics investigation– AAA statement of ethics in anthropological research,

1973

Part Six

Drawing it all Together

The Strong and the Weak

• STRONG Artificial Intelligence: “I can create a machine that can reason as a human mind.”

• WEAK Artificial Intelligence: “I can create a machine that can mimic human techniques for aggregating information.”

• STRONG Computational Social Science: “I can create a virtual society, in which interacting agents produce cultural forms that in turn shape their behaviors.”

• WEAK Computational Social Science: “Within a bounded problem space, I can program agents with simple rules and see what emerges; I can use evolutionary algorithms so my agents can adapt to the emergent phenomenon.”

Modeling and Simulation in Applied Physics and Decision Making

• What’s the purpose of the model?• Who owns the model? Who runs it? Who interprets

its results?• How transparent is the model? • How is the model verified? Validated?• How does the model account for uncertainty in

inputs and outputs?• What is the social location of the model in the

knowledge production process? In decision making?• Is there a formal basis for TRUST in a given

model/simulation?

Questions to consider when developing, using, considering a model

• Why was the model developed? What does it claim?• Which theory? What is theory doing (description,

explanation, prediction?)• What aspect of reality is the model targeting?• What does it mean for data to ‘represent’ some

targeted aspect of social/cultural reality?• What theoretical framework underlies the conceptual

model? • Is the conceptual model being driven by the

computational technique being used to develop the model?

• How do we measure the ‘fit’ between the (rule based) computational format and the (narrative) conceptual model?

• What will the model be used for? ‘Pure’ knowledge production or decision making?

[from Turnley 2004][from Turnley 2004]