Quantitative Methods Processes In “Cases, Numbers, Models: International Relations Research Methods” edited by Detlef F. Sprinz and Yael Wolinsky Reviewed.

Quantitative Methods Processes In“Cases, Numbers, Models:

International Relations Research Methods”edited by

Detlef F. Sprinz and Yael Wolinsky

Reviewed by David HenckeSSCI E100b-2

March 4, 2013

Need for This StudyIn the Interest of Scientific Progress

• An outcome of meeting of the authors at the 1997 International Studies Association (ISA)

• Leading scholars of the field push for a more robust discourse on methodology in International Relations seeking to derive benefit from multiple disciplines (“bridges, not walls”)

• Concern is that serious discussions of methodological problems common to the analysis of International Relations are still comparatively rare (are overshadowed)

• Intent is to fill the gap by presenting theoretical and empirical studies that deal with central methodological issues in the field focusing on 3 common sub-fields:– international political economy– international environmental politics– international security

Need for This StudyIn the Interest of Scientific Progress

• Three major methods used in international relations:– case study (Ch 1-5)– Quantitative (Ch 6-9)– formal methods (Ch 10-14)

“Scientific progress is bolstered by and may in fact require the application of all three methods” (Bueno de Mesquita 2002).

Developing a Research Approach• The state of the art and the intellectual progress of an academic

fields is best reflected in: – the set of empirical phenomena and questions being studied– the state of theory development– the ways in which methodology is used to evaluate theoretical claims and

their empirical implications• Focuses on methodology (and how it affects empiricism and

theoretical development• “Empirical” - depending upon experience or observation alone,

without using scientific method or theory• “Theory”

– provides clear and precise explanations of important phenomena– focuses scholarly attention on important puzzles that set the research

agenda for students

Methodology

• Refers to systematically structured or codified ways to test theories

• Critical in facilitating the evaluation of theory and the evolution of research

• Particularly useful in the context of a progressive research program where hypotheses lend themselves to falsification

• Can assist in testing existing theories (especially case studies and quantitative analysis

• Helps in generating or expanding the scope of received theories as sometimes reflected in formal modeling.

The Joys of Academic ResearchIt All Depends on Your Vantage Point

Undergrad and Post Grad Students

Doctoral Students and TAs

Faculty

• Permits the researcher to draw inferences about reality based on • the data at hand • the laws of probability• permits political scientists to aggregate information from a tremendous number of cases• ability to extract even basic summary statistics from a mass of data conveys a wealth of information in

compact and understandable form

• Ability to draw inferences helps to:– assess the extent to which the empirical expectations generated by theories are consistent with reality (does

what you think would happen logically actually happen?)– uncovering questions or puzzles (when evidence is contrary to prior theories)

• The ability to aggregate information is a potent stimulus for theorizing (to know whether or not it indicates a prevalent pattern of behavior)

• Makes the terms of a given debate more explicit• Entails standards of evidence (coding procedures for researchers to use)• Benefit to reliability and validity: can flag unacceptably “noisy” measures (“statistical

noise/artifacts”)• *Permits us to assess the claim that observed associations among variables are due to chance and thus to draw causal inferences• Conveys the ability to test two explanations against one another with remarkable precision

Empirical-Quantitative Approaches to the Study ofInternational Relations (Ch 6) Bruamoeller & Sartori

ADVANTAGES

• Sometimes far from persuasive because of misunderstanding or ignorance of the underlying purpose of the method

• Two sets of common errors– Errors of Specification: Failure of statistical tests to relate meaningfully to the

causal mechanisms implied by the theories that they purport to evaluate.• Imprecise Theories: many international relations theories are too imprecise or

shallow to be subjected to test against other impossible to test and, in the extreme, may be entirely unfalsifiable

• Has testable implications but fails to explain even a few details of any one type of event.

– Errors of Inference• problem is a marked tendency to ignore some of the thornier problems

involved in integrating data into larger-scale explanations• Basically: statistics can take the place of thinking• Researchers simply accept or reject a theory based upon an assessment of

how likely certain variables are to have non-zero effects (“null value”)

Disadvantages of the Empirical-Quantitative Approach

• The form of a statistical test should be derived from the form of the theory (not vice-versa)

• Finding a statistical test suitable for one’s theory is crucial• Obvious but equally critical: think about the theory• *The eyeball test is part of the intricate interplay between theory and data that

occurs in skillful application of the scientific method• In theorizing, one may miss important nonlinearities• Statistical tests should correspond to a well-developed theory which:

– prompts careful thinking and forces the researcher to specify– many important aspects of the situation

• Game Theory Models– Require basic (raw) elements of a theory (actors, their choices , their beliefs)– Must include all the raw elements or model cannot be solved (implications cannot be drawn )

Developing a Research Approach

1) specify precisely the real-world correspondents of the raw elements of the model (eg “no new weapons”)

2) the model must be complex enough to capture an explanation of an interesting phenomenon

3) resolve indeterminacy that is inherent in some models before turning to empirical tests (a large number of outcomes are logically possible)• “equilibrium refinements” - tools for narrowing down the set of likely outcomes• When game-theoretic models lead to multiple equilibria, consider each as its own

explanation, with its own comparative statics and its own statistical tests

Considerations:– Remember functional form and the nature of the error term– models are simplifications, and the error structure that comes literally

from the model may not be the theorist’s true best guess

3 Steps to Create a Testable (falsifiable) Theory:

Background:• Growing interest in the links between politics and economics; how political factors

shape foreign trade• Literature on this topic has given rise to some of the most influential theories of the

international political economy • Various empirical analyses of these relying on quantitative methods

Purpose• to survey some of the quantitative research conducted on the political

economy of international trade• aspects of international politics have a potent influence on patterns of

overseas commerce• statistical models of trade developed by economists frequently ignore political factors

which can produce biased results and a misleading set of conclusions about the determinants of foreign commerce

Quantitative Approaches to the International PoliticalEconomy

Edward D. Mansfield (Ch 7)

• The Growth of Quantitative Research on the International Political Economy– field of international political economy is only about three decades old– surveyed the articles published between 1970 and 1999 multiple political science

journals• Methods have been used to address a series of core debates over the effects of

the distribution of power, political-military alliances, preferential trading arrangements (PTAs), and interstate conflicts on international trade.

• Quantatative Research Documents the Effects of Hegemony on Trade– A state that is powerful enough and willing to single-handedly manage the global system– Theorists claim that such a state is necessary to promote economic openness.– important to distinguish between single hegemony and the concentration of capabilities – which

is a function of the number of major powers

• Alliances, Preferential Trading Arrangements, and Trade• Findings:

– holding constant economic factors, international trade is strongly influenced by international politics

– empirical research on the international political economy has demonstrated that this omission risks generating biased results

Quantitative Approaches to the International PoliticalEconomy

Structure/Key Discussion Points

Background:• study of international environmental policy has hitherto been dominated by case study

analysis• sub-field is largely organized around substantive concerns rather than methodologically-

oriented groupings

Increasing using of Quantitative Analysis markedly affects development of International Environmental Policy clustered around the themes of:

– ecological modernization– effect of international trade on the environment– Environmental regulation – Environmental security – international regime effectiveness

Results:– most pressing problem for quantitative analyses lie with the difficulties

involved in efforts to create a sufficiently large data collection efforts.– the field may be particularly well suited for multi-method research

The Quantitative Analysis of InternationalEnvironmental PolicyDetlef F. Sprinzl (Ch 8)

Background• Scholars of international conflict concerned with lingering key questions:

– Under what conditions are disputes between states likely to escalate to war?– What impact do alliances have on the outbreak of militarized conflict? – When will a deterrent threat be credible? – How do domestic political institutions affect a state’s propensity to settle disputes non-

violently?

• Proper Research Design and Estimation based on selected issues and submitted to Statistical Analysis yields credible insight

• Statistical tests of the causes of international conflict can be by incorporating from recent formal and game theoretic approaches to the study of international conflict

Testing Theories of International Conflict: Questions of Research Design for Statistical Analysis

Paul Huth and Todd Allee (Ch 9)

• Focus on state leaders and their choices in international disputes as the unit of analysis for building data sets

• Recognize that selection effects and strategic behavior are central concepts for understanding how international disputes evolve into stages where higher levels of conflict occur

• Better account for how policy choices in international disputes are linked across time and space

• Include explanatory variables that better capture and measure the impact of domestic and international conditions during periods of more intense diplomatic and military interactions.

Testing Theories of International Conflict: Questions of Research Design for Statistical Analysis

Paul Huth and Todd Allee (Ch 9)

Results• Research designs will greatly improve statistical tests of theories of

international conflict by better addressing problems of selection bias, non-independent observations, and measurement erro

• There should be a tighter connection between the formal game-theoretic literature and the design of statistical analyses and tests

• Empirical researchers will need to:– devote more time, effort, and resources to developing more

micro-level data sets of international disputes across different issue areas

– developing data on dispute behavior that does not involve military threats and the use of force

Testing Theories of International Conflict: Questions of Research Design for Statistical Analysis

Paul Huth and Todd Allee (Ch 9)

• permits the researcher to draw inferences about reality based on – the data at hand – the laws of probability

The Empirical ApproachDifferent Experiences = Different Results for Some (?)