Scientific Methodology & the Use of Theory

• POLITICAL RESEARCH METHODSProf. P. M. Loveless

Scientific Methodology Scientific Methodology

• & the Use of Theory

Dipartimento di Scienze Politiche e Sociali

University of Bologna

Autumn 2019

Contact Information

• Prof. Matthew Loveless

• Office: via dei Bersaglieri 19 (Bologna)

• Email: [email protected]

� Readings:� Readings:� Box-Steffensmeier, Janet M., Henry E. Brady, and David Collier. 2008. “Political Science Methodology

“ in Janet M. Box-Steffensmeier, Henry E. Brady, and David Collier (eds.) The Oxford Handbook of

Political Methodology. Oxford: Oxford University Press.

� Jackman, Simon. 2008. “Measurement” in Robert E. Goodin (ed.) The Oxford Handbook of Political

Science. Oxford: Oxford University Press, pp: 119-151.

� Brady, Henry E. 2008. "Causation and Explanation in Social Science" in Robert E. Goodin (ed.) The

Oxford Handbook of Political Science. Oxford: Oxford University Press, pp: 217-270.

� Lijphart, Arend. 1971. “Comparative Politics and the Comparative Method” American Political

Science Review 65(3): 682-693.

� Sartori, Giovanni 1970. "Concept Misinformation in Comparative Politics," American Political Science

Review 64(4): 1033-1053.

Why Scientific Methodology?

• So what? • One Reason: A lot of what we know about the

world is discovered and often explained in the

language of sciencelanguage of science

• e.g. Voting: A, B, & C and X, Y, & Z � Voting

• e.g. Conflict: D, E, & F and R, S, & T � Conflict

• The answers are derived from the literature of

accumulated studies that have been conducted empirically

and have drawn conclusions based on those findings.

What’s wrong with Common Sense?

� The first time someone sees the moon. It is a rock that is in a near

perfect orbit around this planet – which is also round – and is held

in place by the same invisible force that holds your feet to the

ground.

� While common sense appeals to logic (inasmuch as it makes � While common sense appeals to logic (inasmuch as it makes

sense) and may even be empirical as it does not contradict actual

observation, there is a limit to the validity of common sense.

� Without the means of scientific investigation, common sense is

more likely to be subject to the errors of inaccurate observation,

overgeneralization, selective observations, and illogical reasoning.

� SM is good in that it is formal: ‘Disciplining our thought’

A few Key Terms

• Empirical:

• Rigorous and formal use of data data to support analysis

• Data:

• Observations on relevant elements (a.k.a. variables) of a research

question to help determine if one explanation is better than anotherquestion to help determine if one explanation is better than another

• Qualitative (Empirical) Research:

• Use of data data such that similar things are grouped and dissimilar things

are separated by qualities or attributes

• Quantitative Research:

• Use of statistics, formal logic, or other computational solutions to

analyse datadata

What is science?

• It is a means to understand the world.

• Two types – but at the core is the assumption that the world is

ultimately knowable and that we can converge on that knowledge

through the application of an agreed upon and rigorous

methodology. methodology.

• Science is an objective and disinterested

method of inquiry

• Scientific research has the following characteristics:

• The content is the method (verifiable and objective)

• The conclusions are uncertain (knowable uncertainty)

• The procedures are public (transmissible)

• The goal is inference (generalization)

I. The Content is the Method

• A uniform, non-normative method of inquiry as a means

to impose objectivity.

• Normative: evaluative, value-laden - How things should be?

• Non-normative: Factual and objective - How things are?Non-normative: Factual and objective - How things are?

• Objectivity an attempt to eliminate or minimize the

influence of our own prejudices, biases, etc…

• Difficulty in social sciences: examining an event and its

significance (the “so what” question): e.g. participation

• Weber: There is no objective science but “the method” (i.e.

science) reduces the bias.

• Thus the “scientific approach” doesn’t describe what we study, it

describes how we study it.

II. The Conclusions are Uncertain

• We must qualify what we say by balancing it with

how certain we are that what we are saying is

correct

• “I think, I feel, I believe”

• “No one cares what we think, others only care what

we can prove (demonstrate).”

• “…and how well”

• Meaning: Though certainty is unattainable, we can

improve the reliability, validity, certainty, and

honesty of our conclusions.

III. The Procedures are Public

• A scientific approach outlines the means for others

to replicate the work - That is, it is transmissible.

• Methodology must be detailed so that replication is

possiblepossible

• The choice of methods is very subjective and can lead

to somewhat different findings

• e.g. Different conclusions - competing methodologies

III. The Procedures are Public

� In one way or another, all paradigms and theories share

the belief in the discovery of truth…...

� When scientists – not interpretivists – reach a disagreement

about a conclusion, they set about the task of finding out who is about a conclusion, they set about the task of finding out who is

wrong (or less wrong, in any case). Further rigorous testing

produces more evidence in which to determine the fitter of the

two explanations (or not!). It is an attempt to approach “truth”.

Not “Truth” but, as Dennett calls it, the “…ho hum truth about

this particular factual disagreement” (2006, 262).

� Academic journals include the peer review process

IV. The Goal is Inference, I

• Levels of Scientific Debates

• Epistemological – the study of knowing

• Scientific method vs. post-modernism

• Methodological – how we find outMethodological – how we find out

• Quantitative vs. Qualitative

• If science is a method, this debate is meaningless

• Research technique - research design

• Idiographic vs. Nomothetic • EX: What can we expect to learn from:

• A case study of the evolution of Benin’s political institutions

• A seven country, cross-national study of sub-Saharan west

Africa’s political institutions

IV. The Goal is Inference, II

• Generalization (Inference and the use of TheoryInference and the use of Theory) seeks to identify

a class of events that have applicability beyond the cases under

investigation

• DESCRIPTION ���� EXPLANATION ���� PREDICTION

• EX: Before the last Bundestag elections in Germany, we ask 1,000 Germans about their education and voting and determine that Germans about their education and voting and determine that within this group, people with higher levels of education voted more often.• Description: In this group of Germans in the past national election, the more

highly educated voted more often than less educated ones.

• Explanation: It is easier for educated Germans to gather information about politics and therefore, I make the argument that they vote more because the costs of voting are lower for them.

• Prediction: What might be one of several predictive statements we could make?

• Scientific knowledge is explanatory and even predictive

IV. The Goal is Inference, III

� The ability to go beyond being mere note-takers of

history, i.e. description (Explanation)

� The ability to be more right than wrong (Certainty)

� The ability to speak beyond the evidence at hand

(Generalization: the use of Theory: Inference)

� The ability to - if only partially - understand how the

world works (Prediction: the use of Theory: Causality)

Confirmation

• One of the elements of the scientific research methodology is the ability to be confident about our findings

• “True” in a qualified manner

• Probabilistic: random variation exists in the world and there is nothing you can do about it.is nothing you can do about it.

• Think: weather

• Deterministic: given the right explanatory variables, the world is entirely predictable

• Think: Billiards

• It is important to understand the difference between a statement of certainty and a statement that we have a great deal of confidence in.

Progress of the Scientific Approach I

� Thomas Kuhn The Structure of Scientific Revolutions (1962) asks the question,

“Brother, can you paradigm?”

� A paradigm represents the body of tested scientific ‘facts’.

� Paradigms tells us how things work, contain the available theoretical constructs,

and provide the appropriate analytical tools and methods

� i.e: “What to look at, how to look at it, and what to expect”� i.e: “What to look at, how to look at it, and what to expect”

� ‘Normal Science’: Paradigm serves as a basic road map to:

� Match facts and new observations with paradigm

� Determine significant facts

� Improve the articulation of theory

� Ground-breaking work finds new applications or increases the precision of

paradigm

� Although innovation is not necessarily rewarded as scientists are vested in

the current paradigm.

Progress of the Scientific Approach II

� However, the shift of paradigms are not incremental, they are revolutionary

� That is, Normal Science → CRISIS → Revolution → New Paradigm� «The Structure of Scientific Revolutions»

� Not gradual – moves more like a religious conversion

� “punctuated equilibrium” - although an evolution/improvement � “punctuated equilibrium” - although an evolution/improvement

� What is the origin of the crisis?

� The accepted paradigm promotes specialization, increasing the chance of noticing anomalies via improved instruments and the ability to control for greater numbers of variables

� Yet, increased specialization reduces the ability to correct for anomalies of scientific discoveries. They are ‘patched over’

� The crisis can be (temporarily) averted if the paradigm adapts, i.e.: loosens of rules for ‘Normal Science’

Progress of the Scientific Approach III

� However, when the existing paradigm can no longer retool to accommodate the anomalies � acceptance of new alternative paradigm (= rejection of the old)

� ‘Why is this idea so powerful?’ � � Revolutions as ‘Changes of Worldview’� � Revolutions as ‘Changes of Worldview’

� Revolutions demonstrate the inadequacy of the existing paradigm� Recourse fails to mend competing views

� Changes in instruments, time, and perception [i.e.: our conceptual parameters]� Not seeing things differently, seeing it ‘more correctly’

� A change in the standard not just gained knowledge

� A new paradigm should answer important questions better.� Some paradigms solve some problems, others solve others… which problems

are you interested in solving?

Conclusion: Scientific Method

• Science is a method (how), approaches are tools

• As Heisenberg said, “We have to remember that what we

observe is not nature herself, but nature exposed to our observe is not nature herself, but nature exposed to our

method of questioning.”

• The scientific method achieves probabilistic,

general knowledge about political and social

phenomena – not through statistics or case

studies but - through the use of THEORYTHEORY

Theory’s Role in Social Science

� Definition: A systematically related set of propositions that

provide possible explanations for a set of phenomenapossible explanations for a set of phenomena. .

� Statements that organize, explain, and predict knowledge

�� It is a necessarily simplified It is a necessarily simplified modelmodel of realityof reality

� “All models are wrong, but some models are useful”� “All models are wrong, but some models are useful”

� ONE: Identifies relationships about the phenomenon that we are

interested in.

� TWO: Summarizes work that has been done on our subject.

� THREE: Suggests where to look for future testing.

� e.g. : Education and Voting

� Implications: years not degrees

Inductive and Deductive Reasoning

� Induction: Specific to general

�� Observation Observation �� TTheoryheory

� EX: Class attendance on a rainy day

� Deduction: General to specific

�� Theory Theory �� OObservationbservation

� EX: The Agenda Setting theory states that the issues

that are most prominent in the news are also the

issues that are most important in public opinion.

A Model of the Role of Theory

Model Deduction Potential

(Theory) Data

Induction

World Observed

Data

Core Characteristics of Theory, I

� ONE: Testable: The concepts under investigation should

� (1) vary

� (2) be measureable (have observable empirical referents)

� (3) be what we are trying to explain

� TWO: Logically sound: Does it correspond with reality?

Does it make sense?

� EX: Height and Presidents

Core Characteristics of Theory, II

� THREE: Parsimonious: An appeal to simplicity as a foundation

for accumulating knowledge (heuristic function)

� This is Leverage (vs. Fit), think: “model” [or ‘Occam’s Razor’]

� EX: MP’s who are already in office, are well-liked by their constituents, have

raised a great deal of money, and face a challenger who has not held office

are more likely to win reelectionare more likely to win reelection

� VS. “Incumbents are more likely to win reelection than challengers

� FOUR: Predictive and fertile: Does it do a good job explaining

what we are interested in and can it keep on explaining?

� Why is globalization occurring?

� Why do countries try to become democracies?

� Why do people distrust the EU?

Criteria for Evaluating Theory

� ONE: Falsification: Falsification: Theories must have defined parameters that

tell us when they will not work.

� Karl Popper’s White/Black Swan example: : theories are not verifiable

because we can never test all observable implications of a theory

� Refinement of theory : Sophisticated Methodological Falsification [Lakatos]

� TWO: Generate as many observable implicationsobservable implications as possible

� EX: Incumbents win reelection more often because they tend to have more

money. vs. Incumbents win reelection more often because they tend to have

more resources.

� THREE: Theory should be as concreteconcrete as possible

� The importance of clearly defined concepts (e.g. je ne sais quoi)

� Theories are not Right or Wrong. They are Weak or StrongWeak or Strong