The positive discourse Re-invention of alternative methodological solutions might open new paths – no panacea but better than the dead-end street The negative discourse - Inherent controversy between quali and quanti – No compromise possible (any combination provides the mix of the downside of both methods)
20
Embed
The positive discourse Re-invention of alternative methodological solutions might open new paths – no panacea but better than the dead-end street The negative.
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
The positive discourseRe-invention of alternative methodological solutions might
open new paths –
no panacea but better than the dead-end street
The negative discourse
- Inherent controversy between quali and quanti – No compromise possible (any combination provides the
mix of the downside of both methods)
The three evaluation criteria: reliability, validity and generalisability
Reliability: the consistency of the measuring instruments and/or anytime repeated – same result
Validity: consistency between the data and ”reality” , between the measuring instrument and the conclusion
and the minimum of unobserved factors
Generalisability: the extent to which research findings can be applied to settings other than that in
which they were originally tested
If all three perfect - predictibility is at the max - the dream of social sciences
but it is impossible
The inherent difference between quali and quanti
Validity
Reliability Generalisability
The optimum
Validity
Reliability Generalisability
The qualitative bias
The quantitative bias
Validity
Reliability Generalisability
The idealtypes of measurement techniques
Official statistics
Survey Standard methods
- many publications,
- often financed,
-- standard classes
Qualitative
Experiment
- laboratory
Experiment
- natural
Non standard
Experiment
- controlled
Non-participant observation
The illustration: the prevalence of discrimination to predict discriminative
behavior• Not the attitudes towards discrimination
(attitude survey)• Not the consequences of discrimination
(wage data)• Not the media representation of discrimination
(content analysis)• Not the unintended-unconscious
(Implicit Association Test)
But the behavior of average actors (employer, fellow employee,
The methodology to be used in situation testing should be rigorously specified in order to neutralise variables that could falsify the analysis or discredit the
operation (Rorive 2009).
A general case:Ethnic discrimination in Israel
Playing „trust”, „dictator”, and „ultimatum” games (Fershtman & Gneezy, 2001)
Natural experiment (V +, R -, G -)
Orchestrating Impartiality (Goldin-Rouse,2000)
Ethnicity and gender in the „Weakest Link” TV Show (Levitt, 2004)
Ethnicity on the New York Stock Exchange during WW I (Moser, 2008)
Controlled (or field) experiment Discrimination testing (V ?, R ?, G ?)
testers are assigned to treatment and controls
- and are randomly assigned to pairs (e.g. one of each race)
- and matched on equivalent characteristics (e.g. socioeconomic status
selected situations in natural settings
Controlled for certain elements of the process and contextual factors
Non-participant observation (V +, R +, G?)
Observe and recorde data concerning the police stops: Two observers focused on benchmarking, one determined the ethnicity, the other one the gender and age of exiting passengers. A third observer recorded data for individuals being stopped by the police.
Ethnic discrimination in the Moscow metro (Ethnic… 2006)Ethnic discrimination in the Moscow metro (Ethnic… 2006)
Reliable data selectionReliable data selection: exits of 15 Moscow Metro stations with the highest : exits of 15 Moscow Metro stations with the highest ridership and stable police presenceridership and stable police presence
Training of the observersTraining of the observers: individuals classified into three ethnic categories: : individuals classified into three ethnic categories: „Slavs”, „Minorities” (from the Caucasus and Central Asia), „Other” „Slavs”, „Minorities” (from the Caucasus and Central Asia), „Other” (African, East European, etc.)(African, East European, etc.)
Benchmarking the „general population”Benchmarking the „general population”: ethnic composition of the : ethnic composition of the population at these locations ( min. 1000 observations)population at these locations ( min. 1000 observations)
Tentative overview by the three evaluation criteria of the idealtypes of measurement techniques
Validity Reliability Generalisability
Official statistics - + ?
Survey - + ?
Qualitative + - -
Experiment
- laboratory
+ + -
Experiment
- natural
+ - -
Experiment
- controlled
? ? ?
Non-participant observation
+ + ?
To sum up
We know everything about
nothing
(R- and G-)
qualitative methods,
laboratory and quasi laboratory
experiment,
natural experiment
We know nothing about
everything (V-)
discrimination statistics,
survey
We might know something about
something (no -)
discrimination testing,
non-participant
observation
Controlled experiment and non-participant observation as best options
Validity
Reliability Generalisability
Controlled experiment and non-participant observation