Building Blocks of Research

BUILDING BLOCKS OF RESEARCHTuesday, September 6, 2011

TODAY:1. Observation assignment: details, how

to calculate2. Design: variables, u of a,

measurement issues3. Discussion on forseeable problems

Observation Assignment: The How-Tos!

OBSERVATION IN RESEARCH Directly observing as it happens Different from a survey: Different from experiment: Different from taking pre-existing data &

analyzing

Primarily qualitative approach

QUANT VS. QUAL QUALITATIVE- description/explanation,

smaller scale, non-prob samples, limited attempts at generalizability (depth vs. breadth). WHY?

QUANTITATIVE-Establishing the existence of a relationship (correlation) b/w variables through large-scale, prob samples, generalizing(breadth vs. depth). WHAT?

TYPES OF OBSERVATION 2 broad approaches:

Structured:

Unstruct’d:

Your projects=STRUCTURED

STRUCT VS. UNSTRUCT OBS In general:

structured=good for testing hypotheses unstructured=good for generating

hypotheses Most of the time when obs is really

used in soc research, it is unstructured field research.

WHEN IS OBSERVATION USEFUL? Useful for same situations in which

you’re not trying to get a randomized sample

What types of studies would this be useful in?

WHEN IS OBS USEFUL, 2 When you want to maintain the complexity of

a situation; not reduce it to just a set of quantifiable variables.

When you want to get an insider’s view of events; different interpretations/meanings.

Example Humphreys

STEPS IN OBSERVATION You will do struct’d field observation Steps: In any obs study, need to decide

on Site (where and why?) Sample (who or what) Access and self-presentation Recording obs/data collection Data analysis

I’ll go through each in turn…

STEPS IN DEPTH Site: How do researchers choose where

to base their study? Convenience Relevance to study

STEPS IN DEPTH, CONT. Sampling: Often less structured for qual

(obs) studies. Usually non-prob. For this project, be very specific

(structured) Access/Self-presentation

Decisions about obtrusive vs. unobtrusive, Ethics and practicalities play in.

STEPS, CONT. Recording obs: Also called “data collection.” Data analysis: After done observing, go home

and figure out what you’ve got.

SPECIFICS OF PROJECT You’ll do all these things in a very

structured way for this 1st assignment. Overall idea: come up with a concept, then

a hypothesized relationship between this concept and a variable, operationalize and observe, analyze data.

THINGS TO OBSERVE What are possibilities for things you could

actually observe? Ex: People in public places, TV shows or ads, News

articles or broadcasts, billboards. What to think about b/f you choose:

CONCEPTS: What broad concept are you interested in studying?

Ex: authority, obedience, courtesy, chivalry, heterosexism/homophobia, generosity,etc.

How to get at this concept? Develop IV and DV

NOTES ON WORKING IN GROUPS Working in grps=key part of this assigment. Reliability Researchers have different ways of handling

this..

RELIABILITY, CONT. Researchers also try to check reliability as

they go by: Triangulating Insider’s view “Inter-coder reliability”

INTER-CODER RELIABILITY Make sure 2+ trained people who are given all the

info on operationalizing, do in fact see the same thing.

Examples?

INTER-CODER RELIABILITY, CONT. What do you think you look for: all

coders having same info, or diff?

One key is that coders work separately. Why?

RELIABILITY IN YOUR PROJECTS

For your projects, this is important, too. Develop a detailed observation plan together Go out and observe separately Come back and see how reliable your measure

turned out to be.

OBSERVING IN GROUPS If observing people:

Observe same place/time, but don’t talk or share notes.

Decide how to choose If observing TV

Make sure to observe same thing at same time If observing documents

Can hand them back & forth

CALCULATION!

ERIK’S NOTES ON HOW TO DO TABLESCalculating Reliability -1N = total number of distinct people seen (by both or either)

S1 = sample difference 1: number of times partner 1 observed a subject partner 2 did not see.S2 = sample difference 2: number of times partner 2 observed a subject partner 1 did not see.

A = number you agree on: same subject, same variable code.C = coding difference: number of times you observed the same subject but coded the variable differently.

Calculating Reliability - 2

Coding Error: CE = C/(A+C) (proportion you both saw that you disagree about in the variable).

Sample selection error: SE = (S1 + S2)/N (proportion of total cases that one person saw but not the other).

Evaluating Reliability

Perfect reliability is the goal, zero errors But for this assignment, need to do a variable

that is complex enough that this is not ŅeasyÓ Even 10% error is fairly high for reliability Try to understand the source of all errors and

how they could be avoided If error is low, discuss what you did well in the

procedures to produce low error

Conditional Percentages

1) Dependent variable is qualitative2) Cross-tabulate the data3) Calculate percentages for the dependent variable separately within each category of the independent variable.4) Compare the percentages across categories of the independent variable

Cross-tabulate the Data

Male Female Total

Bite 11111111 1111

Lick 11111 1111111111

Other 11 111

Total

Independent

Dep

ende

nt

Calculate Cell, Row, Column Totals

Male Female Total

Bite 11111111 1111 12

Lick 11111 1111111111 15

Other 11 111 5

Total 15 17 32

8 4

5

2 3

10

Check the row and column totals against the data beforeproceeding

Calculate Conditional Percentages

Male Female Total

Bite 8/15=.533 4/17=.235 12

Lick 5/15=.333 10/17=.588 15

Other 2/15=.133 3/17=.176 5

Total 15 17 32

Check that each column adds to 1.0 within rounding error..533+.333+.133=.999 .235+.588+.176=.999

IndependentD

epen

dent

Final Table

(17)(15)(N)

101%*99%*Total

18%13% Other

59%33% Lick

24%53% Bite

Female

Male Interpretation:Males bit 53% of the time compared to 24% of the women (a percentage difference of 29%); females licked 59% of the time compared to 33% for males (a percentage difference of 26%). ŅOtherÓwas only slightly different for men and women.

It is OK to use the original proportions or to turn them into percents.

Conditional Means

1) Dependent variable is quantitative2) List the values for the dependent variable separately for each category of the independent variable3) Calculate the mean for the dependent variable separately for each category of the independent variable.4) Compare the means across categories of the independent variable

List Dependent Variable Scores by Independent Variable

Nm = 20 (number of males)3xm = 542 (sum of scores)

Nf = 27 (number of females)3xf = 1093 (sum of scores)

Dependent variable is number of seconds it took to complete sales transaction.

1322263410193933302242

Males172344102955312716

56798223571433

Females47253624316939333022

42172374492955314726

Sex and Time to Complete Sales Transactions

(27)(20)(N)

40.527.1Mean Seconds for Transaction

WomenMen

Interpretation: Women took 13.4 seconds longer than men, on average, to complete their transactions.

Difference of Conditional Means

SET UP YOUR OWN STUFF! Depending upon whether your ind var

is quantitative or qualitative

Design: Variables, Units of Analysis

GO BACK… Process We need to develop the language tools

UNITS OF ANALYSIS Who or what you are actually

observing, recording, etc.

TYPES OF UNITS OF ANALYSIS Individual people Events Social groupings Social roles/relationships Social artifacts

WHY ARE UNITS OF ANALYSIS IMPORTANT?

Helps you make sense! Ex:

VARIABLES A subset of units of analysis

VARIABLES IIYOU define them.

VARIABLES AND ATTRIBUTES Attributes:

Exhaustive mutually exclusive.

EXAMPLE: US CENSUS AND RACE… EXHAUSTIVE? MUTUALLY EXCUSIVE?

1820: “White,” “Colored”

1890: “White,” “Black,” “Mulatto,” “Quadroon,” “Octoroon,” “Chinese,” “Japanese,” or “Indian”

2000: "White,” “Black or African American,” “American Indian and Alaska Native,” “Asian,” “Native Hawaiian and Other Pacific Islander,” “Some other race,” “Two or more races”

2010: “For this census, Hispanic origins are not races.”

SO. In the previous example of the census,

what was the: Unit of Analysis? Variable? Attributes?

Unit of Analysis (who or what)

Variable (what about them)

Attributes (Categories within variable)

Individual Income $ amount ORCats: <$10,000$10,000-24,999

IndividualEye color

Brown, green, blue, etc.

Household Size (# people) 1, 2, 3, 4, …

Organization Gender composition (% female)

Census tract Average income Mean to the nearest dollar

VAR AND ATTRIBUTES, II If there are no attributes, then you don’t

have a variable.

EXAMPLE Study: Interview religious victims of

domestic violence to see coping strategies.

Why are “religion” and “victim” NOT variables?

TYPES OF VARIABLES Nominal/Categorical Ordinal

TYPES OF VARIABLES-2 Interval Ratio

QUALITATIVE AND QUANTITATIVE VARIABLES

Quant: Qual:

MORE ON QUALITATIVE VARIABLES Trick with Qual variables:

STATISTICS WITH VARS Qual: Can do some frequencies, %,

mode. Quant: Can do all the descriptive stats

mentioned above, plus other statistics (re: correlations. More later.)

MEAN, MEDIAN, MODE Remember? Say it with me…

RELATIONSHIPS BETWEEN VARIABLES

The key to the whole thing. That’s what we want.

Explanatory: included in study Extraneous: Outside of study

Intervening An effect of IV that helps cause DV

Antecedent Affects both the DV and IV

CAN WE THINK? Can you think of an intervening

variable? What about an antecedent variable?

Variables II: Operationalization and Measurement Issues

INTRO Operationalization (how you measure)

variables (dimensions and indicators) Measurement issues (how well you

measure) variables (reliability and validity)

OPERATIONALIZING, I Conceptualizing first Operationalizing second

DEFINITION OF OPERATIONALIZATION The rules used to assign each

observation into some category of a variable.

OPERATIONALIZATION, II Remember? Variables have to have…

Exhaustive Mutually exclusive

…attributes.

OP’N III Creating those rules means that the researcher is

classifying, ordering, or quantifying information. Classifying Ordering Quantifying

SOME THINGS THAT ARE DIFFICULT…

INDICATORS Def: “An observation that we choose to

consider as a reflection of the variable we wish to study.”

LEGAL AND MEDICAL ANALOGIES Medical: Legal:

DIMENSIONS For some variables, you need only 1

indicator, or a couple of indicators.

For harder-to-define concepts, in order to operationalize them you need to break them down into different aspects of the variable before you can decide on indicators.

DIMENSIONS, CONT. The idea with dimensions is:

make sure you are covering all your bases Make sure you’re only measuring what you

want to measure

EXAMPLE: CONFORMITY Ex: Someone could show up for school every

day and get good grades, but be all tatooed across the face and neck and reject peers, or vice versa. Conformist?

CONFORMITY, CONT. Institutional dimension: Do they play by the

official rules? Cultural dimension: Do they try to fit in with

those around them?

Inst. Indicators: Cult. Indicators:

VALIDITY! Which of these indicators do you think is best

and why?

RELIABILITY Your goal: the same data/info would be

collected on different days, by different people.

VALIDITY/RELIABILITY TRADEOFFS GRADES:

vs. WEARING CERTAIN CLOTHES:

VALID/RELIABLE CONT.

VARIABLE EXERCISE Two Heads are Better than One A House Divided Against Itself Cannot

Stand Actions Speak Louder Than Words Drastic Times Call For Drastic Measures Haste Makes Waste Variety Is the Spice Of Life

NOW… Let’s brainstorm some other broad concepts

that you might be interested in studying. First brainstorm all possibilities that might fit

under that concept Then decide if they should be grouped along

different dimensions Then come up with indicators