Culture and Survey Response Lecture Week 12: Conceptual Issues By: Shirley Cheng & Adam Branson.

Culture and Survey Response

Lecture Week 12: Conceptual IssuesBy: Shirley Cheng & Adam Branson

Week 12 Discussion Outline

Introduction Cross-Culture background

Paper Review General Discussion

Aggregate the readings

Warm up discussion

What is Culture?

What is the focus of cross-cultural research? What is the general research question?

Cross-cultural comparison of survey response behaviors

Why do we suspect cultural difference?

Cultural Dimensions Cultural Orientation Cultural Subgroups

Impacts various aspects of survey response behaviors

The readings

Survey compliance and nonresponse Cialdini et al. (1999; also Johnson et al. 2002)

Response style Johnson et al. (2005)

Context effect; Response scale effect Haberstroh et al. (2002); Schwarz (2003)

Social desirability responding Lalwani et al. (2006; also Johnson and van de

Vijver 2002)

Cialdini et al. (1999)

Compliance With a Request in Two Cultures What cultural dimensions are studied? What are their findings?

Four Hypothesis: Four Results What are six social influence principles?

What are SP and C/C? What are the effects Social Proof (SP) and

Commitment/Consistency (C/C) on I/C How would reciprocity, authority, scarcity, and

liking affect compliance?

Does it matter whether culture is self-identified, proxy reported, or determined by a Cultural Orientation Scale (COS) or measure? (Handout)

Is it measuring individual respondents, groups of respondents, both? How would we use other survey modes to conduct a COS? Is it a reliable measure of cultural orientation? How would cultural features of the interviewer influence compliance?

Questions and Critiques Is it a problem if the researchers only conducted the follow-up on U.S.

students? What if it is not Coca-Cola? Less well-known? A firm w/ neg. or neut.

connotations? Does it matter that Poland is/was a country undergoing transition?

Would there be other results if we revisit the respondents later? Is there something different or shared about college students in Poland

and the U.S.? Was methodology sound?

Cialdini et al. (1999)

Cialdini et al. (1999)

The principles of SP and C/C appear important determinants of compliance decisions in each society. SP has greater effect on collectivists and C/C has greater effect on individualists.

In a collectivistic society, how one behaved in the past may not be an accurate reflection of one’s own preferences.

Priming procedures not only activate focal constructs in consciousness, but also deactivate competing or incompatible constructs.

Johnson et al. (2002)

Culture and Survey Nonresponse What cultural dimensions are studied? What are their findings?

What are the differences in culture based on non-controllable factors, like race or ethnicity, and controllable factors such as nationality? Will surveys capture biracial, multinational, trilingual respondents?

If it is important for non-response that there has been a greater attrition on panel data collection from certain races and ethnicities, what does this imply for cross-cultural response?

What are the cultural orientations we can unpackage?

Johnson et al. (2002) cont’d. What are implications of:

I/C Power Distance Vertical or Horizontal

Relationships Cultural Differences in

Context Requirements Nonverbal Behavior Self-Disclosure Minority Oppression and

Opposition Social Distance Helping Behavior

on survey response?

Do these items interaction with each other amplify or weaken the cross-cultural variability presented in survey response like they do in nonresponse?

Respondent Culture

CommunicationStyles

Respondent Access/

Cooperation

Socioeconomic Opportunities

Social Participation

Patterns

Future Research Ideas from Cialdini et al. (2002) and Johnson et al. (2002)

Cialdini (2002): Is willingness to comply enough or should we measure actual

survey taking (does anonymity matter). Examine impact of existing I/C orientation on the effectiveness

of C/C and SP principles (explore foot in door and long or short lists).

Examine how interpersonal relations, such as hierarchical authority or social distance impact compliance with a request (what happens if we know the interviewer).

Johnson et al. (2002): We must investigate of limits of residence, race, and ethnic

status measures as proxies of culture. We must explore cultural orientations, such as degree

respondent consider themselves an I/C. We must see if it is possible to map these cultural orientations

onto other constructs.

Johnson et al. (2005)

-Cross-cultural difference in response style

Two forms of response style: Extreme responding Acquiescent responding

Hofstede’s four cultural dimensions

Johnson et al. (2005)

Power distance What are the findings:

Acquiescent responding Extreme responding

What is the rationale? Are you surprised with the findings?

Johnson et al. (2005)

Uncertainty avoidance What are the findings:

Acquiescent responding Extreme responding

What is the rationale?

Johnson et al. (2005)

Individualism What are the findings:

Acquiescent responding Extreme responding

What is the rationale?

Johnson et al. (2005)

Masculinity What are the findings:

Acquiescent responding Extreme responding

What is the rationale?

Likelihood of Cultures to Exhibit Response Styles

Extreme Response

Acquiescence

Power distance + -

Uncertainty avoidance

ns -

Individualism ns -

Masculinity - -

Discussion

We need more specific conceptualization of the effect of each cultural dimension.

How is extreme response operationalized? How do we verify responses?

How are cultural dimensions operationalized? (see Table 1, pg 269) Aggregate (national) level vs. Individual level

Let’s take a short break

Schwarz (2003)

-Cross cultural difference in scale frequency effect

Recap: What is the scale frequency effect? What is the basis of this effect?

It occurs when memory of the behavior is ambiguous

Schwarz (2003)

Memory of other’s behavior What is the cultural difference?

Findings: “Chinese students’ reports of public

behaviors were virtually unaffected by scale range.” (p.591)

Schwarz (2003)

The “illusionary” cultural difference Behavioral frequency (%) of observable behaviors:

Is the use of proxy report better in a collectivist culture?

3229

22

29

38

30

0

5

10

15

20

25

30

35

40

US Chinese

OpenLow FreqHigh Freq

Schwarz (2003)

-Cross cultural difference in question order effect

Recap: Question order effect? What is the basis of this effect? Conversational norms

Schwarz (2003)

Correlation of marital satisfaction(M) and life satisfaction (L):

What is the explanation of this effect?

0.67

0.32

0.18

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

M/L L/M Lead-in

Haberstroh et al. (2002)

Correlations of academic satisfaction (A) and general life satisfaction (L):

0.78

0.36

0.53 0.5

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

A/L L/A

German Chinese

Haberstroh et al. (2002)

“Illusionary” cultural difference in values of achievement: Germans value academic achievement

more than Chinese.

Significance of the priming study Differential sensitivity towards

“context”

Interdependence Priming

Circle the Pronouns We go to the city often. Our anticipation fills

us as we see the skyscrapers come into view. We allow ourselves to explore every corner, never letting an attraction escape us. Our voices fill the air and street. We see all the sights, we window shop, and everywhere we go we see our reflections looking back at us in the glass of a hundred windows. At nightfall we linger, our time in the city almost over. When finally we must leave, we do so knowing that we will soon return. The city belongs to us.

Haberstroh et al. (2002)

Discussion Is it better to prime respondents

with one either independent or interdependent construal before any question?

How to avoid “illusionary” cultural difference in survey results?

Lalwani et al. (2006)

Relation Between Cultural Orientation and Socially Desirable Responding (SDR) What cultural dimensions are studied?

What is SDR? (Handouts) Does it introduce bias? How? Systematic tendency to give answers making the respondent look good. Would it manifest if we had proxy respondents? Does anonymity matter

if it is SDE or SDD? How does SDR introduce systematic or asystematic distortion?

What are the findings? Four Studies How do I/C vary w.r.t. goals, values, etc.? How does it tie to

different types of SDR? Do HC actively or passively seek social acceptance?

Lalwani et al. (2006) cont.

Are impression management (IM) and self deceptive enhancement conscious or unconscious forms of response editing?

How do I/C cultural categories vary horizontally and vertically? How can we measure and what is the reliability? 16-Item Triandis & Gelfand’s (1995) Cultural Orientation

Scale You and your friends decided to spontaneously go out to dinner.

What do you think is the best way to handle the bill? (answer)1) Split it equally without regard to what is ordered2) Split it according to how much each person makes 3) The group leader pays the bill or decides how to split it 4) Compute each person’s charge according to what the

person ordered

Lalwani et al. (2006) cont.

Lalwani: Response tendencies of people with individualistic vs. collectivistic

cultural orientations respond to two distinct SDR. C&HC IM (but VC does not) I&HI SDE (but VI does not)

Is that VC and VI do not have any social desirability responding?

According to the article, and your own intuition, what is the fundamental factor that underlies the different “ways” of SDR?

What are some HC, VC, HI, VI traits? HC = particularly oriented toward sociability and maintaining good

relationships => IM HI = self-reliance, independence, capability, and self directness => SDE

What could possibly be “socially desirable” in VC and VI cultures?

How does SDR scale measurement effect cultures? Are there cultural differences in SDR if we examine

items on a “yes/no,” “true/false” or Likert-Type scale?

Does it bring about SDR if we seek affirmation or negation? Can SDR depend on if we use statements or questions?

Key Positive NegativeTrue I am a saint I am not a sinnerFalse I am not a saint I am a sinner

Yes Are you a saint Are you not a sinner?No Are you not a saint? Are you a sinner?

Johnson and van de Vijver (2002) Social Desirability in Cross-Cultural Research

SD is a universal concept, but there is the possible presence of culture-specific factors. Do you agree?

Do you agree that correction for SD, on occasion, will decrease validity of cross-cultural comparisons.

If SD results from question and administration characteristics, how are different individuals or cultures triggered or primed?

W.R.T. admin., is SD a feature of survey questions, mode, social distance or difference with the interviewer.

W.R.T. personal, is SD a response style (keying), a personality characteristic?

Can we deactivate SD?

Johnson and van de Vijver (2002)

What is the purpose of Lie Scale, Eysenk Personality Questionnaire and Marlow-Crowne Scale measures? Are some tools better assessments of one culture than

another culture? Are the tools consistent with one another inside a

culture?

Treating SD as a person characteristic: Will q’s that do not activate cultural perceptions in one

culture be processed without any SDR? What does this mean if it activates cultural perceptions in only one culture of a cross cultural comparison?

Is individualism in culture A the same as individualism in culture B? Collectivism?

Future Research ideas from Lalwani et al. and Johnson and Van De Vijver

Lalwani et al.: If SDR is a major cause of research bias, what does it mean for surveying

field? We need to look at ways to address the effects of SDR. Will C orientation

only manifest itself with C priming or surroundings? What linkages exist between other aspects of culture and SDR? What

possibilities can be read into the study of Japanese that choose to study in Canada? Is it groups of people (a culture) or individual persons (parts of a culture) that matter?

Johnson and van de Vijver: There is a need for theories explaining cultural differences and SD

tendencies along with methods that measure and control during cross-cultural research. Why?

There is no simple safeguard against SD, but it can often be measured or bias techniques can be used to evaluate if score differences are due to questions or culture.

SD shows systematic cross-cultural difference that are negatively related to affluence of countries and to social power of individuals. Also, individuals from more affluent countries show lower SD scores. Furthermore, there is a strong relationship between SD and GNP (incl: educational level and personal income).

Is it that cultural value systems (e.g. C/I or the need for affiliation, conformity, approval and other psych. Constructs) are related to SD.

What more should we learn in this topic?

Cognitive Stage I/C Racial subgroup

Comprehension Stage

Judgment Stage

Formatting Stage

Editing Stage

Haberstroh et al. (2002)

Haberstroh et al. (2002)

Schwarz (2003)

Johnson et al. (2005);

Johnson and van de Vijver (2002); Lalwani et al. (2006)

General Discussion Any cross-cultural difference in mode effect?

For examples, face-to-face interviews (vs. other modes) induces more compliance among collectivists, but such difference is smaller among individualist?

What other cultural dimensions might effect survey response?

General Discussion

Three ways to investigate cultural differences: Aggregate level (Johnson et al. 2002) Individual level (Cialdini et al. 1999) Within individual (intercultural priming;

Haberstroh 2002)

Pros and Cons of each?

Is culture quantifiable?

Thanks!