The impact of culture on 360 feedback

Taking a closer look at the impact of culture on Multi Rater Feedback

PhD Student

Jouko van Aggelen, MsC

Managing Consultant Cubiks

[email protected]

Promotor

Prof. Paul G.W. Jansen, PhD,

VU University Amsterdam

[email protected]

Copromotor

Josje Dikkers, PhD

VU University Amsterdam

[email protected]

Copromotor

Rob Feltham, PhD,

Director IPT Cubiks

[email protected]

Content

� Introduction, theoretical background

� Hypothesis

� Research method

� Results & Discussion

� Next steps, questions & suggestions

Introduction

• For many organizations it has become common practice to use some kind of Multi Rater Feedback.

• “As these instruments gain popularity around the world the resulting need to conduct multisource research across and within different countries intensifies.” (Atwater, Brett, & Charles, 2007; Atwater, Waldman, Ostroff, Robie, & Johnson, 2005; Brutus, Leslie, & McDonald-Mann, 2001).

• Although research has shed some light on the phenomenon, its use in a cross-cultural environment is less explored. This is even more relevant when one realizes the impact they can have on individuals

Introduction

• This current research combines my daily life as a consultant (working for multinational organisations) with a more scientifically grounded approach.

• I would like to share some first results and collect your feedbackand suggestions.

MRF

• Three-hundred-and-sixty degrees feedback was first used (and detonated as a trademark!) by the American Consultancy Firm ‘Teams inc.’ in 1985 (e.g. Jansen and Vloerbergs, 1998).

• It was Jack Welch who started the glory days of this method in 1994.

Definition: “A process whereby raters from multiple perspectives rate a subjects performance“ (Zimmerman, Mount & Goff III, 2008, p. 123)

Cross-cultural usage of MRF

• People from different groups, or cultures have a different way of acting and performing (Kruger & Roodt (2003).

• Contextual differences have a greater influence on employees’ receptiveness than personality. One of the most impactful contextual differences can be national culture (Funderburg & Levy,1997)

• MRF ratings are based upon interpersonal interactions and shared feedback (Atwater et all 2009).

• These processes, interaction and the sharing of feedback, are highly influenced by culture (Ashford, 1989, Varela & Premeaux, 2008)

Some examples of previous research findings

• Gillespie (2005): Employees from different countries and cultures, working for the same multinational, interpreted and responded differently to the same MRF questionnaire.

• Shipper, Hoffman and Totondo (2007): MRF has the most effect (gaining actionable knowledge out of the process) in individualistic and low power distance cultures.

• Varela and Premeaux (2008): The discrepancy between peer- and self-ratings was the least and direct reports gave the highest ratings to their bosses in high collectivistic and high power distance cultures.

• Atwater, Wang, Smither and Fleenor (2009): High assertiveness and Power Distance seems to stimulate the relationship (= low discrepancy) between self and reports and between self and peer ratings.

• Gentry, Yip, & Hannum, (2010): The discrepancy seems to be wider in high power and individualistic cultures, mainly due to the subjects’ self-ratings, not the ratings of others.

Hofstede’s culture typology

• One of the most popular and most used classifications of cultural differences (Atwater, Waldman, Ostroff, Robie & Johnson, 2005, Shipper, F. Hoffman, R.C. & Totondo 2007)

• An accepted way to study cultural differences.

• Hofstede’s dimensions (1980, 2002, 2010)

– Power Distance: The distribution of power by nature.

– Identity: Individual freedom vs. focus on the collective, group harmony.

– Gender: A caring ‘Feminine’ attitude vs. an assertive Masculine one.

– Truth. Tolerance for ambiguity and uncertainty vs. uncertainty avoidant

– Long term vs. short term orientation.

– Indulgence vs. Restrain

Focus of this study

• Moderating role of culture on with respect to three of Hofstede’scultural Dimensions: Power Distance, Identity & Gender.

The discrepancy between the subjects’ self-ratings

and the ratings of bosses & the possible interaction

effects,

Some examples

PDI COLL-IDV FEM-MAS

Russia 93 39 36

New Zealand 2 79 58

US 40 91 62

Venezuela 81 12 73

Japan 54 46 95

Sweden 31 71 5

Netherlands 38 80 14

HypothesesPower Distance

• Johnson, Kulesa, Cho & Shavitt (2005): PD is positive related with extreme

response styles and negative with acquiescent response behavior (“yah-saying”)

• Balzer et al., 2004, Hofstede 1983, Varela 2008: Reports ratings will be inflated

since they want to maintain the relationship and that believe that their manager

will know best.

• Carl, Gupta, & Javidan: “Under the higher power distance style of supervision,

there is virtually no rapport between the leader and subordinate” ( 2004, p. 535).

1. The discrepancy between self and boss becomes wider when the subject

comes from a (more) high PD culture.

2. The discrepancy between self and boss becomes wider when the boss

comes from a (more) high PD culture

3. The discrepancy is the least wide when both are from a low PD culture and

the discrepancy is the widest when both are from a high PD culture

HypothesesIdentity

• In collectivistic cultures giving feedback is not so easy. People will be less ‘tough’

on each other. In individualistic cultures people are focused on saving own face

vs. saving face of others. This could result in a less critical self-assessment.

• Reports gave the highest ratings to bosses in high collectivistic cultures (Varela

and Premeaux, 2008)

• The discrepancy seems to be wider in individualistic cultures (Gentry, Yip, &

Hannum, 2010)

4. The discrepancy becomes more wide when the subject is from a more

individualistic culture

5. The discrepancy becomes less wide when the boss is from a more

collectivistic culture

6. The discrepancy is widest when both come from an individualistic culture

and smallest when subjects come from a collectivistic and bosses from an

individualistic culture

HypothesesGender

• Goffin and Anderson (2006): High achievement orientation and a high self-esteem

(masculine characteristics) are related to overestimation.

• Another characteristic that is associated with Masculine cultures is assertiveness,

which seems to be related to more comfortable with giving critical feedback.

7. The discrepancy is wider when subjects are from masculine cultures.

8. The discrepancy is lower when bosses are from masculine cultures

9. The discrepancy is widest when both come from an masculine culture and

smallest when subjects come from a femnine and the boss from an

masculine culture

The data

• Three data sets– Company specific MRF’s and competences

– Conducted with the same objective for more or less the same target groups

– All with a developmental focus

– All available in multiple languages

• Three different global operating companies1. Western European Food and beverage

2. Northern European Mechanical Engineering

3. Asian Steel Company

• Three different regional spreads1. Eastern and western Europe + Africa

2. US, Western and Northern Europe + some Asian countries

3. Europe, North and South America and Asia

Cultural spread

Sample 1 Sample 2 Sample 3

Subjects 295 1170 320

Power Distance

min 11 11 35

max 104 104 104

mean 70.50 49.03 61.72

SD 27.68 16.66 13.93

Identity (individualism vs. Collectivism)

min 11 14 12

max 90 91 91

mean 46.54 72.95 62.52

SD 19.05 15.84 17.80

Gender (Masculinity vs. Femininity)

min 14 5 14

max 110 79 88

mean 54.94 49.25 51.64

SD 27.67 19.19 10.45

Sample 1 Sample 2 Sample 3

Reviewers 2719 10228 2931

Power Distance

min 11 11 11

max 104 104 104

mean 60.97 49.57 60.95

SD 27.48 19.51 14.44

Identity (individualism vs. Collectivism)

min 8 13 12

max 91 91 91

mean 47.89 71.70 63.00

SD 17.10 16.37 17,67

Gender (Masculinity vs. Femininity)

min 14 5 5

max 110 110 110

mean 53.33 47.75 51.21

SD 27.12 19.02 10.36

Measures & analytical procedure

• Correlations company specific competences all high, we chose for 1 single overall rating per reviewer

• Calculated discrepancies between subject’s and boss’ rating

• The cultural interaction is calculated by multiplying the ‘Cultural Z-scores’ of subject and boss

• Multiple regression

– Independent variables: The cultural dimensions of the subject, the boss and

their interaction

– Dependent variables: The subjects’ self rating, the reviewer/boss rating and the

discrepancy.

First: Overall analyses Self, boss & discrepancy ratings

Data set 1 Data set 2 Data set 3

Mean self 3.26 3.61 3.12

SD Self .44 .46 .41

Mean Boss 3.25 3.46 3.09

SD Boss .52 .57 .45

Mean discrepancy .024 .141 .055

SD discrepancy .656 .666 .594

T –test discrepancy T = .664

p = .507

T = 7.410

p = .000

T = 2.115

p < .035

• Rating scale 1-5

• The mean discrepancy in data set 2 & 3 varies significantly

ResultsPower Distance as moderator on the self-boss discrepancy

Data set 1 Data set 2 Data set 3

1 Subject (wider if subject is high PD) β = .021

p = .774

β = .123

p. = .090

β = .029

p. = .604

2 Boss (wider if boss is high PD) β = -.074

p =.308

β = -.002

p = .979

β = -.008

p = .979

3 Interaction (least if both low PD + vice versa) β = -.394

p = .000

β = .105

p < .002

β = -.066

p = .186

• (1) and (2) are not supported

• (3) is supported in two of the three samples

Only the interaction between the power distance of self and boss seems to be

significant.

Interaction effectsPower distance

Data set 2

• Most impact: PD Boss

• Discrepancy is most negative if HPD Boss

• Discrepancy is most positive if LPD Boss

• Lowest discrepancy: LPD Boss, LPD self

• Widest discrepancy: HPD Boss

• Discrepancy is most negative: HPD Boss, LPD Self

• Discrepancy is most positive: HPD Boss, HPD Self

ResultsIdentity as moderator on the self-boss discrepancy

Data set 1 Data set 2 Data set 3

4 Subject (more wide if subject is

individualistic)

β = -.172

p <.005

β = -.164

p < .004

β = -.080

p = .207

5 Boss (less wide if boss is collectivistic) β = .024

p = .701

β = .074

p = .193

β = .082

p = .200

6 Interaction (widest if both from

individualistic, least of subject is

collectivistic, boss ind.)

β = .160

p < .032

β = .209

p = .000

β = -.186

p < .001

• (4) is supported in two of the three samples

• (5) is not supported

• (6) is supported in all three samples.

Individuality / collectivism of the subject seems to have an effect

The interaction between the subject and boss also seems to have an effect.

Interaction effectsIdentity

Data set 2

Data set 3

• Most negative discrepancy: IND Boss, IND subject

• Most positive discrepancy : COL subject, Col Boss

• Widest and most positive discrepancy: COL subject, COL boss

• Lowest discrepancy: IND subject

• Widest & most positive discrepancy: COL subject, IND boss

ResultsGender as moderator on the self-boss discrepancy

Data set 1 Data set 2 Data set 3

7 Subject (wider if subject is

Masculine)

β = -.074

p = .276

β = .012

p = .889

β = .049

p = .378

8 Boss (wider of boss is Masculine) β = -.143

p < .034

β = .049

p = .566

β = -.083

p = .132

9 Interaction (widest when both from

masculine, smallest when subjects

is feminine, boss is masculine)

β = -.021

p = .751

β = .126

p = .000

β = -.031

p = .512

• (7) is not supported in all three samples

• (8) is not supported in two of the three samples

• (9) is not supported in two of the three samples

Masculinity / Femininity does not seem to have much impact, most

hypotheses with respect to this dimension are not supported.

Interaction effectsGender

Data set 2• Widest discrepancy: MAS boss

• Lowest discrepancy: FEM boss

and subject

Conclusions

• Power Distance• The cultural background of the subject and his/her boss itself does not seem to have

moderate the wideness of the discrepancy

• However the interaction between the two does seem to moderate the wideness of the discrepancy (2 out of 3). Based upon the current analyses the interaction effect itself is difficult to interpret, it’s really blurry

• Identity (collectivism vs. individuality) • The cultural background of the subject does seem to moderate the wideness of the

discrepancy (2 out of 3 significant)

• The cultural background of the boss does not seem to moderate the wideness of the discrepancy

• The interaction between the two (subject, boss) does seem to moderate the wideness of

the discrepancy, however again the effect is difficult to interpret.

ConclusionsContinued

• Gender (feminine vs. masculine • The cultural background of the subject does not seem to moderate the wideness of the

discrepancy .

• The cultural background of the boss does not seem to moderate the wideness of the discrepancy (1 out of 3).

• The interaction between the two does seem to moderate the wideness of the discrepancy (1 out of 3)

Implications

• Power Distance and Identity seem to moderate the discrepancy between the self and the boss rating in MRF, especially the interaction between the cultural backgrounds of the two.

• This is in line with past research, where these two are seen as having the most impact (also these two are by far the most studied).

• How this interaction works needs to be clarified, further studied, however it does confirm our concerns the usage and interpretation of MRF results worldwide in more or less the same way.

Next steps

1. Expand the data set + with the different rater groups (peers, reports).

2. Expand with other cultural dimensions

3. Analyse the moderating role of the organisational culture (vs national culture).

4. Analyse the moderating role of culture with respect to the used competences (what is actually assessed).

Questions and suggestions……

Thank you!