C2 Protocol CooperativeLearning

1 The Campbell Collaboration | www.campbellcollaboration.org

Protocol: Effects of Cooperative Learning on Inter Ethnic Relations in School Settings Philip Curry1, Leyla De Amicis2, Robbie Gilligan3

Contents Page 1. Background for the Review................................................................................. 2 2. Objectives of the Review..................................................................................... 4 3. Methods 3.1 Criteria for inclusion and exclusion of studies in review............................ 4 3.2 Description of methods used in component studies .................................. 5 3.3 Literature search strategy for identification of appropriate studies .......... 6 3.4 Criteria for determination of independent findings .................................. 9 3.5 Details of study coding categories............................................................ 10 3.6 Data extraction and incomplete reporting of study data.......................... 10 3.7 Statistical procedures and conventions .................................................... 11 3.8 Treatment of qualitative research ............................................................ 11 4. Review Team ..................................................................................................... 11 5. Timeframe.........................................................................................................12 6. Plans for updating the review............................................................................12 7. Statement concerning conflicts of interest ........................................................12 8. References ........................................................................................................13 Appendix A: Coding Sheets ...................................................................................16

1 School of Social Work and Social Policy, Trinity College Dublin, Dublin 1, Ireland [email protected] 2 Children’s Research Centre, Trinity College Dublin, Dublin 1, Ireland [email protected] 3 School of Social Work and Social Policy, Trinity College Dublin, Dublin 1, Ireland [email protected]


1. BACKGROUND

The single most influential text in the modern history of efforts to enhance inter-group and inter-ethnic relationships is a short section of Allport’s (1954) The Nature of Prejudice in which he proposed what is now called the Contact hypothesis. Allport predicted that contact between different groups would lead to reduction in prejudice provided certain conditions were met. The principal conditions were that group members should work together:

• Towards a common goal • With interdependent roles • With equal status • In the context of policies which endorse the contact

A considerable amount of research evidence has supported his position (Binder et al., 2009; Paolini, Hewstone, Harwood, & Cairns, 2006; Pettigrew, 1998; Wagner, Tropp, Finchinescu, & Tredoux, 2008). On one hand it has been found that passive forms of inter-group contact such as simple desegregation do not reduce prejudice and discrimination (Brown & Hewstone, 2005; Pedersen, Walker, & Wise, 2005; Schofield & Eurich-Fulcer, 2003). On the other hand contact under most or all of the conditions above has consistently been found to have positive effects on intergroup attitudes with a wide variety of groups, settings and age groups (see Pettigrew & Tropp, 2006 for meta-analysis). A dilemma for the policy maker however is how to routinely engineer situations which meet the conditions of contact specified above. Cooperative learning refers to small group teaching and learning strategies which actively seek to promote inter-dependence among group members such that “Each member of a team is responsible not only for learning what is taught but also for helping teammates learn” (Ryan, Reid, & Epstein, 2004). Pupils work with little teacher supervision (Deering, 1989, as cited by Cooper & Slavin, 2004) on carefully planned and monitored activities (Slavin & Cooper, 1999) in such a way that each participant can achieve his or her learning goal if and only if the other group members achieve theirs. Cooperative Learning may be distinguished from regular group work by both the level of planning by teachers and by the interdependence of the group members. Cooperative learning strategies have been shown to have positive effects on many learning outcomes such as achievement (Cohen et al., 1997; Cohen & Lotan, 1997; Slavin, 1995), self-esteem (De Vries, Slavin, Fennessey, Edwards, & Lombardo, 1980; Johnson, Johnson, Tiffany, & Zaidman, 1983; Slavin, 1980; Stephen, 1978), social competency (Cohen & Lotan, 1995; Ginsburg-Block, Rohrbeck, & Fantuzzo, 2006; Roseth, Johnson, & Johnson, 2008), the quality of the learning environment (Aronson & Osherow, 1980; Aronson & Patnoe, 1997; Blaney, Stephen, Rosenfield, Aronson, & Silkes,1977; Geffner, 1978, as cited by Aronson & Osherow, 1980) and other domains. Cooperative learning is often promoted on the basis of these benefits, it is not inherently designed to reduce prejudice or improve inter-group attitudes (Mckown, 2005; Singh, 1991). However when used in classrooms which involve different ethnic groups, cooperative learning techniques reproduce many of the conditions specified by the contact hypothesis above4 and have been advanced as tools to improve inter-ethnic relations in schools (McKown, 2005).

4 The relationships between Cooperative Learning theory and inter-group relations can also be understood in terms of Social Interdependence Theory (Deutsch, 1949; 1962; Johnson & Johnson, 1989).


The positive impact of Cooperative Learning strategies on inter-group relations among children has a long documented history (early examples include Cohen & Roper, 1972; Cohen, Lockheed, & Lohman, 1976; Ziegler, 1981; Johnson and Johnson, 1981; Slavin and Oickle, 1981; Sharan, Hertz-Lazarowitz, Bejarano, Raviv, Sharan, 1984). A meta-analysis by Johnson and Johnson (1989) found that cooperation between majority and minority group members was associated with greater inter-group friendships. A review by Ginsburg-Block et al (2006) of Peer Assistant Learning interventions (which include Cooperative learning) found a positive impact on ‘social outcomes’ under which they include positive inter-ethnic attitudes. A review of 19 field experiments using 8 different Cooperative Learning paradigms by Cooper and Slavin (2004) found generally positive effects on prejudice and discrimination among children in 4 to 12th grades where the minority presence was between 10% and 61%. Some conflicting findings have emerged however. The following have been noted as moderating factors by researchers or commentators:

• The type of Cooperative Learning program (Cooper & Slavin, 2004; Singh, 1991). • Context of program (Cooper & Slavin, 2004) • The group looked at (i.e. minority, majority or both) – (Aronson & Osherow, 1980;

Slavin & Oickle, 1981). • Measures used to assess inter-ethnic relations (Slavin, Leavey & Madden, 1984). • Age of participants involved (McKown, 2005).

Teachers require only a small amount of training in order to use Cooperative Learning strategies. Once they have familiarized themselves, teachers can adapt existing lesson plans as well as access those of others. Cooperative Learning strategies are thus appealing in policy terms as they are relatively cheap and easy to implement. They can be used with different ages and abilities and can be incorporated into existing school curricula (Singh, 1991; Slavin, 1985). Moreover, they can be presented to schools and educational policy makers as a means not only of improving interethnic relations but also as a way of improving a variety of more traditional ‘academic’ outcomes (McKown, 2005). A new systematic review of the effect of Cooperative Learning strategies on inter-ethnic relations in schools is needed because the reviews which currently exist are:

1. Not systematic (Cooper and Slavin, 2004; Ginsburg-Block et al. 2006; Roseth et al., 2008)

2. Narrative, non quantitative (McKown, 2005; Pederson et al., 2005; Paluck & Green, 2008)

3. Country specific and out of date (Cohen, Lotan, & Leechor, 1989; Johnson & Johnson, 1989)

4. Focused on one particular Cooperative Learning strategy (Bratt, under review, on Jigsaw).

5. Focused on more general outcomes such as social relationships (Roseth et al., 2008).

6. Include Cooperative Learning as one of a much wider undifferentiated range of teaching strategies or general contact situations (Ginsburg-Block et al., 2006; Pettigrew and Troop, 2006; Paluck & Green (2008).


2. OBJECTIVES

The aim of this review is to synthesize all existing empirical evidence on the effects of Cooperative Learning on inter-ethnic relations in school settings. The review will be:

• Systematic including published and unpublished research • Up-to-date • Inclusive of all school going age groups (4 to 18 years of age) • Inclusive of only school based, valid cooperative learning strategies

Where data is available the review may also use moderator analysis to examine the effect of the following variables:

• Type of cooperative learning paradigm • Characteristics of participants (age, gender, majority-minority status etc.) • Implementation by teacher / researcher / teacher with support • Type of outcome measure used (cross-ethnic friendship, attitudes etc.) • Duration of the program • Point(s) in time at which outcome measurements were taken • Teaching strategy used for control group • Inter-group and national context

The review will focus on outcome measures directly related to inter-ethnic relations but where information is provided on key secondary outcomes these data will also be collected. We will focus on two: academic achievement and self-esteem. The aim is to determine if cooperative learning has an impact on these variables when it is used in multi-cultural settings. Improved academic performance is a key selling point for cooperative learning strategies over and above their impact on inter-ethnic relations. We would expect improved self-esteem to result from enhanced inter-ethnic relations.

3. METHODOLOGY

3.1 Criteria for inclusion and exclusion of studies in the review The aim of this review is to include all studies of valid, school based implementations of Cooperative Learning paradigms which include outcome measures of inter-ethnic relations and which employ experimental and quasi-experimental designs. The inclusion criteria are:

1. Cooperative learning is a generic term that refers to many different instructional practices which share common principals. We will define cooperative learning as utilization of one of the 11 most common approaches (Cooper and Slavin, 2004; Johnson, Johnson and Stanne, 2000) and use a twelfth category to represent all other forms of cooperative learning. The 11 most common approaches are:

• Jigsaw method (Aronson et al., 1978) • Learning Together (Johnson and Johnson, 1983) • Constructive Controversy (Johnson and Johnson, 1985)


• Student Team Achievement Division (STDA, Slavin, 1978) • Complex instruction (Cohen, 1986) • Team-Games-Tournament (DeVries & Edwards, 1974) • Team-Assisted-Individuation (Slavin et al., 1984) • Team-Accelerated Instruction (Slavin, Leavey, & Maddey, 1986) • Group Investigation (Sharan and Sharan, 1976) • Cooperative Learning Structures (Kagan, 1985) • Cooperative Integrated Reading & Composition (CIRC, Stevens et al, 1987)

The implementation of the program will be assessed by comparison with the procedures of the original authors listed above.

In addition a program will be identified as ‘Other Cooperative Learning’ if it does not fall under any of the headings above but exhibits the following characteristics:

• Small group work (from three to maximum 8 members) • Tasks and activities carefully planned in advance • There is interdependence among team members such that “each member of a

team is responsible not only for learning what is taught but also for helping teammates learn” (Ryan et al, 2004, p.332). This means that each member is a necessary source of information / support for all the others’ learning.

2. Full data on at least one outcome measure assessing children’s behavior, attitudes,

emotions or beliefs towards at least one ethnic, religious or linguistic out-group is reported. Outcomes involving relations with other groups (e.g. disabled or overweight students) will not be included.

3. Randomized field experiments and quasi-experimental studies will be included. Quasi-

experimental studies must include a pre-test. All studies must include a control group which received an intervention other than Cooperative Learning or no intervention at all.

4. Intervention is school based (excluding after-school programs).

5. Research published after 1965. This is the year in which the earliest of the Cooperative

Learning programs currently in use, Learning Together, was first developed.

6. Study may be conducted anywhere in the world and reported in any language. On the basis of these criteria an example of a study that would be excluded is Oortwijn, Boekaerts, Vedder and Fortuin (2008) in which no control group is present. An example of a study that would be included is Slavin & Oickle (1981) which investigated the impact of Student Team Achievement Division in a desegregated school utilizing a control group which engaged in traditional group work. Test and control group equivalence was checked at pretest and controlled for. Both pre- and post- treatment measures were taken. 3.2 Description of methods used in component studies We expect the majority of eligible studies will involve randomized classes rather than individuals, pre- and post- test measurements and will use treatment as usual control groups.


We also expect that there will be quite a lot of heterogeneity in outcome measures used, some studies utilizing measures such as cross-group friendships with others measuring more generalized outcomes like prejudice. Measures of prejudice for use with children are controversial and have changed considerably through the years covered by the eligibility criteria (Aboud, 2005; Cameron, Alvarez, Ruble, & Fuligni, 2001; Nesdale, 2001). 3.3 Literature search strategy for identification of appropriate studies Information Retrieval We will use a range of strategies and sources to exhaustively and systematically locate studies which match the eligibility criteria. In order, we will: • Perform a keyword search on several online abstract and literature databases (see list of

keywords and databases below). • Search references provided by previous reviews and pertinent material. • Hand search journals which have published two or more primary studies. • Perform a keyword internet search. • Search conference proceedings and the publications of government and professional

specialized agencies. • We will contact leading scholars in this area - defined as authors involved in two or more

studies which meet the eligibility criteria. The following electronic databases will be searched:

1. C2-SPECTR 2. Cochrane Database of Systematic Reviews 3. PsychARTICLES 4. PsycINFO 5. ASSIA 6. Chicano database 7. WILSON OMNIFILE (including ERIC) 8. Chinese-ERIC 9. Web of knowledge 10. Science Direct 11. Dissertation Abstracts 12. ISI Citation Indexes 13. British Education Index 14. British Humanities Index 15. Australian Education Index

Journals which are identified that have two or more primary studies will be hand searched. At the moment we expect this will include as a minimum:

1. Contemporary Educational Psychology 2. Review of Educational Research 3. Educational Psychology Review 4. Journal of Educational Psychology 5. Journal of Social Issues 6. Journal of Community & Applied Social Psychology 7. Sociology of Education


Internet searches will be performed using Google (www.Google.com; http://scholar.google.com/) and Yahoo (www.yahoo.com). Websites which specialize in the areas of education and inter-ethnic issues will also be explored (www.complexinstruction.org; www.raceproject.com ). Conference proceedings and grey literature will be searched electronically:

• National and regional Departments of Education for OECD countries plus South Africa and Israel.

• Government reports online (www.gpoaccess.gov) • American Evaluation Association • Society for Research on Child Development • Non-profit organizations and societies for human rights which support ethnicity and

immigration issues in children such as UNICEF, SAVE THE CHILDREN and others identifiable through www.charitywatch.org/azlist.html

Electronic strategies will be based on the search of multiple terms that describe intervention, context, and target population of interest as suggested in the IRMG guidelines. We will search for these keywords as topic and subject as included in the title and in the abstract. Different combinations of the specific key-words (e.g., cooperat* AND racial AND school*) will be included in order to produce search results. The work of relevant authors (e.g. Cohen, Slavin) on the field will be also looked at. Listed below are keywords that were identified as key-terms in relevant studies and thesauri of online databases (e.g. PsychInfo). Intervention: cooperat* collaborat* jigsaw* Learning Together Constructive Controversy Team-Accelerated Instruction Cooperative Learning Structures Cooperative Integrated Reading & Composition CIRC STAD TGT TAI small-group-learning student-team-achievement team-games-tournament team-assisted-individuation group-investigation group-instruction complex-instruction team learning interdependen* inter-dependen*


Cohen* Slavin* Cooper* Sharan* Johnson* Aronson* Wiegel* Ziegler* Context: integrat* rac* antirac* anti-rac* intergroup inter-group interethnic inter-ethnic multiethnic multi-ethnic bias* ethnic* diversity minorit* majorit* cultur* prejudice stereotyp* multicultur* migra* indigenous segregate* desegregate* discriminat* conflict tension tolera* intoleran* national pro-social prosocial interracial inter-racial heterogen* anti-Semiti* Target population: child* school* grade*


class* boy* girl* preschool* pre-school* adolescen* preadolescen* pre-adolescen* youth* young pupil* Title and Abstract Screening Procedure At the retrieval stage all titles and abstracts will be evaluated for inclusion by two reviewers (Curry and De Amicis). A full copy of the article will be obtained if both reviewers agree that it is definitely or potentially eligible. If the reviewers disagree on a title or abstract, a full copy of the article will be read by both reviewers to reach a common decision. Where this is not possible a third reviewer (Gilligan) will read the full copy of the article, to reach a conclusion. Full Text Screening Procedure Full-text versions will be obtained through different strategies. If the electronic journal is available at the library of Trinity College Dublin, the full-text version will be obtained through the library research port. If not electronically available, a print version of the journal will be sought. If a printed copy in unavailable, we will use the Interlibrary Loan Office to borrow the journal from another University. If these methods are unsuccessful, we will contact the authors or organization that funded the research. Coding Procedures and Categories for Included Studies The coding of the included studies will be carried out by Curry and De Amicis and discrepancies will be resolved by discussion. If any discrepancy persists, Gilligan will make a decision. The reviewers will meet once a week, to discuss difficulties, discrepancies and if necessary modify the code system. 3.4. Criteria for determination of independent findings Some studies will report multiple findings on different dependent variables and or different samples. When independent samples are present, the results will be treated as separate findings and all results will be included in the analysis. We expect that some studies will report multiple primary outcomes. These will be dealt with in one of two ways:

a. Where primary outcome measures are designed to be equivalent (i.e. to measure the same underlying construct of prejudice but in different ways) they will be examined to determine how to either combine them or select one that best represents the findings. The criteria for doing this will be relevance to the aim of the program and the reliability and validity of the measures used.


b. Where primary outcomes are designed to be non-equivalent (e.g. to measure cognitive, emotional and behavioral dimensions of prejudice) two strategies are possible. If this does not occur too frequently (as we expect) we will select one outcome to represent each study again using the criteria of relevance to the aim of the program and reliability / validity of the measurement. If non-equivalent outcomes arise more frequently we will consider including multiple outcomes and using strategies that compensate for statistical dependence.

3.5 Details of study coding categories All eligible studies will be coded according to the following criteria (see Appendix A for more details):

• Reference information: title, authors, publication type, source of information, year of publication etc.

• Description of cooperative learning intervention plan: type, duration, teacher’s role, implementation, subject, type of interdependence, group size.

• Nature and selection of comparison group • Participants’ characteristics: age, gender, SES, grade level, ability composition. • Location characteristics: country where the study was conducted, percentages of specific

minority/majority group in the samples, type of inter-ethnic relation. • Unit of analysis. • Sample size of experimental and control groups. • Methodological characteristics: design and recruitment strategy. • Description of primary and secondary outcomes. • Implementation difficulties • Statistical test/s used • Report of statistical significance • Effects size/power • Conclusions drawn by the authors

Dr Philip Curry and Dr Leyla De Amicis will independently code each eligible study, and where there are discrepancies, Professor Robin Gilligan will review the study and have the final decision. 3.6 Data extraction and Incomplete Reporting of Study Data The two main reviewers will independently extract the data from each eligible study. The studies including all the required information will be summarized in an “Included Studies” table, with comments about design, participants, interventions and outcome. In the case of missing data in a study, the first author will be contacted to ask for the missing information. If the requested data are available and appropriate, the study will be included and extracted. If data are not available, the study will be reported in the summary table but not included in the final analysis and synthesis. Information about the missing data will be reported in the text of the meta-analysis.


3.7 Statistical procedures and conventions For eligible studies, when sufficient data are available, effect sizes will be calculated with standardized measures of effect sizes as recommended by Lipsey & Wilson (2001). Before calculating weighted ESs, analyses on ES and sample size distribution will be checked to detect outliers. In order to retain as many studies as possible, extreme values will be windsorized rather than eliminated. Mean effect sizes will be calculated across the studies and the associated standard error will be computed, correcting it by the inverse variance weight. We expect to have a sufficient number of studies to provide satisfactory statistical power to use a Q-test to detect heterogeneity of effect size across studies. Our initial assumption is that effect size will be a random factor due to the variety of cooperative learning programs implemented, inter-ethnic contexts studied and outcome measures used. We therefore expect to use either a random or mixed effects model. We hope to be able to explore the effect of several key moderating factors. Different effects are expected by type of cooperative learning method, age group and inter-ethnic context. To do this, we will use the analog to the ANOVA method of moderator analysis for categorical moderator variables and weighted regression analysis for continuous moderator variables. We will consider publication bias and use methods such as the comparison of the mean effect size for published and unpublished studies and a trim-and-fill analysis. 3.8 Treatment of qualitative research No qualitative data will be considered in this meta-analysis.

4. AUTHOR(S) REVIEW TEAM

Lead reviewer: Name: Philip Curry Title: Dr. Affiliation: School of Social Work and Social Policy Address: Trinity College Dublin, 3.07 3-4 Foster Place City: Dublin Postal Code: Dublin 2 Country: Ireland Phone: +353-1-896 3238 Email: [email protected] Co-author(s): Name: Leyla De Amicis Title: Dr. Affiliation: Children’s Research Centre Address: 30 Anglesea Street, Trinity College Dublin


City: Dublin Postal Code: Dublin 2 Country: Ireland Phone: +353-1-896 3239 Email: [email protected] Name: Robbie Gilligan Title: Prof. Affiliation: School of Social Work and Social Policy Address: Trinity College Dublin, City: Dublin Postal Code: Dublin 2 Country: Ireland Phone: +353-1-896 1331 Email: [email protected]

ROLES AND RESPONSIBLIITIES

• Content: De Amicis; Gilligan; Curry. • Systematic review methods: Curry. • Statistical analysis: Curry. • Information retrieval: De Amicis

5. TIMEFRAME

The review will adhere to the following schedule: Search for published and unpublished studies Oct 2009 Relevant assessment Nov 2009 Coding of eligible studies Nov 2009 Statistical analysis Dec 2009 Preparation of report Jan 2010 Submission of completed report Feb 2010

6. PLANS FOR UPDATING THE REVIEW

Curry will be responsible for updating the review every two years. This review of Cooperative Learning is being conducted as an independent piece but our plan is to extent the review in 2010 to allow us to evaluate and compare the effectiveness of other techniques used to enhance inter-ethnic relations in schools (e.g. multi-cultural curricula).

7. DECLARATIONS OF INTEREST

No known conflicts of interest.


8. REFERENCES

Aboud, F. (2005). The development of prejudice in childhood and adolescence. . In J. F. Dovidio, P. Glick & L. Rudman (Eds.), On the nature of prejudice: fifty years after Allport. Malden, MA: Blackwell Publishing.

Allport, G. W. (1954). The nature of prejudice. . Cambridge, MA: Addison- Wesley Press. Aronson, E. (1978). The Jigsaw classroom. Beverley Hills, CA Sage. Aronson, E., & Osherow, N. (1980). Cooperation, pro-social behaviour and academic

performance. Experiments in the desegregated classroom. In Bricrman (Ed.), Applied Social Psychology (Vol. 1, pp. 163-197). Newbury Park, CA: Sage.

Aronson, E., & Patnoe, S. (1997). The Jigsaw classroom New York: Longman. Binder, J., Zagefka, H., Brown, R., Funke, F., Kessler, T., Mummendey, A., et al. (2009). Does

contact reduce prejudice or does prejudice reduce contact? A longitudinal test of the contact hypothesis among majority and minority groups in three European countries. Journal of Personality and Social Psychology 96(4), 843-856.

Blaney, N. T., Stephen, C., Rosenfield, D., Aronson, E., & Sikes, J. (1977). Interdependence in the classroom. a field study. Journal of Educational Psychology, 69(2), 121-128.

Bratt, C. (in review).The questionable promotion of the jigsaw classroom for cooperative groups at schools.

Brown, R., & Hewstone, M. (2005). An integrative theory of intergroup contact In Advances in experimental Social Psychology (Vol. 37, pp. 255-343). San Diego, CA, US: Elsevier Academic Press.

Cameron, J. A., Alvarez, J. M., Ruble, D. N., & Fuligni, A. J. (2001). Children's lay theories about ingroups and outgroups: reconceptualizing research on prejudice Personality and Social Psychology Review, 5(2), 118-128.

Cohen, E. (1986). Designing groupwork: Strategies for the heterogeneous classroom. New York: Teachers College Press.

Cohen, E. G., Bianchini, J. A., Cossey, R., Holthuis, N. C., Morphew, C. C., & Whitcomb, J. A. (1997). What did students learn? In E. G. Cohen & R. A. Lotan (Eds.), Working for equity in heterogeneous classrooms: Sociological theory in practice (pp. 137-165). New York: Teacher College Press.

Cohen, E. G., Lockheed, M., & Lohman, M. (1976). The Center for Interracial Cooperation: A field experiment. Sociology of Education, 49, 47-58.

Cohen, E. G., & Lotan, R. A. (1995). Producing equal status interaction in the heterogeneous classroom. American Educational Research Journal 32, 99-120.

Cohen, E. G., & Lotan, R. A. (Eds.). (1997). Working for equity in heterogenenous classrooms: Sociological theory in practice. New York: Teachers College Press.

Cohen, E. G., & Roper, S. (1972). Modification of interracial interaction disability: An application of status characteristics theory. American Sociological Review 37, 648-655.

Cooper, R., & Slavin, R. E. (2004). Cooperative learning: An instructional strategy to improve intergroup relations. In W. G. Stephan, Vogt, W.P. (Ed.), Education Programs for improving intergroup relations (pp. 323). N.Y., US: Teachers College Press.

Deutsch, M. (1949). A theory of cooperation and competition. Human Relations, 2, 129-151. Deutsch, M. (1962). Cooperation and trust: Some theoretical notes In M. R. Jones (Ed.),

Nebraska symposium on motivation (pp. 275-319). Lincoln: University of Nebraska Press.

DeVries, D., & Edwards, K. (1974). Student teams and learning games: Their effects on cross-race and cross-sex interaction. Journal of Educational Psychology, 66, 741-749.

De Vries, D. L., Slavin, R. E., Fennessey, G. M., Edwards, K. J., & Lombardo, M. M. (1980).


Teams-games tournament. The team learning approach. Englewood Cliffs, NJ: Educational Technology Publications

Ginsburg-Block, M. D., Rohrbeck, C. A., & Fantuzzo, J. W. (2006). A meta-analytic review of social, self-concept, and behavioral outcomes of peer-assisted learning. Journal of Educational Psychology, 98(4), 732-749.

Johnson, D. W., & Johnson, R. T. (1981). Effects of cooperative and individualistic learning experience on interethnic interaction. Journal of Educational Psychology, 73, 444-449.

Johnson, D. W., & Johnson, R. T. (1983). Learning Together and Alone. New Jersey: Prentice Hall.

Johnson, D.W., & Johnson, R.T. (1985). Classroom conflict: Controversy versus debate in learning groups. American Educational Research Journal, 22, 237-256.

Johnson, D. W., & Johnson, R. T. (1989). Cooperation and competition: Theory and research. Edina, MN: Interaction.

Kagan, S. (1985). Cooperative learning resources for teachers. Riverside, CA: University of California at Riverside.

Lipsey, M. W., & Wilson, D. B. (2001). Practical Meta-analysis (Vol. 49). Thousand Oaks, CA: Sage Publications.

McKown, C. (2005). Applying ecological theory to advance the science and practice of school-based prejudice reduction interventions. Educational Psychologist, 40(3), 177-189.

Nesdale, D. (2001). The development of prejudice in children. In M. A. Augoustinos & K. J. Keynolds (Eds.), Understanding prejudice, racism, and social conflict (pp. 57-73). London: London Sage

Oortwijn, M. B., Boekaerts, M., Vedder, P., & Fortuin, j. (2008). The impact of a cooperative learning experience on pupils' popularity, non-cooperativeness, and interethnic bias in multiethnic elementary schools. Educational Psychology, 28(2), 211-221.

Paluck, E., & Green, D. P. (2008). Prejudice reduction: What works? A review and assessment of research and practice Annual Review of Psychology, 60, 339-367.

Paolini, S., Hewstone, M., Harwood, J., & Cairns, E. (2006). Intergroup contact and the promotion of intergroup harmony: The influence of intergroup emotions. In R. Brown & D. Capozza (Eds.), Social identities: Motivational, emotional, cultural influences. Hove, UK: Psychology Press.

Pedersen, A., Walker, I., & Wise, M. (2005). "Talk does not cook rice": Beyond anti-racism rhetoric to strategies for social action. Australian Psychologist, 40(1), 20-30.

Pettigrew, T. F. (1998). Intergroup contact theory. Annual Review of Psychology, 49, 65-85. Pettigrew, T. F., & Tropp, L. R. (2006). A meta-analytic test of intergroup contact theory

Journal of Personality and Social Psychology, 90(5), 751-783. Roseth, C. J., Johnson, D. W., & Johnson, R. T. (2008). Promoting early adolescents'

achievement and peer relationships: The effects of cooperative, competitive, and individualistic goal structures. Psychological Bulletin, 134(2), 223-246.

Ryan, J. B., Reid, R., & Epstein, M. H. (2004). Peer-mediated intervention studies on academic achievement for students with EBD - A review. Remedial and Special Education, 25(6), 330-341.

Schofield, J. W., & Eurich-Fulcer, R. (2003). when and how school segregation improves intergroup relations. In R. Brown, Gaertner, S. (Ed.), Handbook of Social Psychology: Intergroup processes (pp. 475-494). Oxford: Blackwell.

Sharan, S., & Sharan, Y. (1976). Small group teaching. Englewood Cliffs, NJ: Educational Technology Publications.

Sharan, S., Hertz-Lazarowitz, R., Bejarano, Y., Raviv, S., & Sharan, Y. (1984). Cooperative learning in the classroom: Research in desegregated schools. Hillsdale, NJ: Erlbaum.

Singh, B. R. (1991). Teaching methods for reducing prejudice and enhancing academic achievement for all children. Educational studies 17(2), 157-171.


Slavin, R. E. (1980). Cooperative learning. Review of Educational Research 50, 315-340. Slavin, R. E. (1983). When Does Cooperative Learning Increase Student-Achievement.

Psychological Bulletin, 94(3), 429-445. Slavin, R. E. (1978). Student teams and comparison among equals: Effects on academic performance and

student attitudes. Journal of Educational Psychology, 70(4), 532-538. Slavin, R. E. (1995). Cooperative learning and intergroup relations. In J. Banks & C. M. Banks

(Eds.), Handbook of research on multicultural education. (pp. 628-634). New York: Macmillan.

Slavin, R. E., & Cooper, R. (1999). Improving intergroup relations: Lessons learned from cooperative learning programs. Journal of Social Issues, 55(4), 647-663.

Slavin, R. E., Leavey, M., & Madden, N. A. (1984). Combining cooperative learning and individualized instruction: Effects on students' mathematics achievement, attitudes, and behaviours. . Elementary School Journal, 84, 409-422.

Slavin, R., Leavey, M., & Madden, N. (1986). Team accelerated instruction: Mathematics. Watertown, MA: Charlesbridge.

Slavin, R. E., & Oickle, E. (1981). Effects of cooperative learning teams on student achievement and race relations: treatment by race interactions Sociology of Education, 54, 174-180.

Stephen, W. G. (1978). School desegregation. An evaluation of predictions made in Brown vs Board of Education. Psychological Bulletin, 85, 217-238.

Stevens, R., Madden, N., Slavin, R., & Farnish, A. (1987). Cooperative integrated reading and composition: Two field experiments. Reading Research Quarterly, 22, 433-454.

Wagner, U., Tropp, L. R., Finchinescu, G., & C., T. (Eds.). (2008). Improving intergroup relations. Building on the legacy of Thomas F. Pettigrew. Social Issues and interventions. Malden: Blackwell Publishing

Ziegler, S. (1981). The effective of cooperative learning teams for increasing cross-ethnic friendship: Additional evidence. Human Organization 40, 24-268.


APPENDIX A: CODING SHEETS FOR META-ANALYSIS ON EFFECTS OF COOPERATIVE LEARNING ON INTER ETHNIC RELATIONS IN SCHOOL SETTINGS

1. ELIGIBILITY CHECK SHEET

1. Document ID: _________ 2. First author last name (and first name initial): _________

3. Study title: _____________________________________________________

4. Journal Name, Volume and Issue: _____________________________________ 5. Journal ID: ____________________________________________________ 6. Coder’s initials: ________

7. Date eligibility determined: ____________

8. A study must meet the following criteria in order to be eligible. Answer each question

with a “yes” or a “no”

a) The study is an evaluation of one of the following cooperative learning strategies: • Jigsaw method (Aronson et al., 1978) • Learning Together (Johnson and Johnson, 1983) • Constructive Controversy (Johnson and Johnson, 1985) • Student Team Achievement Division (STDA, Slavin, 1978) • Complex instruction (Cohen, 1986) • Team-Games-Tournament (DeVries & Edwards, 1974) • Team-Assisted-Individuation (Slavin et al., 1984) • Team-Accelerated Instruction • Group Investigation (Sharan and Sharan, 1976) • Cooperative Learning Structures (Kagan, 1985) • Cooperative Integrated Reading & Composition (CIRC, Stevens et al, 1987) • Other cooperative learning consisting of small group work (3 -8

members), advance planning, team work that is interdependent. _______

b) Full data on at least one outcome measure assessing children’s behavior,

attitudes, emotions or beliefs towards at least one ethnic, religious or linguistic out-group is reported. _______

c) The study involved a randomized field experiments or quasi-experimental design. Quasi-experimental studies include a pre-test. _______

d) The control group received an intervention other than Cooperative Learning or

no intervention at all. _______


e) The study is based in a school context. _______

f) The study was published after 1965. _______

9. Eligibility status:

Eligible ____

Not eligible ____

Relevant review ____

9a. If the study does not meet the criteria, answer the following question:

The study is a review article that is relevant to this project (e.g., includes references of other useful studies, have pertinent background information) _________

10. Notes:


II. CODING PROTOCOL Reference information 1. Study ID: _________ 2. Study author(s): ______________________________________ 3. Study title: __________________________________________________ 4a. Publication type:

1. Book 2. Book chapter 3. Journal article (peer reviewed) 4. Thesis of doctoral dissertation 5. Unpublished academic manuscripts/data 6. Conference paper/poster 7. Government report (state/local) 8. No-profit organization report 9. Unpublished no-academic data 10. Other (specify)

4b. Specify (other)….?_____________________ 5. Publication date (year) ________ 6a. Journal Name/Book title:________________________________________ 6b. Journal Volume: ______________________________________ 6c. Journal Issue: _________________________________________ 7. Source of funding for study: _______________________________________ 8. Country of publication: ___________________________________________ 9. Date coded: ________________ 10. Coder’s initials: __________________


Location characteristics: 11. Country where the study was conducted: _____________ 12. Location in which the study was conducted:

1. Urban 2. Sub-urban 3. Rural

13. Broad inter-group context: _____________________________________________ ________________________________________________________________________ 14. Classification of broad inter-ethnic context:

1. Majority group and one established minority group (slavery or economical exchange) 2. Majority group and one established minority status group (immigration) 3. Majority group and one/more established minority group/s and other minority status groups, in general (immigration) 4. Majority group and one/more established minority group and one/other specific minority group/s (immigration) 5. Majority group and minority groups, in general 6. Majority group and specific minority groups 7. Equal- status groups

Participants’ characteristics: 15a. Age lowest: _________ 15b. Age highest: _________ 16. Gender:

1. Mixed class, homogenous by CL group 2. Mixed class, heterogeneous by CL group 3. Female 4. Male

17. SES:

1. Low 2. Medium 3. High

18a. Lowest grades in study: _________ 18b. Highest grade in study: _________


19. Ability composition 1. Homogeneous class 2. Heterogeneous, homogenous by group 3. Heterogeneous, heterogeneous by group

Ethnic composition of sample 20. Number of ethnic groups present: ________ 21a. Percentage of largest ethnic group: ________ 21b. Percentage of second largest ethnic group: ________ 21c. Percentage of third largest ethnic group: ________ 21d. Percentage of fourth largest ethnic group: ________ 21e. Percentage of fifth largest ethnic group: ________ 22. Different ethnic composition in experimental and control groups5:

1. Yes 2. No

Intervention’s characteristics: 23a. Start data of research: ________ 23b. Finish date of research: ________ 23c. Point in school cycle: ________ 24a. Type of cooperative learning intervention described?

1. Jigsaw 2. Learning Together 3. Constructive Controversy 4. Student Team Achievement Division 5. Complex instruction 6. Team-Games-Tournament 7. Team-Assisted-Individuation 8. Team-Accelerated Instruction 9. Group Investigation 10. Cooperative Learning Structures 11. Cooperative Integrated Reading & Composition 12. Other

24b. Other, specify: ________________________________________________________________________________________________________________________________________________________________________________________________________________________ 25. How many curriculum subject areas was the intervention used with? ________ 26a. What subjects did the intervention focus on?

5 We expect that there will not be many cases in which detailed data is available on this. If it turns out that such data is routinely reported in a sufficient number of studies we will code it appropriately.


1. Language 2. Literature 3. Maths 4. Science 5. History 6. Arts 7. Gym 8. Extra-curricular activities 9. More than one of these 10. Other

26b. More than one of these, specify: ________________________________________ __________________________________________________________________ 26c. Other, specify: ____________________________________________________ __________________________________________________________________ 27. Did the intervention also include strategies for working with parents?

1. Yes 2. No

28a) In the CL group, how was outcome evaluated?

1. Evaluated on group outcome, individual contributions not differentiated (e.g. project on which all work at the same tasks and are evaluated on group outcome) 2. Evaluated on group outcome, individual contributions clearly differentiated (e.g. project in which group members work on different parts, all parts needed for successful outcome) 3. Evaluated on individual contribution only (e.g. group work together and share information but complete own task and are evaluated on it)

28b) In the CL group, what reward were children supposed to get?

1. Rewarded with prize, recognized inside and outside of class 2. Rewarded with prize, recognition inside of class only 3. No concrete reward, recognition only 4. No reward or recognition

28c) In the CL group, what resources did the children share?

1. None 2. Information 3. Competence 4. Material 5. Space

28d) In the CL group, how were children interdependent? 1. Needed information which others had 2. Each had specific role

28e) In the CL group, how were children encouraged to support one another?

1. Instructed how to support one another before starting 2. Teacher / researcher gave encouragement during the activities 3. Instructed before and encouraged during 4. Encouraged informally


5. No encouragement was given 6. No information is reported on this

29. Were groups/teams in competition with other teams? 1. Yes 2. No

30a. Lowest group size: __________ 30b. Largest group size: __________ Implementation 31. Overall how long did the intervention last for? ________weeks 32. How often did the intervention sessions occur in a week? _________ 33. How long did each session last for? ______minutes 34. How much were researchers involved in the implementation?

1. No researcher involvement 2. Researcher provided training but no active support 3. Researcher provided training and some active support 3. Implemented entirely by researchers

35. How much training was given to teachers? __________________________________ __________________________________________________________________ 36. What role did the teacher have in the intervention program?

1. None 2. Instructor 3. Assessor 4. Support of group dynamics 5. Instructor + Assessor 6. Instructor + support of group dynamics 7. Assessor + support of group dynamics 8. Instructor+ Assessor+ support of group dynamics (All roles)

37. Success of implementation: 1. High (implemented as planned or nearly so) 2. Moderate (partially implemented as planned) 3. Little (implemented in radically different way) 4. Unclear/No process evaluation reported

38. If intervention is a replication, rating of fidelity of replication:

1. High 2. Moderate 3. Little 4. Very poor

39. Curriculum across target/ control groups

1. Same


2. Similar 3. Different 4. No reported

40. Number of researchers directly involved in intervention: _________ Methodological/research design descriptors: 41. Unit of assignment to conditions:

1. Individuals 2. Groups 3. Classes

42. Assignment to conditions:

1. Random, simple (also includes systematic sampling) 2. Random, after matching or stratification 3. Non-random, matching 4. No clear information is provided 5. Non random, other

43. Total number of text and control groups: _________ 44. Number of control groups: __________ 45. Type of control group:

1. Treatment as usual 2. Individualistic strategy learning 3. Competitive strategy learning 4. Different cooperative learning strategy 5. Not specified

46. Was the equivalence of the groups tested at pretest?

1. Yes 2. No

47. Total initial sample size: __________ 48. In total how many outcomes are reported in the study? __________ 49a. Other than prejudice what outcomes are reported in the study:

1. School achievement 2. Self-esteem 3. Other

49b. Other, please specify: ___________________________________ ___________________________________ ___________________________________ ___________________________________ ___________________________________


Primary outcome: Prejudice 50. How many variables were used to measure prejudice? ____________ 51. How was prejudice measured? COGNITIVE

1. Trait attributions 2. Racial awareness 3. Perception of physical differences 4. Appreciation of diversity 5. Knowledge of stereotypes’ effects 6. Other cognitive measure

EMOTIONAL 7. Empathy 8. Specific negative emotions 9. Specific positive emotions 10. Liking for unknown or known ingroup / outgroup members 11. Preference for ingroup and outgroup stimuli 12. Other measure of emotions

BEHAVIORAL 13. Social distance 14. Intentions/memories of discouraging peers’ racist remarks 15. Standing up for bullied outgroup members 16. Supporting pro-inter-ethnic contact discussions 17. Other measure of behavior

52. Form of data collection:

1. Self-report questionnaires 2. Structured interviews 3. Observational data 4. Test scoring 5. Other

53. Source of data:

1. Participants 2. Peers 3. Teachers 4. Researchers 5. Other

54. Did the researcher express any concerns about the quality of the collected data?

1. Yes 2. No 3. None reported


Sample size 55. Sample size of the cooperative learning group: ______________ 56. Sample size of the control group: ______________ 57a. Was attrition a problem in the analysis for this outcome?

1. Yes 2. No

57b. If attrition was a problem, provide details (how many cases and why were lost) Effect size data 58. Did a test of statistical significance indicate significant differences between the control and the experimental group?

1. Yes 2. No 3. No testing reported

59a. Was/were a standardized effect size/s reported?

1. Yes 2. No

59b. If yes, what was the effect size? ____________________________ 59c. If yes, page number where the effect size data was reported: ______________ 59d. If not, is there data available to calculate the effect sizes?

1. Yes 2. No

59e. Type of data effect size can be calculated from:

1. Means and standard deviations 2. t-value or F-value 3. Chi-square (df=1) 4. Frequencies or proportions 5. Pre and post (and/or during counts) 6. Other (specify)

59f. Specify Other: __________________


60. Raw difference favors (e.g. more success on): 1. Experimental group (or post period) 2. Control group (or pre-period) 3. Neither

Means and standard deviations 61a. Intervention group mean: ___________ 61b. Control group mean: ____________ 61c. Intervention group standard deviation: _________ 61d. Control group standard deviation:__________ Proportions/frequencies 62a. N of intervention group with a successful outcome________ 62b. N of control group with a successful outcome__________ 62c. Proportion of intervention group with a successful outcome _________ 62d. Proportion of control group with a successful outcome __________ Significance tests 63a. t-value _______ 63b. F-value ________ 63c. Chi-square value (df=1)__________ Calculate effect Size 64. Effect size ___________ Conclusions made by the author(s) about prejudice outcome 65. Conclusions about the impact of the intervention:

1. The intervention worked in the expected direction 2. The intervention did not work in the expected direction 3. Unclear or no conclusions stated by the authors

66. Did the author(s) conclude that the cooperative learning intervention was effective?

1. Yes 2. No 3. Can’t tell

67. Additional notes on conclusions_________________________________________ __________________________________________________________________ _________________________________________________________________


Secondary outcome 1: School achievement If school achievement was assessed: 68. How many variables were used to measure school achievement? ___________ 69. How was school achievement measured?

1. Standardized test scores 2. School tests 3. Number of correct answers 4. Response accuracy 5. Response rapidity 6. Other

70. Form of data collection:

1. Self-report questionnaires 2. Structured interviews 3. Observational data 4. Test scoring 5. Official records 6. Other

71. Who was the source for school achievement data?

1. Participants 2. Peers 3. Teachers 4. Researchers 5. Records





1. Yes 2. No

76b. If attrition was a problem, provide details (how many cases and why were lost) Effect size data 77. Did a test of statistical significance indicate significant differences between the control and the experimental group?



1. Yes 2. No


1. Yes 2. No



78f. Specify Other: __________________


79. Raw difference favors (e.g. more success on): 1. Experimental group (or post period) 2. Control group (or pre-period) 3. Neither

Means and standard deviations 80a. Intervention group mean: ___________ 80b. Control group mean: ____________ 80c. Intervention group standard deviation: _________ 80d. Control group standard deviation:__________ Proportions/frequencies 81a. N of intervention group with a successful outcome________ 81b. N of control group with a successful outcome__________ 81c. Proportion of intervention group with a successful outcome _________ 81d. Proportion of control group with a successful outcome __________ Significance tests 82a. t-value _______ 82b. F-value ________ 82c. Chi-square value (df=1)__________ Calculate effect Size 83. Effect size ___________ Conclusions made by the author(s) about school achievement outcome 84. Conclusions about the impact of the intervention:




86. Additional notes on conclusions_________________________________________ __________________________________________________________________ __________________________________________________________________


Secondary outcome 2: Self esteem If self-esteem was assessed: 87. How many variables were used to measure it? ___________ 88. What specific dependent variables were used to measure self-esteem?

1. Standardized test scores 2. Ad-hoc Likert scale 3. Other

89. What type of data was used to measure self-esteem?

1. Self-report questionnaires 2. Structured interviews 3. Observational data 4. Test scoring 5. Other

90. Who was the source for self esteem data?

1. Participants 2. Peers 3. Teachers 4. Researchers 5. Other




1. Yes 2. No

94b. If attrition was a problem, provide details (how many cases and why were lost)


Effect size data 95. Did a test of statistical significance indicate significant differences between the control and the experimental group?



1. Yes 2. No


1. Yes 2. No



96f. Specify Other: __________________ 97. Raw difference favors (e.g. more success on):

1. Experimental group (or post period) 2. Control group (or pre-period) 3. Neither

Means and standard deviations 98a. Intervention group mean: ___________ 98b. Control group mean: ____________ 98c. Intervention group standard deviation: _________ 98d. Control group standard deviation:__________ Proportions/frequencies 99a. N of intervention group with a successful outcome________ 99b. N of control group with a successful outcome__________ 99c. Proportion of intervention group with a successful outcome _________


99d. Proportion of control group with a successful outcome __________ Significance tests 100a. t-value _______ 100b. F-value ________ 100c. Chi-square value (df=1)__________ Calculate effect Size 101. Effect size ___________ Conclusions made by the author(s) about self esteem outcome 102. Conclusions about the impact of the intervention:




104. Additional notes on conclusions________________________________________ __________________________________________________________________ _________________________________________________________________

C2 Protocol CooperativeLearning

Documents

trinity college dublin

school of social work

social policy

review team

contact hypothesis

interethnic relationships

group members

research evidence