BEYOND THE SOUNDBITE: Arts Education and Academic Outcomes

Soundbite Design_newestAr ts Educat ion and Academic Outcomes
C o n f e r e n c e P r o c e e d i n g s f r o m
B e yo n d t h e S o u n d b i t e :
W h a t t h e R e s e a r c h A c t u a l l y
S h ow s A b o u t A r t s E d u c a t i o n a n d
A c a d e m i c O u t c o m e s
E l l e n W i n n e r a n d L o i s H e t l a n d
E d i t o r s
T h e G e t t y C e n t e r
L o s A n g e l e s , C a l i f o r n i a
A u g u s t 2 4 – 2 6 , 2 0 0 0
B E
Y O
N D
T H
E S
O U
N D
B IT
es
Beyond the Soundbi te : Ar ts Educat ion and Academic Outcomes
C o n f e r e n c e P r o c e e d i n g s f r o m
B eyo n d t h e S o u n d b i t e : W h a t t h e R e s e a r c h A c t u a l l y
S h o w s A b o u t A r t s E d u c a t i o n a n d A c a d e m i c O u t c o m e s
T h e G e t t y C e n t e r • L o s A n g e l e s , C a l i f o r n i a • A u g u s t 2 4 – 2 6 , 2 0 0 0
Ellen Winner and Lois Hetland Editors
Beyond the Soundbi te : Ar ts Educat ion and Academic Outcomes
© 2001 The J. Paul Getty Trust 1200 Getty Center Drive Suite 400 Los Angeles, California 90049-1681 www.getty.edu/
Publisher Christopher Hudson Managing Editor Mark Greenberg
Editors Ellen Winner and Lois Hetland Project Coordinator Josephine Ramirez Manuscript Editor Mollie Holtman Production Coordinator Elizabeth Burke Kahn Designer Hespenheide Design
Table o f Contents
Preface v i i J a c k M eye r s
Acknowledgments ix E l l e n W i n n e r a n d L o i s H e t l a n d
Conference P roceed ings
I . Meta-Analysis : I ts Use and Value in Ar ts Education Research 1
R o b e r t R o s e n t h a l a n d L o i s H e t l a n d
I I . The Re lat ionsh ip Between Ar ts and Academic Ach ievement : No Ev idence (Yet ) fo r a Causa l Re lat ionsh ip . A Summar y of a Meta -Analyt i c Study 17
E l l e n W i n n e r
C O M M E N TA R I E S
Main Po ints in Response to “Mute those C la ims : No Ev idence (Yet ) fo r a Causa l L ink Between the Ar ts and Academic Ach ievement” 32
J a m e s S . C a t t e r a l l
Response to James Catte ra l l 38 E l l e n W i n n e r a n d L o i s H e t l a n d
Comments on the Quest ion o f Transfe r 41 E l l i o t E i s n e r
What Research in the Uni ted K ingdom Shows
About Trans fe r f rom the Ar ts 49 J o h n H a r l a n d
I I I . The Re lat ionsh ip Between Mus ic and Spat ia l Reasoning : A Summar y o f Two Meta -Analyt i c Stud ies 55
L o i s H e t l a n d
C O M M E N TA R I E S
Research and Just i f i cat ion in Ar ts Educat ion :
An I l l - Fated Romance 71 B e n n e t t R e i m e r
Revo lut ion in Math Educat ion Fue led by Mus ic Tra in ing 79 G o r d o n L . S h aw
The E f fects o f Ear ly Mus ic Exper iences 89 R i c h a r d C o l w e l l
IV. St rengthening Verba l Sk i l l s Through the Use o f C lass room Drama: A C lear L ink . A Summar y o f a Meta -Analyt i c Study 99
A n n Po d l o z ny
C O M M E N TA R Y
Learn ing in Drama 108 J o h n S o m e r s
V. Embrac ing Babe l : The P rospects o f Ins t rumenta l Uses o f the Ar ts fo r Educat ion 117
D av i d Pe r k i n s
VI . Future Research Di rect ions
What Are the Most P romis ing Di rect ions fo r Future
Research in Ar ts Educat ion? 125 J a m e s S . C a t t e r a l l
A Hero ic E f fo r t to Make a P ract i ca l D i f fe rence in Ar t Educat ion 129
E l l i o t E i s n e r
I Can See C lear ly Now: Poss ib le Seque lae o f the Rev iewing Educat ion and the Ar ts P ro ject (REAP) 132
C o n s t a n c e B u m ga r n e r G e e
A Per sonal and Non-Academic Repor t on the Getty Center Rev iewing Educat ion and the Ar ts P ro ject (REAP) Confe rence 136
P a t t e r s o n S i m s
Fo l lowing the Rev iewing Educat ion and the Ar ts P ro ject (REAP) 140
S a m u e l H o p e
VI I . Research in Ar ts Educat ion : D i rect ions fo r the Future 143 E l l e n W i n n e r a n d L o i s H e t l a n d
vi
Preface
As part of its ongoing work in education and the arts, the Getty Trust joined with Harvard’s Project Zero to convene a conference at the Getty Center from August 24 to 26, 2000, entitled “Beyond the Soundbite: What the Research Actually Shows About Arts Education and Academic Outcomes.” The goal of the conference was to examine, in all their complex implications, the many answers that have been given by researchers to the question of whether and how the study of the arts can improve a student’s academic performance.
The focus of the conference was a three-year study, Reviewing Education and the Arts Project, also known as . The study was directed by Ellen Winner and Lois Hetland of Project Zero, funded by the Bauman Foundation, and presented first as a series of papers in a special Fall/Winter issue of the Journal of Aesthetic Education. set out to evaluate the hundreds of studies examining the relationship between some form of arts study and some form of academic outcome. The project examined several key relationships, from broader issues, such as the relationship between the study of the arts in general and a student’s academic achievement as measured by standardized tests and grades, and the relationship between arts study and cre- ativity as measured by various creativity tests; to more particular effects, such as the relationship of music and spatial reasoning, and the relationship between visual arts and reading.
The conference took up these same issues, but provided the opportu- nity for scholars from widely divergent perspectives to discuss critically and reflect upon the findings. What follows are the proceedings of this con- ference, including the talks, commentaries, and suggestions for further research. Although the points of view on the findings varied greatly
vii
among the thirty-six conference participants, virtually all of the participants came to agree that the answers to the central question about the relationship between arts study and academic achievement cannot be characterized as simple or straightforward. In some cases, it was clear that positive relation- ships were demonstrated and in others they were not. Often, it appeared that more research was necessary before any definitive conclusions could be drawn or, in still other cases, that different questions really needed to be asked.
Unfortunately, however, the desire, particularly on the part of the media and policy makers, for a single, definitive answer to the art-and- academic-achievement question is a powerful one. We can only hope that the conference, and this present volume of its proceedings, will act as a counter- weight by taking the issues beyond soundbite simplicity and by helping to convey the true complexity of the issues involved.
On the Getty Trust’s behalf, I would like to take this opportunity to thank Ellen Winner and Lois Hetland for all their hard work in helping to organize and chair the conference, and to thank the conference participants for their thoughtful contributions. This conference represents the first in what we hope will be a series of convenings at the Getty Center that will examine cen- tral questions related to research in arts education, and we look forward to the opportunity to work with a wide range of scholars in this field.
Jack Meyers Deputy Director Getty Grant Program The J. Paul Getty Trust
PREFACE
viii
Acknowledgments
We would like to thank John Landrum Bryant, codirector of the Bauman Foundation, for funding the Reviewing Education and the Arts Project () work and Ralph Smith, former editor of the Journal of Aesthetic Education, for publishing the collected papers in a special issue. We express our special thanks as well to the advisors: Howard Gardner, John H. and Elisabeth A. Hobbs Professor of Cognition and Education, Harvard Graduate School of Education; Richard Light, Professor, Harvard Graduate School of Education and John F. Kennedy School of Government; David N. Perkins, Professor, Harvard Graduate School of Education; Robert Rosenthal, Edgar Pierce Professor of Psychology, Emeritus, Harvard University, Distinguished Professor, University of California, Riverside; and Judith D. Singer, Academic Dean and Professor of Education, Harvard Graduate School of Education.
We are most grateful that the President and Chief Executive Officer of the J. Paul Getty Trust, Barry Munitz, was intrigued enough by our work to offer to host a conference on the topic of arts education research. We are espe- cially grateful to Jack Meyers, Deputy Director of the Getty Grant Program, for his guidance in helping to plan this conference. We thank Josephine Ramirez, Program Officer, and Jenny Goto, Senior Staff Assistant, of the Getty Grant Program for their help in planning and running the conference on which this volume is based and for their assistance in preparing this publication.
We would like to acknowledge the other researchers whose papers were drawn upon in the summary of the findings: Kristin Burger, Ron Butzlaff, Monica Cooper, Mia Keinanen, Erik Moga, Ann Podlozny, and Kathryn Vaughn. Additionally, we thank Ron Butzlaff for serving as the statis- tical advisor to the project.
ix
We also want to thank the research assistants who helped us at various times during the project: Kristin Burger, Lisa French, Kimberlee Garris, Nandita Ghosh, Maxwell Gomez-Trochez, Jessica Gordon, Joanna Holtzman, Jenny Martin, Elisabeth Moriarty-Ambrozaitis, Brian Moss, Melissa Mueller, Leah Okimoto, Nina Salzman, and Daniel Schneider.
And finally, for their willingness to answer all our questions about details of their work, we thank the researchers whose work we reviewed.
Ellen Winner and Lois Hetland
ACKNOWLEDGMENTS
x
Met a-Analys i s : I t s Use and Va lue in Ar ts Educat ion Research
RO B E R T R O S E N T H A L
University of California, Riverside
LO I S H E T L A N D
Project Zero, Harvard University
1
I .
The research conducted by the Reviewing Education and the Arts Project (), summarized in this volume, is a set of meta-analyses that synthesized empirical evidence about transfer effects from arts to non-arts cognition and learning.1 Because meta-analysis is a relatively unfamiliar tool in the field of arts education research, a description of the nature and value of this quantita- tive technique for synthesizing empirical research should help readers better understand the summaries that follow.
The fundamental goal of meta-analytic procedures is the cumulation of evidence for the purpose of furthering understanding of past research and guiding future inquiry. Meta-analytic methods are similar to those of other quantitative research procedures. First, meta-analysts define a research domain. Then the search for studies (the “subjects” of a meta-analysis) com- mences in a comprehensive and systematic manner so as to achieve a sample as unbiased as possible to represent the population of studies that explores the question. Next, studies are coded for qualities whose variation may be associ- ated with the size of effect. Average effects and combined significance levels are computed through standard statistical procedures. Studies can then be grouped and compared to explain whether effects vary systematically by iden- tified “moderator” variables. Finally, results are clearly reported according to standard criteria for empirical research and interpreted in relation to the body of literature and to the field at large.
There has long been underlying pessimism in the younger social, behavioral, educational, and biomedical sciences that our progress has been exceedingly slow and less orderly than we would like. Work in physics and
chemistry, for example, seems to build directly upon the older work of these sciences, whereas the social, behavioral, educational, and biomedical sci- ences seem often to be starting from scratch. In the arts, the issue of cumu- lation may be especially controversial, since some maintain that arts or aesthetics cannot even be studied empirically, much less quantitatively sum- marized. Debate on this question has persisted for at least half a century, despite serious attention to subtle human behaviors that are relevant to artistic experience (e.g., exploration, pleasure, flow, preference) and the development of methods suited to the study of aesthetic and artistic out- comes.2–6 Measures still may be too blunt, treatments ill-defined, or inter- pretation flat-footed, but all these elements can be improved through the reflective process of ongoing research, synthesis, and redefinition of the questions of interest. Methods only improve through creative thinkers’ efforts to solve interesting puzzles.
Not surprisingly, the researchers found nuanced, conflicting results in studies of variable quality, often incompletely reported. Although it may be cold comfort, such problems are not specific to research in the arts but characterize a great deal of social science and biomedical research. While meta-analysis cannot completely solve these problems of cumulation, it can help fields to address them by bringing methodological and theoretical issues to light and by specifying what has and has not been accomplished by research conducted to date.
The problems of cumulation do not seem to be due to lack of replica- tion, or to the failure to recognize the need for replication. Indeed, there are many areas of the social, behavioral, educational, and biomedical sciences— including the arts, as demonstrated through ’s efforts—for which the results of many studies, all addressing essentially the same question, are avail- able. However, reviews of research in education have not been nearly as informative as they might have been, either with respect to summarized signif- icance levels (i.e., how confidently we can assert that there is some non-zero effect of an intervention) or with respect to summarized effect sizes (i.e., how strongly any intervention is related to the outcome as measured). Even the best reviews of research by the most sophisticated scholars have been primar- ily qualitative narratives and have rarely told us much more about each study in a set of studies than the direction of the relationship between the variables investigated (e.g., smaller classes improve student learning), and whether or not a given significance level was attained (i.e., usually whether an effect found in a given study showed p # .05). If traditional narrative reviews only summa- rize researchers’ conclusions rather than synthesizing and interpreting study data, the actual findings may be misinterpreted.
META-ANALYSIS: ITS USE
2
This state of affairs is beginning to change, however. More and more reviews of the literature in the social sciences are moving from the traditional literary approach toward quantitative approaches to research synthesis described in an increasing number of textbooks of meta-analysis.7–14 Because meta-analytic procedures and methods are codified and described explicitly, meta-analytic reviews are more replicable than traditional reviews and allow reviewer bias to be revealed over time through the scientific process.
Meta-analytic syntheses do not intend to be the final word on a research area, but rather to clarify what has been learned thus far from the studies conducted and to determine what remains to be learned. The
research summarized here assesses what we know to date from quantitative studies about cognitive transfer to non-arts learning that results from arts edu- cation. Its results can be used to guide future studies on this complex question.
Defining Research Results Before we can consider various issues and procedures in the cumulation of research results, we must become quite explicit about the meaning of the con- cept “results of a study.” It is easiest to begin with what we do not mean. We do not mean the prose conclusion drawn by the investigator and reported in the abstract, results, or discussion sections of a research report. We also do not mean the results of an omnibus F test with df . 1 in the numerator or an omnibus x2 test with df . 1. Such statistics are commonly reported when research designs employ more than two conditions (i.e., a “treatment” and one “control”), but they do not answer precise research questions and can obscure important trends that data might reveal when more focused methods of analy- sis (i.e., contrasts) are used.
What we do mean by results is the answer to the question: What is the relationship between any variable X (e.g., arts instruction) and any variable Y (e.g., test scores or grades)? The variables X and Y are chosen with only the constraint that their relationship be of interest to us. In the case of the
analyses, X is some form of arts intervention (e.g., dance, drama, music, visual art, or a combination of these areas that the researchers call “multi-arts”), and Y is a score on some measure of cognition or…