Observations of Effective Teacher-Student Interactions ... - ERIC

!!!

Observations!of!effective!teacher0student!interactions!in!secondary!school!classrooms:!predicting!student!achievement!with!the!classroom!assessment!scoring!

system!–!Secondary!!

Joseph!Allen!University!of!Virginia!

!Anne!Gregory!

Rutgers!University!!

Amori!Mikami!University!of!British!Columbia!

!Janetta!Lun!

University!of!Maryland!!

Bridget!Hamre!and!Robert!Pianta!University!of!Virginia!

!2013!!

Acknowledgements:!Institute!for!Education!Science!(R305A100367)!

Increasing!Adolescent!Engagement,!Motivation,!and!Achievement:!Efficacy!of!a!Web0Based,!Teacher!Professional!Development!Model!

!!

School Psychology Review,2013, Volume 42, No. 1, pp. 76-98

RESEARCH INTO PRACTICE

Observations of Effective Teacher-Student Interactionsin Secondary School Classrooms: Predicting Student

Achievement With the Classroom AssessmentScoring System—Secondary

Joseph AllenUniversity of Virginia

Anne GregoryRutgers University

Amor i MikamiUniversity of British Columbia

Janetta LunUniversity of Maryland

Bridget Hamre and Robert PiantaUniversity of Virginia

Abstract. Multilevel modeling techniques were used with a sample of 643 studentsenrolled in 37 secondary school classrooms to predict future student achievement(controlling for baseline achievement) from observed teacher interactions with stu-dents in the classroom, coded using the Classroom Assessment Scoring System—Secondary. After accounting for pdor year test performance, qualities of teacherinteractions with students predicted student performance on end-of-year standardizedachievement tests. Classrooms characterized by a positive emotional climate, withsensitivity to adolescent needs and perspectives, use of diverse and engaging instruc-tional leaming formats, and a focus on analysis and problem solving were associatedwith higher levels of student achievement. Effects of higher quality teacher-studentinteractions were greatest in classrooms with fewer students. Implications for teacherperformance assessment and teacher effects on achievement are discussed.

Correspondence regarding this article should be addressed to Joseph Allen, University of Virginia,Department of Psychology, Box 400400, Charlottesville, VA 22904-4400; e-mail: [email protected]

Copyright 2013 by the National Association of School Psychologists, ISSN 0279-6015

76

Predicting Student Achievement

CO

<oQ

(0UJ

ClassroomOrganization

instructionalSupport

ORS

DICA

T

Z

! Affe

ct,

>ns.

ila i

ill0) o 0)

o

et, P

uniti'

ë<

gativ

e

Z

spec

t

(0Q

ntro

l,

oÜ

IßIß

i

espo

nsiv

ceoitacgï

Com

fort

, Stu

dent

IßIßc

ectiv

e

it:UJ

Idea

s,ib

ility

lom

y,St

uden

ton

s. F

lex

° ^'^3 0 ^

111tn o s.

ve,

dent

act

Stu

n. P

roct

ion.

ar e

xpec

tsec

tive

Red

îavi

or

^ 3= <DU UJ m

.1

ning

Ttio

nsM

axim

izing

LRo

utin

es, T

ra

co•ss

niz tior

si iä O Rt

arnin

g Ta

rerie

ty, A

ctiv

ectiv

e En

g

ai ea it:;

08.2..« E

¡il

III

> à« c

III

f i l

n u

Figure 1. Classroom Leaming Assessment Scoring System—Secondary frame«work for assessing effective teacher-student Interactions

Improving the quality of teacher-stu-dent interactions within the classroom dependsupon a solid understanding of the nature ofeffective teaching for adolescents. A numberof descriptions of classroom environments orquality teaching have been put forth in theeducational and developmental literatures list-ing factors likely to be related to student leam-ing (e.g., Brophy, 1999; Eccles & Roeser,1999; Pressley et al., 2003; Soar & Soar,1979). Hamre and Pianta (Hamre & Pianta,2010; Hamre, Pianta, Burchinal, & Downer,2010) developed an assessment approach thatorganizes features of teacher-student inter-actions into three major domains: emotionalsupports, classroom organization, and instruc-tional supports. This approach to assessingclassroom interaction qualities has been testedand validated for the grades of prekindergartento five, with evidence supporting this latentstructure of dimensions and domains across

grades and across content areas (Hamre et al.,2010).

The Classroom Leaming AssessmentScoring System—Secondary (CLASS-S; Pi-anta, Hamre, Hayes, Mintz, & LaParo, 2008)was developed for secondary schools as anupward extension of previous work. Withineach domain considered are specific dimen-sions of classroom interactions that past re-search suggests are likely to be important tostudent leaming (see Figure 1). The impor-tance of qualities of emotional and relationalsupport in the classroom is suggested by bothattachment and self-determination theories(Bowlby, 1969/1982; Connell & Wellbom,1991; Pianta, 1999), and is captured via as-sessments of the dimensions of positive class-room climate, teacher sensitivity, and regardfor adolescent perspectives. Although the needfor emotional support of students is perhapsmore self-evidently important in the lower

77

School Psychology Review, 2013, Volume 42, No. 1

grades (Ladd, Birch, & Buhs, 1999), adoles-cents are highly sensitive to the emotionalrapport they establish with adults in schoolsettings, and the experience of strong connec-tions to adults has been consistently linkedto long-term academic success (Allen, Kuper-minc, Philliber, & Herre, 1994; Bell, Allen,Häuser, & O'Connor, 1996). The organiza-tional support domain encompasses dimen-sions of classroom management, productivity,and use of varied instructional learning for-mats, which facilitate the development of ad-olescent self-regulation skills and enhancelearning (Blair, 2002; Cameron, Connor, &Morrison, 2005; Emmer & Stough, 2001;Paris & Paris, 2001; Raver, 2004). The in-structional support domain reflects teachers'content understanding, focus on analysis andproblem solving, and quality of feedback,which are areas that have long been recog-nized to allow students to learn on a deep level(Marton & Saljo, 1976; National ResearchCouncil, 2005).

The domain approach used by theCLASS-S aligns well with constructs fromseveral existing theoretical and practical ap-proaches. For example, Brophy (1999) de-scribed 12 principles of effective teaching en-compassing many of the same dimensions asthe CLASS-S, including supportive classroomclimate, opportunities to learn, curricularalignment, thoughtful discourse, scaffoldingengagement, and achievement expectations.Similarly, Pressley and colleagues (2003)draw from their studies of effective teachers(e.g., Bogner, Raphael, & Pressley, 2002;Wharton-McDonald, Pressley, & Hampston,1998) to suggest that effective teaching strat-egies can be organized into decisions regard-ing motivational atmosphere, classroom man-agement, and curriculum and instruction.Eccles and Roeser (1999) suggest that school-ing is optimally characterized by organiza-tional, social, and instructional processes thathelp regulate children and adolescents' devel-opment across cognitive, social-emotional,and behavioral domains. A similar approachhas been taken by McCaslin, Burross, andGood (2005) in their examination of class-room setting effects on student motivation.

The CLASS-S draws upon constructs fromeach of these frameworks; however, a keychallenge raised by all of these theoreticalframeworks is how best to operationalize andassess the constructs they delineate in ongoingteacher-student interactions. The CLASS-Sseeks to meet this challenge by conceptualiz-ing and operationalizing these constructs interms of observable, ongoing qualities ofteacher-student interactions.

In terms of actual assessment ap-proaches, a long line of research suggests thevalue of observation in classrooms as a meansfor capturing the social assets in those settings(Brophy & Good, 1986; Pianta & Hamre,2009; Shinn & Yoshikawa, 2008; Tseng &Seidman, 2007). The CLASS-S builds on arich tradition of approaches to observing in-structional environments. The InstructionalEnvironment System—11 (Ysseldyke & Chris-tenson, 1987,1993) approach outlines an arrayof qualities of instructional support and notesthe role of positive emotional climate, al-though it has not been linked to achievementoutcomes in typical classroom environments.Similarly, the Ecobehavioral Assessment Sys-tems Software (Greenwood, Carta, & Dawson,2000; Greenwood, Carta, Kamps, & Terry,1994), initially developed for use in alterna-tive school classrooms, addresses many simi-lar constructs to the CLASS-S (Wallace, An-derson, Batholomay, & Hupp, 2002; Watson,Gable, & Greenwood, 2011). One key distinc-tion, however, is that the CLASS-S focuses onbroad patterns of interaction assessed at a mo-lar level, as opposed to the time-sampling andcounting of discrete behaviors as in the Ecobe-havioral Assessment Systems Software. Thus,the CLASS-S is in keeping with principlesof developmental psychology that suggest theimportance of a focus on the broader organi-zation of molar patterns of behavior as ameans to get at subtle processes not easilycaptured via counts of discrete behaviors(Sroufe, Egeland, Carlson, & Collins, 2005),such as emerging self-regulation skills andresilience. At the younger grades, there isemerging evidence that a broad range of class-room interaction qualities can be observed andlinked to student learning gains using this

78


more global yet standardized approach to as-sessment (Landry et al., 2006; Pianta, Belsky,Vandergrift, Houts, & Morrison, 2008). Sim-ilarly, an intervention approach based on theCLASS-S framework has demonstrated evi-dence of efficacy along with evidence thatprogram effects may be at least partially me-diated via a single global measure of teacher-student interaction qualities from this system(Allen, Pianta, Gregory, Mikami, & Lun, 2011).

Unfortunately, outcome-based researchon classroom observation as a predictor ofactual student achievement is still relativelyrare in secondary education. A modest amountof research beyond the elementary grades hassuccessfully observed teaching in specific con-tent areas or for specific lessons (Johnson,Kahle, & Fargo, 2007; Roth et al, 2006;Weiss, Pasley, Smith, Banilower, & Heck,2003). Seidel and Shavelson (2007) conducteda broader meta-analysis of 125 studies using avariety of methodologies to link teacher be-havior to student achievement. They did notbreak out the tiny amount of observationalresearch from nonobservational research atthe secondary level, but did note that overall,assessments of teaching relied heavily upondistal and proxy variables (e.g., survey data)of questionable reliability and validity. Theyconcluded that effects on secondary studentacademic performance from current ap-proaches to measuring teaching quality weretoo small to be of practical significance (e.g.,effect sizes s 0.04). Thus despite the strongtheoretical interest in identifying qualities ofteacher-student interactions linked to studentachievement, scientific evidence is quitesparse regarding our capacity to identify andobserve the critical features of teacher-studentinteractions that actually predict student leam-ing within the secondary school classroom.Virtually no evidence exists regarding the ef-fectiveness of assessment systems designed tocapture broad interactional patterns and applyacross diverse content areas at the secondarylevel.

One significant potential confound in ef-forts to identify qualities of effective teacher-student interactions linked to student achieve-ment is the likelihood that high-quality inter-

actions may come more easily among studentswho are already academically motivated andsuccessful. Given the likelihood that studentsare to some degree tracked into higher andlower achieving groups in secondary schools(either explicitly or implicitly), differentteachers are likely to face students with verydifferent characteristics at the start of an aca-demic year. End-of-year student test scoresare highly dependent on preexisting studentlevels of academic proficiency and are typi-cally highly correlated with prior year testscores. Failure to account for prior year testscores would thus misattribute variance in stu-dent achievement that would be more directlyaccounted for by background student profi-ciencies. In keeping with the growing recog-nition of the importance of "value-added"approaches to assessing student learning (Ha-nushek, Rivkin, Figlio, & Jacob, 2010; Roth-stein, 2010), we assess end-of-year test scoresafter first accounting for prior year test scores,which we consider to be an indicator of stu-dent academic proficiency independent of thecurrent classroom environment.

Purpose

Some qualities of teacher-student inter-actions may primarily reflect student charac-teristics as they enter class at the start of theyear, particularly to the extent that students areimplicitly or explicitly grouped into higherand lower achieving classes. Without aware-ness of this possibility, it would be all too easyto misattribute the qualities of classroom in-teractions to teacher skill levels, rather thanrecognizing that they may primarily reflect theacademic characteristics of the students theyare teaching. Thus, to provide the most helpfuland balanced information to school personnel,this study also focused on identifying qualitiesof teacher-student interaction that are linkedto preexisting student characteristics at thebeginning of a school year, which are alsoqualities that might otherwise be misattributedto teacher skill. By identifying such student-driven qualities, this study seeks to provideappropriate contextual balance to our emerg-ing picture of the role of teacher-student in-

79


teractions: showing not only where these in-teractions predict future achievement, but alsowhere they may also simply reflect preexistingstudent characteristics, rather than simpleteacher skill.

This study addressed three overarchinggoals;

1. Examine whether a broad, molar, ob-servational measure of the quality ofteacher-student interactions could pre-dict achievement in classrooms at thesecondary level across diverse contentareas.

2. To the extent the ñrst goal was met,examine the 10 individual dimensionsof the CLASS-S to try to identify spe-ciflc qualities of teacher-student inter-actions linked to future studentachievement and to do so at a level ofgranularity sufficient to guide future re-search on specific teaching practices.

3. Provide a balanced picture to schoolpersonnel by assessing not only quali-ties of teacher-student interactions thatpredict student performance (after ac-counting for prior performance levels),but also to identify the extent to whichcertain qualities of teacher-student in-teractions appear largely determined bystudent academic skills upon entry intothe teacher's classroom.

In pursuing these goals, we account forkey contextual and background factors as bothcovariates and potential moderators (e.g., classsize, and student gender, grade level, andfamily poverty status). Analyses also consid-ered whether the predictive efficacy of theCLASS-S might be moderated by classroomcontent domain (e.g., math/science vs. Eng-lish/social studies).

Method

Participants

Participants were 643 students enrolledin 37 classrooms (across 11 schools in sixdistricts), which served as the control, teach-ing-as-usual condition classrooms in a largerstudy of an intervention to improve classroom

interaction qualities. The larger interventionstudy included 78 classrooms altogether and1267 students in Year 1 and is further de-scribed in Allen, Pianta, Gregory, Mikami, &Lun (2011). Students were eligible for partic-ipation if they (a) were in a classroom partic-ipating in the control condition of the inter-vention study, (b) had parental consent, and(c) provided their own informed assent to par-ticipate. Teacher informed consent to peirtici-pate in the study was also obtained. Eachteacher was asked to identify one class that heor she considered "challenging" to teach, andonly one classroom per teacher was assessedin the study. Classes averaged 23 students{SD = 6.1) in size and an average of 76.4% ofstudents participated from each class. Class-rooms were approximately equally divided be-tween math or science courses {N = 17), andhistory, social studies, or English courses(A = 20). Table 1 presents descriptive infor-mation regarding both students and teachers;the table indicates that both groups consistedof racially and ethnically diverse samples, andthat the participating teachers reflected a broadrange of experience levels.

Measvires

The CLASS-S was the primary sourceof standardized observational data on teacher-student interactions in the classroom. Thepre-K and elementary versions of the CLASSare among the most current and widely usedstandardized assessments of social and in-structional interactions in classrooms (Burchi-nal et al., 2008; Howes et al., 2008; McCaslinet al., 2005; Rimm-Kaufman, Curby, Grimm,Nathanson, & Brock, 2009). The CLASS-Sversion was modified to capture preciselythose aspects of classroom interactions thatwe hypothesize to be critical resources foreducational achievement in adolescence. TheCLASS-S consists of a set of global 7-pointrating scales (one rating scale for each dimen-sion below) with behaviorally anchored scalepoints providing detailed descriptions of a spe-cific dimension of classroom process and itsscaling from low to high. The CLASS-S scalesare organized into three overarching domains.

80


Table 1Means and Standard Deviations of Primary Measures and

Demographic Variables

Student Grade in SchoolTeacher AgeTeacher Education LevelNumber of Years TeachingClass SizePrior Year Achievement Test ScoreEnd-of-year Achievement Test Score

Student Gender

Student Race/Ethnicity

Students' Family Poverty Status

Teacher Gender

Teacher Race/Ethnicity

Teacher Education Level

Mean

8.341.25.4

10.822.8

461.6456.9

MaleFemaleAsianAfncan-AmericanHispanicWhiteLow-incomeNon-low-incomeMaleFemaleAfrican-AmericanWhitePacific IslanderMulti-racialBachelor's DegreePost Bachelor's Training

SD

1.510.61.2

10.76.1

75.8574.15N

348295

715426

456200420

1720

132

13

1126

similar to those reported in factor analyses ofthe elementary version;

Emotional Support. The EmotionalSupport domain is composited from subscalesfor Positive Climate, reflecting warmth andsense of connectedness in classroom; NegativeClimate, reflecting expressed negativity inclassroom; Teacher Sensitivity, reflecting re-sponsiveness to student academic/emotionalneeds; and Regard for Adolescent Perspec-tives, reflecting the teacher's ability to recog-nize and capitalize on student needs for auton-omy, active roles, and peer interaction in theclassroom.

Classroom Organization. The Class-room Organization domain is compositedfrom subscales for Behavior Management, re-

flecting teacher ability to use effective meth-ods to encourage desirable behavior and pre-vent/redirect misbehavior; Productivity, re-flecting teacher ability to manage theclassroom so as to maximize instructionaltime; and Instructional Leaming Formats, re-flecting teacher use of varied and interestingmaterials and teaching techniques in an orga-nized fashion.

Instructional Support. The third do-main is Instructional Support, which is com-posited from subscales for Content Under-standing, reflecting teacher presentation ofcontent within a broader intellectual frame-work; Analysis and Problem Solving, reflect-ing emphasis upon engaging students in higherorder thinking skills; and Quality of Feedback,

81


reflecting provision of contingent feedback de-signed to challenge students and expand theirunderstanding of a concept.

Student achievement. Student aca-demic achievement was assessed using theStandards of Learning (SOL; Commonwealthof Virginia, 2005). The SOL is the state-man-dated accountability measure for the Com-monwealth of Virginia that is administered tomeet the requirements of No Child Left Be-hind It was first used in 1998, with a 7-yearperiod for schools to align their curricula andadjust to the testing requirement before theschool accreditation process began (Common-wealth of Virginia, 2005). The SOL programhas now been in place for almost a decade,making it one of the oldest such programs inthe nation. Students take SOL tests (whichconsist of between 45 and 63 multiple-choicequestions, depending upon the specific testused) at the end of the course in core subjectstaught by their teacher, and each is standard-ized on a 200-600 point scale, with 400 de-fined as a passing score with real-world impli-cations both for student graduation and schoolaccreditation. In terms of validity, the subjecttests have strong unidimensionality and corre-late between .50 and .80 with the Stanford 9achievement tests (Hambleton et al., 2000).In terms of reliability, high school assess-ments had KR-20 coefficients of .87 and .91(Hambleton et al., 2000).

External reviewers have also reviewedthe tests and found that the "reliability evi-dence for the SOL assessments is solid andtypical of high quality assessments" (Hamble-ton et al., 2000, p. 8). Although tests use thesame scales across subject matter, studentscores for a given test were adjusted usingstatewide normative data for each test to as-sure that scores could be fully equated acrossdifferent tests. We obtained each students'baseline SOL score, which was taken from asimilar course the year before and each stu-dents' end-of-year SOL score, which was di-rectly linked to the instructional content ofthe classrooms under examination (i.e., mathcourses were paired with prior year mathcourses, social studies courses were paired

with prior year social studies courses, and soon).

Student, teacher, and classroomcharacteristics. School records were used toidentify students' gender, race/ethnicity, andgrade level. Records also indicated whetherstudents came from low-income families(coded based on student eligibility for free andreduced-price lunch, which extends to familieswith incomes up to 185% of federal povertyline). Teachers reported on their gender, race/ethnicity, years of experience teaching, andeducation level. Class size was obtained fromteachers' enrollment rosters.

Procedures

Student SOL data were collected viastandardized end of course assessments bothfor the study year and for the year prior to thestudy. Classroom observation data were col-lected within a limited window between the4th and 8th weeks of the school year. Teacherswere asked to record a typical classroom ses-sion and coding was performed based on vid-eo-recording of this classroom interaction fora teacher, with the camera positioned to cap-ture both the teacher as well as a significantnumber of students. Teachers were told torecord a class session in which they wereactually teaching (as opposed to giving a test,watching a video, and so on). The window ofobservation at the beginning of the school yearwas deliberately selected to minimize the de-gree to which qualities of student-teacher in-teractions might be influenced by ongoing stu-dent engagement and achievement (as op-posed to predicting such achievement).

A team of advanced undergraduate andgraduate student coders were trained in a two-day workshop on the CLASS-S system. Cod-ers learned to rate each of the ten specificCLASS-S dimensions along a 1-7 scale, witha 1 or 2 indicating low quality; 3, 4, or 5indicating midrange quality; and 6 or 7 indi-cating high quality. At the end of the work-shop, each coder passed a reliability test, inwhich they scored within one point of themaster-coded tapes on 80% of scores, acrossfive video segments. The master coders had

82


extensive knowledge of the CLASS-S instru-ment. In addition, team members met regu-larly during the year to jointly code mastertapes in order to prevent drift and increasecoding agreement. The recorded classroomsession was divided into two 20-min segmentsand each segment was coded independently bytwo trained coders, with a different pair ofcoders used for each of the two segments.

Scores were averaged across raters. Re-liabilities between the two raters for domains,assessed via intraclass correlations, using Cic-chetti's (1981) norms for interpreting intra-class correlation coefficients, ranged fromgood (.73 for Instructional Support) to excel-lent (.77 and .82, for Emotional Support andClassroom Organization, respectively). Forthe specific dimensions, with the exception ofNegative Climate (intraclass correlation coef-ficient [ICC] = .50—in the "fair" range), allother dimensions also had ICC s in the good toexcellent range (ranging from .64 to .78).

Analytic Strategy

This study used a nested design thatincluded multiple students within each class-room. Initial analyses revealed a significantICC at the classroom level and related designeffect (.52 and 9.43, respectively). Classroomswere also nested within schools, althoughnumbers of schools and classrooms withinthem were too small to permit adequate mod-eling of school level effects. Hierarchical lin-ear modeling (Raudenbush & Bryk, 2002) wasthus used as the conceptual and analyticframework for specifying two-level modelsthat examined the association between mea-sures of classroom quality and individual-levelchild outcomes (end-of-year achievement testscores), after accounting for student gradelevel, gender, family poverty status, and class-room size. PROC Mixed in SAS, using re-stricted maximum likelihood estimation, wasused to specify the models derived from thefollowing equations (Singer, 1998). In the firstlevel of the two-level model (Equation 1), anend-of-year test score {Y) for a student (0 whois in classroom (/) is a function of the meanpost-test score for students in this class (ßq,)

after adjusting for pretest scores (ßm) anddemographic characteristics of students (ßo2-4)>and the error term associated with this esti-mated mean {r¡j).

Yij = ßo; + ßoi (pretest)

-I- ßo2 (student gender)

+ ßo3 (student grade level)

+ ßo4 (student poverty status) -I- ry ( 1 )

Equation 2 specifies in the second-levelmodel that the adjusted mean post-test scorefor students in each classroom (ßo ) is a func-tion of the grand mean post-test score (7oo)'classroom interaction quality ("Voi)' rid classsize (702) and the error term associated withthis estimated mean

ßoy = 7oo + 7oi (quality)

+ 7o2 (class size) + MQ,- (2)

Because all Level 1 coefficients for thecontrols were fixed, the only substitution is forßoy, resulting in Equation 3:

Yij = 7oj + 7oi (quality) + 702 (class size)

+ ßoi (pretest) + ßo2 (student gender)

-I- ßo3 (student grade level)

+ ßo4 (student poverty status)

+ '•y + "oy (3)

All Level 1 coefficients aside from ßoywere fixed, meaning that they were not al-lowed to vary across classes. For ease of in-terpretation, the outcome variable was stan-dardized at the grand mean for all analyses.

We conducted three sets of primaryanalyses. The first examined whether eachglobal domain of teacher-student interactionquality predicted student achievement, afterfirst taking into account student-level andclassroom-level demographic characteristics.

83


Second, we conducted a fine-grained analysisof each dimension within each global domainof teacher-student quality as earlier, enteringstudent and classroom demographic factorsfirst in the model. This identified which spe-cific dimensions were significant predictors ofachievement. Finally, we ran similar analysesto assess the extent to which certain observedqualities of student-teacher interactions wereactually predictable from students' prior levelsof achievement.

For each set of these primary analyses,we also tested to see whether the findingsobtained might be moderated by classroom orstudent characteristics. Equations 4 and 5 pro-vide examples of the models used for testingmoderation of the relation of CLASS-S scoresto achievement via variables at the classroomlevel (Equation 4) and at the student level(Equation 5). Equation 4 specifies adding aninteraction term to the second level model sothat the adjusted mean post-test score for stu-dents in each classroom (ßpy) is a function ofthe grand mean post-test score (700). class-room interaction quality (701). class size (702)'the interaction between both Level 2 maineffects (703), and the error term associatedwith this estimated mean

ßo; = 7oo + 7oi (quality) + 702 (class size)

+ 7o3 (quality X class size) + UQJ (4)

Equation 5 instead specifies a cross-level interaction between student poverty sta-tus (as one example) and teacher-student in-teraction quality by combining Equation 1 andLevel 2 equations, where ßq, = 7oo + 7oi(quality) + UQJ and ßj^ = 7IO + 7,, (quality)+ Uij to yield the following:

y¡j = 7oo + 710 (pretest)

+ 720 (student gender)

+ 730 (student grade level)

+ 740 (student poverty status) + 701 (quality)

84

+ 7,1 (student poverty status X quality)

+ {UQJ + Ua¡ (poverty status) + r^) (5)

Results

Preliminary Analyses

Means and standard deviations for allprimary variables of interest in the study arereported in Table 1. Table 2 presents a matrixof the bivariate correlations between the threeoverarching domains of Emotional Support,Classroom Organization, and InstructionalSupport along with the specific dimensionalscales for measures of teacher-student inter-action quality. Confirmatory factor analysis,using maximum likelihood analysis, supportedthe loading of scales onto the domains aspresented earlier, with all factors loadingabove 0.4 on the relevant factor (x^ = 39.60,p = .06, confirmatory factor analysis = 0.97,root mean square error of approxima-tion = 0.07; exact factor loadings are avail-able from the authors upon request). As ex-pected, prior year achievement test scoreswere substantially correlated with current yearachievement test scores (r =.69, p < .001).All analyses also accounted for a range ofpotentially confounding variables: contextualfactors, including student grade level, gender,and family poverty status, and classroom size.Potential moderating effects of these contex-tual factors were also examined (via interac-tion terms created from centered versions ofthese variables) and yielded no significant in-teractions, with the exception of a moderatingeffect of classroom size, discussed furtherlater.

Qualities of Teacher-Student ClassroomInteractions to Predictors StudentAchievement

Table 3 presents results of three hierar-chical linear modeling models, one for each ofthe three overarching domains assessingteacher-student interaction quality. In thesemodels, end-of-year student achievement testscores are predicted from baseline scores,from student grade in school, gender, and fam-


* * *•—' m o^r¡ v^ v^

IPi

uali

ui2

(Z)

&K

§

X

0U

•sB

Iu

cca

* * * I<N " —[~-- • * od

* * * \in en a•* in M

* * I *ON r-- 00in H r-

* * j *• ^ ON| vD

en \o

ON - H

NO - ^ e ni n NO NO

r-; q 00' t in en

en "in en

I

O es 00 t^vO t^ "

* *t^ —' esen in en

o 00 en oen eN ^ -^

* * * * * *en r- ON en eN c^i n • * • * - ^ • * i n

ON es ren m r-

* *CM r^ "^ enCM m en -H

* *— ON - ^NO CM NO

^ i n — 0 0 ND t—r- en 00 00 r-- vo — m es esr^ r-- ON 00

-^ *^ en in inoe) CM ONen en en

Ic

os•a

•M

g-

•s

s

S

. (Din "Oq I

I S85


Table 3Predicting End-of-year Achievement from Teacher-Student InteractionQuality (Covarying Prior Year Achievement and Demographic Factors)

Predicting End-of-year Achievement Scores From:

Emotional Support Classroom Organization Instructional Support

SE SE SE

InterceptBaseline Achievement ScoreStudent Characteristics

Student Grade-LevelStudent GenderStudent Family Poverty Status

Classroom SizeInteraction Quality Variable

31.30***.61***

5.401.90

-3.31-0.0631.30**

10.63.029

3.603.394.321.03

10.63

359.8***.61***

4.502.05

-3.23-.04721.65*

59.42.029

3.733.394.331.07

10.35

372.6***.61***

6.331.95

-3.00-0.5625.58**

46.64.029

3.773.394.331.04

10.13

' p < .001. ** p £ .01. * p < .05.

ily poverty status (at the child level), and fromclassroom size (at the classroom level), fol-lowed by the classroom interaction qualityvariable of interest. The domains of teacher-student interactions overlapped sufficientlythat when analyses examined all three simul-taneously as predictors, it was not possible toidentify significant unique predictive variancefrom any single domain, after accounting forthe others. Thus analyses were conducted sep-arately by domain.

Results indicate that each of the threedomains of teacher-student interaction werepredictive of higher student achievement testscores at the end of the year, even after cova-rying prior-year scores and relevant studentand classroom characteristics. Using the ap-proach suggested by Peugh (2010), we exam-ined the proportional reduction in classroomlevel variance obtained by using this compos-ite, relative to the classroom variance remain-ing after considering the effects of studentgrade, gender, and family poverty status, andclassroom size. Results indicated a propor-tional reduction in classroom-level variance instudent outcomes of 12.8%, 5.3%, and 8.9%,respectively, from the single assessments ofemotional, organizational, and instructionalsupport. For ease of general interpretation, we

examined effects not simply in terms of rawscore equivalents, but in percentile terms (ex-amining the change in percentile terms in testscores associated with a one standard devia-tion increment in an observed domain score).The magnitude of the strongest prediction,from the Emotional Support domain, indicatesthat, after accounting for other measured vari-ables, a student entering with average priortest scores (i.e., 50th percentile) in a class thatwas one standard deviation below the meanin Emotional Support would on average placein the 41st percentile in end-of-year tests;whereas an average student with the samebackground characteristics in a class that wasone standard deviation above the mean inEmotional Support would on average place inthe 59th percentile in end-of-year tests.

Class size interacted with EmotionalSupport {B = -4.81, SE = 2.00, p = .02)and with Instructional Support {B = —3.54,SE = 1.78, p = .046). These interactions aredepicted in Figure 2 using standardized scoresfor all variables and plotting lines for class-rooms that were one standard deviation aboveand below the mean in class size (reflectingclass sizes of approximately 29 and 17, re-spectively). Figure 2 shows that measuredEmotional and Instructional Support in the

86


Interaction of Class Size and Emotional Support in ClassroomPredicting Relative Achievement Gains Over Time

[ïain

s\c

hiev

emen

t <R

elat

ive

i

0.60.50.40.30.20.1

0-0.1 --0.2-0.3 --0.4

Smaller >*Class S i z e v ^

"" J' Larger^r Class Size

-1 0 1Classroom Emotional Support

Interaction of Class Size and Instmctional Support inClassroom Predicting Relative Achievement Gains Over Time

ins

A

a

Lchi

evem

ent

•«t ;

iativ

Rel

0.60.5

0.4

0.3

0.2

0.1

0

-0.1

-0.2

-0.3

SmallerClass Size

LargerClass Size

- 1 0 1Classroom Instructional Support

Figure 2. Interaction of class size and emotional support in classroom predict-ing future student achievement (controlling for baseline achievement)

classroom was of greatest predictive value forstudent academic achievement in smaller ascompared to larger classrooms.

No moderator effects of course contentarea were found, which indicates that relationsbetween past and current test scores were not

87


significantly different for math/science versusEnglish/social studies courses and that resultsdid not significantly differ depending on thetype of course being taught. (There were alsono differences in the predictive power of prioryears' test scores for math versus sciencecourses or English versus social studiescourses, indicating it was appropriate to col-lapse into these two broad categories for ana-lytic purposes).

Specific Effects for Teacher-StudentInteractions

For descriptive purposes, analyses nextfollowed up on the findings regarding the threeglobal domains of teacher-student interactionin finer grained detail, by assessing the spe-cific dimensions that were coded and compos-ited into each construct. As with the domainscores discussed earlier, the specific dimen-sions overlapped sufficiently that when analy-ses examined them simultaneously as predic-tors, it was not possible to identify significantunique predictive variance from any singledimension, after accounting for the others.These analyses were conducted separately foreach dimension and should be interpreted aspost hoc follow-up tests to the overall resultsnoted earlier demonstrating significance at thedomain level. Results are presented in Table 4These results indicate significant predictionsof achievement from observed positive cli-mate, teacher sensitivity, and regard for ado-lescent perspectives in the Emotional Supportdomain, instructional learning formats in theClassroom Organization domain, and analysisand problem solving in the Instructional Sup-port domain.

Classroom Qualities as Predicted byPrior Student Levels of Achievement

Finally, analyses sought to determinethe extent to which measures of observedteacher-student interactions could be pre-dicted from levels of student achievementprior to entering the class. Such predictionscan identify which aspects of teacher-studentinteraction may be influenced heavily by stu-dent entry characteristics. These results are

presented in Table 5. Higher quality behaviormanagement, instructional learning formats,content understanding, and quality of feed-back were significantly predicted by students'baseline achievement test scores, along withthe domain-level scales for Classroom Orga-nization and Instructional Support. However,no effects of baseline levels of studentachievement on Emotional Support were ob-served.

Post Hoc Analyses

To see whether one might utilize thesefindings to construct a measure of observedteacher-student interactions that would opti-mize prediction (at least in this somewhatsmall sample) on a post hoc basis, we nextcreated a composite comprised solely of thefive individual dimensions of observed inter-action that were significantly predictive ofend-of-year test scores: positive climate,teacher sensitivity, regard for adolescent per-spectives, instructional learning formats, andanalysis and problem solving. When consid-ered in models identical to those in Table 3,this composite yielded a significant B of 30.2(SE = 8.91, p < .001). Results indicateda 16.3% reduction in unexplained variance atthe classroom level from this composite mea-sure, even after accounting for the other cova-riates in the model. Alternatively, the effectsize of this association is equivalent to a find-ing that a student entering with average testscores (i.e., 50th percentile) in a classroomone standard deviation below the mean on thiscomposite reflective of overall quality ofteacher-student interactions would on averageplace in the 37th percentile in end-of-yeartests; whereas a student entering with averagetest scores in a class that was one standarddeviation above the mean on this scale wouldon average place in the 63rd percentile onend-of-year tests.

Discussion

This study identified specific features ofteacher-student interactions in the classroom,observed using standardized measurements,that were directly linked to student achieve-


0U

CO

¿^

2 w

-fi %

o,o.D

g

8

iio-àW00c

oa«5

U

£

^ I . S

Ö0csi§ i "

u -S

ao.3

(/2

O

W

|CQ

III

11 "g

(Su

* oo * ^^ en en O\en O ON en

* —en .en r;

ON

00

en

Í Í 00 ON2 ON r~;^ en en

O\ O ^ " ' " ^ 1-H • ' " ^

°^ ^ -^ es

•aBoa¿.

ali

s

00 vo —' OS — en00 r^ p en • en

• en en es en r Tf

c^Ttri

en en -H en

** S *NO *

- VO -H8 en

*

- i n en —< en-^ en—' vo

00

enNO

I dT2

Í- p

^ Oë s s

00 00 es ON" (^r- ^ P ^ oi•^ m cN n^ I — en

- ^ (N

s.

-à&I

pVQ.

pVIQ.

Va.

89


CA

1

í

2

'>3

SíCo

g

2

ta cM 5

£ ö

5-

NO

NOes

ON

*00oo0¿(N

* ^So "0O N —ON C -

» V)ON O

CO ^• ^ i n

NO —XT) - ^

m „ „

t—o

i

C~ ON Í- '

qV

VIo.

n^ OO esil, ^ O

I 2o S» s-g 3 .32 3u o

3 a¿3 C/3

90


ment over the course of an academic year,even after accounting for prior levels of stu-dent achievement, demographic characteris-tics, and classroom size. Moreover, these find-ings applied regardless of student grade levelsand content area of instruction; in short, "goodteaching" was good regardless of content orgrade level. These results have implicationsnot only for theories regarding the role andvalue of interactions and proximal processesin social settings serving youth (Tseng & Seid-man, 2007), but also for contemporary policiesand practices related to the assessment andimprovement of teacher performance (Weis-berg. Sexton, Mulhem, & Keeling, 2009).

These findings are consistent with theo-retically driven expectations regarding themechanisms through which secondary schoolinstruction affects adolescents' leaming, asoperationalized via the CLASS-S observa-tions. These results suggest that the "value-added" of classroom settings may in part beattributable to qualities of teacher-student in-teractions. These results also extend similarfindings obtained in prekindergarten and ele-mentary school settings, in which observa-tional approaches using the elementary ver-sion of the CLASS have been used to predictmeasures of student leaming (e.g., Hamre &Pianta, 2005; Rimm-Kaufman et al., 2009).Collectively, these studies provide empiricalsupport for the argument that critical teacherbehaviors can be measured in a standardizedobservational assessment.

Although studies of student achieve-ment have been important in laying a founda-tion for inquiry into classroom effects (Ladd,2008; Nye, Konstantopoulos, & Hedges,2004; Rivkin, Hanushek, & Kain, 2005), theyhad not yet succeeded in identifying specificprocesses that may lead to student learningand positive social adjustment across an arrayof content areas. As a result, the field hasthus far been left to rely largely upon an atheo-retical, post hoc approach as refiected inHanushek's (2002) definition of teacher qual-ity: "Good teachers are ones who get largegains in student achievement for their classes;bad teachers are just the opposite" (p. 3). Thisdefinition, although apt as far as it goes.

provides only limited guidance to efforts toproduce effective teaching (Cochran-Smith &Zeichner, 2005). The present study extendsour understanding of the connection betweenspecific observed teacher-student interactionsand student achievement into secondaryschool classrooms and supports the theoreticalproposition that properties of these interac-tions have value for student leaming and de-velopment regardless of the content beingtaught.

In the present study, aspects of teacher-student interactions refiecting instmctionalsupport and classroom organization were pre-dicted by the baseline level of achievement ofstudents in a given class (although these stillwent on to predict achievement after account-ing for these baseline levels). In contrast, how-ever, features of teachers' emotional supportappeared independent of students' baselinelevels of achievement. These findings haveimplications for the observation of instmction(i.e., teacher oversight) in contexts in whichvalue-added achievement test analyses cannotbe readily conducted. These findings suggestthat observed levels of instructional supportand classroom organization, although impor-tant in predicting future student achievement,are likely to reflect both teacher skill and stu-dent background. Thus, to the extent studentsare implicitly or explicitly grouped or trackedinto higher achieving and lower achievingclasses, it would be important not to assumethat observed qualities of instructional supportand classroom organization were solely a re-sult of teacher skills or efforts. Observed emo-tional support, in contrast, also predicts futurestudent achievement after accounting for base-line achievement but appears relatively inde-pendent of student background characteristics,and thus may be more likely to be determinedby individual teacher qualities. Notably, thereis evidence now from experimental studiesthat teachers' behavior in all three CLASSdomains can be improved with teacher train-ing that targets these interactive behaviorsvia training either through college courses(Hamre et al., 2010) or through ongoing, job-embedded consultation (Allen, Pianta, Greg-ory, Mikami, & Lun, 2011; Brown, Jones,

91


LaRusso, & Aber, 2010; Pianta, Belsky et al.,2008; Raver et al., 2008).

With regard to emotional support, at-tachment theorists posit that when adults pro-vide emotional support and respond contin-gently, children develop self-reliance and arebetter able to explore (Ainsworth, Blehar,Waters, & Wall, 1978; Bowlby, 1969/1982),a premise that has been repeatedly validatedin school environments (Birch & Ladd, 1998;Hamre et al., 2010; Howes, Hamilton, &Matheson, 1994). Self-determination theory(Connell & Wellborn, 1991; Skinner & Bel-mont, 1993) posits that motivation to learn isin part related to adults' support for compe-tence, relationships, and autonomy (Roeser,Eccles, & Sameroff, 1998). Instructional inter-actions have been put in the spotlight in recentyears as more emphasis has been placed on thetranslation of cognitive science, leaming, anddevelopmental research to educational envi-ronments (Bransford, Brown, & Cocking,1999; Carver & Klahr, 2001; National Re-search Council, 2005). Teachers who usestrategies that focus students on higher orderthinking skills, give consistent, timely, andprocess-oriented feedback, and work to extendstudents language skills, tend to have studentswho achieve more academically (Hamre &Pianta, 2005; Justice, Meier, & Walpole,2005; Meehan, Hughes, & Cavell, 2003; Tay-lor, Pearson, Peterson, & Rodriguez, 2003).

Emotional support may be particularlyimportant in instructional settings serving ad-olescents. Unlike in elementary education, inwhich most students can be expected to have adisposition toward seeking to please teachersand comply with authority, engaging adoles-cent students emotionally may be critical tomaximizing their academic motivation in theclassroom. Further, as adolescents seek greaterautonomy with respect to parents, havingother settings in which they can receive emo-tional support from adults may provide a pow-erful motivation to engage in those settings(National Research Council, 2004).

That adolescents achieve at higher lev-els across a range of content areas in thecontext of more positive classroom interac-tions, although not yet well documented in the

educational literature, is actually quite consis-tent with developmental theory. In contrast toassessments targeted at specific content areas(e.g., seventh-grade math instruction), this as-sessment broadly targeted instruction acrossthe full range of core secondeiry instructionalcontent areas. Moderator analyses provided noevidence that predictions differed in magni-tude across different content areas, suggestingthat the generality of the approach to assessinginstruction across widely different content ar-eas was successful in this instance.

Follow-up descriptive analyses shedlight on specific dimensions of teacher-stu-dent interaction that were most predictive ofstudent outcomes. These analyses were explic-itly descriptive and exploratory in nature;however, of necessity, the dimensions exam-ined contained a degree of redundancy, bothwith the larger domains examined, but alsowith the other dimensions observed. Althoughwe present them in terms of conventional sig-nificance levels, we recommend they be inter-preted with caution, requiring further replica-tion. In the Emotional Support domain, teach-ers' ability to establish a positive emotionalclimate (Positive Climate), their sensitivity tostudent needs (Teacher Sensitivity), and theirstructuring of their classroom and lessons inways that recognize adolescents' needs for asense of autonomy and control, for an activerole in their leaming, and for opportunities forpeer interaction (Regard for Adolescent Per-spectives) were all associated with higher rel-ative student achievement, after covaryingbaseline levels of such achievement. Simi-larly, use of instructional leaming formats thatencouraged active participation by studentsand that provided variety in classroom ap-proaches (Instmctional Leaming Formats) wasalso predictive of student achievement, aswere lessons that required high levels of anal-ysis and problem solving by students (Analy-sis and Problem Solving).

Overall, the particular constellation ofinteractions that was most linked to futureachievement seemed to focus upon tailoring aclassroom experience to be maximally emo-tionally and intellectually engaging to the ad-olescent. In post hoc analyses, this constella-

92


tion could account for a difference in studentachievement test performance spanning the37th-63rd percentiles, a magnitude that hasconsiderable ramifications for teachers, class-rooms, and schools in terms of accountability.Stated differently, on the particular tests used,the swing in scores reflected in moving froman average classroom to one that was onestandard deviation above the mean score inoverall teacher-student interaction qualitywould be sufficient in the abstract to reducefailure rates on these high-stakes tests from17% to 11%.

Classroom observations have been usedas measurement tools in educational researchfor more than three decades (Gage & Needels,1989). In most studies using observationalmethods, approaches typically focused on spe-cific teacher pedagogical behaviors. The pres-ent study results and other related findings(Pianta, Belsky et al., 2008) demonstrate thepredictive utility of a focus on interactions,given the fact that they can be assessed insecondary classrooms via 40 min of video, andthat global ratings of interactions, as theydemonstrate patterns of behavior and responseover the segment, can be coded reliably andaccount in some part for the value of thatclassroom setting for student learning. Nota-bly, two of the domains of teacher-studentinteraction quality that were assessed, emo-tional and instructional support, were morestrongly related to achievement in smaller asopposed to larger classrooms. One explanationfor these findings is that qualities such assensitivity to student needs or provision ofhigh-quality feedback to students might havethe greatest effect when they are concentratedamong relatively fewer students. Conversely,the effect of these factors might be relativelydiluted in very large classrooms.

The particular methods for observationused in this study also warrant discussion asthey have implications for the use of standard-ized observations as measures of teacher per-formance in states and in districts. Resultswere obtained based on observations of just40 min of a single classroom session early inthe school year using global, judgment-basedstandardized ratings of teacher-student inter-

action. Under these conditions, the CLASS-Sis likely capturing only a modest portion of thetrue variance in the quality of teacher-studentinteractions that exists across an entire year(Mashbum, Downer, Rivers, Brackett, & Mar-tinez, 2013). Thus, the estimates of the effectsof classroom interaction quality in this study,although substantial, should be taken not onlyas a likely minimum estimate of the effects ofthe specific qualities observed, but also as anindication that stable and reliable between-teacher differences in interaction can be ob-served in a fairly efficient manner.

At the same time, it should also be ac-knowledged that this was not an experimentalstudy, and that even longitudinal analyses ofend-of-year achievement controlling for base-line levels are not logically sufficient to dem-onstrate causal relations. It could be, for ex-ample, that other unmeasured factors werepromoting both student achievement and high-quality instruction in the classroom. Even ifthis were the case, however, the results of thisstudy would still indicate that the domainsassessed were sensitive proxies of importantclassroom processes related to studentachievement. Also, as achievement data couldnot be obtained on 24% of students in class-rooms (because of a lack of parental consent),it remains possible that such additional datamight have altered study findings, although itis not clear whether they would have biasedfindings in any particular direction. Teacherswere also asked to select one of their morechallenging classes to teach for this study, andthus results should properly be generalized tosuch classrooms. It is also possible that biasexisted in which classroom session teachersselected for video recording, thus introducingunmeasured bias into the assessments ob-tained. Finally, it was not possible to assesseffects of classroom nesting at the schoollevel, which introduces an additional potentialsource of unmeasured bias into the results.

In sum, it is striking that given the im-portance of classroom settings as vehicles forthe transmission of knowledge and skill in oursystem of education, that little-to-no popula-tion level data exist pertaining to adolescents'exposure to specific practices in the classroom

93


that are (a) known to relate to academic suc-cess or failure, (b) desired on the basis ofcertain policies or values, or (c) even hypo-thetically expected to relate to outcomes. Insecondary classrooms there is no current workthat provides national-level, observationaldata on these environments. The Measures ofEffective Teaching Study (Bill and MelindaGates Foundation, 2010) will likely produce anumber of important findings in this regard incoming years. The MET study includes obser-vations of over 3,000 teachers in Grades 4-9and is using several observational protocols.Several, like CLASS-S, focused on global in-structional process, and some focused specif-ically on content. We argue that designingstandardized observation protocols into cur-rent value-added state-standards tests andlarge-scale student assessments could leveragenot only our understanding of classroom ef-fects (Early et al., 2005; National Institute ofChild Health and Human Development EarlyChild Care Research Network, 2005; Pianta etal, 2005; Pianta, La Paro, & Hamre, 2008) butalso the capacity to systematically produceeffective teaching.

Potential Implications for Practitioners

In the midst of concem about standard-ized tests, block versus regular class schedul-ing, school size and budgets, this study em-phasizes the fundamental importance of theemotional quality of the classroom as a key toadolescent leaming. Although we can think ofthem as "leamers," adolescents are first andforemost highly social and emotional beings.The role of school psychologists and admin-istrators in both supporting this recognitionand in helping teachers understand and enact itis potentially paramount. The elements of apositive classroom climate—teacher sensitiv-ity to adolescents' needs, and recognition oftheir desire for peer interaction and for a senseof autonomy regarding classroom activities—are not simply the niceties in a functionalclassroom, they are key predictors of adoles-cent leaming, even regarding otherwise "dry"subjects such as algebra or geometry. An ad-vantage of the CLASS-S is that it provides

highly specific markers of these qualities,which can be used not only by those evaluat-ing teacher-student interactions, but also byteachers seeking to enhance them. For exam-ple, teachers may enhance the positive climateof a class by laughing with students, engagingthem as they enter the room (vs. sitting behinda desk at the front of the room), and askingabout events outside of class at the beginningand end of a lesson period. They may givestudents choices (within a prespecified range)of ways of approaching leaming assignments.They may use varied instructional leamingformats—moving beyond lecturing to alargely silent class, for example, to askingstudents to work briefly in small teams, pres-ent ideas to the class, and so on—and focus onstudent analysis and problem solving by ask-ing "why" questions not simply factual ques-tions. All of these elements were part of theratings of interactions linked to greater studentleaming in this study.

Equally important, for those chargedwith evaluating teachers, this study identifiedqualities of teacher-student interactions thatmay appear to be linked to quality teachingbecause they occur more in high-achievingclassrooms, but which appear primarily to re-flect qualities of students entering the class-room than to predict future achievement gainsof those students. The quality of behaviormanagement in a classroom is associated withstudent achievement, for example; however,the association is primarily to preexisting lev-els of student achievement rather than to fu-ture gains in achievement. This is not to saythat behavior management is unimportant, butrather to suggest that great care be used inassessing teachers on this quality unless onealso accounts for the qualities of the studentsbeing taught.

Perhaps the most important reason toconduct observational assessment of class-rooms is for the purposes of professional de-velopment. Professional development typi-cally occurs in the absence of a direct link toactual teaching behavior in classrooms, partic-ularly for already-trained and certified teach-ers (Caspary, 2002). Systematic classroom ob-servation systems provide a standardized ap-

94


proach to measuring and noting teachers'strengths and weaknesses and evaluatingwhether professional development activitiesare actually helping improve the classroominteractions responsible for learning. The cur-rent study suggests that professional develop-ment focused specifically on teachers emo-tional, organizational, and instructional inter-actions with students may enhance teachereffectiveness in ways that have a direct effecton student learning (Mashbum, Downer,Hamre, Justice, & Pianta, 2010), while clearlyraising important questions for future researchto consider.

References

Ainsworth, M. D. S., Blehar, M. C, Waters, E., & Wall,S. (1978). Patterns of attachment: A psychologicalstudy of the strange situation. Hillsdale, NJ: Erlbaum.

Allen, J. P., Kuperminc, G., Philliber, S., & Herre, K.(1994). Programmatic prevention of adolescent prob-lem behaviors: The role of autonomy, relatedness, andvolunteer service in the Teen Outreach Program. Amer-ican Journal of Community Psychology, 22, 617-638.

Allen, J. P., Pianta, R. C, Gregory, A., Mikami, A. Y., &Lun, J. (2011). An interaction-based approach to en-hancing secondary school instruction and studentachievement. Science, 333(6045), 1034-1037.

Bell, K. L., Allen, J. P., Häuser, S. T., & O'Connor, T. G.(1996). Family factors and young adult transitions:Educational attainment and occupational prestige. InJ. A. Graber & J. Brooks-Gunn (Eds.), Transitionsthrough adolescence: Interpersonal domains and con-text (pp. 345-366). Mahwah, NJ: Erlbaum.

Bill and Melinda Gates Foundation. (2010). Workingwith teachers to develop fair and reliable measuresof effective teaching. Available online at http://www.metproject.org/downloads/met-framing-paper.pdf

Birch, S. H., & Ladd, G. W. (1998). Children's interper-sonal behaviors and the teacher-child relationship. De-velopmental Psychology, 34, 934-946.

Blair, C. (2002). School readiness: Integrating cognitionand emotion in a neurobiological conceptualization ofchildren's functioning at school entry. American Psy-chologist, 57, 111-127.

Bogner, K., Raphael, K. M., & Pressley, M. (2002). HowGrade-1 teachers motivate literate activity by theirstudents. Scientific Studies of Reading, 6, 135-165.

Bowlby, J. (1969/1982). Attachment and loss. Vol. 1. NewYork: Basic Books.

Bransford, J. D.. Brown, A. L., & Cocking, R. R. (1999).How people learn: Brain, mind, experience, andschool. Washington, DC: National Academy Press.

Brophy, J. (1999). Teaching (Vol. Educational PracticesSeries—1). Geneva, Switzerland: International Acad-emy of Education and International Bureau of Educa-tion.

Brophy, J., & Good, T. L. (1986). Teacher behavior andstudent achievement. Handbooic of Research on Teach-ing, 3, 328-375.

Brown, J. L., Jones, S. M., LaRusso, M. D., & Aber, J. L.(2010). Improving classroom quality: Teacher intiu-ences and experimental impacts of the 4Rs Program.Journal of Educational Psychology, 102, 153-167.

Burchinal, M., Howes, C, Pianta, R. C , Bryant, D., Early,D., Clifford, R., & Barbarin, 0. (2008). Predictingchild outcomes at the end of kindergarten from thequality of pre-kindergarten teacher-child interactionsand instruction. Applied Developmental Science, 12,140-153.

Cameron, C. E., Connor, C. M., & Morrison, F. J. (2005).Effects of variation in teacher organization on class-room functioning. Journal of School Psychology, 43,61-85.

Carver, S. M., & Klahr, D. (2001). Cognition and instruc-tion: 25 years of progress. Mahwah, NJ: Erlbaum.

Caspary, K. (2002). California's pioneering training andretention initiative for early childhood educators.Berkeley, CA: PACE.

Cicchetti, D. V. (1981). Testing the normal approximationand minimal sample size requirements of weightedkappa when the number of categories is large. AppliedPsychological Measurement, 5(1), 101-104.

Cochran-Smith, M., & Zeichner, K. (2005). Studyingteacher education: The report of the AERA panel onresearch and teacher education. Mahway, NJ: Erl-baum.

Commonwealth of Virginia. (2005). The Virginia Stan-dards of Learning: Technical report. Richmond, VA:Author.

Connell, J. P., & Wellborn, J. G. (1991). Competence,autonomy, and relatedness: A motivational analysisof self-system processes. In M. Gunnar & L. A. Sroufe(Eds.), Self processes in development: Minnesota sym-posium on child psychology (Vol. 23, pp. 43-77).Hillsdale, NJ: Erlbaum.

Early, D., Barbarin, 0., Bryant, B., Burchinal, M., Chang,F., Chfford, R., Crawford, G., Howes, C, Sharon, R.,Kraft-Sayre, M. E., Pianta, R. C , Barnett, S. W., &Weaver, W. (2005). Pre-iàndergarten in eleven states:NCEDL's multi-state study of pre-kindergarten andstate-wide early educational programs (SWEEP)study. Washington, DC: National Institute for EarlyEducation Research.

Eccles, J. S., & Roeser, R. W. (1999). School and com-munity influences on human development. In M. H.Bomstein & M. E. Lamb (Eds.), Developmental psy-chology: An advanced textbooic (4th ed.) (pp. 503—554). Mahwah, NJ: Erlbaum.

Emmer, E. T., & Stough, L. M. (2001). Classroom man-agement: A critical part of educational psychology,with implications for teacher education. EducationalPsychologist, 36, 103-112.

Gage, N. L., & Needels, M. C. (1989). Process-productresearch on teaching: A review of criticisms. TheElementary School Journal, 89, 253-300.

Greenwood, C. R., Carta, J. J., & Dawson, H. (2000).Ecobehavioral Assessment Systems Software(EBASS): A system for observation in education set-tings. In T. Thompson, D. Felce, & F. J. Symons(Eds.), Behavioral observation: Technology and appli-cations in developmental disabilities (pp. 229-251).Baltimore, MD: Paul H. Brookes.

Greenwood, C. R., Carta, J. J., Kamps, D., & Terry, B.(1994). Development and validation of standardclassroom observation systems for school practitio-

95


ners: Ecobehavioral Assessment Systems Software(EBASS). Exceptional Children, 61, 197-210.

Hambleton, R. K., Crocker, L., Cruse, K., Dodd, B.,Plake, B. S., & Poggio, J. (2000). Review of selectedtechnical characteristics of the Virginia Standard ofLeaming (SOL) assessments. Richmond, VA: Com-monwealth of Virginia Department of Education.

Hamre, B. K., & Pianta, R. C. (2005). Can instructionaland emotional support in the first-grade classroommake a difference for children at risk of school failure?Child Development, 76, 949-967.

Hamre, B. K., & Pianta, R. C. (2010). Classroom envi-ronments and developmental processes: Conceptual-ization & measurement. In J. Meece & J. Eccles (Eds.),Handbook of research on schools, schooling and hu-man development (pp. 25-41). New York, NY: Rout-

Hamre, B. K., Pianta, R. C, Burchinal, M., & Downer,J. T. (2010). A course on supporting early languageand literacy development through effective teacher-child interactions: Effects on teacher beliefs, knowl-edge and practice. Paper presented at the annual meet-ing of the Society for Research on Educational Effec-tiveness, Washington, DC.

Hanushek, E. A. (2002). 77!« long run importance ofschool equality. Cambridge, MA: National Bureau ofEconomic Research.

Hanushek, E. A., Rivkin, S. G., Figlio, D., & Jacob, B.(2010). Generalizations about using value-added mea-sures of teacher quality. American Economic Review,100, 261-21 \.

Howes, C , Burchinal, M., Pianta, R., Bryant, D., Early,D., Clifford, R., & Barbarin, 0. (2008). Ready to learn:Children's pre-academic achievement in pre-kinder-garten programs. Early Childhood Research Quarterly,2J(1), 27-50.

Howes, C, Hamilton, C. E., & Matheson, C. C. (1994).Children's relationships with peers: Differential asso-ciations with aspects of the teacher-child relationship.Child Development, 65, 253-263.

Johnson, C. C , Kahle, J. B., & Eargo, J. D. (2007).Effective teaching results in increased science achieve-ment for all students. Science Education, 91, 371-383.

Justice, L. M., Meier, J., & Walpole, S. (2005). Learningnew words from storybooks: Findings from an inter-vention with at-risk kindergarteners. Language, Speech,and Hearing Services in Schools, 36, 17-32.

Ladd, G. W., Birch, S. H., & Buhs, E. S. (1999). Chil-dren's social and scholastic lives in kindergarten:Related spheres of influence? Child Development, 70,1373-1400.

Ladd, H. (2008). Teacher effects: What do we know. Paperpresented at the Teacher Quality Conference, North-western University, Chicago, IL.

Landry, S. H., Swank, P. R., Smith, K. E., Assel, M. A.,& Gunnewig, S. B. (2006). Enhancing early literacyskills for preschool children. Journal of Learning Dis-abilities, 39, 306.

Marton, F., & Saljo, R. (1976). On qualitative differencesin leaming: I. Outcome and process. British Journal ofEducational Psychology, 46, 4-11.

Mashburn, A. J.. Downer, J. T., Hamre, B. K., Justice, L.,& Pianta, R. C. (in press). Consultation for teachersand children's language and literacy development dur-ing pre-kindergarten. Applied Developmental Psychol-ogy-

Mashburn, A. J., Downer, J. T., Rivers, S. E., Brackett,M. A., & Martinez, A. (2013). Improving the power ofan experimental study of a social and emotional leam-ing program: Application of generalizability theory tothe measurement of classroom-level outcomes. Preven-tion Science.

McCaslin, M., Burross, H. L., & Good, T. L. (2005).Change and continuity in student achievement fromGrades 3 to 5: A policy dilemma. Education PolicyAnalysis Archives, 13{\).

Meehan, B. T., Hughes, J. N., & Cavell, T. A. (2003).Teacher-student relationships as compensatory re-sources for aggressive children. Child Develop-ment, 74, 1145-1157.

National Research Council. (2004). Engaging schools:Fostering high school students' motivation to learn.Washington, DC: National Academies Press.

National Research Council. (2005). How students team:History, mathematics, and science in the classroom.Washington, DC: National Academies Press.

National Institute of Child Health and Human Develop-ment Early Child Care Research Network. (2005). Aday in third grade: A large- scale study of classroomquality and teacher and student behavior. The Elemen-tary School Journal, 105, 305-232.

Nye, B., Konstantopoulos, S., & Hedges, L. V. (2004).How large are teacher effects? Educational Evaluationand Policy Analysis, 26, 237-257.

Paris, S. G., & Paris, A. H. (2001). Classroom applicationsof research on self-regulated learning. EducationalPsychologist, 36(2), 89-101.

Peugh, J. L. (2010). A practical guide to multilevel mod-eling. Journal of School Psychology, 48, 85-112.

Pianta, R., Howes, C, Burchinal, M., Bryant, D., Clifford,R., Early, D., & Barbarin, O. (2005). Features of pre-kindergarten programs, classrooms, and teachers: Dothey predict observed classroom quality and child-teacher interactions? Applied Developmental Sci-ence, 9, 144-159.

Pianta, R. C. (1999). Enhancing relationships betweenchildren and teachers. Washington, DC: AmericanPsychological Association.

Pianta, R. C, Belsky, J., Vandergrift, N., Houts, R., &Morrison, F. J. (2008). Classroom effects on children'sachievement trajectories in elementary school. Ameri-can Educational Research Journal, 45, 365-397.

Pianta, R. C, & Hamre, B. K. (2009). Conceptualization,measurement, and improvement of classroom process-es: Standardized observation can leverage capacity.Educational Researcher, 38, 109-119.

Pianta, R. C , Hamre, B. K., Hayes, N., Mintz, S., &LaParo, K. M. (2008). Classroom assessment scoringsystem—Secondary (CLASS-S). Charlottesville, VA:University of Virginia.

Pianta, R. C, La Paro, K. M.. & Hamre, B. K. (2008).Classroom Assessment Scoring System (CLASS): Man-ual—Pre-K. Baltimore, MA: Brookes Publishing.

Pressley, M., Roehrig, A., Raphael, L., Dolezal, S., Bohn,C, Mohan, L., . . . Hogan, K. (2003). Teaching pro-cesses in elementary and secondary education. InW. Reynolds & G. Miller (Eds.), Handbook of psy-chology: Educational psychology (Vol. 7, pp. 153-176). Hoboken, NJ: John Wiley & Sons.

Raudenbush, S. W., & Bryk, A. (2002). Hierarchicallinear models: Applications and data analysis meth-ods. Thousand Oaks, CA: Sage Publications.

96


Raver, C. C. (2004). Placing emotional self-regulation insociocultural and socioeconomic contexts. Child De-velopment, 75, 346-353.

Raver, C. C, Jones, A. S., Li-Grining, C. P., Metzger, M.,Smallwood, K., & Sardin, L. (2008). Improving pre-school classroom processes: Preliminary findings froma randomized trial implemented in Head Start settings.Early Childhood Research Quarterly, 25(1), 10-26.

Rimm-Kaufman, S. E., Curby, T. W., Grimm, K. J.,Nathanson, L., & Brock, L. L. (2009). The contributionof children's self-regulation and classroom quality tochildren's adaptive behaviors in the kindergarten class-room. Developmental Psychology, 45, 958-972.

Rivkin, S. G., Hanushek, E. A., & Kain, J. F. (2005).Teachers, schools, and academic achievement. Econo-metrica, 73, 417-458.

Roeser, R. W., Eccles, J. S., & Sameroff, A. J. (1998).Academic and emotional functioning in early adoles-cence: Longitudinal relations, patterns, and predictionby experience in middle school. Development & Psy-chopathology, 10, 321-352.

Roth, K. J., Dmker, S. L., Gamier, H. E., Lemmens, M.,Chen, C, Kawanaka, T., . . . Okamoto, Y. (2006J.Teaching science in five countries: Results from theTIMSS 1999 Video Study (NCES 2006-011). U.S.Department of Education, National Center for Educa-tion Statistics. Washington, DC: U.S. GovernmentPrinting Office.

Rothstein, J. (2010). Teacher quality in educational pro-duction: Tracking, decay, and student achievement.The Quarterly Joumal of Economics, 725(1), 175-214.

Seidel, T., & Shavelson, R. J. (2007). Teaching effective-ness research in the past decade: The role of theory andresearch design in disentangling meta-analysis results.Review of Educational Research, 77, 454-499.

Shinn, M., & Yoshikawa, H. (2008). Toward positiveyouth development: Transforming schools and commu-nity programs. New York, NY: Oxford UniversityPress.

Singer, J. D. (1998). Using SAS PROC MIXED to fitmultilevel models, hierarchical models, and individualgrowth models. Joumal of Educational & BehavioralStatistics, 23, 323-355.

Skinner, E. A., & Belmont, M. J. (1993). Motivation in theclassroom: Reciprocal effects of teacher behavior andstudent engagement across the school year. Journal ofEducational Psychology, 85, 571-581.

Soar, R., & Soar, R. (1979). Emotional climate and man-agement. In P. Peterson & H. Walberg (Eds.), Re-search on teaching: Concepts, findings, and implica-tions (pp. 97-119). Berkeley, CA: McCutchan.

Sroufe, L. A., Egeland, B., Carlson, E. A., & Collins,W. A. (2005). The development of the person: TheMinnesota Study of Risk and Adaptation from Birth toAdulthood New York, NY: Guilford.

Taylor, B. M., Pearson, P. D., Peterson, D. S., & Rodri-guez, M. C. (2003). Reading growth in high-povertyclassrooms: The infiuence of teacher practices thatencourage cognitive engagement in literacy leaming.The Elementary School Journal, 704(2), 3-28.

Tseng, V., & Seidman, E. (2007). A systems frameworkfor understanding social settings. American Joumal ofCommunity Psychology, 39, 217-228.

Wallace, T., Anderson, A. R., Bartholomay, T., & Hupp,S. (2002). An ecobehavioral examination of highschool classrooms that include students with disabili-ties. Exceptional Children, 68, 345-359.

Watson, S. M. R., Gable, R. A., & Greenwood, C. R.(2011). Combining ecobehavioral assessment, func-tional assessment, and response to intervention to pro-mote more effective classroom instruction. Remedialand Special Education, 32, 334-344.

Weisberg, D., Sexton, S., Mulhem, J., & Keeling, D.(2009). The widget effect: Our national failure to ac-knowledge and act on differences in teacher effective-ness. Retrieved June 2, 2009, from http://widgeteffect.org/

Weiss, I. R., Pasley, J. D., Smith, P. S., Banilower, E. R.,& Heck, D. J. (2003). Looking inside the classroom: Astudy of K-12 mathematics and science education inthe United States. Chapel Hill, NC: Horizon Research.

Wharton-McDonald, R., Pressley, M., & Hampston, J. M.(1998). Outstanding literacy instruction in first grade:Teacher practices and student achievement. Elemen-tary School Joumal, 99, 101-128.

Ysseldyke, J., & Christenson, S. (1993). The InstructionalEnvironment System—// (TIES-II). Colorado Springs,CO: Sopris West.

Ysseldyke, J. E., & Christenson, S. L. (1987). Evaluatingstudents' instructional environments. Remedial andSpecial Education, 8(3), 17-24.

Date Received: June, 13, 2012Date Accepted: December 5, 2012

Action Editor: Amanda VanDerHeyden •

97


Joseph Allen, PhD, is the Hugh P. Kelly Professor of Psychology at the University ofVirginia, specializing in the study of adolescent social development.

Anne Gregory, PhD, is an associate professor of psychology in the Graduate School ofApplied and Professional Psychology at Rutgers University, specializing in the study ofdisproportionality in implementation of disciplinary procedures across racial/ethnicgroups in schools.

Amori Mikami, PhD, is an assistant professor of psychology at the University of BritishColumbia, specializing in the study of ways in which school and home environmentsinfluence social development.

Janetta Lun, PhD, is currently a research associate in the Department of Psychology at theUniversity of Maryland, specializing in the study of intergroup relations and culturalpsychology.

Bridget Hamre, PhD, is a research scientist at the Curry School of Education at theUniversity of Virginia and currently serves as associate director of the Center forAdvanced Study of Teaching and Learning at the University of Virginia.

Robert Pianta, PhD, is the Novartis U.S. Foundation Professor of Education and currentlyserves as dean of the Curry School of Education and director of the Center for AdvancedStudy of Teaching and Learning at the University of Virginia.

98

Copyright of School Psychology Review is the property of National Association of School Psychologists and itscontent may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder'sexpress written permission. However, users may print, download, or email articles for individual use.

Observations of Effective Teacher-Student Interactions ... - ERIC

Documents