Multivariate relationships among developmental age, global engagement, and observed child engagement

Multivariate Relationships Among Developmental Age,Global Engagement, and Observed Child Engagement

Renee E. L. de Kruif and R. A. McWilliamUniversity of North Carolina at Chapel Hill

The purpose of this study was to explore patterns of relationships amongdevelopmental age, teacher ratings of global engagement, and observed engage-ment in the classroom setting. Sixty-two children (age 9.5 to 63.6 months) wereobserved during free play, structured activities, and mealtimes. All children wereadministered a developmental test to assess their developmental age. In addition,teachers completed a rating scale to assess children’s typical engagement pro-files. Canonical correlation analysis revealed two uncorrelated patterns of rela-tionships among the variables. Function I reflected the positive relationshipbetween children’s developmental age and high levels of engagement, and thenegative relationship with lower levels of engagement. Function II representedthe bivariate relationship between high levels of engagement regardless ofdevelopmental age.

Early childhood researchers (e.g., Buysse & Bailey, 1993; Jones & Warren, 1991;McWilliam, Trivette, & Dunst, 1985; McWilliam & Bailey, 1992, 1995) haveproposed that high quality of engagement with the environment is a potentiallycritical mediating factor in young children’s learning. Specifically, these research-ers suggest that when engagement is systematically promoted in the classroom,children are more likely to participate in developmentally appropriate activitieswith peers, adults, and materials (McWilliam et al., 1985; see Whaley & Bennett,1991). Higher levels of engagement have also been related to high achievementlevels and aptitude scores (e.g., McGarity & Butts, 1984; Rosenshine, 1978).

Engagement has typically been defined as “the amount of time children spendinteracting appropriately with the environment at different levels of competence”(McWilliam & Bailey, 1992, p. 234). The potential importance of engagement hasled to extensive research over more than a decade, resulting in the conceptual-

Direct all correspondence to: R. A. McWilliam, University of North Carolina at Chapel Hill, FrankPorter Graham Child Development Center, CB#8180, Chapel Hill, NC 27599-8180; Phone: (919)966-7485; E-mail: [email protected].

Early Childhood Research Quarterly, 14, No. 4, 515–536 (1999) © 1999 Elsevier Science Inc.ISSN: 0885-2006 All rights of reproduction in any form reserved.

515

ization and measurement of child engagement in increasingly elaborate ways. Thepurpose of this study was to add to the literature of child engagement by exploringmultivariate relationships among developmental age and components of engage-ment as measured by two different methods: an observational method (i.e., theE-Qual; McWilliam, 1995) and a rating scale of “global” engagement (i.e., theCEQ; McWilliam, 1991).

In early studies, engagement was conceptualized as participation in plannedactivities (Twardosz, Cataldo, & Risley, 1974) or as participation in manipulativeactivities (McClannahan & Risley, 1975), and researchers used the percentage ofchildren participating in activities as an outcome measure (e.g., Krantz & Risley,1977; Montes & Risley, 1975). A number of studies found that factors contrib-uting to high percentages of engaged children included the physical environment,the social environment, and the teaching method. More specifically, researchersreported that a modified open classroom arrangement (Dunst, McWilliam, &Holbert, 1986; Twardosz & Risley, 1982), providing a variety of developmentallyappropriate materials accessible to children (Krantz & Risley, 1977; Montes &Risley, 1975), opportunities for children to make choices (see Ostrosky & Kaiser,1991; Whaley & Bennett, 1991), smooth transitions between activities (Doke &Risley, 1972; Rogers-Warren, 1982), carefully sequencing activities (Krantz &Risley, 1977), and incidental teaching (Warren & Kaiser, 1986) promote chil-dren’s engagement. These studies were helpful in determining which environ-mental factors influenced engagement but focused less on how children spendtheir time and how time spent was related to learning.

More recently, research on engagement has shifted its emphasis from assessingthe quantity of engagement to also investigating the quality of children’s engage-ment behaviors. Contemporary conceptualizations of engagement no longer viewengagement as a dichotomous construct (i.e., time engaged versus time nonen-gaged). Rather, researchers are now more interested in thefocus of a child’sengagement (e.g., with adults, with peers, with materials) and thelevel of thisengagement. McWilliam (1995) has defined engagement levels with behaviors,such as pretend play, persistence, attention, participation, undifferentiated behav-ior, and nonengagement. Mastery-motivation researchers have included similarbehaviors in their studies of this construct (see Brockman, Morgan, & Harmon,1988; Messer, Rachford, McCarthy, & Yarrow, 1987). They have not, however,linked mastery motivation to engagement, and this literature has not yet led toimprovements in instructional practices (McWilliam & Bailey, 1992). Whereasmastery motivation is generally operationalized as the time the child spends ingoal-directed behavior, engagement also includes less goal-directed behaviors.Mastery motivation can therefore be considered to be a particular form ofengagement and may be an important key to what happens during engagementthat leads to learning (McWilliam & Bailey, 1992).

The inclusion of quality or mastery levels in the engagement construct hasaided researchers in better understanding how child engagement can be influencedby a number of different factors internal and external to the child. Researchershave investigated how child engagement is affected by involvement in activities(McWilliam & Bailey, 1995), and how engagement may differ across settings

516 de Kruif and McWilliam

(Blasco, Bailey, & Burchinal, 1993; McWilliam & Bailey, 1991, 1995), types ofactivities (McCormick, Noonan, & Heck, 1998), social and nonsocial behavior(McWilliam & Bailey, 1991), and mastery levels (MacTurk, Hunter, McCarthy,Vietze, & McQuiston, 1985).

Measures of engagement are typically obtained through observation; research-ers observe live or videotaped classroom sessions, using a coding scheme con-sisting of specific behaviors to determine the focus and level of engagement (seeCarta & Greenwood, 1985; Greenwood & Carta, 1987; McWilliam, 1995; Ma-lone, Stoneman, & Langone, 1994). Although this approach provides very specificinformation about children’s behaviors, it may not be the only relevant index ofa child’s engagement. Another approach is to conceptualize engagement as achild’s typical or “global” behavioral pattern when interacting with the environ-ment. Parents, teachers, or other adults who have broad knowledge of the child,gathered over long periods in a variety of settings, could provide informationabout this dimension of engagement. The potential for adults’ perceptions of thechild’s engagement to influence their interactions with the child emphasizes theimportance of an instrument to assess a child’s global engagement as viewed bythe adult. Morgan and his colleagues (see Morgan, Maslin-Cole, Harmon, Busch-Rossnagel, Jennings, Hauser-Cram, & Brockman, 1993) have taken such anapproach to the assessment of mastery motivation. They developed the Dimen-sions of Mastery Questionnaire (DMQ; Morgan, Maslin, & Harmon, 1987), whichasks parents and teachers to rate how typically their children engage in behaviorsthat reflect mastery motivation.

Preliminary work on measuring typical or global engagement levels has in-volved the use of the Children’s Engagement Questionnaire (CEQ; McWilliam,1991), an instrument based on the DMQ. In a validation study, parents andprofessionals completed the 32-item CEQ for 108 children under the age of 6years (McWilliam, Snyder, & Lawson, 1993). Results showed high congruencebetween parent and professional ratings of children, indicating that adults whohave an ongoing opportunity to observe children’s behavior can judge theirengagement equally well. Factor analysis of the CEQ items resulted in fourunderlying factors: Competence, Persistence, Undifferentiated Behavior, and At-tention. No investigations have been published examining multivariate relation-ships between engagement components measured through global methods (e.g.,the CEQ; McWilliam, 1991) and engagement components measured throughobservational methods (e.g., E-Qual; McWilliam, 1995).

In addition to the influence of the environment on engagement, as mentionedearlier, the child’s intra-individual characteristics (e.g., mastery behavior, tem-perament, disability) influence engagement also. A number of studies haveexamined the relationship between developmental age and engagement. Malone etal. (1994) investigated how categorical and sequential play levels were related todevelopmental age and chronological age in a home context versus a classroomcontext. Patterns of association in both conditions were the same: nonplay,functional play, and exploratory play behavior (which were considered to be lesssophisticated behaviors) were negatively related to developmental age, whereasmore sophisticated behaviors such as constructive play and pretend play were

517Engagement Relationships

positively related to developmental age. In addition, developmental age wasrelated to more sophisticated play sequences.

Blasco et al. (1993) also reported a relationship between developmental age andthe sophistication of play behavior. Typically developing children and childrenwith disabilities in mixed-age classrooms who were developmentally more ad-vanced, spent more time in purposeful play and social play and less time simplymanipulating materials than children who were less developmentally mature.Finally, McWilliam and Bailey (1995) found that developmental age was posi-tively associated with more interactions with peers. As developmental age in-creased, children with disabilities spent more time attentionally engaged withpeers than did children without disabilities. Developmental gain appeared toreduce the effects of disability with regard to nonengagement. In conclusion, thesestudies indicate the importance of developmental age as a predictor in children’sengagement.

The purpose of this study was to investigate patterns of relationships betweentwo sets of variables including developmental age and components of the engage-ment construct as measured by the CEQ (McWilliam, 1991), and an observationalmethod (i.e., the E-Qual; McWilliam, 1995). Specifically, the question explored inthis paper is which combinations of developmental age and teacher ratings ofglobal engagement components are associated with similar combinations ofobserved engagement components.

METHOD

This study was part of a broader study investigating teachers’ interaction stylesand the influence of these different styles on children’s engagement. All teacherswere videotaped, and their behaviors as well as the behaviors of the children intheir presence were coded through a computerized coding system. Although thepresent investigation focused on children’s engagement behaviors, we acknowl-edge that these observed behaviors might have been influenced by the presence ofthe teacher. A brief description of the teachers, in addition to the information weprovide on the children who participated in the study, is included.

Participants

Children. Sixty-two children (31 males, 31 females) enrolled at a universitysupported childcare center participated in the study. The children ranged in agefrom 9.5 to 63 months. Forty-nine of the children were typically developing, and13 children were classified as having special needs. Disabilities or conditionsrepresented by these children included developmental delay, cerebral palsy,Apert’s Syndrome, autism, Prader Willi syndrome, and Williams syndrome. Table1 presents the demographic information and the number of children in eachclassroom. Fifty percent of the children were Caucasian, 44% were AfricanAmerican, and 5% were from some other ethnic background, as determined fromparents’ self-identification on center registration forms. The average SES of thechildren’s families was calculated with the Hollingshead (1975) four-factor for-


Tab

le1

Dem

ogra

phic

Info

rmat

ion

for

Chi

ldre

nin

Diff

eren

tC

lass

room

s

Nur

sery

(n5

10)

Tod

dler

A(n

57)

Tod

dler

B(n

57)

Tod

dler

C(n

57)

Pre

scho

olA

(n5

16)

Pre

scho

olB

(n5

15)

Gen

dera

Mal

e5

4(1

)4

(1)

18

(4)

9(4

)F

emal

e5

3(1

)3

(1)

68

(1)

6E

thni

city

a

Afr

ican

Am

eric

an4

3(1

)3

(1)

37

(1)

8(1

)W

hite

64

(1)

4(1

)4

7(3

)6

(3)

Oth

er0

00

02

(1)

1M

SE

Sb45

.849

.637

.147

.944

.641

.6R

ange

27.0

–58.

518

.5–6

3.0

19.0

–66.

041

.5–5

7.0

14.0

–66.

027

.0–5

3.0

MA

gein

Yea

rs9.

519

.219

.233

.650

.448

.0R

ange

6.8–

11.8

14.4

–26.

413

.2–2

1.6

30.0

–37.

232

.4–6

3.6

33.6

–63.

6M

Bat

telle

(BD

I-D

A)c

18.5

24.6

(18.

0)26

.4(2

1.0)

45.4

57.9

(45.

4)58

.3(4

1.5)

Ran

ge7.

0–21

.022

.0–2

9.0

24.0

–31.

039

.0–5

1.0

21.0

–70.

052

.0–6

5.0

(17.

0–19

.0)

(14.

0–28

.0)

(16.

0–43

.0)

(27.

0–56

.0)

No

tes:

aN

umbe

rsin

pare

nthe

ses

repr

esen

tth

enu

mbe

rof

child

ren

with

spec

ialn

eeds

.b

Hol

lings

head

(197

5)F

our-

fact

orin

dex.

cN

umbe

rsin

pare

nthe

ses

repr

esen

tM

and

SD

for

child

ren

with

spec

ialn

eeds

.


mula and ranged from 14 to 66 (the ceiling). Since this was a sample ofconvenience, we had no control over the heterogeneity of the children. Althoughthis variability could confound our findings, we appreciated the diversity pre-sented.

Teachers. The adults participating in the study consisted of 10 female teach-ers and 1 male teacher working in six classrooms at a university-supported childcare center. Four teachers were African American and seven were Caucasian.Two teachers had master’s degrees, one had an associate’s degree, and the rest hadbachelor’s degrees. Teachers ranged in teaching experience from 2 to 22 years(M 5 13.64,SD 5 7.99), and their experience with children with special needsranged from 1 to 12 years (M 5 8.20, SD 5 4.62). Each classroom had anassistant, and the number of children in each classroom ranged from 7 to 16 (M 59.00).

Instrumentation

Battelle Developmental Inventory (BDI). The BDI (Newborg, Stock, Wnek,Guidubaldi, & Svinicki, 1984) was administered to determine the children’sdevelopmental age. The BDI is a standardized instrument for assessing children’sdevelopment in five domains (personal-social, adaptive, communication, motor,and cognitive domain). Test procedures are designed to collect informationthrough presentation of a structured test format; interviews with parents, caregiv-ers, or teachers; and observations of the child in natural settings. Each child wasassessed using all five domains in order to establish the children’s approximatedevelopmental age.

Children’s Engagement Questionnaire (CEQ).Teachers completed theCEQ (McWilliam, 1991) for each child. The CEQ is a 32-item instrumentdesigned to rate children’s global engagement. It is completed by an adult familiarwith the child (in this case, the teacher) and demands their free-recall impressionof the child’s levels of engagement with peers, adults, and materials (i.e., therating is independent of time or context). The CEQ has a four-point rating scaleto record whether the child’s behavior is (1) not at all typical, (2) somewhattypical, (3) typical, or (4) very typical. The instructions specify that “typical”means that the child spends quite a lot of time in the activity. Behavioral examplesare provided for each item on the CEQ to further clarify the intent of the item.Previous research on the CEQ indicated the existence of four underlying factors:Competence, Persistence, Undifferentiated Behavior, and Attention. These factorshave been found to explain 62.1% of the variance, and parents’ and professionals’ratings of children were found to be highly congruent (McWilliam et al., 1993).The generalizability coefficient was found to be .84, and the alpha coefficient forinternal consistency was .96. Coefficient alpha in the present study was .93. Table2 presents example items for each of the factors.

E-Qual Observational Coding System (E-Qual).Observed engagement wasmeasured using a version of the E-Qual (McWilliam, 1995). Two kinds ofengagement variables were coded: level of engagement (i.e., attention, undiffer-


entiated behavior, nonengagement, participation, pretend, and persistence), and amodifier indicating the type or focus of engagement (i.e., with the teacher, anotheradult, peers, materials, and self). To reduce the number of variables only level ofengagement was used for analyses in the present study. All engagement codeswere mutually exclusive; raters could not assign two codes at the same time. Twoadditional codes were used to indicate that the child was no longer present and toindicate when the child was in transition between two locations without beingotherwise engaged. Neither of these two codes was used in the analysis. Table 3presents brief descriptions of the observed behaviors included in the analysis.Detailed operational definitions may be requested from the second author.

Procedures

Videotaping. Observational data were collected through videotaped sessionsof teachers interacting with the children. Each teacher in the study was videotapedfor at least eight 20-minute indoor sessions. The camera focused on the teacherand a 6- to 10-foot area in front of him or her to capture the children most likelyto be affected by teacher behavior. To obtain a balanced sample of teacherbehaviors across activities, two of the sessions were taped during mealtime, two

Table 2Sample Items on the CEQ Factors

CEQ Factors CEQ Item Item Example

Competence ● Tries to complete things even if ittakes a long time to finish.

● The child knows how to put togethersimple jigsaw puzzles, sticks with ituntil it is completed.

● Plays appropriately for his or herage.

● The child who does most of thethings at the 2-year-old level playswith objects and people at the 2-year-old level.

Persistence ● Plays with other children who tryto play with him or her.

● When another child approaches, thechild will talk to or play with him orher.

● Tries to get adult to do things. ● The child tries to get the teacher togive him or her a toy.

UndifferentiatedBehavior

● Plays with objects in a simplemanner (i.e., repetitive, changing).

● The child bangs the toy car over andover again on the high-chair tray.

● Uses repetitive vocalization. ● The child says “Ba-ba-ba-ba-ba.”Attention ● Watches or listens to adults. ● When the mother moves about the

kitchen, talking to the child, thechild watches her.

● Watches or listens to otherchildren.

● When other children are playing, thechild follows their movements withhis eye-gaze.

Note: A copy of the full instrument may be obtained from the second author.


during structured class activities, and the remainder of the sessions duringunstructured class time. In the nursery, structured sessions were replaced withsessions taped during unstructured class time. The sampling times were not intendedto represent exact proportions of the day during which each activity occurred (i.e.,free play occurring exactly twice as often as meals or structured sessions), becauseeach classroom and age level differ in their appointment of time.

Structured sessions were considered to be sessions in which the teacher (a)selected materials, (b) had expectations for sustained play, (c) encouraged alimited range of behaviors, and (d) spent at least 2 minutes directing the activity.Large and small group times are examples of structured sessions. Anotherexample we observed was when a toddler teacher sat at the table with all childrenin the room. Each child had a peg-board and a bowl or plate with pegs. Thechildren were not allowed to leave or take out other toys, but were directed backto the table and the pegboard in front of them. Unstructured sessions were sessionsin which the teacher (a) allowed free choice of materials, (b) allowed the childrenthe freedom to come to and leave the activity, and (c) encouraged a variety ofbehaviors. These sessions varied from activities in which children asked theteacher to read a book to them, to activities in which children who were interestedcame to the art table to make animal masks of their choice with materials of theirchoice. Our intent was to capture the normal flow of the activities, and as a result,some of the videotaped sessions included more than one type of activity. Forexample, we once captured teacher-child interactions during a session startingwith the last minutes of free play and the beginning of large group time. When thisoccurred, the session type was determined by the activity that took up 50% ormore of the total session time.

Every child in the study was taped in at least four sessions and for a minimumof 10 and a maximum of 60 minutes across all sessions. Since the sessions werenot systematically organized by child but by teacher, the presence of the childrenin the sessions was somewhat random. Every child, however, was filmed at leastonce during a structured activity, once during an unstructured activity, and once

Table 3Brief Definitions of Levels of Observed Child Engagement from the E-Qual

Level Definition

Persistence Goal directed problem solving or repeated attempt.Pretend Talking in character, substituting objects or acting out a

scenario.Participation Actively involved with the environment (i.e., busy), but not in

pretend play, not persistently, and not repetitively.Undifferentiated Behavior Interacting with the environment without differentiating his or

her behavior (i.e., in a repetitive manner).Attention Watching or listening to features in the environment for at

least 3 seconds.Nonengagement Unoccupied, waiting, staring, wandering, crying or aggressive

behavior. None of the other behaviors are occurring.


during mealtime. Three teachers needed 9 sessions, and another teacher needed 10sessions, to obtain adequate samples of child behavior (i.e., enough time). All 93sessions were taped within 8 weeks. Videotaping occurred between 8:00 a.m. and12:30 p.m. Teachers were notified in the morning that they would be taped thatday but had the option to defer taping at their convenience. The teachers were toldthat observations would focus on all children’s engagement and that group,individual, or caregiving activities were acceptable. Activities in the classroomwere taped as they naturally occurred. After the third session, however, teacherswere informed of the type of activity that still needed to be captured, prior to theday the taping would take place.

Coding. The Observer™, a computer software program for observationalstudies with a Video Tape Analysis System (Noldus Information Technology,1993), was used to code both teacher and child behaviors. In this study, analyseswere limited to the coding of the level of child engagement behaviors. Exactly 900seconds from each 20-minute taped session were electronically time-coded. Fourgraduate students were trained for 6–8 weeks to master the technique of microcoding on The Observer™ and to establish a minimum 80% level of agreementon all categories of the E-Qual Observational Coding System (McWilliam, 1995).Coders rated continuous behavior, using one of six levels of engagement followedby the type of engagement. Coders were trained, using sessions similar to thoseused in the actual study, until they had reached acceptable levels of inter-rateragreement. The coding team met regularly to answer questions about the codingsystem. Twenty-five percent of the formal observations were double-coded tomonitor inter-rater agreement and to prevent coder drift. Table 4 presents inter-rater agreement for each code across the double-coded sessions. Low expectedagreement and low kappas occurred in the codes for behaviors observed veryinfrequently. This phenomenon is characteristic of expected agreement and kappacoefficients.

Computation of Scores. Factor scores on the CEQ were calculated by takingthe mean of the items constituting each factor. Observed child engagementbehaviors were coded from the videotaped teacher sessions. Because the focus of

Table 4Inter-rater Agreement—Observed Engagement Codes (E-Qual)

CodeObserved

AgreementExpected

AgreementMeankappa

Attention .94 .69 .76Participation .94 .80 .84Nonengagement .97 .34 .43Undifferentiated Behavior .98 .40 .49Persistence .99 .30 .37Pretend .98 .32 .37High Level Engagement .99 .32 .38


the taping was on the teacher and some classrooms had more than one teacher inthe room, some children were coded as many as 21 times (children in classroomswith three teachers and who were present for the sessions of almost all of theteachers), and others only 4 or 5 times (children with just one teacher in the room).Specifically, children in classrooms with one teacher were taped an average of4.79 times (range 4.00–7.00), children in classrooms with two teachers weretaped an average of 9.86 times (range 8.00–11.00), and children in classroomswith three teachers were taped an average of 14.41 times (range 4.00–21.00, withone child being taped only 4 times). No differences in sessions were found as afunction of ability status or gender. Since more frequent codes only increases thereliability of the observed behavior, we used all sessions in which a child wascoded to compute an overall mean score for each observed behavior per child.Scores on each of the engagement behaviors represent the percentage of time thechild was engaged in this behavior across sessions.

Data Analysis

To reduce the number of variables, the observed pretend and persistencecategories, both cognitive advanced skills occurring very infrequently (0.41% and1.14% respectively), were combined to create a single variable: high levelengagement. Correlations were used to examine bivariate relationships betweenvariables in the analysis. Canonical correlation analysis was then used to explorepatterns of multivariate relationships between developmental age, four CEQglobal engagement variables, and five observed engagement variables. Canonicalcorrelation analysis may not be familiar to some readers (for an introduction seeThompson, 1991, in press) but can be related to regression. The basic strategy incanonical correlation analysis is to derive a weighted sum of linear combinations(or canonical variates) of two sets of variables in such a way that the linearcombinations are maximized (Thompson, 1984). The first pair of canonicalvariates (i.e., function) selected has the highest intercorrelation possible. Thesecond pair of canonical variates then selected accounts for the maximum amountof the relation between the two sets of variables unaccounted for by the firstcanonical function, and so forth. This process is repeated until the number ofcanonical correlations established equals the number of variables in the smallerset.

In this study, developmental age and the four CEQ factors (competence,persistence, undifferentiated behavior, and attention) were selected to form thepredictor set in the canonical analysis. Given the age variability in the sample,developmental age was included in this first set of variables. Five observedengagement variables (attention, undifferentiated behavior, nonengagement, par-ticipation, and high engagement) served as the second or criterion set of variables.

RESULTS

Means and standard deviations for all 10 variables are presented in Table 5.Inspection of the correlation matrix (see Table 6) revealed a number of moderate


to high correlations within and between the two sets of variables. Before describ-ing the results of the canonical correlation analysis, we will provide a briefoverview of the most important correlations. Developmental age was, as expected,highly negatively correlated with global undifferentiated behavior (i.e., teachers’ratings of children’s typical repetitive behavior). It was moderately positivelycorrelated with global competence and observed participation and moderatelynegatively correlated with observed nonengagement and observed undifferenti-ated behavior. Finally, developmental age was somewhat associated with globalpersistence. These relationships were all expected.

We will focus on three other particularly salient variables: global competence,observed participation, and observed nonengagement. Global competence was thefirst factor derived from the CEQ (McWilliam et al., 1993). As in the originalstudy, global competence was most strongly associated with global persistence,the second factor on the CEQ. Moderate positive correlations were found withdevelopmental age and observed participation. Global undifferentiated behavior,observed nonengagement, and observed undifferentiated behavior were modestlynegatively correlated with global competence. Again, these relationships were allexpected.

Among the observed scores, participation was the most commonly seen be-havior (see Table 5). The relationships with developmental age and globalcompetence reported above show that this variable is associated with maturity. Itwas modestly correlated with global persistence. Moderate negative relationshipswith global undifferentiated behavior, observed undifferentiated behavior, ob-served attention, and observed nonengagement confirm that this variable isassociated with appropriate active play.

Table 5Means and Standard Deviations on the Battelle, CEQ and E-Qual Variables

M SD

BattelleDevelopmental Age 39.79 17.97

Child Engagement QuestionnaireAttention 3.05 0.67Competence 2.65 0.78Persistence 2.92 0.63Undifferentiated Behavior 2.54 0.92

E-Qual Observed EngagementAttention 33.13 9.29Nonengagement 7.13 4.74Participation 52.79 11.03Undifferentiated Behavior 3.58 6.04High Level Engagement 1.56 1.30

Note: Means for the CEQ are presented in terms of the Likert scale options of the CEQ scale (i.e., 15 not atall typical, 2 5 somewhat typical, 35 typical, and 45 very typical). Means for the E-Qual scoresrepresent percentages of time observed.


Tab

le6

Cor

rela

tions

Am

ong

Bat

telle

,C

EQ

,an

dE

-Qua

lVar

iabl

es

Var

iabl

es1

23

45

67

89

10

Dev

elop

men

talA

ge/G

loba

lEng

agem

ent

1.D

evel

opm

enta

lAge

1.00

2.C

EQ

Atte

ntio

n2

.23

1.00

3.C

EQ

Com

pete

nce

.68

.35

1.00

4.C

EQ

Per

sist

ence

.46

.57

.86

1.00

5.C

EQ

Und

iffer

entia

ted

Beh

avio

r2

.87

.33

2.5

72

.29

1.00

Obs

erve

dE

ngag

emen

t6.

Obs

erve

dA

ttent

ion

2.0

8.1

9.1

4.1

8.0

21.

007.

Obs

erve

dN

onen

gage

men

t2

.66

2.1

12

.66

2.6

3.5

22

.21

1.00

8.O

bser

ved

Par

ticip

atio

n.6

92

.19

.43

.30

2.5

72

.60

2.5

21.

009.

Obs

erve

dU

ndiff

eren

tiate

d2

.59

.14

2.5

22

.40

.54

2.2

0.4

92

.54

1.00

Beh

avio

r10

.O

bser

ved

Hig

hLe

vel

Eng

agem

ent

2.1

0.2

4.2

4.3

8.1

4.0

22

.20

2.0

22

.08

1.00


Nonengagement is the negative equivalent of total observed engagement; if thechild were engaged in any way, nonengagement would not have been coded. Themoderate negative correlations with developmental age, global competence, andglobal persistence indicated that observed nonengagement is not associated withratings of typical mature engagement. It is, however, associated with globalundifferentiated and observed undifferentiated behavior, indicating that childrenjudged to spend much time in repetitive behavior are also likely to be observed ina nonengaged state.

Five canonical functions (i.e., pairs of variates) were derived from the canonicalcorrelation analysis. Results of the analysis were interpreted according to theguidelines provided by Thompson (1991), who cautioned researchers not to relysolely on statistical tests when deciding which canonical functions to interpret. Hepointed out that tests for significance as offered in statistical packages are not testsof the significance of single functions. Rather, in a set with three canonicalcorrelations, the first test statistic is used to evaluate whether the complete set offunctions is zero, or equivalent, that there are no linear relationships between thefirst and the second set of variables. The second test statistic evaluates the secondand third canonical correlation coefficients, and only the third test statistic is a testof a single correlation coefficient (Thompson, in press).

Because the significant tests do not provide information about the importanceof individual functions, it is suggested that researchers use the squared canonicalcorrelation coefficient (RC

2) (describing the proportion of variance that a pair ofcanonical variates shares) as an indicator of effect size in addition to the signif-icance test (Thompson, 1991). This coefficient is similar to a multiple R2 inregression. In the present study Wilks’ lambda prior to extraction of the firstfunction was statistically significant,F (25, 194.67)5 5.08,p 5 .001 (see Table7). Inspection of the canonical correlation and the squared canonical correlationindicated that the first pair of variates was strongly correlated and accounted for70% of the shared variance. Wilks’ lambda prior to extraction of the secondfunction was also statistically significant,F (16, 162.56)5 2.49,p 5 .002. Thevariates were moderately correlated and accounted for 37% of the shared vari-ance. The high canonical correlation coefficients indicated that relationshipsbetween the two sets were unlikely to have occurred by chance.

Applying the Wherry (1931) correction formula to the obtained canonicalfunctions to account for “shrinkage” (Thompson, 1990), the first squared canon-ical correlation was .64. This indicated that, even with this conservative correctionof the effect size applied, the variates of Function I still shared 64% of theirvariance. Applying the Wherry correction formula to the second function yieldeda shrunken squared canonical correlation coefficient of .25.

To interpret the extent to which the pairs of canonical variates contribute to themultivariate relationship, we used both standardized function coefficients andstructure coefficients (see Table 7). Standardized function coefficients or canon-ical weights are similar to beta weights in regression and indicate the contributionof each variable to the variance of its respective within-set variate (Thompson,1991). In both analyses these weights are usually not correlation coefficients andtherefore do not necessarily range between21 and11. A given variable can have


Tab

le7

Can

onic

alC

orre

latio

nA

naly

sis

for

Bat

telle

,C

EQ

,an

dO

bser

ved

Eng

agem

ent

Var

iabl

es

Fun

ctio

nI

Fun

ctio

nII

Fun

ctio

nC

oeffi

cien

tS

truc

ture

Coe

ffici

ent

Squ

ared

Str

uctu

reC

oeffi

cien

tF

unct

ion

Coe

ffici

ent

Str

uctu

reC

oeffi

cien

t

Squ

ared

Str

uctu

reC

oeffi

cien

t

Dev

elop

men

talA

ge.7

7.9

7.9

42

1.22

2.2

4.0

6A

ttent

ion

2.0

52

.06

.00

2.7

1.4

6.2

1C

ompe

tenc

e2

.01

.81

.66

.46

.42

.18

Per

sist

ence

.34

.66

.43

1.25

.66

.44

Und

iffer

entia

ted

Beh

avio

r2

.05

2.8

2.6

8.0

6.2

8.0

8R

c2

.70

.37

Atte

ntio

n.7

42

.01

.00

2.8

2.4

4.2

0N

onen

gage

men

t2

.14

2.8

8.7

72

.76

2.2

7.0

7P

artic

ipat

ion

1.25

.80

.64

21.

562

.28

.08

Und

iffer

entia

ted

Beh

avio

r.1

62

.74

.55

2.7

32

.16

.02

Hig

hLe

velE

ngag

emen

t.0

2.0

3.0

0.6

7.8

9.8

0


a function coefficient of near-zero when the variable does not contribute to therelationship between the variable sets or when variables in the same set aremoderately interrelated. In the latter case, one or more other variables may containthe same information and the variable may be arbitrarily denied credit forproviding this information (Thompson, in press).

Structure coefficients are the correlations between measured or observed vari-ables (e.g., developmental age, observed attention) and scores on the canonicalfunctions derived by applying the canonical weights to the measured variables(Thompson, 1991). Thus structure coefficients help us to understand the nature (orstructure) of the variables being related in the analysis. A measured variable is notnoteworthy on a given function if both the standardized function coefficients andstructure coefficients are near zero. As previously described and shown in Table6, variables included in the present analysis within the given sets were relativelyhighly related with each other, suggesting that merely inspecting the functioncoefficients may not be reliable in indicating variable contributions. Hence,structure coefficients and squared structure coefficients were consulted for inter-pretation of the canonical variates. Because only the first two RC

2’s were note-worthy and only the first two lambdas were statistically significant, Table 7 onlypresents the function coefficients, structure coefficients, and squared structurecoefficients for these first two functions.

Variables within the predictor set (developmental age and CEQ factors) thatwere correlated most highly with the first canonical variate were developmentalage and global competence, global persistence, and global undifferentiated be-havior (acceptable cut-off for correlation at .30; see Tabachnick & Fidell, 1996).Among the variables in the observed engagement or the criterion set, nonengage-ment, participation, and undifferentiated behavior were correlated with the firstcanonical variate. Specifically, this first pair of canonical variates indicated thatchildren who were developmentally more mature and were rated by their teachersas typically engaging in competent and persistent behavior, but not typicallyengaging in undifferentiated behavior, were frequently observed participating.These children tended to display less nonengaged and undifferentiated behaviorwhen observed by raters in the classroom. Near-zero values on both standardizedfunction coefficients and structure coefficients indicated that global attention andobserved high level engagement contributed little to the first canonical function.The high function coefficient and near-zero structure coefficient for observedattention indicated that this variable actually functioned as a suppressor variable.This means that children’s observed attention contributed little to the multivariaterelationship between the two sets of variables but contributed an appreciableamount indirectly by making the other measured variables more related to eachother. In conclusion, as indicated by large and homogenous squared structurecoefficients ranging from .43 to .94, Function I appeared to be a general functionincluding most of the measured variables.

Variables most highly related to Function II were global attention, globalcompetence, global persistence, observed attention, and observed high levelengagement. Compared to other variables, global competence, global attention,and observed attention had relatively small function coefficients (e.g., .46,2.71,


2.82 vs. 1.25,21.22.21.56), and relatively small squared structure coefficients(.18, .21, and .20 respectively), indicating that these variables contributed little tothe second canonical function. Function II therefore primarily reflected the biva-riate relationship between global persistence and observed high level engagement.

DISCUSSION

The results of this study indicate the existence of two uncorrelated patterns ofrelationships among developmental age, global engagement, and observed en-gagement variables. First, more developmentally mature children who were ratedby their teachers as typically engaging in high level engagement behaviors andless typically engaging in lower level engagement behaviors were indeed moreactively involved with their environment and engaged less in lower level behav-iors when observed in the classroom. Maturity and teacher-rated high levelengagement, however, were not related to observed high level engagement. Thisfinding was supported by the second function, which indicated that children whowere rated as typically engaging in persistent behaviors tended to engage inobserved high engagement, regardless of developmental age. In addition, chil-dren’s attention did not contribute directly to the multivariate relationship amongdevelopmental age and the engagement variables. In this section we will list eachof these findings, point out possible limitations to the study, and discuss impli-cations for practitioners, administrators, researchers, and personnel developmentfaculty.

The nature of the relationship of developmental age with the variables onFunction I is generally consistent with findings from previous engagement studiesthat included developmental age as an independent variable (e.g., Blasco et al.,1993; Malone et al., 1994; McWilliam & Bailey, 1995). In these studies, devel-opmental age was positively related to more sophisticated engagement behaviorsand negatively to lower level engagement behaviors (e.g., nonengagement, un-differentiated behavior). Based on the literature, we hypothesized that observedhigh engagement, consisting of persistence and pretend play, would be related tothe first function as well, together with CEQ persistent play, CEQ competence,and developmental age.

A possible explanation why this was not the case is the frequency of occurrenceof observed persistence and observed pretend play. Both were observed infre-quently, which limited their range and may have influenced the low correlationsof high engagement with the other variables in the analysis. Low occurrence ofpersistent behavior may have been an artifact of both our observational codingsystem and our videotaping methods. For example, a child who is completing apuzzle, looking for pieces, trying to fit pieces in the puzzle over and over againto find the right one is clearly engaging in persistent behavior. Our second-by-second coding of each behavior, however, may have caused us to lose sight of thisbroader picture, leading us to alternate short intervals of persistence with shortintervals of any of the other engagement levels, thereby reducing the time thechild spent in persistent behavior. The age range of the children we were studying


might explain low occurrence of pretend behavior. Although most preschoolersengage in pretend play, most infants and toddlers do not. In addition, we observedthat when teachers approached children who were engaged in pretend play andtried to join them in their play, the children often turned to the teacher, whichresulted in a different level of engagement (e.g., attention, participation).

Observed high level engagementwas included in the second function, as wasglobal persistence. The functions in a canonical correlation are uncorrelated.Function II, therefore, reflected the relationship between the variables after thevariance associated with Function I was accounted for. Thus, Function II tappedinto different relationships across the sets of variables from those represented inFunction I. While Function I reflected an increase in sophisticated behavior anda decrease of lower level engagement behaviors when children become moredevelopmentally mature, Function II primarily involved the bivariate relationshipbetween global persistence and observed high engagement. The absence ofdevelopmental age in this second function suggested that there is a relationshipbetween global persistence and observed high level engagement that is notage-related. This makes conceptual sense since both younger and older childrenmay engage in persistent behavior (high level engagement consisted of observedpersistence and pretend play).

At the same time this finding is somewhat worrying given the fact that observedhigh level engagement only occurred 1.56% of the time. One would expect moremature children to engage in these behaviors at a much higher rate. One expla-nation might be the fact that children’s engagement was observed in the presenceof the teacher. Few studies have examined the influence of adult involvement, andthose that have typically examine adult involvement in structured or teacher-ledactivities (e.g., Karners, Johnson, & Beauchamp, 1989; McWilliam & Bailey,1995). Because the current study was part of a research project designed toexamine the influence of teacher interaction behaviors on child engagement(McWilliam, Scarborough, & Kim, 1999), the teacher was the focus of all tapedobservations and was therefore present (i.e., interacting with the child or withchildren in the same activity) in all sessions (i.e., in structured activities as wellas free play and mealtimes). Teachers’ interactions with children may haveequalized typical age and maturity advantages in terms of higher level engage-ment behaviors. As mentioned before, we observed how teachers changed thenature of the children’s activity when trying to join children in their play, oftenwithout being aware of it. This raises the question whether the presence of theteacher reduced the amount of time developmentally more mature children spentin sophisticated behaviors, increased the amount of time less mature childrenspent in more sophisticated behaviors, or both. Moreover, these findings empha-size the influence of teacher interaction behaviors on children’s engagement.Given the assumption that the quality of engagement is a critical mediating factorin children’s learning, it is important to help teachers to become aware of howtheir interaction behaviors may help children to improve the quality of theirengagement.

The finding that neither global attention nor observed attention directly con-tributed to the multivariate relationship among the variables included in the


analysis was surprising, especially because the children in this study spent aconsiderable amount of time attending (i.e., 33.13%) to a teacher or a peer. Basedon the literature it was expected that an increase in developmental age would berelated to more frequent engagement in sophisticated behaviors and less frequentengagement in lower level behaviors such as undifferentiated behavior andattention (McWilliam & Bailey, 1995; Blasco et al., 1993). Reviewing theliterature on the development of attentional behavior in young children revealedreports of conflicting results as to how attentional behavior is related to moresophisticated behaviors. For example, Bornstein and Colombo and their col-leagues (see Bornstein & Sigman, 1986; Colombo & Mitchell, 1990) suggestedthat the duration of looking is negatively related to better functioning in terms ofnovelty preferences, language development, and later performance on cognitivetests. These findings have been interpreted in terms of the speed and efficiency ofprocessing; children who spend less time looking are more efficient in processinginformation. Or it may be that children who are more competent to act on theirenvironment are less content passively observing (Ruff & Saltarelli, 1993). Otherresearchers, however, have reported a positive relationship between children’sattention and competence in play, future performance, and academic achievement(Jennings, Harmon, Morgan, Gaiter, & Yarrow, 1979; Palisin, 1986; Ruff, 1988).Greater attentiveness is likely to lead to new information about the environmentthat, in turn, provides a better foundation for children’s developing knowledgeand cognition (Ruff, 1990).

The confusion in the findings of the role of attention in young children’slearning is not new and is related to the way attention is defined in differentstudies. When attention is defined as the duration of looking at, but not manip-ulating or interacting with, external events or objects, attention is negativelyrelated to better functioning. But, when attention is defined as looking at an objectwhile manipulating it, duration of attention is a predictor of better cognitivefunctioning at later ages (Ruff & Saltarelli, 1993; Tamis-LeMonda & Bornstein,1993). Since we defined attention as the amount of time children look at and listento features in the environment without manipulating toys, our findings are notconsistent with what is reported in the literature. Although attentional behavior isat times appropriate behavior for all children, it is a less active level of engage-ment and we would not expect more mature children to spend a large percentageof their time merely looking. Again, however, these findings may be typical whenobserving children’s engagement in the presence of the teacher. Observed atten-tion was not directly related to the other variables in the analysis, but it contrib-uted an appreciable amount by making the relationships among the other variablesstronger.

The findings of this study must be interpreted with regard to some limitationsnot yet addressed. First, our method of data collection (i.e., videotaping) may nothave been as natural as intended. Although the teachers were free to choose whattypes of session were taped in the first couple of sessions, they were subsequentlytold which sessions were needed. In addition, the teachers were asked duringmealtime to move around instead of sitting at one table to capture the interactionwith as many children as possible in one session. On the other hand, we taped all


sessions in natural environments (as opposed to laboratory situations) and atnatural times of the day (vs. out-of-context situations).

Second, it has been suggested that canonical weights and loadings are mostreliable for detecting the most important variables and for interpreting the canon-ical variates when there is a large ratio of sample size to the number of variables.Barcikowski and Stevens (1975) suggested a ratio of at least 42:1 for interpretingthe first two sets of canonical variations and about 20:1 for interpreting only thefirst set of canonical covariates. Thompson (1990), however, reported that esti-mates of the canonical correlations and eigenvalues appear to be reasonably stableif the researcher uses at least 5–10 subjects per variable. With a subject to variableratio of 6.2:1 we can be relatively confident that the results of our analysis are“reasonably stable.”

In conclusion, this study shows that, as children mature, their engagementobserved in the presence of the teacher becomes more competent and they shedsome lower level behaviors. This seemingly obvious finding is important forassessment and intervention planning. With regard to assessment, teachers shouldassess both global and observed engagement. Rating children’s global engage-ment and observing their engagement in classroom settings gives caregivers(parents and teachers) information about functional activities from which they candevelop intervention or curricular goals. With regard to intervention planning,teachers should be aware that children rated as less competent and persistent arelikely to have problems with participation (i.e., teachers should plan activities thatencourage children’s active involvement with adults, peers, and materials in theclassroom). In addition, if a child’s global rated persistence is high, teachersshould ensure that the child has lots of opportunities for high level engagement.These opportunities can include such practices as using problem solving materi-als, challenging activities, “sabotaged” situations, housekeeping centers, dramaticplay activities, and informal pretend play interactions.

Furthermore, these findings have implications for faculty involved in personneldevelopment, administrators, and researchers. Personnel development facultyshould (a) help teachers become aware of the importance of engagement inchildren’s learning, (b) introduce teachers to global and observational measuresthat are helpful in determining children’s level of engagement, (c) introduceteachers to developmentally appropriate practices that promote high levels ofengagement for all children, and (d) help teachers become aware of the factors(such as their interaction behaviors) that influence engagement. Administratorsshould provide a structure for assessment of global engagement and observedengagement. Researchers should continue to explore links among intra-individualfactors influencing engagement, especially between ratings of child propensityand observations of behavior.

Acknowledgments: This research was funded by a grant from the Office ofSpecial Education and Rehabilitative Services, U. S. Department of Education,Grant No. HO23C40015. The authors thank the participating teachers and chil-dren. Thanks also to Katherine Harville, Ellen Reilly, Anita Scarborough, Kim-berley Sloper, Virginia Smith, Alyse Sweeney, Don Trull, and Rebecca Zulli.


Finally, the authors are appreciative of specific comments by two anonymousreviewers on an earlier version of this article.

REFERENCES

Barcikowski, R., & Stevens, J. P. (1975). A Monte Carlo study of the stability of canonicalcorrelations, canonical weights and canonical variate-variable correlations.MultivariateBehavioral Research, 10, 353–364.

Blasco, P. M., Bailey, D. B., & Burchinal, M. A. (1993). Dimensions of mastery in same-age andmixed-age integrated classrooms.Early Childhood Research Quarterly, 8, 193–206.

Bornstein, M. H., & Sigman, M. D. (1986). Continuity in mental development from infancy.ChildDevelopment, 57, 251–274.

Brockman, L. M., Morgan, G. A., & Harmon, R. J. (1988). Mastery motivation and developmentaldelay. In T. D. Wachs & R. Sheenan (Eds.),Assessment of young developmentally disabledchildren. Perspectives in developmental psychology(pp. 267–284). New York, NY: PlenumPress.

Buysse, V., & Bailey, D. B. (1993). Behavioral and developmental outcomes in young children withdisabilities in integrated and segregated settings: A review of comparative studies.Journal ofSpecial Education, 26, 434–461.

Carta, J. J., & Greenwood, C. R. (1985). Eco-behavioral Assessment: A methodology for expandingthe evaluation of early intervention programs.Topics in early Childhood Special Education,5(2), 88–103.

Colombo, J., & Mitchell, D. W. (1990). Individual differences in early visual attention: Fixationtime and information processing. In J. Colombo & J. W. Fagan (Eds.),Individual differencesin infancy: Reliability, stability, and prediction.Hillsdale, NJ: Erlbaum.

Doke, L. A., & Risley, T. R. (1972). The organization of daycare environments: Required versusoptional activities.Journal of Applied Behavior Analysis, 5, 405–420.

Dunst, C. J., McWilliam, R. A., & Holbert, K. (1986). Assessment of preschool classroomenvironments.Diagnostique, 11, 212–232.

Greenwood, C. R., & Carta, J. J. (1987). An ecobehavioral interaction analysis of instruction withinspecial education.Focus on Exceptional Children, 19(9), 1–12.

Hollingshead, A. B. (1975).Four-factor index of social status.Unpublished manual. Department ofSociology, Yale University, New Haven, CT.

Jennings, K. D., Harmon, R. J., Morgan, G. A., Gaiter, J. L., & Yarrow, L. J. (1979). Exploratoryplay as an index of mastery motivation: Relationships to persistence, cognitive functioning,and environmental measures.Developmental Psychology, 15, 386–394.

Jones, H. A., & Warren, S. F. (1991). Enhancing engagement in early language teaching.TeachingExceptional Children, 23(4), 48–50.

Karnes, M. B., Johnson, L. J., Beauchamp, K. D. F. (1989). Developing problem-solving skills toenhance task persistence in handicapped preschool children.Journal of Early Intervention,13(1), 61–72.

Krantz, P., & Risley, T. R. (1977). Behavioral ecology in the classroom. In K. D. O’Leary & S. G.O’Leary (Eds.),Classroom management: The successful use of behavior modification(2nded., pp. 349–367). New York: Pergamon.

MacTurk, R. H., Hunter, F. T., McCarthy, M. E., Vietze, P. M., & McQuiston, S. (1985). Socialmastery motivation in Down syndrome and nondelayed infants.Topics in Early ChildhoodSpecial Education, 4(4), 93–109.

Malone, D. M., Stoneman, Z., & Langone, J. (1994). Contextual variation of correspondencesamong measures of play and developmental level of preschool children.Journal of EarlyIntervention, 18(2), 199–215.

McClannahan, L. E., & Risley, T. R. (1975). Design of living environments for nursing-homeresidents: Increasing participation in recreation activities.Journal of Applied BehaviorAnalysis, 8(3), 261–268.


McCormick, L., Noonan, M. J., & Heck, R. (1998). An exploration of variables affecting engage-ment in inclusive preschool classrooms.Journal of Early Intervention, 21(2), 160–176.

McGarity, J. R., & Butts, D. P. (1984). The relationship among teacher classroom managementbehavior, student engagement, and student achievement of middle and high school sciencestudents of varying aptitude.Journal of Research in Science Teaching, 2(1), 55–61.

McWilliam, R. A. (1991).Children’s Engagement Questionnaire. Chapel Hill, NC: Frank PorterGraham Child Development Center, University of North Carolina at Chapel Hill.

McWilliam, R. A. (1995).E-Qual Observational Coding System. Chapel Hill, NC: Frank PorterGraham Child Development Center, University of North Carolina at Chapel Hill.

McWilliam, R. A., & Bailey, D. B. (1991).The effects of activity structure and age mixture on childengagement. Chapel Hill, NC: Frank Porter Graham Child Development Center, Universityof North Carolina at Chapel Hill.

McWilliam, R. A., & Bailey, D. B. (1992). Promoting engagement and mastery. In D. B. Bailey &M. Wolery (Eds.),Teaching infants and preschoolers with disabilities(2nd ed., pp. 230–255). New York: MacMillan Publishing Company.

McWilliam, R. A., & Bailey, D. B. (1995). Effects of classroom social structure and disability onengagement.Topics in Early Childhood Special Education, 15(2), 123–147.

McWilliam, R. A., Scarborough, A. A., & Kim, H. (1999).Adult interactions and child engagement.Manuscript under review.

McWilliam, R. A., Snyder, P., & Lawson, S. (1993).Dependability and factorial validity of theChildren’s Engagement Questionnaire: A broad-based child status measure for early inter-vention. Paper presented at the DEC Conference, San Diego, CA.

McWilliam, R. A., Trivette, C. M., & Dunst, C. J. (1985). Behavior engagement as a measure of theefficacy of early intervention.Analysis and Intervention in Developmental Disabilities, 5,33–45.

Messer, D. J., Rachford, D., McCarthy, M. E., & Yarrow, L. J. (1987). Assessment of masterymotivation behavior at 30 months: Analysis of task-directed activities.Developmental Psy-chology, 23, 771–781.

Montes, F., & Risley, T. R. (1975). Evaluating traditional day care practices: An empirical approach.Child Care Quarterly, 4, 208–215.

Morgan, G. A., Maslin-Cole, C., Harmon, R. J., Busch-Rossnagel, N. A., Jennings, K. D.,Hauser-Cram, P., & Brockman, L. M. (1993). Parent and teacher perceptions of youngchildren’s mastery motivation. Assessment and review of research. In D. J. Messer (Ed.),Mastery motivation in early childhood. Development and measurement and social processes(pp. 109–122). London: Routledge.

Morgan, G. A., Maslin, C. A., & Harmon, R. J. (1987).Dimension of mastery questionnaire.Unpublished instrument. Fort Collins: Department of Human Development and FamilyStudies, Colorado State University.

Newborg, J., Stock, J. R., Wnek, L., Guidubaldi, J., & Svinicki, J. (1984).Battelle DevelopmentalInventory. Allen, TX: DLM Teaching Resources.

Ostrosky, M. M., & Kaiser, A. P. (1991). Preschool classroom environments that promote com-munication.Teaching Exceptional Children, 23(4), 6–10.

Palisin, H. (1986). Preschool temperament and performance on achievement tests.DevelopmentalPsychology, 22, 766–770.

Rogers-Warren, A. K. (1982). Behavioral ecology in classrooms for young handicapped children.Topics in Early Childhood Special Education, 2(1), 21–32.

Rosenshine, B. V. (1978). Academic engaged time, content covered, and director instruction. In H.Leitenberg (Ed.),Handbook of behavior modification and behavior(pp. 453–474). Engle-wood Cliffs, NJ: Prentice-Hall.

Ruff, H. A. (1988). The measurement of attention in high-risk infants. In P. Vietze & H. G. Vaughan(Eds.),Early identification of infants with developmental disabilities. Philadelphia: Grune &Stratton.

Ruff, H. A. (1990). Individual differences in sustained attention during infancy. In J. Colombo & J.Fagen (Eds.),Individual differences in infancy: Reliability, stability, prediction(pp. 247–270). Hillsdale, NJ: Erlbaum.


Ruff, H. A., & Saltarelli, L. M. (1993). Exploratory play with objects: Basic cognitive processes andindividual differences. In M. H. Bornstein & A. W. O’Reilly (Eds.),The role of play in thedevelopment of human thought(pp. 5–16). San Francisco, CA: Jossey-Bass Publishers.

Tabachnick, B. G., & Fidell, L. S. (1996).Using multivariate statistics(3rd ed.), New York NY:Harper Collins.

Tamis-LeMonda, C. S., & Bornstein, M. (1993). Play and its relations to other mental functions inthe child. In M. H. Bornstein & A. W. O’Reilly (Eds.),The role of play in the developmentof thought. San Francisco, CA: Jossey-Bass Publishers.

The Observer 3.0 [Computer software]. (1993). Wageningen, The Netherlands: Noldus InformationTechnology.

Thompson, B. (1984).Canonical correlation analysis: Uses and interpretation. Newbury Park, CA:Sage.

Thompson, B. (1990). Finding a correction for the sampling error in multivariate measures ofrelationship: A Monte Carlo study.Educational and Psychological Measurement, 50, 15–31.

Thompson, B. (1991). Methods, plainly speaking: A primer on the logic and use of canonicalcorrelation analysis.Measurement and Evaluation in Counseling and Development, 24,80–93.

Thompson, B. (in press). Canonical correlation analysis. In L. Grimm & P. Yarnold (Eds.),Readingand understanding multivariate statistics(Vol. 2). Washington, DC: American PsychologicalAssociation.

Twardosz, S., & Risley, T. R. (1982). Behavioral ecological consultation to daycare centers. In A.Jean & R. Slotnick (Eds.),Community mental health and behavioral ecology: A handbook oftheory, research, and practice(pp. 147–159). New York: Plenum.

Twardosz, S., Cataldo, M. F., & Risley, T. R. (1974). Open environment design for infant andtoddler day care.Journal of Applied Behavior Analysis, 7, 529–546.

Warren, S. F., & Kaiser, A. P. (1986). Incidental language teaching: A critical view.Journal ofSpeech and Hearing Disorders, 51, 291–298.

Whaley, K., & Bennett, T. C. (1991). Promoting engagement in early childhood social education.Teaching Exceptional Children, 23(4), 51–54.

Wherry, R. J. (1931). A new formula for predicting the shrinkage of the coefficient of multiplecorrelation.Annals of Mathematical Statistics, 2, 440–451.


Multivariate relationships among developmental age, global engagement, and observed child engagement

Documents