Advancing a multidimensional, developmental spectrum approach to preschool disruptive behavior

82

EW RESEARCH
N
Advancing a Multidimensional,Developmental Spectrum Approach to

Preschool Disruptive BehaviorLauren S. Wakschlag, PhD, Margaret J. Briggs-Gowan, PhD, Seung W. Choi, PhD,

Sara R. Nichols, PhD, Jacqueline Kestler, MPH, James L. Burns, MS,Alice S. Carter, PhD, David Henry, PhD

Objective: Dimensional approaches are gaining scientific traction. However, their potentialfor elucidating developmental aspects of psychopathology has not been fully realized. The goalof this article is to apply a multidimensional, developmental framework to model the normal–abnormal spectrum of preschool disruptive behavior. The Multidimensional Assessment ofPreschool Disruptive Behavior (MAP-DB), a novel measure, was used to model dimensionalseverity across developmental parameters theorized to distinguish the normative misbehaviorof early childhood from clinically salient disruptive behavior. The 4 MAP-DB dimensions areTemper Loss, Noncompliance, Aggression, and Low Concern for Others. Method: Parents ofa diverse sample of 1,488 preschoolers completed the MAP-DB. Multidimensional itemresponse theory (IRT) was used for dimensional modeling. Results: The 4-dimensional,developmentally informed model demonstrated excellent fit. Its factor loadings did not differacross demographic subgroups. All dimensions provided good coverage of the abnormal endof the severity continuum, but only Temper Loss and Noncompliance provided good coverageof milder, normatively occurring behaviors. The developmental expectability and quality ofbehaviors distinguished normative from atypical behaviors. The point at which frequency ofbehaviors was atypical varied based on dimensional location for Temper Loss, Noncompliance,and Aggression. Conclusion: The MAP-DB provides an innovative method for operation-alizing developmentally specified, dimensional phenotypes in early childhood. Establishing thevalidity of these dimensional phenotypes in relation to clinical outcomes, neurocognitivesubstrates, and etiologic pathways will be a crucial test of their clinical utility. J. Am. Acad.Child Adolesc. Psychiatry, 2014;53(1):82–96. Key Words: developmental psychopathology,dimensional, disruptive behavior, externalizing spectrum, preschool

isruptive behavior (DB) plays a central rolein developmental sequences of psychopa-
D thology. It is antecedent to up to 60% of
common mental disorders across the lifespan,1,2

often emerges in early childhood,3 and is themost prevalent disorder of the preschool period.4

One reason for DB’s centrality to both internal-izing and externalizing disorders is the hete-rogeneity of its defining features. For example,irritability is a key feature of oppositional defiantdisorder (ODD) and is also central to a number of

Supplemental material cited in this article is available online.

JOURN

www.jaacap.org

other disorders with disrupted emotion regulation(e.g., depression).5 In contrast, aggressive behav-iors, particularly callous aggression, are associatedwith a distinct antisocial pathway.6 The utility of amultidimensional approach to ODD for clinicalprediction has been robustly demonstrated: dis-aggregation into dimensions such as “irritable,”“headstrong,” and “hurtful” differentiates inter-nalizing and externalizing patterns and their co-occurrence.7-11 Conduct disorder (CD) subtypes,including aggressive and nonaggressive rulebreaking and callousness, also distinguish variedclinical risk profiles.12,13 Parsing the heterogeneityof emergent disruptive behavior may illuminateearly markers of divergent developmental andclinical pathways. In particular, modeling the

AL OF THE AMERICAN ACADEMY OF CHILD & ADOLESCENT PSYCHIATRY

VOLUME 53 NUMBER 1 JANUARY 2014

http://www.jaacap.org

DIMENSIONS OF EARLY DISRUPTIVE BEHAVIOR

dimensional structure of early childhood DB isimportant for mapping the prodromal phasesof clinical patterns, linking them to underlyingmechanisms, and targeting prevention beforethe onset of psychopathological conditions.14

Application of a multidimensional, develop-mental framework for early childhood DB is thegoal of the present paper.

An important first step toward testing DBdimensional models in young children wastesting the application of existing dimensionalframeworks to early childhood. This has beendemonstrated in clinical and community samplesof preschoolers with patterns similar to those inolder youth.15,16 There is also some evidencethat callous behaviors are distinguishable at pre-school age, with short-term predictive utility.17-19

However, most dimensional approaches to datehave used DSM symptom sums; their deve-lopmental impossibility/improbability (CD) and/or imprecision (ODD) for capturing early child-hood clinical patterns has been previously noted.3

A key limitation is that dimensions comprisingsymptoms that capture only severe behaviorscannot provide an ordered metric for character-izing the dimensions along a normal–abnormalcontinuum.20 Thus, an important follow-on foradvancing a developmental framework is oper-ationalizing dimensions in a manner that ad-dresses measurement challenges inherent inassessing psychopathological processes in earlychildhood. These challenges include the highlevel of behavioral variability and the overlapbetween normative misbehavior and disrup-tive behavior. Such a developmental approachcaptures the developmental variation of earlychildhood by characterizing a continuum of nor-mal to abnormal behaviors, with atypicality de-rived from deviation from expectable patternswithin the age period.3,21,22 This requires concep-tualization of dimensions that are linked tonormative developmental processes and oper-ationalization of age-typical behavioral expression.

We previously proposed and provided a pre-liminary test of a developmentally-informedmodel of DB in early childhood with 4 distinctdimensions linked to core developmental pro-cesses of the preschool period:3,15 Temper Lossand regulation of frustration; Noncompliance andinternalization of rules; Aggression and capac-ity to modulate aggressive tendencies; and LowConcern for Others and the emergence of em-pathy and conscience (the theoretical rationalefor these dimensions has been extensively

JOURNAL OF THE AMERICAN ACADEMY OF CHILD & ADOLESCENT PSYCHIATR


discussed).3,15 Using secondary data analysis,this prior study demonstrated the superior fitof a 4-dimensional model compared to aDSM-oriented 2-dimensional ODD/CD model,an irritable/headstrong/hurtful model, and aDB/callous model.15 However, prior testing ofthis multidimensional model was constrainedby the use of existing measures that were notdeveloped for this purpose. For example, the useof DSM symptoms to comprise dimensions has aconstricted range (focuses on extreme behaviors),has the same symptoms for all age periods, anddoes not provide full coverage across dimen-sional spectra.

Here we use Item Response Theory (IRT)23 totest the 4-dimensional, developmentally-informedmodel with a novel measure, the Multidimen-sional Assessment of Preschool Disruptive Be-havior (MAP-DB). IRT is useful for dimensionalmodeling because it can map the locations of bothitems and respondents on an underlying latenttrait continuum, scaled from mild, commonlyoccurring behaviors to severe, rarely occurringbehaviors. Within the framework of IRT, behav-iors are psychometrically defined as “abnormal”or severe when they are rarely occurring (e.g., inless than 5% of the population).

To operationalize behaviors along the normal–abnormal spectrum for early childhood, theMAP-DB incorporates assessment of behavioralfrequency, quality, and context. These parametersmay provide more nuanced distinction betweennormative and clinically-concerning behaviorsin this age period.24-26 Milder, normative mis-behaviors were theorized to occur in devel-opmentally expectable contexts (e.g., “whenfrustrated”), whereas atypical behaviors weretheorized to occur in developmentally unexpect-able contexts (e.g., “out of the blue”). Qualita-tively atypical behaviors were conceptualizedin terms of intensity (e.g., “hurt someone onpurpose”), dysregulation (e.g., “difficulty calm-ing down after tantrum”), intransigence (e.g.,“refuse to do as asked, no matter what”), andprovocativeness (e.g., “persist in scaring or up-setting someone”). To test the theory that evennormative misbehaviors would be atypical if theyoccurred at higher than average frequencieswe used an objective frequency format (i.e., rat-ings of how often the behavior actually occurred).These are in contrast to subjective ratings (e.g.,“never,” “sometimes,” “often”), which may givethe same rating (e.g., “often”) to varying fre-quencies depending on factors influencing the

Y

www.jaacap.org 83


WAKSCHLAG et al.

judgment of the rater. Given the rapidity ofchange in developmental capacity across thepreschool period, and the centrality of genderdifferences in psychopathology expression, test-ing for age and gender differences is also animportant aspect of this developmental frame-work.27,28

The MAP-DB was used to test the theorizeddimensional spectrum of preschool disruptivebehavior in a large, sociodemographically diversesample of preschool children. The aims of thepresent study were as follows: to test whether thedata supported the hypothesized 4-dimensional,developmentally informed model and its robust-ness to sociodemographic variation, and to testthe hypothesis that location along the dimen-sional severity continuum would vary based oncontext, quality, and frequency.

METHODSample and ProceduresThe Multidimensional Assessment of Preschoolers(MAPS) Study comprises a large, sociodemogra-phically stratified sample of preschoolers recruitedfrom 5 pediatric clinics in the Chicago area.29 All studyprocedures were approved by institutional reviewboards, and parental informed consent was obtained.

Parents were eligible for MAPS if they were thelegal guardian of a 3- to 5-year-old child present at theclinic. These children received well-child care at thepediatric clinic. All children were eligible provided thattheir parent had not already participated in the studyfor a sibling. Psychiatric referral history was notcollected at screening; however, any psychiatricallyreferred children who met other eligibility criteriawere eligible to participate. Of 1,814 parents whowere eligible for the survey, 1,606 (88.5%) consentedand 1,516 completed surveys (94.4% completion rateof all consented; 83.6% of all eligible). Compared withthe 298 eligible parents who did not participate, the1,516 participants were significantly more likely to befemale (94.3% versus 88.6%), of minority ethnicity(68.6% versus 59.9%), or from 1 of the Chicago-basedclinics (85.6% versus 76.8%) (all p < .01). However,results of a logistic regression model revealed that onlyrecruitment source and female caregiver uniquelypredicted participation. Sample size was designed toprovide power of at least 0.8 to detect small (0.10)differences in loadings across the 4 key sociodemo-graphic strata: child age, gender, poverty status, andethnicity.

A total of 26 children with autism spectrum disorderwere excluded (due to insufficient numbers to provideinformative data on this important subgroup) and 2participants with more than half the MAP-DB datamissing were also excluded. Thus, the analytic sample

JOURN

84 www.jaacap.org

was 1,488. Of In all, 97% were biological parentsand 91% were mothers. Consistent with the MAPSstratification goals, the sample was fairly evenlydistributed by child gender (49% girls, 51% boys), age(35% 3-year-olds, 36% 4-year-olds, 29% 5-year-olds),race/ethnicity (36% African American, 36% Hispanic,27% non-Hispanic white, 1% other), and poverty status(42% below federal poverty level based on annualhousehold income and household size).30 A demo-graphically comparable test–retest sample was alsoassessed (n ¼ 76). This sample size is sufficient for apower of 0.8 to detect a test–retest correlation of 0.28,a magnitude that indicates adequate test–retest reli-ability of construct stability. The survey, which con-sisted of the MAP-DB, demographic information, andbrief information on disruptive behavior correlates,was administered in English or Spanish. A $20 incen-tive was provided for survey completion, with a $10in-clinic completion bonus.

MAP-DB MeasureThe MAP-DB was developed by a team of experts inearly childhood, clinical assessment and treatment, anddevelopmental epidemiology. The iterative measuredevelopment process included theoretical delineation ofthe core dimensions,3 preliminary validation usingsecondary data,15 pilot testing, focus groups, and re-view of extant measures. Measurement developmentwas led by the first and second authors and includedthe following: construct delineation; item generation;review of item pool to assess how well the itemcaptured the intended dimension; ease of comprehen-sion and wording; and deciding which items to retain inthe final pool. Items for which there was disagreementwere retained for empirical determination of fit. A rangeof interactional contexts (e.g. with parents, with otheradults) and contextual antecedents (e.g. “when tired,hungry or sick, ” for no reason or out of the blue”) wereincluded, with variability by dimension depending oncontextual salience. The final item pool for the presentanalyses comprised 111 MAP-DB items. A Spanishversion was generated via certified translation andback-translation. Ratings were done on a 6-point scale(0 ¼ never; 1 ¼ rarely [less than once per week]; 2 ¼some [1–3] days of the week; 3 ¼ most [4–6] days of theweek; 4 ¼ every day of the week; 5 ¼ many times eachday), within the range of optimal number of responseoptions for the use of IRT in health assessment.31

Temper Loss items ranged from normative expres-sions, such as tantrums in the face of frustration, tointense, dysregulated tantrums (22 items).29 Noncom-pliance items ranged from normative refusal to followdirections to provocative and recalcitrant disobedience(30 items). Aggression items included normal reactiveaggression and abnormal manifestations of inten-tional, hostile aggression (44 items). Low Concern forOthers items included disregard for others’ feelingsand pleasure in others’ distress (15 items).





Data Reduction and ModelingModeling of the MAP-DB was guided by both theoryand data to extract factors as parsimoniously aspossible, to minimize interfactor correlations, to bestrepresent the dimensional spectrum, and to discrimi-nate well between children with high versus low levelsof each dimension. Of the original 111 items, 78 wereretained via this process. The fit of the 4-dimensiontheoretical model was tested with confirmatory factoranalysis (CFA), a method for testing theories about thestructure of items, conducted in Mplus 6.11,32 testedthe fit of the 4-dimension theoretical model. Weexamined the comparative fit index (CFI) and the rootmean square error of approximation (RMSEA) statisticsto assess the extent to which the model was a good fitto the data. RMSEA values less than 0.08 and CFIvalues greater than 0.90 are considered indicators ofacceptable fit.33 We also calculated Cronbach’s alpha(a) coefficient as an indicator of the internal reliabilityof the dimensions. The Bayesian information criterion(BIC) statistic was used to compare whether the samefactor structure fit best across demographic subgroups(differences of 10 on the BIC are considered strongevidence of improved fit, with lower BIC scoresreflecting improvement34,35). Finally, the IRTPRO36

program used IRT methods to estimate the severityof items along each dimension. Higher severity scoresindicate “more difficult” items that are less commonlyendorsed.

RESULTSAim 1: To Examine the Fit of the 4-Dimensional,Developmentally-Informed ModelStatistical “fit” refers to the extent to which amodel is an adequate representation of the actualdata. When fit is not adequate, alternativemodels should be explored. Fit is measured inthis study using the RMSEA on which values lessthan 0.05 are considered to indicate excellent fitand the CFI on which values greater than 0.90suggest good fit and values greater than 0.95excellent fit. The 4-dimensional model fit the datawell according to both indicators (RMSEA ¼0.048; CFI ¼ 0.936). All 4 dimensions demon-strated excellent internal consistency (TemperLoss, a ¼ 0.97; Noncompliance, a ¼ 0.96;Aggression, a ¼ 0.96; Low Concern, a ¼ 0.92)(item-level loadings are provided in Table S1,available online). Multigroup comparisons acrosschild age, gender, race/ethnicity, and povertystatus demonstrated equivalent fit across sub-groups (indicated by the lower adjusted BICshown in Table S2, available online), suggestingthe generalizability of this 4-dimensional modelacross variable sociodemographic groups. There



were, however, mean differences in scores acrossthe subgroups by child gender, age, and ethnicity(Table S3, available online). In particular, therewere consistent differences by gender (boys’scores higher on all dimensions), age (3- to 4-year-olds had higher scores than 5-year-olds onTemper Loss, Noncompliance, and Aggression;and 3-year-olds had higher scores than 5-year-olds on Low Concern), and ethnicity (children ofwhite ethnicity had higher means on all di-mensions, relative to Hispanic or African Amer-ican children [who did not differ from eachother]). There were no differences by povertystatus. Mothers and other informants did notdiffer in response patterns (data available fromfirst author).

The 4-dimensional model also demonstratedsuperior fit relative to established alternativemodels constructed by regrouping MAP-DB di-mensions (e.g., a 3-dimensional irritable [Tem-per Loss]/headstrong [Noncompliance]/hurtful[Aggression þ Low Concern] model) (Table S4,available online), consistent with prior work inseveral independent samples.15,37

Test–retest reliability of the dimensions wasbased on completion of the MAP-DB twice overan average period of 6 months (mean ¼ 3.44months, SD¼ 1.27months). Intraclass correlationsrevealed good to excellent stability for the di-mensions (Temper Loss ¼ 0.80, Noncompliance ¼0.81, Aggression ¼ 0.85, Low Concern ¼ 0.83).

Aim 2: To Model the Dimensional SeveritySpectrumThe severity continuum is like a ruler measuringthe overall severity of a child’s symptoms on eachdimension. The dimensions themselves are latentvariables that cannot be directly measured. Theseverity of a child’s behaviors can only be indi-cated by the reports collected on each item. Just aseach child can be given a score on the severitycontinuum, each item can be assigned a place onit. The category thresholds of each item representthe severity of the behavior measured at differentfrequencies of occurrence. Table 1 provides thecategory thresholds and item severity (“location”)parameter for items along each dimension. Thecategory thresholds (b1–b5) represent the esti-mated level of the latent trait at which the proba-bility exceeds 50% that a respondent will choosethe next higher category (e.g., choosing “everyday” rather than “on most days”). Together, theyprovide a measure of the item location (indicatingits severity) on the latent trait scale. The item

Y

www.jaacap.org 85


TABLE 1 Multidimensional Assessment of Preschool Disruptive Behavior Dimensional Severity Thresholds

Category Thresholds

Item LocationMean (b)

Rarely orHigher b1

Some Days ofWeek orHigher b2

Most Days orHigher b3

Every Day orHigher b4

Many Times aDay b5

Temper Lossa (95th percentilethreshold ¼ 1.60)

1. Lose temper or have atantrum when frustrated,angry, or upset

1.24 �0.30 0.72 1.43 1.97c 2.38

2. Lose temper or have atantrum to get somethings/he wanted

1.29 �0.26 0.74 1.49 2.05 2.45

3. Have a temper tantrum,fall-out, or melt-down

1.29 �0.17 0.78 1.50 1.96 2.41

4. Lose temper or have atantrum when tired,hungry, or sick

1.31 �0.31 0.68 1.53 2.13 2.53

5. Become frustrated easily 1.33 �0.52 0.65 1.61 2.21 2.696. Lose temper or have a

tantrum during dailyroutines

1.34 �0.26 0.71 1.62 2.05 2.56

7. Lose temper or have atantrum with parents

1.36 �0.17 0.81 1.56 2.06 2.54

8. Yell angrily at someone 1.43 �0.42 0.80 1.71 2.29 2.789. Have a short fuse 1.45 �0.09 0.89 1.64 2.17 2.6510. Have difficulty calming

down when angry1.58 �0.11 0.97 1.81 2.40 2.82

11. Have a hot or explosivetemper

1.66 0.42 1.23 1.80 2.21 2.64

12. Get extremely angry 1.66 0.30 1.18 1.89 2.27 2.6613. Act irritable 1.66 �0.27 0.93 1.94 2.60 3.1114. Stamp feet or hold breath

during a temper tantrum1.71 0.09 1.04 1.92 2.55 2.96

15. Keep on having a tempertantrum even when youtried to help calm down

1.76 0.40 1.32 2.00 2.39 2.71

16. Have a temper tantrumlasting >5 min.

1.79 0.20 1.17 2.12 2.57 2.90

17. Lose temper or have atantrum with other adults

1.91 0.36 1.43 2.23 2.62 2.93

18. Break or destroy thingsduring a temper tantrum

1.92 0.60 1.49 2.13 2.50 2.89

19. Have a temper tantrum untilexhausted

1.95 0.67 1.49 2.14 2.54 2.90

20. Lose temper or have atantrum “out of the blue”

1.96 0.64 1.49 2.17 2.56 2.96

21. Hit, bite, or kick during atemper tantrum

2.04 0.78 1.58 2.12 2.69 3.05

22. Stay angry for a long time 2.30 0.58 1.91 2.62 3.06 3.34

Noncompliance (95th percentilethreshold ¼ 1.61)

23. Break rules even when s/heknew you were watching

0.52 �1.93 �0.38 1.08 1.67 2.18

24. Argue when asked to dosomething

0.82 �0.99 0.10 1.09 1.64 2.26

JOURNAL OF THE AMERICAN ACADEMY OF CHILD & ADOLESCENT PSYCHIATRY

86 www.jaacap.org VOLUME 53 NUMBER 1 JANUARY 2014

WAKSCHLAG et al.


TABLE 1 Continued

Category Thresholds


Rarely orHigher b1




Many Times aDay b5

25. Act stubborn 0.96 �0.93 0.25 1.21 1.78 2.4926. Disobey or break rules with

parents1.05 �0.80 0.44 1.35 1.89 2.38

27. Say “no” when told to dosomething

1.14 �0.74 0.47 1.35 1.94 2.69

28. Disobey or break rulesduring daily routines

1.15 �1.01 0.37 1.54 2.07 2.78

29. Act sassy, talk back, or havea “smart mouth”

1.15 �0.60 0.54 1.39 1.93 2.52

30. Disobey or break rules whenfrustrated, angry, or upset

1.19 �0.59 0.48 1.49 1.97 2.63

31. Do exactly what you justsaid not to do

1.25 �0.71 0.78 1.63 2.08 2.46

32. Refuse to follow directions 1.37 �0.42 0.79 1.64 2.17 2.6633. Disobey or break rules with

other adults1.37 �0.75 0.61 1.84 2.31 2.85

34. Ignore directions 1.39 �0.58 0.77 1.73 2.24 2.8135. Disobey or break rules “out

of the blue”1.53 �0.21 0.91 1.83 2.27 2.86

36. Disobey or break rules whentired, hungry or sick

1.59 �0.20 0.92 1.83 2.37 3.03

37. Disobey or break rules toget something s/hewanted

1.66 0.08 1.11 1.92 2.35 2.85

38. Not do what you asked nomatter what

1.68 �0.29 0.92 1.99 2.52 3.25

39. Argue about just aboutanything

1.69 0.18 1.14 1.85 2.36 2.91

40. Automatically resistwhatever you ask

1.73 0.05 1.17 1.99 2.42 3.00

41. Take things s/he was notallowed to have

1.76 �0.21 1.10 2.12 2.60 3.18

42. Show off or laugh whilemisbehaving

1.99 0.28 1.33 2.22 2.85 3.26

43. Do risky things s/he knewwere not allowed

2.05 0.16 1.40 2.37 2.88 3.42

44. Misbehave in ways that aredangerous or unsafe

2.52 1.08 2.15 2.77 3.12 3.49

Aggression (95th percentilethreshold ¼ 1.50)

45. Act aggressively whenfrustrated, angry, or upset

1.50 �0.06 1.03 1.80 2.17 2.58

46. Act aggressively to try to getsomething s/he wanted

1.62 0.11 1.13 1.94 2.30 2.62

47. Act aggressively with you orother parent

1.82 0.38 1.37 2.08 2.42 2.84

48. Throw something atsomeone

1.93 0.19 1.32 2.17 2.77 3.20

49. Try to hurt someone to getback at them

2.03 0.59 1.42 2.22 2.76 3.14


VOLUME 53 NUMBER 1 JANUARY 2014 www.jaacap.org 87



TABLE 1 Continued

Category Thresholds


Rarely orHigher b1




Many Times aDay b5

50. Hit someone with an object 2.14 0.40 1.65 2.44 2.83 3.3651. Act aggressively “out of

the blue”2.16 0.97 1.85 2.38 2.74 2.89

52. Break or ruin things onpurpose

2.19 0.74 1.67 2.37 2.84 3.31

53. Act aggressively with otheradults

2.29 0.82 1.80 2.54 2.98 3.34

54. Act aggressively towardother children

2.33 0.81 1.83 2.56 2.96 3.47

55. Do or say mean or “notnice” things to otherchildren

2.33 0.69 1.86 2.65 3.09 3.38

56. Hit, shove, or kick parents 2.38 0.99 1.94 2.58 3.03 3.3557. Get into fights 2.40 0.80 1.88 2.64 3.10 3.5858. Pinch, scratch, or pull

someone’s hair2.42 0.58 1.78 2.84 3.28 3.62

59. Call another child names 2.42 0.78 1.88 2.76 3.09 3.6160. Hit, shove, or kick other

children2.44 0.84 1.87 2.79 3.26 3.45

61. Say or domean or “not nice”things to other childrenbehind their backs

2.49 0.54 1.83 2.85 3.38 3.84

62. Bully someone 2.50 1.17 2.04 2.69 3.02 3.5863. Hurt someone on purpose 2.57 1.01 1.95 2.94 3.30 3.6464. Refuse to let other children

play with him/her2.66 0.43 1.96 3.19 3.62 4.11

65. Threaten 2.71 1.29 2.18 2.96 3.37 3.7566. Hit, shove, or kick other adults 2.73 1.34 2.20 2.90 3.43 3.7967. Tell others not to let someone

play with them2.73 �0.12 1.71 3.21 3.92 4.95

68. Curse 3.14 1.57 2.66 3.52 3.79 4.1869. Spit 3.66 1.56 2.86 4.04 4.50 5.32

Low Concern for Others (95thpercentile threshold ¼ 1.53)

70. Not care about other’sfeelings when frustrated,angry, or upset

1.97 0.52 1.60 2.23 2.58 2.94

71. Not seem to care aboutparent’s feelings

2.02 0.73 1.70 2.32 2.57 2.79

72. Keep on doing somethingthat was scaring orupsetting someone

2.07 0.44 1.47 2.35 2.83 3.26

73. Not seem to care aboutother adults’ feelings

2.10 0.77 1.74 2.48 2.61 2.92

74. Act like s/he did not careabout pleasing otherpeople

2.11 0.58 1.85 2.34 2.77 3.01

75. Act like s/he did not carewhen someone was mador upset

2.17 0.71 1.67 2.44 2.78 3.22


88 www.jaacap.org VOLUME 53 NUMBER 1 JANUARY 2014

WAKSCHLAG et al.


TABLE 1 Continued

Category Thresholds


Rarely orHigher b1




Many Times aDay b5

76. Enjoy making others mad 2.39 0.98 1.85 2.60 3.11 3.4377. Do things to humiliate or

embarrass others2.55 1.36 2.12 2.76 2.96 3.55

78. Act like s/he did not carewhen someone felt bador sad

2.64 1.02 2.18 2.91 3.31 3.81

Note: Data are derived from IRT-graded response model. Numbers in boldface type indicate category thresholds above the 95th percentile.aIn prior work, we documented similar parameters for Temper Loss derived from a unidimensional model29. Temper Loss parameters reported here differ

slightly as they are derived from a multidimensional model that accounts for correlation with the other dimensions.


location (first column in Table 1) represents theaverage of these category thresholds for the item.When this item location value exceeds thethreshold marking the 95th percentile of the sam-ple of children for the dimension, this is an indi-cator of “item severity” in psychometric terms (i.e.,it is atypical because it occurs in <5% of the pop-ulation). Category thresholds and item locationsthat exceed the 95th percentile are shown inboldface type in Table 1. The boldfaced categorythresholds indicate at what specific frequency thatparticular item becomes atypical. For example,item 1, “Lose temper or have a tantrum whenfrustrated, angry or upset,” tends to be severe onlywhen it occurs “every day” or more; item 22, “Stayangry for a long time,” tends to be severe when itoccurs “some days of the week” or higher.

Across the dimensions, items ranged in se-verity from just above the mean (0.52) to morethan 3 standard deviations above the mean (3.66).Across all dimensions, items provided goodcoverage of the severe end (e.g., Aggressionlocation values ranged from 1.5 to 3.66). How-ever, the dimensions varied in the extent to whichthey encompassed the “normal” end of the di-mensional continuum. Temper Loss and Non-compliance provided coverage across this fullspectrum, including 45% to 64% of items belowthe 95th percentile severity threshold. In contrast,behaviors on the Aggression and Low Concerndimensions were more severe and not commonlyoccurring, with all items having item severity atgreater than the 95th percentile.

The ordering of item severities in the left-handcolumn of Table 1 demonstrates the presence of thetheorized severity continuum within each dimen-sion. For example, developmentally-expectable



behaviors (e.g., behaviors that occurred whenfrustrated or during daily routines, includingtantrums, disobedience, aggression, or not caringabout others’ feelings) tended to have lower itemseverity (indicated by their severity threshold inthe right-hand column, many at less than the95th percentile). In contrast, behaviors thatoccurred in developmentally-unexpected con-texts (e.g., hitting a nonparental adult, having atantrum out of the blue) tended to have higheritem severity (>95th percentile). Similarly, acrossall dimensions, behaviors that were qualitativelyatypical, i.e., intense (e.g., hurt someone onpurpose), dysregulated (e.g., have a tantrumlasting >5 min.), intransigent (e.g., automaticallyresist whatever you ask), or provocative (e.g.,enjoy making others mad), were more likely tohave item severity scores at greater than the 95thpercentile.

Examination of the category thresholds forTemper Loss, Noncompliance, and Aggressionrevealed within-dimension variation in terms ofthe response category at which items crossed intoatypicality, which became lower as items pro-gressed across the severity continuum for thedimension. Taking Noncompliance as an example,“Say “no” when told to do something” has a lowitem severity (1.14) and is severe when exhibited“every day” or higher. In contrast, “Misbehaves inways that are dangerous or unsafe” has high itemseverity (2.52) and is severe when exhibited “somedays a week” or higher. In contrast, Low Concerndid not demonstrate this same variation in fre-quency at different levels of severity, perhapsbecause of its more restricted range.

Review of category thresholds also revealedvariation across dimensions in the frequency at

Y

www.jaacap.org 89


FIGURE 1 Ruga and dot plot illustrating higher category thresholds for normative misbehavior versus problemindicator from the Noncompliance dimension. Note: (a) Normative misbehavior. (b) Problem indicator. a“Rug” along thex-axis of each plot is the severity distribution of the dimension being measured. Dots represent the position of each itemcategory threshold along the dimension, with the threshold of atypicality (95th percentile) marked with the dashedvertical line.

WAKSCHLAG et al.

which severity at the 95th percentile tended to bereached. For Noncompliance and Temper Loss,most items were severe when exhibited “most” or“every day” of the week. In contrast, Aggressionand Low Concern items tended to be severe whenexhibited “some” or “most” days of the week.Two Aggression items had a severity threshold ata frequency of “rarely.”

Figure 1a and 1b through Figure 4a and 4bhighlight how item category thresholds vary onthe dimensions according to the quality and fre-quency of a behavior (normative misbehaviorversus qualitatively atypical behaviors). The“rug” along the x-axis of each plot is the severitydistribution of the dimension being measured.The dots represent the position of each itemcategory threshold along the dimension, with thethreshold of atypicality (95th percentile) markedwith the dashed vertical (red) line. In theNoncompliance dimension (Figures 1a–1b) forthe normative misbehavior (“Say “no” when toldto do something”), the 95th percentile (1.61) fallsat daily frequency. In contrast, the 95th percentileoccurs at weekly frequency for a qualitativelyatypical Noncompliance behavior (“Misbehavesin ways that are dangerous or unsafe”). This

JOURN

90 www.jaacap.org

means that engaging in dangerous misbehavioron a weekly basis is as severe as saying no torequests on a daily basis. Figures 2a and 2bthrough 4a and 4b illustrate differences inseverity thresholds for items on the Aggression,Temper Loss, and Low Concern dimensions. Inthe Aggression (Figures 2a and 2b) dimension,the 95th percentile for the normative misbehavior(“Act aggressively when frustrated, angry orupset”) falls at the weekly frequency, whereas the95th percentile for the atypical misbehavior (“Actaggressively to get something s/he wanted”)occurs rarely. In the Temper Loss dimension(Figures 3a–3b), the 95th percentile for thenormative item (“Lose temper or have a tantrumwhen tired, hungry, or sick”) falls at the almostdaily frequency, whereas the 95th percentile forthe atypical item (“Have a tantrum that lastedmore than 5 minutes”) occurs only weekly. In theLow Concern dimension (Figures 4a and 4b), the95th percentile of the comparatively normativemisbehavior (“Not seem to care about your orother parents’ feelings”) falls at weekly, whereasthe 95th percentile for the more atypical misbe-havior (“Act like s/he didn’t care when someoneelse felt bad or sad”) only needs to occur rarely.




FIGURE 2 Ruga and dot plot illustrating higher category thresholds for normative misbehavior versus problemindicator from the Aggression dimension. Note: (a) Normative misbehavior. (b) Problem indicator. a“Rug” along thex-axis of each plot is the severity distribution of the dimension being measured. Dots represent the position of each itemcategory threshold along the dimension, with the threshold of atypicality (95th percentile) marked with the dashedvertical line.


Thus, these figures illustrate how the frequencydiffers for the threshold of typicality for norma-tive versus atypical misbehaviors.

To test the extent to which there was distinct-ness in severity patterns across dimensions, weexamined cross-dimensional severity overlap. Atotal of 118 preschoolers (7.8%) had scores in theatypical range on at least 1 dimension. Within thissubgroup, 31% had atypical scores on 1 dimen-sion, 32% on 2 or 3 dimensions, and 37% on all 4dimensions.

DISCUSSIONOur study findings provide psychometric supportfor a developmentally-informed, dimensionalmodel of preschool disruptive behavior. IRTmethods were used to demonstrate that theoret-ically derived developmental indicators of atypi-cality differentiate behaviors that are commonlyoccurring misbehaviors at preschool age fromthose that are abnormal. In particular, we haveshown that normative misbehaviors can bedistinguished from abnormal behaviors inearly childhood in terms of the developmentalexpectability of the context in which they occur



and their qualitative features. Normative mis-behaviors, i.e., those that occur in expectablecontexts and/or are qualitatively better modu-lated and flexible, fell at the milder end of thedimensional spectrum. In contrast, behaviors thatwere qualitatively intense, dysregulated, intran-sigent, provocative, and/or occurred in devel-opmentally unexpectable contexts demonstratedheightened severity along the dimensional spec-trum. Furthermore, there were distinct dimen-sional patterns in terms of severity thresholds andin the moderate degree of cross-dimensionaloverlap. Although the majority of preschoolersexhibited some of the behaviors tapped intoby the MAP-DB dimensions, less than 10% fell inthe atypical range on any dimension. This sug-gests that the MAP-DB is effectively distinguish-ing the normative:atypical distinction for this agegroup.

The generalizability of our findings for broad-based clinical application is highlighted by the fitof the 4-dimensional model across variations ingender, ethnicity, and child age. Mean differencesfor some subgroups (e.g., uniformly higher scoresfor boys on all dimensions), however, raise thepossibility that clinical thresholds may need to be

Y

www.jaacap.org 91


FIGURE 3 Ruga and dot plot illustrating higher category thresholds for normative misbehavior versus problemindicator from the Temper Loss dimension. Note: (a) Normative misbehavior. (b) Problem indicator. a“Rug” along thex-axis of each plot is the severity distribution of the dimension being measured. Dots represent the position of each itemcategory threshold along the dimension, with the threshold of atypicality (95th percentile) marked with the dashedvertical line.

WAKSCHLAG et al.

determined by subgroup. This can be determinedonly based on norming in large representativesamples with sufficient numbers in each sub-group and across strata to ensure that meaningfuldifferences have been accurately characterizedand validated in relation to clinical endpoints.This will be an important step for developingthe clinical potential of the MAP-DB in diversepopulations.

Demarcation of the developmentally informed,behavioral distinctions was further evident inthe different frequency thresholds at which theybecame psychometrically atypical for TemperLoss, Noncompliance, and Aggression. In gen-eral, milder, normative misbehaviors had to occurvery regularly to be severe (i.e., daily or more forTemper Loss and Noncompliance, weekly forAggression). This emphasizes that even thecommon misbehaviors of early childhood areatypical if they occur at high frequencies. Forexample, in previous findings from this sample,we have shown that although tantrums are anormative misbehavior, less than 10% of pre-schoolers have tantrums daily and those whodo are more likely to have problems in other

JOURN

92 www.jaacap.org

areas.29,38 In contrast, qualitatively atypical be-haviors in each dimension exceeded the severitythreshold with relatively low occurrence.

Low Concern did not demonstrate this samevariation in frequency at different levels ofseverity and had a more restricted range, withaverage severity values of all items greater thanthe 95th percentile threshold (connoting thatthey are rare and serious behaviors). Further,virtually all Low Concern items had the samerelative frequency at which the 95th percentilethreshold was reached. This is not surprisinggiven that this dimension was conceptualized asa developmental substrate of later callous/un-emotional (C/U) patterns.39 C/U is an atypicalvariant of antisocial behavior6 and may not havenormative variations. The MAP-DB Low Concerndimension represents one of the first attempts tooperationalize callous traits in a clinically anddevelopmentally salient manner specific to earlychildhood.19

A primary limitation of the present findingsis that only psychometric (internal) validity wasestablished. Psychometrically, atypicality is delin-eated in terms of rare occurrence as manifested




FIGURE 4 Ruga and dot plot illustrating higher category thresholds for normative misbehavior versus problemindicator from the Low Concern dimension. Note: (a) Normative misbehavior. (b) Problem indicator. a“Rug” along thex-axis of each plot is the severity distribution of the dimension being measured. Dots represent the position of each itemcategory threshold along the dimension, with the threshold of atypicality (95th percentile) marked with the dashedvertical line.


by scores that occur in a low percentage of thepopulation. In contrast, clinical (external) validityis defined by association with impairment, familyhistory, caseness, differentiated etiologic markers,response to treatment, and prognostic utility.Further research is needed to establish the rela-tionship between MAP-DB psychometric severityand clinical severity. Ultimately, it is this jointconsideration that determines clinical utility. Inparticular, the added value of this dimensional,developmentally-informed approach will reston the following: establishing whether psycho-metrically demarcated atypicality (as differenti-ated by the distinct dimensions), meaningfullypredicts varied clinical symptoms and impair-ment; increases the accuracy of differentiationof normative versus clinically significant be-haviors in early childhood relative to traditionaldiagnostic categories; and links established riskprocesses to distinct dimensions. Furthermore,future research that includes sufficient numbers



of children with developmental disabilities formodeling individual differences in patterns willimportantly inform understanding of atypicalmanifestations of behavior (e.g., Low Concern)in children with developmental disabilitiesrelative to children with emotional and behav-ioral syndromes.

Another limitation is the cross-sectional natureof the data. Recent findings from large repre-sentative samples are promising in demon-strating the developmental continuity of ODDdimensions from school age to adolescence, aswell as their predictive utility.40 Establishingcontinuity in MAP-DB dimensions over time willbe important. Finally, as the majority of the in-formants were mothers, calibration of the MAP-DB with fathers and teachers is needed.

Visions for DSM-5,41 as well as neuroscience-based classifications of psychopathology such asthe Research Domain Criteria (RDoC),42 convergeon the importance of developmentally-informed,

Y

www.jaacap.org 93


WAKSCHLAG et al.

dimensional phenotypes for pinpointing unfold-ing clinical sequences as they progress from earlyprodromal manifestations to frank disorder.14,43

Our 4-dimension framework integrates key ele-ments of leading dimensional models that havebeen validated for ODD and CD.6,8,12 Findingssuggest that their varying components may bestbe considered distinctly, consistent with work byothers.9,16,40,44

There are disadvantages and benefits to bothcategorical and dimensional models of childhoodpsychopathology. One is not intrinsically prefer-able to the other;45 they may best be understoodas complementary. Evidence is emerging thatsome types of child psychopathology (such asmelancholic depression) may belong to discretetaxa, whereas others (such as attention-deficit/hyperactivity disorder [ADHD]) may best beunderstood as dimensions.45 The present findingshighlight the potential utility of psychometricmethods for operationalizing the vision of adimensional classification approach. In partic-ular, IRT methods revealed a developmentalspectrum of behavior and identified varyingthresholds for individual behaviors along adimensional severity continuum. We theorizethat this combination of developmental differen-tiation and articulation of the normal–abnormalcontinuum will enhance specificity of predictionand linkage to mechanisms because it removesmuch of the “noise” of developmental variabilityin early childhood that has impeded accurateclinical identification when traditional non-developmental categorical nosologies have beenapplied. Variation in dimensional patterns foundhere were also consistent with revisions of DBsymptom criteria in DSM-5, i.e., differentiatingsymptom dimensions within ODD and recog-nizing the importance of a callous trait specifierfor CD.43 If clinical and incremental utility of thisapproach is established relative to traditionalpsychiatric classification systems, the develop-mental severity continuum delineated here sug-gests that early childhood classification mightbenefit from greater specification. Such specifi-cation may include incorporation of symptomsthat differentiate normative misbehaviors fromqualitatively atypical behaviors, and incorpora-tion of specific frequency thresholds basedon deviation from expectable developmentalpatterns of occurrence.

Multidimensional measures, such as the MAP-DB, ultimately have the potential to provide

JOURN

94 www.jaacap.org

a psychometrically robust, developmentally-informed metric for characterizing behavior asnormative, atypical, or “at the boundaries” foran age period. This has the potential to providean empirical basis for determining whether“watchful waiting,” psychoeducational guid-ance, pharmacologic, or more intensive treat-ments are warranted. Dimensional measurementtools that enable precise developmental char-acterization of latent dimensions can advanceour understanding of the unfolding of psycho-pathology and etiologic distinctions. Severalrecent studies provide an intriguing glimpseof the potential clinical utility of this dimen-sional approach for prevention. Low-intensityinterventions that target children (includingpreschoolers) based on temperamental or per-sonality risk have shown promise for reducingthe likelihood of developing clinical disor-der.46,47 These studies suggest that targetedprevention along dimensional continua may befruitful for altering the risk of developmentalpsychopathology. &

AL

Accepted November 4, 2013.

Dr. Wakschlag is with the Feinberg School of Medicine and Institute forPolicy Research, Northwestern University. Dr. Briggs-Gowan is with theUniversity of Connecticut. Dr. Choi is with CTB/McGraw Hill andNorthwestern University. Dr. Nichols, Mr. Burns, and Ms. Kestler arewith the Feinberg School of Medicine, Northwestern University. Dr.Carter is with the University of MassachusettseBoston. Dr. Henry iswith the Institute for Health Research and Policy, University of Illinois atChicago.

Drs. Wakschlag, Briggs-Gowan, Choi, Carter, Nichols, and Henrywere supported by the National Institute of Mental Health (NIMH)grants R01MH082830 and R01MH090301. Dr. Wakschlag wasalso supported by the Walden and Jean Young Shaw Foundation.

The contributions of Patrick Tolan, PhD (University of Virginia), RobertGibbons, PhD (University of Chicago), Barbara Danis, PhD (FamilyInstitute of Chicago), and Carri Hill, PhD (Rush University), to thedevelopment of the Multidimensional Assessment of Preschool Disrup-tive Behavior (MAP-DB) are gratefully acknowledged. The authorsthank the pediatric clinics and participants from Rush University, theUniversity of Illinois at Chicago, and the following Pediatric PracticeResearch Group practices for their participation: Healthlinc in Valpar-aiso, IN, Healthlinc in Michigan City, IN, and Associated Pediatri-cians in Valparaiso, IN. The authors also thank David Cella, PhD, forhis inspiring leadership and scientific support.

Disclosure: Drs. Wakschlag, Briggs-Gowan, Choi, Nichols, Carter,and Henry, Ms. Kestler, and Mr. Burns report no biomedical financialinterests or potential conflicts of interest.

Correspondence to Lauren S. Wakschlag, PhD, Department of Medi-cal Social Sciences, Feinberg School of Medicine, NorthwesternUniversity, 633 N. St. Clair Street, 19th Floor, Chicago, IL 60611;e-mail: [email protected]

0890-8567/$36.00/ª2014 American Academy of Child andAdolescent Psychiatry

http://dx.doi.org/10.1016/j.jaac.2013.10.011

OF THE AMERICAN ACADEMY OF CHILD & ADOLESCENT PSYCHIATRY


mailto:[email protected]

http://dx.doi.org/10.1016/j.jaac.2013.10.011



REFERENCES
1. Kim-Cohen J, Caspi A, Moffit T, Harrington H, Milne BJ,
Poulton R. Prior juvenile diagnoses in adults with mental disorder:developmental follow-back of a prospective longitudinal cohort.Arch Gen Psychiatry. 2003;60:709-717.

2. Nock M, Kazdin A, Hiripi E, Kessler R. Lifetime prevalence, cor-relates, and persistence of oppositional defiant disorder: resultsfrom the National Comorbidity Survey replication. J Child PsycholPsychiatry. 2007;48:703-713.

3. Wakschlag L, Tolan P, Leventhal B. “Ain’t misbehavin”: towards adevelopmentally-specified nosology for preschool disruptivebehavior. J Child Psychol Psychiatry. 2010;51:3-22.

4. Bufferd SJ, Dougherty L, Carlson G, Rose S, Klein D. Psychiatricdisorders in preschoolers. Am J Psychiatry. 2012;169:1157-1164.

5. Leibenluft E. Severe mood dysregulation, irritability, and thediagnostic boundaries of bipolar disorder in youths. Am J Psy-chiatry. 2011;168:129-142.

6. Frick P. Developmental pathways to conduct disorder: implica-tions for future directions in research, assessment and treatment.J Clin Child Adolesc. 2012;41:378-389.

7. Stringaris A, Zavoa H, Leibenluft E, Maughan B, Eley T. Adoles-cent irritability: phenotypic associations and genetic links withdepressed mood. Am J Psychiatry. 2012;169:47-54.

8. Stringaris A, Goodman R. Three dimensions of oppositionality inyouth. J Child Psychol Psychiatry. 2009;50:216-223.

9. Burke JD, Hipwell AE, Loeber R. Dimensions of oppositionaldefiant disorder as predictors of depression and conduct disorderin preadolescent girls. J Am Acad Child Adolesc Psychiatry. 2010;49:484-492.

10. Rowe R, Maughan B, Costello J, Angold A. Defining opposi-tional defiant disorder. J Child Psychol Psychiatry. 2005;46:1309-1316.

11. Drabick D, Gadow K. Deconstructing oppositional defiant disor-der: clinic-based evidence for an anger/irritability phenotype.J Am Acad Child Psychiatry. 2012;51:384-393.

12. Burt S. How do we optimally conceptualize the heterogenetywithin antisocial behavior? An argument for aggressive versusnon-aggressive behavioral dimensions. Clin. Psychol. Rev. 2012;32:263-279.

13. Frick PJ, White S. Research review: The importance of callous-unemotional traits for developmental models of aggressiveand antisocial behavior. J Child Psychol Psychiatry. 2008;49:359-375.

14. Insel T. Disruptive insights in psychiatry: transforming a clinicaldiscipline. J Clin Invest. 2009;119:700-705.

15. Wakschlag L, Henry DB, Tolan PH, Carter AS, Burns JL, Briggs-Gowan MJ. Putting theory to the test: modeling a multidi-mensional, developmentally-based approach to preschooldisruptive behavior. J Am Acad Child Psychiatry. 2012;51:593-604.

16. Ezpeleta L, Granero R, de la Osa N, Penelo E, Domenech JM.Dimensions of oppositional defiant disorder in 3-year-old pre-schoolers. J Child Psychol Psychiatry. 2012;53:1128-1138.

17. Ezpeleta L, de la Osa N, Granero R, Penelo E, Domenech JM. In-ventory of callous-unemotional traits in a community sample ofpreschoolers. J Clin Child Adolesc. 2013;42:91-105.

18. Kimonis E, Frick P, Boris N, et al. Callous-unemotional features,behavioral inhibition, and parenting: independent predictors ofaggression in a high-risk preschool sample. J Child Fam Stud.2006;15:745-746.

19. Hyde L, Shaw D, Gardner F, Cheong J, Dishion T, Wilson M.Dimensions of callousness in early childhood: links to problembehavior and family intervention effectiveness. Dev Psychopa-tholol. 2013;25:347-363.

20. Achenbach T. Assessment of psychopathology. In: Sameroff A,Lewis M, Miller S, eds. Handbook of Developmental Psychopa-thology. 2nd ed. New York: Kluwer Academic; 2000:41-56.

21. Wakschlag L, Briggs-Gowan M, Carter A, et al. A develop-mental framework for distinguishing disruptive behavior fromnormative misbehavior in preschool children. J Child PsycholPsychiatry. 2007;48(special issue on preschool psychopathol-ogy):976-987.

22. Markon K, Krueger R, Watson D. Delineating the structure ofnormal and abnormal personality: an integrative hierarchicalapproach. J Pers Soc Psychol. 2005;88:137-157.



23. Reise S, Waller N. Item response theory and clinical measurement.Annu Rev Clin Psychol. 2009;5:25-46.

24. Tremblay R, Nagin D, Seguin J, et al. Physical aggression duringearly childhood: Trajectories and predictors. Pediatrics. 2004;114:43-50.

25. Hay DF, Castle J, Davies L. Toddlers’ use of force against familiarpeers: a precursor of serious aggression? Child Dev. 2000;71:457-467.

26. Kochanska G, Aksan N. Children’s conscience and self-regulation.J Personal. 2006;74:1587-1617.

27. Rutter M, Caspi A, Moffitt T. Using sex differences in psycho-pathology to study causal mechanisms: unifying issues andresearch strategies. J Child Psychol Psychiatry. 2003;44:1092-1115.

28. Baillargeon HR, Morrisset A, Keenan K, Normand LC, Seguin JR,Japel C, Cao G. Development of disruptive behaviors in youngchildren: a prospective population-based cohort study. Inf MentHealth J. 2012;33:633-650.

29. Wakschlag L, Choi S, Carter A, et al. Defining the developmentalparameters of temper loss in young children: implications fordevelopmental psychopathology. J Child Psychiatry Psychol.2012;53:1099-1108.

30. Barajas RG, Philipsen N, Brooks-Gunn J. Cognitive and emotionaloutcomes for children in poverty. In: Crane D, Heaton T, eds.Handbook of Families and Poverty. Newbury Park, CA: Sage;2008:311-333.

31. Bode R, Lai J, Cella D, Heinemann A. Issues in the development ofan item bank. Arch Phys Med Rehabil. 2003;84:S52-S60.

32. Muth�en LK, Muth�en BO. Mplus User’s Guide. Vol 6. Los Angeles,CA: Muth�en & Muth�en; 2010.

33. Hu L, Bentler P. Cutoff criteria for fit indexes in covariancestructure analyses: conventional criteria versus new alternatives.Struct Equat Model. 1999;6:1-55.

34. Eaton N, Krueger R, Markon K, et al. The structure and predictivevalidity of the internalizing disorders. J Abnorm Psychol. 2013;122:86-92.

35. Raftery A. Bayesian model selection in social research. SocMethod. 1995;25:111-163.

36. IRTPRO: Flexible, multidimensional, multiple categorical IRTmodeling [computer program]. Lincolnwood, IL: Scientific Soft-ware International; 2011.

37. Wakschlag L, Henry D, Blair R, Dukic V, Burns J, Pickett K.Unpacking the association: individual differences in the relation ofprenatal exposure to cigarettes and disruptive behavior pheno-types. Neurotoxicol Teratol. 2011;33:145-154.

38. Hullsiek H, Briggs-Gowan M, Wakschlag L. Temper tantrumfrequency and associated problems in a diverse sample of pre-school children. American Academy of Child and AdolescentPsychiatry and Canadian Academy of Child and AdolescentPsychiatry Joint Annual Meeting; October 18-23;2011; Toronto,ON, Canada.

39. Briggs-Gowan MJ, Nichols SR, Voss J, et al. Punishmentinsensitivity and impaired reinforcement learning in pre-schoolers. J Child Psychol Psychiatry. 2013. [Epub ahead ofprint].

40. Whelan Y, Stringaris A, Maughan B, Barker ED. Developmentalcontinuity of oppositional defiant disorder subdimensions atages 8, 10, and 13 years and their distinct psychiatric outcomes atage 16 years. J Am Acad Child Adolesc Psychiatry. 2013;52:961-969.

41. Helzer J, Kraemer HC, Krueger RF, Wittchen HU, Sirovatka PJ,Regier DA. Dimensional approaches in diagnostic classification:refining the research agenda for DSM-V. American PsychiatricPublishing; 2008.

42. Insel T, Cuthbert B, Garvey M, et al. Research domain criteria(RDoC): toward a new classification framework for research onmental disorders. Am J Psychiatry. 2010;167:748-751.

43. Frick P, Nigg J. Current issues in the diagnosis of attention deficithyperactivity disorder, oppositional defiant disorder, and conductdisorder. Annu Rev Clin Psychol. 2012;8:77-107.

44. Stringaris A, Goodman R. Longitudinal outcome of youth oppo-sitionality: irritable, headstrong, and hurtful behaviors havedistinctive predictions. J Am Acad Child Psychiatry. 2009;48:404-412.

Y

www.jaacap.org 95


WAKSCHLAG et al.

45. Coghill D, Sonuga-Barke EJ. Annual research review: Categoriesversus dimensions in the classification and conceptualisation ofchild adolescent mental disorders—implications of recent empir-ical study. J Child Psychol Psychiatry. 2012;53:469-489.

46. Yeager DS, Miu AS, Powers J, Dweck CS. Implicit theories ofpersonality and attributions of hostile intent: a meta-analysis, an

JOURN

96 www.jaacap.org

experiment, and a longitudinal intervention. Child Dev. 2013;84:1651-1667.

47. Rapee RM. The preventative effects of a brief, early interven-tion for preschool-aged children at risk for internalising:follow-up into middle adolescence. J Child Psychol Psychiatry.2013;54:780-788.




TABLE S1 Multidimensional Assessment of Preschool Disruptive Behavior (MAP-DB) Item Factor Loadings From4-Dimensional Developmental Confirmatory Factor Analysis (CFA) Model

Item Description Factor Loading

Temper Loss1. Lose temper or have a tantrum when frustrated, angry, or upset 0.892. Have a temper tantrum, fall-out, or melt-down 0.873. Lose temper or have a tantrum to get something he or she wanted 0.874. Lose temper or have a tantrum when tired, hungry, or sick 0.835. Become frustrated easily 0.736. Lose temper or have a tantrum during daily routines such as bedtime,

mealtime, or getting dressed0.84

7. Lose temper or have a tantrum with you or other parent 0.888. Yell angrily at someone 0.779. Have a short fuse (become angry quickly) 0.8110. Have difficulty calming down when angry 0.7811. Have a hot or explosive temper 0.7512. Get extremely angry 0.8613. Act irritable 0.8714. Stamp feet or hold breath during a temper tantrum, fall-out, or melt-down 0.7215. Keep on having a temper tantrum, fall-out, or melt-down, even when you tried

to help him/her calm down0.87

16. Have a temper tantrum, fall-out, or melt-down that lasted more than 5 minutes 0.8317. Lose temper or have a tantrum with other adults 0.8118. Break or destroy things during a temper tantrum, fall-out, or melt-down 0.8119. Have a temper tantrum, fall-out, or melt-down until exhausted 0.8320. Lose temper or have a tantrum “out of the blue” or for no reason 0.8521. Hit, bite, or kick during a temper tantrum, fall-out, or melt-down 0.8122. Stay angry for a long time 0.75

Mean (SD) 0.82 (0.05)Noncompliance23. Break rules even when he or she knew you were watching 0.6724. Argue when asked to do something 0.7325. Act stubborn 0.7626. Disobey or break rules with you or other parent 0.8427. Say no when told to do something 0.7728. Disobey or break rules during daily routines, such as bedtime, mealtime, or

getting dressed0.76

29. Act sassy, talk back or have a smart mouth 0.7330. Disobey or break rules when frustrated, angry or upset 0.8331. Do exactly what you just said not to do 0.8032. Refuse to follow directions 0.6933. Disobey or break rules with other adults 0.8434. Ignore directions 0.8135. Disobey or break rules for no reason or out of the blue 0.7936. Disobey or break rules when tired, hungry, or sick 0.8237. Disobey or break rules to get something he or she wanted 0.8638. Not do what you asked no matter what 0.7239. Argue about just about anything 0.7640. Automatically resist whatever you ask 0.8241. Take things he or she was not allowed to have 0.7842. Show off or laugh while misbehaving 0.7443. Do risky things s/he knew were not allowed 0.7344. Misbehave in ways that were dangerous or unsafe 0.74

Mean (SD) 0.77 (0.05)


VOLUME 53 NUMBER 1 JANUARY 2014 www.jaacap.org 96.e1



TABLE S1 Continued

Item Description Factor Loading

Aggression45. Act aggressively when frustrated, angry, or upset 0.8046. Act aggressively to try to get something he or she wanted 0.8147. Act aggressively with you or other parent 0.8348. Throw something at someone (not as part of a game) 0.8049. Try to hurt someone to get back at them 0.8050. Hit someone with an object 0.7951. Act aggressively out of the blue or for no reason 0.8452. Break or ruin things on purpose 0.7953. Act aggressively with other adults 0.8254. Act aggressively toward other children 0.8055. Do or say mean or “not nice” things to other children 0.8456. Hit, shove, or kick you or other parent 0.7857. Get into fights 0.7758. Call another child names 0.6259. Pinch, scratch, or pull someone’s hair 0.6960. Hit, shove, or kick other children 0.7761. Say or do mean or “not nice” things to other children behind their backs 0.7162. Bully someone 0.7963. Hurt someone on purpose 0.8164. Refuse to let other children play with him/her 0.6665. Threaten someone 0.7366. Tell others not to let someone play with them 0.4767. Hit, shove, or kick other adults 0.7768. Curse at someone 0.6269. Spit at someone 0.62

Mean (SD) 0.75 (0.09)Low Concern for Others70. Not care about other’s feelings when frustrated, angry or upset 0.8671. Not seem to care about your or other parent’s feelings 0.8772. Keep on doing something that was scaring or upsetting someone 0.7673. Not seem to care about other adults’ feelings 0.8774. Act like s/he did not care about pleasing other people 0.8375. Act like s/he did not care when someone was mad or upset 0.8776. Enjoy making others mad 0.8277. Do things to humiliate or embarrass others 0.7978. Act like s/he did not care when someone else felt bad or sad 0.79

Mean (SD) 0.83 (0.04)

WAKSCHLAG et al.

JOURN

96.e2 www.jaacap.org


TABLE S2 Modeling Factor Invariance Across Demographic Subgroups

Model Log Likelihood No. of Free Parameters AIC BIC Sample-Size Adjusted BIC

By child gender (760 boys, 727 girls)Factor Variant model �88280 943 178446 183449 180453Factor invariant model �88559 479 178086 180617 179095By child age (523 3-year-olds, 533 4-year-olds, 432 5-year-olds)Factor variant model �88463 1391 179709 187088 182669Factor invariant model �89099 477 179151 181682 180166By child race/ethnicity (533 African American, 530 Hispanic, 406 white)Factor variant model �86519 1358 175754 182941 178627Factor invariant model �87807 466 176546 179012 177532By poverty status (852 not poor, 618 poor)Factor variant model �86923 917 175681 180534 177621Factor invariant model �87439 466 175809 178276 176795

Note: Robust maximum likelihood estimation with logit link (Mplus MLR option); Scaling correction factors ranged from 1.00 to 1.02. In factor variantmodels, item thresholds and factor loadings are estimated for each group and factor means are fixed at 0. In factor invariant models, item thresholdsand factor loadings are equal across groups and factor means are fixed at 0 in 1 group and freely estimated in others. AIC ¼ Akaike InformationCriterion; BIC ¼ Bayesian Information Criterion.

TABLE S3 Comparisons of Subgroup Differences in Dimensional Scoresa

GroupMultivariate F

(by Wilks’ lambda) df

TL NC AG LC

Mean SD Mean SD Mean SD Mean SD

Boys, n ¼ 760 4.33** 4,1483 0.07a 0.93 0.05 0.93 0.10a 0.92 0.10a 0.89Girls, n ¼ 728 �0.07b 0.95 �0.06 0.97 �0.07b 0.92 �0.04b 0.913-Year-olds, n ¼ 523 5.02*** 8,2964 0.10a 0.94 0.10a 0.94 0.11a 0.90 0.08 0.874-Year-olds, n ¼ 533 0.01ab 0.95 0.00ab 0.98 0.02ab 0.94 0.04 0.935-Year-olds, n ¼ 432 �0.13b 0.93 �0.14b 0.92 �0.10b 0.91 �0.04 0.89African American, n ¼ 533 13.89*** 8,2926 �0.11a 1.05 �0.14a 1.03 �0.03a 1.04 �0.06a 1.01Hispanic, n ¼ 530 �0.09a 0.88 �0.12a 0.88 �0.09a 0.83 �0.06a 0.81White, n ¼ 406 0.27b 0.83 0.32b 0.86 0.23b 0.85 0.27b 0.83Not poor, n ¼ 852 8.61*** 4,1465 0.00 0.85 0.02 0.87 �0.01 0.84 0.02 0.82Poor, n ¼ 618 0.00 1.06 �0.04 1.06 .06 1.03 0.05 1.00

Note: Means with different superscript letters differ at p < .01. AG ¼ Aggression; LC ¼ Low Concern; NC ¼ Noncompliance; TL ¼ Temper Loss.aDerived from multivariate analysis of variance.*p < .05; **p < .01; ***p < .001.

TABLE S4 Comparison of Model Fita: 4-Dimensional Developmental Model Versus Established Models

Dimensional Modelsb Log Likelihood No. of Free Parameters AIC BIC Sample-Size Adjusted BIC

Developmental model e87536 474 176021 178536 177030DSM ODD/CD model e88603 469 178143 180631 179141DB/callous model e89373 469 179683 182171 180681IHH model e88524 471 177990 180489 178993

Note: aRobust maximum likelihood estimation with logit link (Mplus MLR option). Scaling correction factors ranged from 1.01 to 1.09. From weightedleast-squares estimation with probit link (Mplus WLSMV option), c2919

2 ¼ 13007. AIC ¼ Akaike Information Criterion; BIC ¼ Bayesian InformationCriterion.

bModel structure: Developmental Model (4 dimensions) ¼ Temper Loss þ Aggression þ Noncompliance þ Low Concern dimensions. DisruptiveBehavior (DB)/Callous Model (2 dimensions) ¼ DB Dimension (Temper Loss þ Noncompliance þ Aggression) and Callous Dimension (LowConcern). DSM oppositional defiant disorder (ODD)/conduct disorder (CD) Model (2 dimensions) ¼ ODD Dimension (Temper Loss þ Noncom-pliance) and CD Dimension (Aggression þ Low Concern). Irritable/Headstrong/Hurtful (IHH) Model (3 dimensions) ¼ Irritable (Temper Loss),Headstrong (Noncompliance), Hurtful (Aggression þ Low Concern).


VOLUME 53 NUMBER 1 JANUARY 2014 www.jaacap.org 96.e3



Advancing a multidimensional, developmental spectrum approach to preschool disruptive behavior

Documents