Evaluating Community Inclusion: A Novel Treatment Program and Skills Measurement Tool for Children with Autism Spectrum Disorders and Other Developmental Delays

Evaluating community inclusion: A novel treatment program and skills measurement tool for children with autism spectrum disorders and other developmental delays

Keywords: Autism spectrum disorders, community-based, social inclusion, skills measurement tools, intervention Abstract Traditional mainstream and inclusion programs for children with Autism spectrum disorders

(ASD) and developmental delays typically provide shadow aides for some children, often

leaving out many children who do not qualify. A limited number of programs provide inclusive

“leisure-time” (out-of-school) recreational and socialization opportunities for developmentally

delayed youth. This paper presents a novel treatment program and skills measurement tool for

ASD and other developmental delays. Including Special Kids, a program developed and run by a

nonprofit organization, offers activities for children with developmental delays alongside

typically developing children in collaboration with well-established local youth programs.

Preliminary research suggests the program has a meaningful effect on the inclusion of ASD

children. This initial assessment with encouraging results for community-based inclusion

programs merits further, in-depth study.

EVALUATING COMMUNITY INCLUSION

2

Promoters of World Autism Awareness Day, April 2, 2012, recently announced that one

in 88 American-born children have been diagnosed with autism spectrum disorders (ASDs), and

countless others live with other types of developmental delays (Department of Health and

Human Services, 2012). Worldwide estimates indicate 4.3 million people had a diagnosis of

ASD in 2009, which is predicted to increase to seven million by 2017. (Global Data, 2010). In

2008, the UN adopted World Autism Day (United Nations, 2008), and the UN Secretary, Ban

Ki-moon stated that the day “should be dedicated to cultivating awareness of this difficult

lifelong developmental disorder and spur such action and draw attention to the unacceptable

discrimination, abuse and isolation experienced by people with autism and their loved ones”

(Global Data, 2010). In addition to discriminatory and exclusion issues, government leaders,

mental health care providers and parents alike worry about the futures of these children and the

societies in which they live; Knapp et al (2009) suggest the lifetime cost for someone with ASD,

after discounting, varies between £0.8 million ($1.25 million) and £1.23 million ($1.92 million).1

It likely benefits all concerned to intervene early with programs that improve outcomes for ASD

children and lower the likelihood of them needing intensive care as they move into adulthood.

Behavior and Interventions

Among the biggest issues that define children with ASD are impaired social functioning

skills that interfere with typical social activities. Social skills are behaviors that result in positive

social interactions, which include verbal and non-verbal behaviors needed for individuals to have

positive interpersonal communication (Gresham F. M., 1987). Rao et al (2008, p. 353) report

that, “children who are deficient in social skills lack the behavioral repertoire necessary to

interact with others according to social convention, a deficit that affects both academic and social

development.” However, Hartup (1989) showed that social skills learned in childhood have


3

consistently been linked to positive developmental outcomes including peer acceptance, mental

health and wellbeing, and academic achievement. (See also Kuhithau et al (2010).) Rao et al

(2008, p. 354) also note that if a child can acquire these skills, he or she should be able to

assimilate into a group of peers and to interact with adults, both familiar and unfamiliar

(Gresham F. R., 1986).

Children with ASD also face difficulties due to their behavior. To be included with their

peers and in society at large, these children must learn behaviors that foster group inclusion.

Recent studies attempt to link behavioral interventions and inclusion outcomes for children with

ASD. For example, Magiati et al (2011) studied children with ASD who attended intensive

community based intervention pre-school programs. They found, among other results, that a

child’s ability to change and adapt predicted the child’s successful outcomes seven years later.

Eldar et al (2010) examined successes and difficulties of including children with ASD in regular

school classes. They note that children with ASD behave in a more social way when they interact

among typical children than when they interact with children who have autism (2010, p. 98). In

addition, Jones and Frederickson (2010) recently examined effects of behavior on social

acceptance and rejection in a school setting, using parent, teacher and peer ratings. Their study

found that cooperative behavior was associated with greater peer acceptance, while higher scores

of shyness were associated with lower peer acceptance, and lower levels of cooperation were

associated with higher levels of social rejection (p. 1098). Looking back over the past couple of

decades, other researchers (see, for example, Fryxell and Kennedy (1995), and Guralnick,

Gottman and Hammond (1996)) suggested that participating in inclusive programs with typically

developing peers improves outcomes for ASD children, and Carr et al. (2002) suggested that

activities for people with diagnosed delays must move beyond education and into other


4

community activities that provide opportunities for participation and social interaction with a

range of typically developing peers.

Intervention programs for children with ASD have largely focused on modifying

behavior. Vismara and Rogers (2010, p. 447) state that, to date, behavioral interventions at

young ages present the only treatments shown to effectively improve core autism symptoms:

abnormal social interactions, impaired communication, and repetitive behaviors. Most programs

stem from applied behavior analysis (ABA), which is based on the science of behavior analysis.

ABA focuses on training individuals to change their behavior by reinforcing incremental

approximations of the desired behaviors and withdrawing reinforcement from undesired

behaviors. These programs typically occur in formal settings and are administered by mental

health professionals. Of particular interest to this research is the ABA-based model of positive

behavior support (PBS). An empirically validated, function-based approach, PBS replaces

challenging behaviors with prosocial skills. PBS trainers focus on modifying the child’s

environment so challenging behaviors become ineffective and inefficient while socially

appropriate adaptive skills become effective and efficient in meeting the child’s needs. PBS

reduces the likelihood that problem behaviors will occur and increases the likelihood the child

will learn prosocial behaviors that increase inclusion and quality of life (Kanne, 2011). In a

recent study, von der Embse et al (2011) surveyed best practices in reducing problem behavior

and promoting inclusion for students with ASD and found that “In most cases, a PBS model is

recommended for students with ASD (Leach, 2009).

Despite weak evidence that specific programs attempting to teach children social skills

have effects outside of the therapy setting, these programs flourish (Agency for Healthcare

Research and Quality, 2011). Often, children can demonstrate specific social skills in the setting


5

in which it was learned but cannot use the skills functionally in their real worlds of school, after

school activities, and family interactions (Lopata, 2006). Thus the targeted social skills training

groups have little impact on the overall quality of life for the child or the child’s family and

community.

No study concretely proves greater inclusion resulting from behavior and social skills

training, however, various programs focus on the development of social skills in children

especially children with ASD. This study adds to other researchers’ observations, but in a

different context, adding to the evidence that specific ways to modify behavior lead to increased

quality of life and greater inclusion in the future.

Instruments

Many types of instruments provide information on social skills and adaptive behavior for

children and adults who have, or are suspected of having an ASD. These include three adaptive

behavior instruments, the first of which is the Adaptive Behavior Assessment System – Second

Edition (ABAS-II), which measures children or adults, examining 10 adaptive skill areas. The

Scales of Independent Behavior-Revised (SIB-R; Bruininks, Woodcock, Weatherman and Hill

(1996)) measures adaptive and problem behaviors, assessing functional independence and

adaptive settings including home, school, job and community. Finally, the Vineland Adaptive

Behavior Scales-Second Edition (VABS-II; Sparrow, Cicchetti, & Balla (2005)) measure

personal and social skills needed for everyday life, and include a socialization area designed to

measure social impairments associated with autism (Department of Education, State of Texas,

2007).

Other instruments measure social function. The first category measures social pragmatics

and includes (1) the Social Responsiveness Scale (SRS) (Constantino, 2003); (2) the Children’s


6

Social Behavior Questionnaire (CSBQ, Hartman et al (2006); and (3) the Children’s

Communication Checklist (CCC: Bishop (1998)). The second category measures quality of life.

These include the SF-36, which provides the “gold standard” in quality of life and contains a

section with social functioning measures (SF-36 Health Services Update); the Health-Related

Quality of Life (HRQoL) (Fayers, 2007) and the Pediatric Quality of Life Inventory 4.0

(PedsQL) (Varni, 1998).

These instruments provide comprehensive measures for various aspects of behavior and

quality of life; however, they measure different skills for different purposes than the ones needed

to evaluate the effects of a community-based, leisure (out-of-school) program where children

with developmental delays participate alongside typical peers. As noted above, this research

assumes that measurable changes in certain skills and behaviors result in increased quality of

life, and perhaps to greater social acceptance. This paper details an evolving program that

assesses, measures and reports on skill development for children with ASD and other

developmental delays.

History of the program

In 2007, parents of children with developmental delays in Monterey County, California,

created Special Kids Crusade (SKC), a 501(c)(3) organization, to make a positive impact on the

lives of children with disabilities and their families (Special Kids Crusade, 2012). As part of the

organization, the parents began creating programs like Including Special Kids (ISK) to help

children of different abilities with their challenges and achievements.

The impetus for ISK came from an unusual grant opportunity offered by the California

Department of Developmental Services (DDS) through the San Andreas Regional Center

(SARC).2 In 2007, DDS offered to provide start-up funds to stimulate new ideas and new types


7

of program models for individuals with developmental delays; however, DDS stipulated no

funding for traditional program models where one person shadows a developmentally delayed

individual or five or six people with developmental delays move together as a group so that the

group "stands out."

Monterey County mainstream and inclusion programs for children with developmental

delays offer traditional services, providing shadow aides for some children, keeping them safe,

but little or no other support to develop an accommodating environment for the child. Several

excellent organizations provide services exclusively for children with developmental delays

including Easter Seals, Monterey County Office of Education, and Special Olympics. However,

until the DDS grant opportunity arose, little collaboration among agencies existed, and no

organization or group offered formal, inclusive “leisure-time” (out-of-school) recreational and

socialization opportunities for developmentally delayed youth.

Recognizing the possibility to combine experience and insight of SKC’s parent-founded

organization with the quality programs, reputation, and resources of Monterey County host sites

provided SKC leaders a unique opportunity for collaboration. ISK leaders quickly began a

campaign to partner with existing and well-established organizations to help children and

community members (developmentally-delayed or typical) acquire new skills and gain

understanding of one another. Building on a relationship between SKC and the Boys & Girls

Club of Monterey County (BGCMC), SKC and BGCMC leaders agreed to the use of BGCMC

space for an inclusion program if SKC staff did the “legwork” to start the project.3 Finding a

good fit among existing programs and BGCMC professional staff initially proved challenging;

yet, the partnership moved forward. Today, ISK provides children with developmental delays

from ages five through high school the opportunity to interact with a typical mix of peers in


8

small and large group recreational activities within four California organizations: the Seaside,

Salinas and Smythe (San Jose) units of the Boys & Girls Club, the First Tee of Monterey County

(a nonprofit centered around teaching youth life skills through the game of golf). ISK leaders4

are actively working with the staff of Porter Youth Center (part of the U.S. Army Child, Youth &

School Services) to bring the program to this site.

Methods

Clients and Controls

The 39 children in the program included in this study range in age from six to 18, and all

have a diagnosis of a developmental delay. Thirty have a primary diagnosis of ASD, five have an

intellectual delay (ID) such as Down’s syndrome, and four have other (Other) developmental

delays including Prader-Willi Syndrome, fetal alcohol exposure and chromosomal abnormalities.

All participants are clients of the SARC. A few have been in the program for six months, while

others have been in it for as long as 27 months. No individual comparison children were chosen

as controls. The context-specific nature of interactions among children provided some baseline

information that when used with research findings on child development, provided the basis for

“typical” skills and behavior for children, by age and gender.

The intervention: approaches

As noted, research suggests that children with ASD can benefit from participating in

inclusive programs with typically developing peers (Fryxell & Kennedy, 1995; Guralnick,

Gottman, & Hammond, 1996; Halvorson & Sailor, 1990). In addition, research suggests,

“coordination between professionals and families, system responsiveness, and “buy-in” from

professionals within the system are important features of community-based programs”

(Brookman, 2003) (Hieneman & Dunlap, 2001). Thus, ISK staff worked to establish and


9

maintain a good relationship with unit directors at the Boys & Girls Clubs and other programs.

This helped program directors be more agreeable to meeting the unique needs of children with

developmental delays. The ISK program director and clinical director began by discussing

different approaches to include children with developmental delays most likely to result in

positive experiences for the children and their more typical peers. They determined that the most

important outcome all for program participants and for the typical peers is for each child to have

the skills to interact appropriately in natural social contexts. (For related goals on social skills

training, see DiSalvo, 2002.)

Using tenets of ABA and PBS, the ISK leaders began to observe the relationship of

participant behavior to the environment, to functionally assess targeted behavior in the

environment, and to help program participants learn, adapt and maintain behavior closer to the

targets. ISK incorporates aspects of different strategies to help assist social interactions among

children with developmental delays and their typically developing peers. For example, ISK

combines peer-mediated strategies, adult-facilitated strategies, and strategies designed to

increase initiating and autonomy in a child with autism. (See, for example, Haring & Breen

(1992), Kamps, Kravits, & Gonzalez-Lopez (1998), Kennedy (2001), Shukla, Kennedy, &

Cushing (1998), and Weiss & Harris (2001).)

Once ISK leaders began to observe and work with participants, they began to construct

methods to assess and rate behavior. Initially, the program director gathered information

provided by parents and professionals on functional and social skills. They then used a

“naturalistic” (Gerhardt, 2010, p. 202) setting, and direct observation of the children. Gerhardt

(2010, p. 202) notes “Direct observation of individuals with ASD in social environments can be

one of the best means of conducting detailed assessment of particular social behavior of interest,


10

as well as interpreting how responsive an individual is to contextual variables regarding

exhibiting particular social skills.” They closely followed Jones’ “three desirable conditions: “(a)

observation and recording of behaviors at the time of occurrence in their natural settings; (b) the

use of trained, objective observers; and (c) a behavioral description system involving a minimal

level of inference by the observers” (Jones R. R., 1979). As found by other researchers, the use

of multi-informant behavior ratings gave ISK better information on social inclusion. (Verhulst,

2008). ISK leaders assumed, as supported in the literature, that multi-informant ratings provide

especially useful and relevant information, particularly in identifying individuals with ASD more

susceptible to social exclusion due to behavior and other characteristics (Jones A. F., 2010).

Thus, the ISK program followed existing research to frame ISK actions, assessments, and, over

time, instruments for evaluation.

Training of personnel and participants

To directly observe program participants and intervene in skills training, the ISK

program director hired inclusion specialists (ISs) and inclusion assistants (IAs) to operate

programs at the four host sites. Inclusion assistants provide direct support to program

participants, while ISs have mastered IA skills and perform additional responsibilities including,

talking with parents, and functioning as on-site managers. IAs and ISs passed strict background

testing and underwent training, including in-service training and training on developmental

disabilities, tools of inclusion, privacy regulations, measurements and assessments and other

competencies needed to work the inclusion program. Each attended 15 hours of continued

training, before being assigned to a child. After the initial training, the IS observed and coached

each IA weekly, and the clinical director provided similar support and guidance on a quarterly

basis or more frequently, as needed.


11

Inclusion specialists and assistants began training BGC staff in how to include children

with developmental delays in daily activities. For example, ISK staff taught host site staff

procedures to facilitate social interactions among all children, to include children with

developmental delays in host site activities, and to reduce challenging behaviors. The staff

learned to use positive behavioral supports such as speaking slowly, moving close by when

giving directions, commenting on what the children do correctly rather than spotlighting

missteps and simplifying games so all children can participate. In addition, those involved in the

program received information from research on typical social and emotional development. Staff

implemented strategies appropriate to the goals of each child on an ongoing basis throughout the

time period the child attended the program.

Early observations

As ISK leaders began to gain better understanding of different characteristics of program

participants and the effects of these characteristics or behaviors on typical children, they quickly

observed that three behaviors most upset typically developing peers and the staff of their

programs: inability to use the restroom independently5, running and yelling. Thus the leaders

focused teaching adaptive skills to overcome those three behaviors, initially using adult-

facilitated activities.

Although inappropriate use of the restroom, running through the setting, and yelling were

most distressing to the staff and members of the club, age appropriate use of these skills did not

necessarily mean that the children would be included in activities or make friends. ISK

conducted a simple stakeholder survey, asking host site children and staff, "What makes it hard

to be friends with ______?" Using the responses, ISK staff began to create additional skill

targets, grouping them into nine adaptive skills.


12

Skills Definitions

Based on research in the fields of applied behavior analysis and child development in

addition to on site observations, ISK leaders focused on the following adaptive skills:

Using the Restroom

IAs focused on teaching participants how to behave appropriately when in a public

restroom. Participants learned how to open and close the locks of the stalls, to use the appropriate

number of paper towels, to check for an empty stall and so on.

Modulating Volume

Participants learned to judge the appropriate volume, depending upon the setting and to

use that volume. IAs guided participants to gradually reduce and eventually eliminate loud,

unpredictable noises.

Moving Safely

The IAs spent a great deal of time walking with individual participants around the setting,

especially at the beginning of each participant’s inclusion. While doing so, each participant

learned to move in a coordinated fashion with another person and to avoid objects/people in the

path. After mastering walking with an IA, each participant worked on moving at the same speed

and frequency as other children of the same age and gender.

Referencing

Referencing refers to the skill of looking to adults or other competent children for

nonverbal cues that help participants know how to behave at a given moment in a given setting.

Initially the participants learned to socially reference by turning towards someone calling his or

her name. Then they learned to use social referencing to seek out guidance in uncertain

situations, and to seek affirmation from significant adults and children6.


13

Engaging in Activities

Participants frequently entered the program with a small number of preferred, often

repetitive activities. IAs carefully scaffolded activities and skills so the participants developed

the competencies to engage in a range of host site activities. Participants learned to sample new

activities and to be flexible in their choices of activities. In addition, they learned to follow the

rules of the activities, both stated and implied.

Communicating

IAs worked with the participants to increase their ability to communicate clearly their

needs and wants with both familiar and unfamiliar people. This communication occurred in the

form of gestures, icons, signs or spoken language, depending upon the participant’s preference.

For participants comfortable using words and discussing ideas, IAs guided them to talk with and

to listen to other people. Participants were gradually guided to expand their choice of

conversational topics.

Cooperating

At the very beginning stages of cooperation, the IAs engaged in very simple play patterns

with each participant. These beginning patterns require the IA and participant to do something

like roll a ball back and forth or place cards on a stack, where the pattern is a very simple version

of turn taking. These patterns helped the participant learn to cooperate with adults and other

children; they also formed the basis for all types of group play. As a participant progressed, the

IAs expected the child to cooperate, even on non-preferred tasks. The “habit” of cooperation was

developed by initially inviting the child to do things that he or she had a high likelihood of doing,

and incrementally adding activities that the child had avoided or did not like.

Regulating Emotions


14

Most young children have temper tantrums; as they mature, they learn to manage their

emotions so as to have few, if any, such outbursts. Children with developmental delays often

have intense emotional outbursts long after their peers have learned to express their frustration

and anger in socially acceptable ways. ISK participants learned to reduce the frequency, duration

and intensity of such emotional outbursts so as to more closely resemble the outbursts of other

children of the same age and gender. At the same time they learned to express their emotions in

ways so that others could understand and respond.

Making Friends

This is the most complex skill that the participants developed. Initially the IAs worked

with a participant to enter a group of children who were engaged in an activity; simultaneously

the IA worked with the group to welcome the participant into the group. The overall goal was to

develop a stable and inclusive group (meaning more than one person) that welcomed the

participant and with whom the participant learned to enjoy spending time.

ISK program staff members saw these nine skills as being hierarchical, arranged in order

from simplest to most complex. In general, participants mastered basic skills before more

complex skills were emphasized. However, as with typical development, children and the IAs

worked on all skills all the time.

From lowest competency to highest, ISK designated three categories of skills based on

the nine individual skills. Attending skills came from the first three individual skills (Using the

Restroom, Modulating Volume and Moving Safely), and must have been high enough that a

child could show up to the program and handle the group setting. Participating skills, the middle

three, (Referencing, Engaging in Activities and Communicating) allowed children to more fully

engage and participate with the activities and routines in the setting. Collaborating skills


15

(Cooperating, Regulating Emotions and Making Friends) described the skills required for

working with or collaborating with other people within the setting.

Assessment

Having identified skills children needed to be included in a community-based setting and

specialists to help with inclusion, ISK program leaders began to combine their goals for children

with the defined skills needed to create an assessment, measurement and reporting system. This

system was meant to allow ISK leaders to determine improvement (or lack thereof) in the

ISKer’s skills. Following Gerhardt (2010, p. 197) who noted that assessing the range of factors

of social functioning and behavior should include rating scales, functional assessment and

behavioral observation, the ISK program included aspects of all three. Although the instruments

typically used to assess and rate functionality and behavior existed, ISK leaders observed that

these instruments measured different skills for different purposes than the ones ISK needed.

Thus, they began to construct their own system to assess, measure and report on skill

development for the participants with ASD and other developmental delays.

The system needed to serve many purposes including allowing: ISK to report quarterly

to SARC on progress of the children; ISK program leaders to understand what skills

interventions work with children having different diagnosis codes; IAs and ISs to also

understand and work to improve their skills and interventions; and families to track meaningful

progress in their children and to ask intelligent questions about progress. Initially, the program

director began assessing skills by collecting very basic data. However, these data did not show a

relationship with progress. Both the program director and adaptive skills consultant studied their

own emerging program and assessment system against several dozen programs for children with

developmental delays. They observed that for most programs, although reports stated children


16

were making progress on their goals, the progress seemed to have no correlation with

improvements in quality of life. They found many issues – complex goals, confusing

terminology, irrelevant outcomes, etc. – and decided to create their own measures that they

believed would reflect the overall desired outcome of children, of all types, being able to

participate together in a community.

Thus, in refining their assessment method, the program director and the behavioral

specialist constructed an easy-to-understand measurement system allowing individuals to work

on the nine skills, describing each skill using 10% improvement increments. The nine skills and

10 levels, or a total of 90 descriptions, became the “ISKipedia” used to create what they called

Individual Profiles. Table 1: Evaluation criteria for each skill shows the levels, where Level 10

means low or very weak skills and Level 1 signifies high or strong skills.

[Table 1 goes approximately here]

Using the evaluation scores, ISK staff assigned each child an initial classification of “attender”

(low skill level), “participator” (medium skill level), or “collaborator” (high skill level) based on

his or her skill levels. Over time and for each skill, IAs evaluated an individual’s progress against

the levels, measuring the frequency of appropriate use of each skill. IAs learned to evaluate using

skills of typically developing children of the same age and gender as the reference group for high

(Level 1) skills. Using the Individual Profiles and ISKipedia, the program director instructed the

IAs on what to do with each child at each skill level. The instructions provided the basis for the

ISK Response Guide, and over time, staff created Individualized Response Guides.

Data collection

When children entered the ISK program, the program director constructed an intake

interview package with a parent interview including the parents’ assessment of the child’s skills


17

and a record review of educational and regional center documents. If a child’s parents provided

assessments that varied more than 20% from the program director’s record review assessment,

the clinical director assessed the child’s skills using a video taken of the child on the first day he

or she participated in the program.

Once a child began the program, IAs completed a daily report form on him or her. For all

children, the IA provided qualitative measures of each of the nine adaptive skills indicating

whether the children’s skills were as expected for their level (check), above expectation (check

plus), or below expectation (check minus). When a child received three check minuses or six

check plusses, the program director evaluated these “tipping points” to determine whether to

record a change in the child’s level for a particular skill. A child’s average skill level determined

the most appropriate number of days the child should attend and the ideal length of time attended

on those days.

During the time of this study, the ISK program director evaluated each child every three

months, comparing her assessments to those of the IAs and ISs. She also monitored daily reports;

when a child hit a tipping point in a month, she combined her own observations and/or those of

the adaptive skill consultant’s to decide whether to promote or demote a skill level score for a

child. In this study, we used the three-month assessments because it minimized inter-rater

reliability and provided enough time between assessments to show real progress.

One of the program goals is to help children build the skills so that they can become fully

independent members at their host sites. This involved advancing children by changing support

and desired skill mastery as they mastered lower-level skills. At the time of the quarterly

assessment, each child who met the criteria to “graduate” to the next level of independence (from

1:1 support to 2:1 support, for example), “graduated” to the next level. ISK staff collaborated


18

with parents and SARC to create a graduation plan, which included a transition timeline, peer

matching, implementation guidelines and evaluation criteria. On the selected start date, the IAs

modified their roles. A lead IA guided children and completed daily report forms while a support

IA helped children learn to navigate the next level successfully. All ISK staff monitored progress

or decline in skill levels, providing additional support and information for parents over the

transition period. After four successful weeks at the new level, ISK staff set up an official

graduation where the child, members of the child’s family and the SARC coordinator attended an

appropriate ceremony and/or celebration.

Analyses, Results and Discussion

Using the measured levels for nine skills and composite indicators developed from them,

we constructed fixed-effects models from the panel data to study the changes within each child

in the program. Fixed effects models are appropriate because we assume that something within

the individual, the unique characteristics of the individual, may impact or bias the predictor or

outcome variables that measure program effect. We needed to control for this to evaluate the

program, thus we held constant (or “fixed”) characteristics of each child that did not change over

the time period of the study. These were medical diagnoses, level of intelligence or other

individual characteristics that did not change or changed very little over two years.

For the first set of models, we examined changes in skills for all children in the program

then grouped children by their primary medical diagnosis. Table 2: Fixed effects models

estimating one-month effect of program participation on skills, all participants and by primary

diagnosis shows the findings from four regression models.



19

The first two columns show parameters and statistical significance for the effect of one

additional month of program participation on skills, using the nine skills and five composite

scores defined above. The second set of columns shows the effect of one additional month of

program participation on children whose primary diagnosis is ASD. The next two sets of

columns show similar results for children with intellectual delays and with other primary

diagnoses.

Each estimation showed changes in the average effect on skill score for children in the

respective group by a one-month increase in program participation. (We used the average change

in score from evaluation to evaluation, a period of three months for each child.) We found that

overall, children participating in the program showed average improvement in scores in moving

safely, modulating volume, referencing and regulating emotions (p<0.05). We also saw

improvement in all compiled scores. For example, the effect of average monthly skill

improvement for moving safely was -0.061 (p<0.01). To make this easier to think about, if

children participated in the program for 24 months, we would expect, on average, to see a change

in the moving safely score of approximately 1.5 points (1=top score; 10=lowest score, so

negative coefficients mean improvement in skill).

When we examined children by primary diagnosis, we found that children with ASD

improved in using the restroom, modulating volume, regulating emotion and making friends

(p<0.05). They improved in the composite measurement when the restroom score was not

included (p<0.05) and in the composite measure for collaborating skills (p<0.05). Interestingly,

this suggests ASD children made their most improvements in very simple skills (restroom use

and modulating volume) and the more complex skills (regulating emotions, making friends and

collaborating).


20

Children with a primary diagnosis of intellectual delay only showed improvement in

moving safely (p<0.01). We found no other statistically significant effects of the program on

these children’s skill scores. Similarly, we found weak improvement in restroom use and

engaging skills (p<0.10) for the children with other primary diagnoses. Both of these findings

suggest that the children who participated during the time period of this study did not show

general improvement as a result of program attendance. These two groups looked markedly

different from the ASD group, suggesting that the program worked better for children on the

autism spectrum, and/or our sample size for the other two groups was small enough to show no

comparative results.

For the second analyses, we calculated the average effect on skill score for children in the

program by categorizing them in terms of their initial skills rather than primary diagnoses. Table

3: Fixed effects models estimating one-month effect of program participation on skills, all

participants and by entering skill category shows the effects on skill score using the attenders

(low), participators (middle) and collaborators (high) categories (representing the spectrum of

skills from simplest to most complex).


The estimations for children who entered the program with low skill levels, the attenders,

show improvements in average skill scores for most of the simplest skills; the effects on

modulating volume, moving safely and referencing show improvement, but are not strongly

significant (p<0.10). The score for restroom use showed a decline in skills (weakly significant;

p<0.10). We found no significant effect on any other skill score.

Interestingly, for children who entered the program with mid-range skills, the

participators, we found strong average effects on the more complex skills of cooperating,


21

regulating emotion, collaborating compiled scores and the overall composite measure. We

observed weaker average change on the composite score without restroom use (p<0.10),

suggesting that the progress in basic skill development, when controlled for restroom use,

improved only weakly (p<0.10). This is the group that showed the most overall improvement

based on composite scores and the number of individual skill scores showing statistically

significant improvement.

For those who entered the program as collaborators (higher skill levels), we observed

strong effects in two of the simpler skills; effects were statistically significant for modulating

volume (p<0.01) and the composite attending score (p<0.05). We also saw weak average effects

on moving safely (p<0.10). These counterintuitive results suggest that basic skills remained

important to improvement and/or perhaps the program needs to be adjusted for those who enter

the program with reasonably good social skills. Restroom use was not statistically significant,

most likely because these children came into the program with good restroom use skills.

Our results seem to support the finding by Rao et al (2008) that social skills training

programs should differ in their approaches to learning and adaptation of skills relative to

cognitive and verbal skills of children with ASD. Certainly our analyses show very different

patterns if children are grouped by their initial skill levels.

In summary, we captured the effectiveness of multiple aspects of the ISK program across

different types of participants. Children with ASD showed the most improvement, scoring much

higher on average than those with different primary diagnoses. In addition, although attenders,

participators and collaborators showed some improvement relative to their entry-level skills,

those children who began the program with mid-range skill levels tended to show the most

improvement. These initial findings support the idea that the methods, measures and evaluation


22

techniques created for the ISK program result in positive outcomes in terms of being included in

mainstream, out-of-school activities for some children. The findings also directed attention to the

areas where children did not show statistically significant improvements. These results suggested

that ISK leaders should examine their approaches to children with intellectual delays, other

developmental delays, and those with any delay who fell at the low or high end of the skill

spectrum. Perhaps these parts of the program need to be modified to better help those children

succeed in community-based, mainstream leisure programs.

There are a number of limitations in this work that should be acknowledged. First,

observed improvements in friendships, social skills and other skill measures do not prove that

children actually are more included or have higher quality of life. We have every reason to

believe this is true, but this study does not provide empirical evidence making that connection.

Secondly, the program is not a clinical trial, and the “control group” consists only of “typical”

behavior based on research and development literature and the behavior of “typical” children at

the community-based facilities. Thirdly, many children in the program had secondary diagnoses

that likely affected their ability to learn and adapt behavior. Fourthly, the small sample of

children with intellectual delay or other (non-ASD) diagnoses may be a concern regarding

observed behaviors and study results. Fifthly, overall group improvements do not show the

highly variable changes occurring in individual children. Finally, the data for this research came

from multiple sources. While ISK leaders made every effort to maximize inter-rater reliability of

learning and adaptive behaviors, the possibility of rater bias exists. In addition, initial data for

each child came from various sources. ISK leaders assumed that these tools measured the same

constructs, but have no method to assess inter-rater reliability in diagnoses or other

characteristics of children when they entered the program. Nonetheless, the study benefits by


23

time series data on children who participated in a community-based, inclusion program,

illustrating program effects not readily found in the literature.

Conclusion

In this study, we examined a novel treatment program and skills measurement tool for

developing better social skills and inclusion among children ages five through high school with

Autism spectrum disorders (ASD) and developmental delays. We examined the Including

Special Kids (ISK) Program, a collaborative effort with well-established community youth

programs in central California. We empirically tested and evaluated aspects of the program, and

controlled for the individual characteristics of each child. Using evidence-based indicators, we

measured progress of a population of 39 children over 6-24 months. We provided information on

areas in which the program was succeeding, notably in children with ASD rather than other

developmental delays, and those at the mid-level of skill achievement. We also identified where

either measurement or learning and adapting skills needed improvement and where the sample

size may have been small enough to affect the results. Our study of this community-based

inclusion program suggests that ISK had a meaningful effect on the skills ASD children need to

be accepted in and to participate with groups of typical children in out-of-school activities. This

initial assessment yielded encouraging results for community-based inclusion programs, which

merits further, in-depth study.


24

Bibliography Agency for Healthcare Research and Quality. (2011). Therapies for children with autism

spectrum disorders. Rockville, MD: Agency for Healthcare Research and Quality, Comparative Effectiveness Review 26, AHRQ Pub. No. 11-EHC029-1.

Bishop, D. (1998). Children's Communication Checklist - 2 (CCC-2). Product . Pearson.

Brookman, L. B. (2003). Facilitating social interactions in a community summer camp setting for children with Autism. Journal of Positive Behavior Interventions , 5(4):249-252.

Bruininks, R. W. (1996). Scales of Independent Behavior - Revised. Assessment system .

Carr, E. D. (2002). Positive behavior support: Evolution of an applied science. Journal of Positive Behavior Interventions , 4 (1), 4-16, 20.

Constantino, J. D. (2003). Validation of a brief quantitative measure of autistic traits: Comparison of the Social Responsiveness Scale with the Autism Diagnostic Interview - Revised. Journal of Autism and Developmental Disorders , 33 (4), 427-433.

Department of Education, State of Texas. (2007). Texas Guide for Effective Teaching Adaptive Behavior Assessment.

Department of Health and Human Services. (2012, March 30). Prevalence of Autism Spectrum Disorder - Autism and developmental disabilities monitoring network, 14 sites, United States, 2008. Morbitity and Mortality Weekly Report .

DiSalvo, D. &. (2002). Peer-mediated interventions to increase the social interaction of children with autism: Consideration of peer expectancies. Focus on Autism and Other Developmental Disabilities , 17 (4), 198-208.

Eldar, E. T.-Z. (2010). Successes and difficulties in the individual inclusion of children with Autism Spectrum Disorder (ASD) in the eyes of their coordinators. International Journal of Inclusive Education , 14 (1), 97-114.

Fayers, P. M. (2007). Quality of life: The assessment, analysis and interpretation of patient-reported outcomes. John Wiley & Sons, Inc.

Fryxell, D. K. (1995). Placement along the continuum of services and its impact on students' social relationships. Journal of the Association for Persons with Severe Handicaps , 20 (4), 259-269.

Gerhardt, P. M. (2010). Assessment of social skills and social competence in learners with autism spectrum disorders. In Practitioner's guide to empirically based measures of social skills, ABCT Clinical Assessment Series (pp. 193-205). Springer Science+Business Media, LLC.


25

Global Data. (2010). Autistic Disorder and Market Forecasts to 2017. Autistic Disorder and Market Forecasts to 2017 , 37.

Gresham, F. M. (1987). The relationship between adaptive behavior and social skills: Issues in definition and assessment. Journal of Special Education , 21, 167-181.

Gresham, F. R. (1986). Social skill deficits and low peer acceptance of mainstreamed learning disabled children. Disability Quarterly , 9 (1), 23-32.

Guralnick, M. G. (1996). Effects of social setting on the friendship formation of young children differing in developmental status. Journal of Applied Developmental Psychology , 17 (4), 625-651.

Haring, T. B. (1992). A peer-mediated social network intervention to enhance the social integration of persons with modereate and severe disabilities. Journal of Applied Behavior Analysis , 25 (2), 319-333.

Hartman, C. L. (2006). Refinement of the Children's Social Behavior Questionnaire (CSBQ): An instrument that describes teh divser problems seen in milder forms of PDD. Journal of Autism and Developmental Disorders , 36 (3), 325-42.

Hartup, W. (1989). Social relationships and their developmental significance. American Psychologist , 44, 120-126.

Jones, A. F. (2010). Multi-informant predictors of social inclusion for students with autism spectrum disorders attending mainstream school. Journal of Autism and Developmental Disorders , 40, 1094-1103.

Jones, R. R. (1979). Naturalistic observation in clinical assessment. In P. McReynolds (Ed.), Advances in psychological assessment (Vol. 3, pp. 42-95). San Francisco: Jossey-Bass.

Kamps, D. K.-L. (1998). What do the peers think? Social validity of peer-mediated programs. Education & Treatment of Children , 21 (2), 107-134.

Kanne, S. G. (2011). The role of adaptive behavior in autism spectrum disoerders: Implications for functional outcome. Journal of Autism and Developmental Disorders , 41, 1007-1018.

Kennedy, C. (2001). Social interaction interventions for youth with severe disabilities should emphasize interdependence. Mental Retardation and Developmental Disabilities Research Reviews , 7 (2), 122-127.

Knapp, M. R. (2009). Economic cost of autism in the UK. Autism , 13 (3), 317-336.

Kuhithau, K. O. (2010). Health-Related Quality of Life in Children with Autism Spectrum Disorders: Results from the Autism Treatment Network. Journal of Autism and Developmental Disorders , 40, 721-9.


26

Leach, D. D. (2009). Supporting students with autism spectrum disorders in inclusive settings. Intervention in School and Clinic , 45 (1), 31-37.

Lopata, C. T. (2006). Effectiveness of a cognitive-behavioral treatment on the social behaviors of children with Asperger disorder. Focus on Autism and Other Developmental Disabilities , 21(4):237-244.

Magiati, I. M. (2011). Patterns of change in children with Autism Spectrum Disorders who received community based comprehensive interventions in their pre-school years: A seven year follow-up study. Research in Autism Spectrum Disorders , 5 (3), 1016-1027.

Rao, P. B. (2008). Social skills interventions for children with Asperger's Syndrome or high-functioning autism: A review and recommendations. Journal of Autism and Developmental Disorders , 38, 353-61.

Shukla, S. K. (1998). Adult influence on the participation of peers with out disabilities in peer support programs. Journal of Behavioral Education , 8 (4), 397-413.

Sparrow, S. C. (2005). Vineland Adaptive Behavior Scales, Second Edition (Vineland-II). Product . Pearson.

Special Kids Crusade. (2012). Special Kids Crusade. Retrieved June 19, 2012, from Special Kids Crusade: specialkidscrusade.org

United Nations. (2008, Jan 21). General Assembly Resolution A/RES/62/139. Retrieved July 9, 2012, from United Nations : http://www.un.org/ga/search/view_doc.asp?symbol=A/RES/62/139

Varni, J. (1998). Pediatric Quality of Life Inventory 4.0. PedsQL .

Verhulst, F. &. (2008). Rating scales. In M. B. Rutter (Ed.), Rutter's child and adolescent psychiatry (pp. 70-86). Oxford: Blackwell.

Vismara, L. R. (2010). Behavioral treatments in autism spectrum disorder: What do we know? Annual Review of Clinical Psychology , 6, 447-468.

von der Embse, N. B. (2011). Reducing problem behavior and promoting inclusion for students with Autism Spectrum Disorders. Intervention in School and Clinic , 47:22-30.

Ware, J. (n.d.). SF-36 Health Services Update. Retrieved May 20, 2012, from SF-36.org: http://www.sf-36.org/tools/

Weiss, M. H. (2001). Teaching social skills to people with autism. Behavior Modification , 25 (5), 785-802.


27

Table 1: Evaluation criteria for each skill

Level Adverb %

observed #

Days Ratio Skills Focus

1 Typically 91-100 3-5 3:1 Attendance, Participation & Collaboration

2 Usually 81-90 3-5 2:1 Attendance, Participation & Collaboration

3 Frequently 71-80 3-5 2:1 Attendance, Participation & Collaboration

4 Often 61-70 3 1:1 or 2:1

Attendance & Participation

5 Half of the Time

51-60 2 or 3 1:1 or 2:1

Attendance & Participation

6 Sometimes 41-50 2 or 3 1:1 Attendance & Participation 7 Occasionally 31-40 1 or 2 1:1 Attendance 8 Sporadically 21-30 1 or 2 1:1 Attendance 9 Seldom 11-20 1 1:1 Attendance w/accommodation 10 Rarely 1-10 1 1:1 Attendance w/accommodation


28

Table 2: Fixed effects models estimating one-month effect of program participation on skills, all participants and by primary diagnosis


29

Table 3: Fixed effects models estimating one-month effect of program participation on

skills, all participants and by entering skill category


30

1 Dollar conversions calculated using Yahoo Currency Converter, July 4, 2012. 2 SARC provides services as required by the Lanterman Developmental Delays Services Act, a California law stating that people with developmental delays and their families have a right to get the services and supports they need to live like people without delays. SARC, one of 21 regional, community-based, private nonprofit corporations funded by the State of California to serve people with developmental delays, supports individuals and their families who reside within Monterey, San Benito, Santa Clara, and Santa Cruz counties. 3 SKC wrote the grant proposal in March of 2007, went through a series of interviews, and was ultimately awarded nearly half of the SARC total funds available. 4 In this paper, we use “ISK leaders” to mean the program director and adaptive skills consultant. 5 Neither SKC nor BGCMC hold daycare licenses, requiring all program participants to use the restroom without assistance. If a child brought his or her own assistant, he or she was allowed to participate. 6 “Significant” individuals include parents, teachers and other adults who have an emotional connection with the child. These are the adults that children learn to reference first; later the children learn to discern who is “in the know” in different environments and to reference them.

Evaluating Community Inclusion: A Novel Treatment Program and Skills Measurement Tool for Children with Autism Spectrum Disorders and Other Developmental Delays

Documents