School-Based Autism Assessments: Review of Current ...casponline.org/event-handouts/fall17/W-20.pdf · School-Based Autism Assessments: Review of Current Practices & Guidelines for

School-Based Autism

Assessments: Review of Current

Practices & Guidelines for a

Comprehensive Assessment

Angela Davis, Ph.D.

Jeanne Anne Carriere, Ph.D.

Kelly Kennedy, Ph.D.

Goals for Today’s Workshop

1. Introduction to ASD/assessment

2. Review data on a survey completed by California school psychologists

3. Discuss relevant case law for school-based autism assessments

4. Openly discuss planning and conducting of a comprehensive school-based autism evaluation

Why conduct research in this area?

• A meta-analysis examined the content of ASD

research in six school psychology research

journals (McKenney et al., 2015).

• Researchers found that only 38 articles over a

10-year span (2002-2012) discussed autism-

related topics. Out of these articles, only five

reviewed assessment and identification of ASD.

Reasons for Increased Attention &

Improved Identification of ASD

• Early identification and treatment can change the

course of the disorder.

• Given “increased rates”, educators are more likely to

encounter students with ASD.

• Students under an Autism eligibility have increased four fold between

1999 and 2012 (currently 10% of SWD and 1% of K-12 population)

• The majority of children with autism are identified

through school resources.

• Average age of ASD identification is about 4 years of age.

• African American & Latino children often diagnosed 1 ½ years later than

the average.

(Brock, 2009; Durand, 2014; Mandall, etal, 2002;Wilkinson, 2009)

Semantics Matters: Eligibility &

Diagnosis

Eligibility Diagnosis

Educational Educational Eligibility Diagnosis

Educational Clinical Evaluations Evaluations

When we try to equate these things we confuse ourselves and confuse families. This causes problems

for everyone.

Federal: IDEA 2004

(i) Autism means a developmental disability

significantly affecting verbal and nonverbal

communication and social interaction, generally evident before age three that adversely affects a

child's educational performance.

Other characteristics often associated with autism are engagement in repetitive activities and

stereotyped movements, resistance to

environmental change or change in daily routines, and unusual responses to sensory experiences.

California Code of Regs

Autism means a developmental disability significantly

affecting verbal and nonverbal communication and

social interaction; generally evident before age three

that adversely affects a student's educational

performance. Other characteristics often associated

with autism are engagement in repetitive activities and

stereotyped movements, resistance to environmental

change or change in daily routines, and unusual

responses to sensory experiences. The term does not

apply if a student's educational performance is

adversely affected primarily because the student has

an emotional disturbance. A student who manifests the

characteristics of autism after age three could be

identified as having autism if the criteria are satisfied.

DSM-5 (must meet crtieria A, B, C, D, & E)

A. Persistent deficits in social communication and

social interaction across multiple contexts, as

manifest by the following, currently or by history

(examples illustrative, not exhaustive):

• Deficits in social-emotional reciprocity

• Deficits in nonverbal communicative behaviors

used for social interaction

• Deficits in developing, maintaining, and

understanding relationships

DSM-5 (must meet crtieria A, B, C, D, & E)

B. Restricted, repetitive patterns of behavior, interests, or activities as manifested by at least two of the following, currently or by history (examples illustrative, not exhaustive):

• Stereotyped or repetitive motor movements, use of objects, or speech

• Insistence on sameness, inflexible adherence to routines, or ritualized patterns of verbal or nonverbal behavior

• Highly restricted, fixated interests that are abnormal in intensity or focus

• Hyper-or hypo-reactivity to sensory input or unusual interest in sensory aspects of the environment

DSM-5 (must meet crtieria A, B, C, D, & E)

C. Symptoms must be present in early developmental period (but may not become fully manifest until social demands exceed limited capacities, or may be masked by learned strategies in later life)

D. Symptoms cause clinically significant impairment in social, occupational, or other important areas of current functioning

E. These disturbances are not better explained by intellectual disability (intellectual developmental disorder) or global developmental delays. Intellectual disability and autism spectrum disorder frequently co-occur, to make comorbid diagnoses of autism spectrum disorder and intellectual disability, social communication should be below that expected for general developmental level.

Are there meaningful similarities & differences?

CA Ed Code 1. Verbal and nonverbal communication and social interaction;

2. Generally evident before age three that adversely affects a student's educational performance.

3. Other characteristics often associated with autism are engagement in repetitive activities and stereotyped movements, resistance to environmental change or change in daily routines, and unusual responses to sensory experiences.

4. A student who manifests the characteristics of autism after age three could be identified as having autism if the criteria are satisfied.

DSM-5 A. Persistent deficits in social

communication and social interaction

across multiple contexts, as manifest by

the following, currently or by history…

B. Restricted, repetitive patterns of

behavior, interests, or activities as

manifested by at least two of the

following, currently or by history

C. Symptoms must be present in early

developmental period (but may not

become fully manifest until social

demands exceed limited capacities, or

may be masked by learned strategies

in later life)

D. Symptoms cause clinically significant

impairment in social, occupational, or

other important areas of current

functioning

“My child has a diagnosis of Autism from his

doctor. Are they eligible for Special Education?”

Special Education Eligibility

Evaluation purpose:

Determination of eligibility for special education services.

Development of individualized educational program to meet unique needs of student

IDEA eligibility criteria used:

Child meets the eligibility criteria

Disability must adversely affect child’s educational achievement/performance

Medical/Clinical Diagnosis

Evaluation purpose:

Treatment planning

DSM-5 criteria used:

Child meets specific diagnostic criteria

Differential diagnosis (Ruling in or out co-morbid conditions)

A medical or clinical diagnosis based on DSM-5 is relevant but not sufficient to qualify a child for Special Education.

What does NASP Best Practices Say?

• A multidisciplinary approach should be utilized

• Parent/teacher/student interview

• Conduct behavioral observations across several

environments

• The NASP Best Practices Series autism assessment authors

recommend school psychologists utilize a structured autism

observation method (e.g., ADOS-2) and autism-specific

checklists or rating scales for school-based assessments

• Areas that are typically included in clinical and school-

based evaluations are speech/language/communication,

cognitive/intelligence, adaptive behavior, occupational

therapy/sensory, behavioral, and academic Clark et al., 2014; Schwartz & Davis, 2014

Survey of Current Practices on

School-Based Autism Assessments

Survey Contents

Consisted of 50 questions- 49 close-ended

questions and 1 open-ended question

Created based on the NASP Best Practice

series guidelines on autism assessments,

California Code of Regulations for the autism

eligibility criteria, California Statutes

Education Code for assessment reports, and

general impressions/practices of school-

based autism assessments

Data Collection Procedures

Emails were sent to CASP members from July

to August 2016

The survey was on the CASP website for 7

weeks

A total of 7 emails were sent to CASP

members to ask them to complete the survey

Demographic Characteristics

Autism Assessments Completed in the

2015-16 School Year

Current Assessment Practices for CA School

Psychs

Parent, Teacher & Student Interview

Information gathered from outside

service providers

Observations

What rating scales were used for

autism assessments?

Other rating scales used

Scale used most frequently

Use of ADOS-2

Areas Assessed

Time Spent with Parents During the

Evaluation

General Impressions

Litigious Reasons for ASD cases

amount of services requested by

parent/advocate/attorney;

factors regarding the district (perception,

relationship, and resources);

factors regarding parents (resources, education

level, grieving, understanding their rights/laws);

outside agencies/professionals advocating for

services deemed inappropriate per school district;

adequacy of autism assessments.

Legal Trends/Case Law

OAH Case Search

July 1, 2005-July 20, 2016

Search terms:

Autism

Assessment

Best Practice

A total of 233 cases were found

10 cases met the criteria- issues regarding the appropriateness of the psych eval

80% of the cases met the legal requirements

Discussion

Considering the litigious reasons for ASD cases, discuss

the follow questions with the person(s) seated next to

you:

1. Did all of the cases include a parent interview?

2. How about the ADOS?

3. Was adaptive behavior assessed in each case?

Themes of OAH Cases

Best Practices Terminology

School Psychologist’s Training and Expertise

Inadequate Assessments

“Best Practices”

Each case had a expert witness that discussed “best

practices”

Some judges sided with these “best practices”

recommendations and stated “not adhering to best

practice did not render the psychoeducational

assessment as legally noncompliant” but other

judges decided the case was inadequate if the

psychologist did not follow “best practice”

recommendations

School Psych’s Training & Experience

In the cases reviewed, the school psychologist’s

training and experience in administering autism-

specific measures (i.e., ADOS-2) was challenged in

two OAH cases

Both psychs did not attend the ADOS-2 workshop

ALJ deemed they were qualified to administer the

ADOS-2

Not Recommended

Inadequate Assessments

Two cases that found assessment inadequate:

1. The school psychologist did not adhere to best

practice assessments per the testimony of a

clinical psychologist

2. The school psychologist incorrectly used a

rating scale and used terminology not

included in the autism eligibility criteria when

determining if a student qualified for special

education services

Conduction Comprehensive

Assessments

We are evaluating… “A singular presentation of a global disorder”

Marilyn J. Monteiro, PhD Autism Conversations, 2013

A Comprehensive Assessment, according to IDEA, CA Education Code, NASP & APA Ethical Guidelines

1. Assessment tools are chosen and used so that they are not

racially, sexually, or culturally discriminatory.

2. Assessments must be conducted in the language and form most

likely to yield accurate information on what the student knows and can do academically, developmentally and functionally unless

the assessment plan indicates why it is not feasible to provide or

administer.

3. For students with Limited English Proficiency the assessment instruments are selected and administered in a manner that

ensures that the assessment measures what is intended rather than

the student’s English language skills.

4. For students with sensory, motor, or speaking impairments the assessment instruments are selected and administered in a

manner that ensures the assessment measures what is intended

rather than the student’s sensory, motor, or speaking skills.

A comprehensive assessment

continued… 5. Assessment tools and strategies are selected to provide information

on the student’s specific areas of educational need.

6. Assessment instruments are reliable and have been validated for the purposes for which they are going to be used.

7. Assessments are only conducted by qualified professionals.

8. No single measure or assessment is used as the sole criterion for determining whether a student has a disability. The assessment covers all areas related to the student’s suspected disability and all areas of need.

9. A variety of assessment tools and strategies should be used to gather relevant functional and developmental information about a child.

10. Assessment results should be provided in written language that is understood by the consumer.

11. Reports should emphasize meaning and make recommendations that help the readers in their work with children

In summary…

The evaluation should…

• be comprehensive

• use a variety of evaluation tools or approaches that gather functional and relevant data

• be fair

• use valid and reliable tools and procedures

• be completed by a competent evaluator

• be written into a report that is useful and understandable

Conceptualizing the Evaluation

Cycle

Conduct Assessment

Interpretation, Analysis & Synthesis

of Data

Eligibility/ Diagnosis

Intervention Services

Treatment

Questions & Hypotheses

Assessment Planning

Common mistakes in ASD assessment…

• Examiner not knowledgeable about general developmental milestones

• Not using multiple informants

• Using multiple informants, but not synthesizing/explaining similarities and differences

• Atypical behaviors in observations & other assessments are described as typical (over-normalizing…)

• Not "digging" deeper into student's perceptions of relationships

• Missing subtle symptoms that have a lasting impact

• Not taking history and/mental health issues into account

• Over-reliance on standardized assessments-no functional skills information

• Limited “optimal” or typical environmental observational opportunities

• Rigidity in thinking about what a child with ASD "looks like“

• Confusion of educational achievement/performance

• Using measures that are not appropriate/valid/reliable

Commonly Used Tools for Autism

Assessments

*GARS-3 published in 2014 and non-GARS published data not available for

review.

Working

Collaboratively to

create a

Comprehensive

Autism

Assessment

Collaboration During Assessment

An interactive process where professionals work together and share their diversity of knowledge and expertise to define needs, plan, implement, assess and follow up on ways of helping children develop to their fullest.

Comprehensive Assessment

Parent(s)

Teachers

School Service

providers

Others who have

specialized knowledge about the

child

Clinical/

Medical Providers

Private

Service Providers

Student

Assessment Planning

When reviewing records and making initial contact with families and teachers where ASD is a concern, ask yourself:

• What is the specific referral concern? (i.e., social interaction, cognition, behavior, language development)?

• Developmentally, what are the signs associated with ASD?

• If previous evaluations have been completed, are the findings valid and or useable?

• What eligibilities/diagnoses best explain the student’s profile?

• What additional information is needed to determine the student’s disabilities?

RIOT- Comprehensive Assessment

Model

Interview

Observe Test

Review

RIOT: Review

Review: Preschool Records, School Cumulative Files &

Health File, Prior Assessments, Outside Assessments

and Treatment Summaries

• Signs of behavior patterns present in early childhood

• History of social communication/interaction deficits,

• Signs of other risk factors (e.g., siblings with autism,

other family members with autism)

• Potential complicating factors

Developmental Milestones Resources

CDC Birth to 5 milestones, videos, concern,

Autism case training for professionals

http://www.cdc.gov/ncbddd/actearly/index.

html

CDC Autism signs and symptoms

http://www.cdc.gov/ncbddd/autism/facts.html

RIOT: Interview

Child

ability to warm to interviewer and socially connect

eye-contact, voice, restricted reciprocal interactions

social experiences

Parents

structured developmental histories or more formalized questionnaires

history of restricted/ repetitive behaviors and social communication in early years

structured and semi-structured clinical interview format

functional skills in the community and at home

Teachers

social communication

self-advocacy/initiation behavior

patterns of restricted interests and/or repetitive behaviors

classroom adaptive skills

academic achievement and performance

Parent input is crucial!

Use both informal & structured interviews to gather information about…

Detailed developmental history

Current adaptive functioning

Behavioral concerns

Communication delays and concerns

Friendships/Social skills

Parent Interview: Open Ended ?

Examples… • What are your concerns? When did you first become concerned?

• How did your child’s development differ from that of his siblings?

• How does he let you know when he needs something? How did he let you know when he first started communicating?

• Tell me about his imitation skills; does he spontaneously copy what you do or do you have to teach him? When did he begin imitating you? How did this look?

• What are his interests? What are his favorite toys and games?

• Describe what he did/does with toys and how he played/plays? (Probe for pretend, sequencing, variety, interactions with dolls, animal or action figures)

• When and how does he interact with others?

• How does he get along with his siblings?

Parent Interview Questions Cont.

• Tell me about his friendships and interest in other kids? How does he play with other kids?

• What does he look like at the park or recess? How does he respond when other children approach or invite him to play?

• How does he communicate his feelings to you and others? How does he respond to others when they are sad or upset?

• Describe any sensitivities (sound, touch, texture, food) and lack of sensitivity (pain, temperature).

• How does he respond to changes in routines and schedules?

• Are there any things he seems to have to do in a particular way or order?

• What motivates him/her? What do you enjoy doing together?

• What are his/her strengths? Special skills?

Teacher input is crucial!

• Strengths/concerns

• Behavioral concerns

• Communication delays and concerns

• Friendships/Social skills (classroom and recess)

• Attention

• Transitions between activities

• Understanding abstract concerns (reading comp, Common core math)

• Generalizing skills and concepts

• Performing upon request

• Motivation to perform

• Classroom adaptive skills

• Problem solving skills

• Fine motor, writing, and drawing skills

“Interview”: Rating Scales

Adaptive

Individuals with ASD have adaptive skill deficits

Comprehensive Examples: Vineland-3, Adaptive Behavior

Assessment Scale (ABAS-3)

Social Communication and Skills

Development

Examples: Social Communication Questionnaire-Current or

Lifetime (SCQ), Social Responsiveness Scale-2 (SRS-2), Social Skills Improvement System (SSIS)

“Interview”: Rating Scales cont…

Behavior and Social Emotional Functioning

• We are evaluating for behavioral difficulties and social skills

though poor emotional regulation, limited coping strategies, self

awareness, insight, ability to attend to others’ reactions /

preferences, and flexibility may also impact overall mental

health

• Broad Band Examples: Behavior Assessment System for Children-

3 (BASC-3), Child Behavior Checklist (CBCL), Social Skills

Improvement System (SSIS), Beck Youth Inventories-2 (BYI-2)

• Singular Examples: Multidimensional Anxiety Scale for Children

(MASC)

RIOT: Observations

• General:

• Observe the child in multiple settings over

multiple days:

• arrival and departure, transitions, structured

and unstructured activities, and

classroom/non-classroom settings, with adults

and peers

• Where are there problems? Why?

• Where is the child successful? Why?

Example: Observations

Social Interaction &

Communication • Ability to initiate and sustain social

interactions (reciprocal communication)

• Failure to initiate or respond

• Poorly integrated verbal and nonverbal communication

• Abnormalities in eye contact (too much or too little)

• Ability to express/read emotions

• Tone of voice

• Difficulty with idioms or pragmatic language

• One-sided conversation patterns

• “Directing” play

Behavior Patterns

• Selected interests

• Need for

routines/sameness

• Difficulty with transitions

• Fine and/or gross motor

difficulties

• Stereotypical behaviors:

flapping, rocking,

posturing, etc.

• Other sensory seeking or

stimulating behaviors

Informal (On your own…) Formal (Published)

Anecdotal/Naturalistic

Time Sampling

Event/Frequency

Recording

Environmental/Setting

BASC-2, Structured Observation System

Behavior Observation of Students in School (BOSS)

Childhood Autism Rating Scales-2 (CARS-2)

Autism Diagnostic Observation Schedule-2 (ADOS-2)

Systematic Direct Observations

Structured Observations:

The ADOS-2 ?

Why Use It?

Psychometrically strong

Pulls for ASD-specific

behaviors

Creates context to observe

diagnostic behaviors

Takes into account

expressive language level

and age of child

Yields rich clinical data

(qualitative & quantitative)

Cautions

Requires extensive training

Requires practice and

routine use

Scores based on behavior

exhibited during

administration only

Results in classification, not diagnosis

Use of an incorrect module

can result in over or under

classification

RIOT: Test

Standardized assessment tools alone will not be sufficient in diagnosing or assessing fully

for ASD.

Don't neglect the RIO!

After Review-Interview-Observe (RIO),

determine:

• What areas need to be directly assessed or test?

• How much formal/standardized testing is needed?

• How do we maximize the overall evaluations validity?

BUT…

STUDENTS WITH ASD TEND TO HAVE DIFFICULTY WITH ESSENTIAL TEST

TAKING BEHAVIORS:

Attention span

Response on demand

Imitation

Motivation

Desire to please

Assessment support tools and

strategies

• Give Choices

• Visual Schedule

• Sticky Pads

• White Boards

• Timer

• Tally Marks

• First-Then cards

• Reinforcers/Sticker Charts

• Game/Play/Movement Breaks

Cognitive Assessment Selection

• All instruments are not created equally

• Choose a test that taps into the child’s current abilities

and developmental level

• Measure both verbal and nonverbal skills

• Look at the pattern of strengths and weaknesses

Cognitive Assessments

Preschool Examples

Differential Ability Scales-2, Early Years

Mullen Scales of Early Learning

Bayley Scales of Infant and Toddler Development-III (Bayley–III)

School Age Examples

Differential Ability Scales-2, School Age

Kaufman Assessment Battery for Children-2 (KABC-2)

Wechsler Intelligence Scale for Children-5 (WISC-5)

Woodcock Johnson-Tests of Cognitive Ability (WJ-3)

School Age + Examples

Stanford-Binet Intelligence Scales, Fifth Edition (SB-5)

Leiter International Performance Scale-3 ( Leiter-3

Assessment Planning/Data Synthesis: School-

based “organization” of an assessment

Why is this important?

Individually…

Take a moment to think about your current practice or a recent

case and fill in as much of the Assessment Planning/Data

Synthesis matrix as possible.

In small groups, discuss…

What does your assessment of

autism look like?

What are you doing to support

a comprehensive assessment?

Model (Planning tool)?

Convergent data?

Multiple sources/informants?

Is data relevant and

functional?

Supports

intervention/treatment

planning?

What steps or changes can

you make toward a more

comprehensive

evaluations?

What will you take back to

your school, team, and

colleagues?

Questions, comments, concerns?

Angela Davis [email protected]

Jeanne Anne Carriere [email protected]

Kelly Kennedy [email protected]