Assessment, Evaluation, and Programming System for Infants and Children (AEPS™)-AEPSi Part 1 KDEC: 2013 Nancy Miller, M.Ed., OTR/L Stephanie Parks, Ph.D.,

Assessment, Evaluation, and Programming System for Infants and Children (AEPS™)-AEPSi

Part 1

Administering with Fidelity

KDEC: 2013

Nancy Miller, M.Ed., OTR/L

Stephanie Parks, Ph.D., OTR/L

• Authentic assessment• AEPS Overview (brief)• Fidelity in Assessment• Scoring, Team Consensus,

and Efficiency• Pitfalls to Fidelity• Resources

The root of the word assessment is assidere, which means “to sit beside and get to know.”

Authentic Assessment

Authentic Assessment

o Familiar people….o In familiar settings…o With familiar objects/toys…o Doing familiar things.

Adapted from: Sophie Hubbell, M.A.T Kent State University

Assessment Evaluation and Programming System

(AEPS) for Infants and Children (Second Edition)

AEPS Interactive (AEPSi)• Secure, web-based tool • Allows for easy means to record, score, track,

aggregate, archive, and report on the results of the AEPS Test

• http://aepsinteractive.com• http://www.aepsi.com

How are you currently using AEPS (CBA)

•Screening•Eligibility•Program Planning •Progress Monitoring•Program Evaluation

What is the AEPS• It is a comprehensive system that ties

together assessment, goal development, intervention, and ongoing monitoring and evaluation.

It is • Criterion-referenced• Curriculum-based • Domain Specific• Developmental• Can be used to corroborate eligibility

decisions• Programmatic: can help you determine

priority goal areas and focus your interventions

linked system framework

• Intervention• Evaluation

• Goal Development

• Assessment

Collecting Information ObservationsDirect Tests

Report

Summarizing Information Student goals

Family outcomes

Curricular Approach

Intervention based on Extended

StandardsInstruction

MonitoringWeekly

QuarterlyAnnual

This

Not This

Fine motor

Gross motor

Adaptive

Cognitive

Social Communi-

cation

Social

AEPS DOMAINS

AREA

STRAND A STRAND B

Goal 1 Goal 1 Goal 2Goal 2 Goal 3

Obj. 2.1

Obj. 2.3

Obj. 1.2 Obj. 2.2

Obj. 1.1

Obj. 2.2Obj. 1.2

Obj. 3.1 Obj. 2.1Obj. 1.1

Obj 2.3

Obj. 3.2

Obj. 1.3

Strands: Easy to More Difficult

Goals: Easy to More Difficult

Objectives become more difficult as the goal is approached.

© Jennifer Grisham-Brown, Kentucky Early Childhood Data Project, 2007

Organizational Structure of AEPS test items

“One mark of excellent teachers is the frequency with which they evaluate and change children’s instructional programs continually adapting them to meet the needs of each child.”

Bailey and Wolery, 1992

FIDELITYEvidence is needed that an assessment such as a CBA [AEPS] is administered, summarized, interpreted, and used in the way that it was designed, intended and validated.

Grisham-Brown, J., & Pretti-Frontczak, K. (2011). Assessing Young Children in Inclusive Settings: The Blended Practices Approach. Baltimore: Paul H. Brookes Publishing Co.

• ONGOING Observation (PREFERRED)• Within routines/activities

• Direct Test*• Report

Collecting Assessment and Evaluation Information with Fidelity-Scoring

*Note: Scoring guidelines when using the Direct Test method are not the same as the Observation guidelines. Refer to page 47 in Volume 1.

• 2 = Consistently meets criterion

• 1 = Inconsistently meets criterion;

emerging

• 0 = Does not meet criterion; currently does not exhibit the skill; (in preschool aged child-may not yet be expected)

Scoring Guidelines: Observation or Direct Test

• Child consistently performs the item as specified in the criterion

• Child performs the item independently• Behavior is a functional part of the child’s repertoire• Child uses the skill across time, materials, settings, and

people

2 Consistently

meets criterion

• Child does not consistently perform the item as specified in the criterion• Child performs the item with assistance• Child does not perform all components of the item or does not meet all

aspects of the specified criterion (i.e., the behavior is emerging• Child performs the item only under specific situations or conditions (i.e.,

with certain people or in certain settings)

1 Inconsistently

meets criterion

• Child does not perform the item as specified in the criterion when given repeated opportunities or assistance or when modifications and adaptations are made.

• Child was not observed performing the item because it is not expected based on knowledge of development (e.g., the child’s chronological age is 6 months and he or she would not be expected to perform such items as categorizing similar objects, copying simple shapes, or walking up and down stairs)

0 Does not

meet criterion

From: Bricker, D. (2002). Assessment, evaluation, and programming system for infants and children. Baltimore: Paul H. Brookes Publishing Co.

Observation

Observation

• Child performs the item as specified in the criterion on at least two out of three trials

• Child performs the item independently on two out of three trials

• Child uses the skill on two out of three trials across time, materials, settings, and people

2 Consistently

meets criterion

• Child performs the item as specified in the criterion on one out of three trials

• Child performs the item with assistance on one out of three trials• Child performs the item under one situation or one set of

conditions

1 Inconsistently

meets criterion

• Child does not yet perform the item as specified in the criterion on zero out of three trials when assistance is provided or when modifications and adaptations are made

• Child was not observed performing the item because it is not expected based on knowledge of development (e.g., the child’s chronological age is 6 months and he or she would not be expected to perform such items as categorizing similar objects, copying simple shapes, or walking up and down stairs)

0 Does not

meet criterion

From: Bricker, D. (2002). Assessment, evaluation, and programming system for infants and children. Baltimore: Paul H. Brookes Publishing Co.

DIRECT TESTING

DIRECT TESTING

Scoring with Fidelity …. It’s not just a 0, 1, 2

Use of Notes and Comments to enhance your assessment data

• A = assistance provided (1 or 0)• B = behavior interfered (1 or 0)• D = direct test (2, 1, 0)• M = modification/adaptations (2, 1, 0)• Q = quality of performance (2, 1)• R = report (2, 1, 0)

A

• Assistance Provided:• When a child is provided some form of assistance, an “A”

should be noted in the space next to the performance score box. If assistance is provided, then the only scores allowed are 1 and 0 because a score of 2 indicates full independent performance.

• Assistance includes any direct verbal or physical prompt, cue or model that assists the child in initiating or performing the desired behavior.

B

• Behavior Interfered• At times a child’s behavior may interfere with the

demonstration of the desired skill. In such cases, the item may be scored 1 or 0 with a “B” noted next to the performance score.

• This note indicates that the child may have the skill, but disruptive or noncompliant behavior interfered with its demonstration.

From: Bricker, D. (2002). Assessment, evaluation, and programming system for infants and children. Baltimore: Paul H. Brookes Publishing Co.

D• Direct Test:• When the examiner directly elicits a behavior, a “D” is noted

next to the performance score and the guidelines for determining the score for “Direct Test” are followed.

M

• Modification/Adaptation:• At times, an examiner may need to modify the stated criteria

(e.g., rate or mode of response) or adapt the environment/materials (e.g., adaptive equipment is necessary) to asses children with sensory or motor disabilities.

• When modifications are made in gathering child performance information, an “M” is noted next to the performance score and a 2, 1, or 0 is used.

From: Bricker, D. (2002). Assessment, evaluation, and programming system for infants and children. Baltimore: Paul H. Brookes Publishing Co.

Q

• Quality of Performance:• At times, a child is able to perform a skill independently,

but the team feels the quality of the performance hinders the ability to meet criteria. At other times, a child is able to meet or partially meet the criteria, but the team wishes to continue strengthening the quality of performance.

• When the quality of the performance is in question, teams are encouraged to use a “Q” in the notes accompanied by a score of 2 or 1.

From: Bricker, D. (2002). Assessment, evaluation, and programming system for infants and children. Baltimore: Paul H. Brookes Publishing Co.

R

• Report:• When an item is assessed by report, an “R” is noted next to the

performance score. Report is used under one of three conditions:• When assessment information is collected by another person or

documented source (e.g. written evaluation), the item is scored 2, 1, or 0, and an “R” is noted.

• When the item is judged inappropriate because it assesses a primitive or developmentally easier response (e.g., sucking on a nipple when the child is able to drink from a cup), the item is scored 2 and an R is noted.

• When the item is judged inappropriate because it is too advanced or beyond the child’s developmental level (e.g., walking when the child is unable to stand), the item is scored 0 and an “R” is noted.

From: Bricker, D. (2002). Assessment, evaluation, and programming system for infants and children. Baltimore: Paul H. Brookes Publishing Co.

Last but not least…

Your written comments to quantify, describe, enhance the observations you have made.

Team Consensus Builds Scoring Fidelity

Team Discussions

• What do these scores mean to you?• Consistently/Inconsistently

• The power of the Notes section• With assistance• Modifications/Adaptations

• Team use of the Comment Section• Under what conditions/strategies etc.

• Team responsibilities discussion

Don’t forget the Family Report• 2 Levels (birth to three and three to six)• 2 sections

• Family Routines • Often done through

conversation/interview• Family Observations

• Scored: Yes, Sometimes, Not Yet

Consider using the second section of the Family Report with Community and Daycare Providers since the skills parallel the AEPS CODRF across the developmental domains.

Score as “Report”

Ways to gather and document

• Individually• CODRF

• Group• Group routine/activity

matrix• Family Report

• I and II (Spanish version available)

Assessment Activity Plans

• AEPS comes with 12 pre-written activities to assess a variety of children across developmental areas (see Volume 2)

• OR you can create your own that parallel existing planned activities or those provided in the AEPS.

Adapted from: Sophie Hubbell, M.A.T Kent State University

Administering the AEPS with Groups of ChildrenAdministering the AEPS with Groups of Children

AEPSi Assessment Activities: Center-based (Level 1 and Level 2)

• Book About Me• Classroom Transitions and Routines

• Dramatic Play• Meals and Snack• Story Time• Playdough and Manipulatives

• Outdoor Play• Conversation with Familiar AdultsJennifer Grisham-Brown, Kentucky Early Childhood Data Project, 2007

AEPSi Assessment Activities: Routine-Based(Level 1)

• Rough & Tumble• Quiet Time• Mystery Box• Feeding & Meals• Daily Transitions & Routines

• Conversations with Caregivers

• Busy Hands

© Jennifer Grisham-Brown, Kentucky Early Childhood Data Project, 2007

Reasons why CBAs are not implemented with fidelity

• Teachers may find the actual implementation of a CBA to be overwhelming (particularly in classrooms where large amounts of data have to be collected on many children).

• Sometimes teachers lack the training or support to administer the CBA. (teachers may become frustrated and implement the assessment with low fidelity)

Grisham-Brown, J., & Pretti-Frontczak, K. (2011). Assessing Young Children in Inclusive Settings: The Blended Practices Approach. Baltimore: Paul H. Brookes Publishing Co.

• REMEDIES:• use the authentic assessment

fidelity measure by Grisham-Brown and colleagues (2008)

• develop a fidelity measure that relates to procedures of the assessment being used by your program

Grisham-Brown, J., & Pretti-Frontczak, K. (2011). Assessing Young Children in Inclusive Settings: The Blended Practices Approach. Baltimore: Paul H. Brookes Publishing Co.

Pitfall # 1: Most ECI assessments, do not include administration checklists that can be used for

integrity/fidelity checks.

http://www.ehhs.kent.edu/ceecrt/index.php/research/current

• REMEDIES:• utilize a coaching system in which

teachers check on another and provide support for those who are new to using the assessment.

• engage in ongoing professional development to ensure accuracy in scoring and use and avoid drift over time.

Grisham-Brown, J., & Pretti-Frontczak, K. (2011). Assessing Young Children in Inclusive Settings: The Blended Practices Approach. Baltimore: Paul H. Brookes Publishing Co.

Pitfall #2: Teams administer assessments without sufficient training or ongoing support

Grisham-Brown, J., & Pretti-Frontczak, K. (2011). Assessing Young Children in Inclusive Settings: The Blended Practices Approach. Baltimore: Paul H. Brookes Publishing Co.

“Regardless of how information is gathered, what information is gathered, or even how the information is summarized, if it isn’t used to plan and guide instruction, then the process is a waste of the teacher’s time and provides no advantage for young children.” (p, 170)

RESOURCES

http://www.kskits.org/ta/Packets/embedAssessment.shtml

KITS/TASN TA Packet:

Embedding Assessment into Daily

Activities and Routines

http://aepsblog.blogspot.com/

AEPS BLOG: Screencast Series

http://www.cde.state.co.us/resultsmatter/RMVideoSeries_PracticingObservation.htm#top

Colorado Department of Education: Results Matter

http://eclkc.ohs.acf.hhs.gov/hslc/tta-system/teaching

National Center on Quality Teaching and Learning NCQTL

http://eclkc.ohs.acf.hhs.gov/hslc/tta-system/teaching/center/practice/ISS

NCQTL

http://www.aepsinteractive.com/

AEPS/AEPSi Community of Practice?

How can we collaborate and share across the state?

Nancy Miller, M.Ed., OTR/L

[email protected]

Stephanie Parks, Ph.D., OTR/L

[email protected]