Colorado Reading First - CDE...Colorado Reading First, 2004 Outcome Assessment What is the administration window for the BEAR? Districts will be given a month (appx. April 15-May 15)

ColoradoColorado Reading Reading First, 2004First, 2004

Colorado Reading FirstColorado Reading FirstCohort 2Cohort 2

Assessment OverviewAssessment OverviewCRFCRF Outcome, Diagnostic, and Outcome, Diagnostic, and

Informal AssessmentsInformal AssessmentsJames EckJames Eck

Assessment CoordinatorAssessment Coordinator


Purposes of Assessment Purposes of Assessment Instructional PlanningInstructional Planning

Screening assessmentsScreening assessmentsDiagnostic assessmentsDiagnostic assessmentsClassroomClassroom--based assessments (progress based assessments (progress monitoring)monitoring)

Program EvaluationProgram EvaluationOutcome assessmentsOutcome assessments

(Assessment Committee, 2002)(Assessment Committee, 2002)


Outcome Assessment Outcome Assessment

Third Grade CSAPThird Grade CSAPCSAP is the outcome assessment for third CSAP is the outcome assessment for third grade, the culminating year for grade, the culminating year for Reading Reading FirstFirst..The third grade CSAP assesses Colorado The third grade CSAP assesses Colorado Standard #1: Reading Comprehension.Standard #1: Reading Comprehension.Refer to the Assessment Framework for Refer to the Assessment Framework for performance levels, proficiencies.performance levels, proficiencies.



BEAR Summative AssessmentBEAR Summative Assessment

The Basic Early Assessment of Reading The Basic Early Assessment of Reading (BEAR) Summative Assessment is the (BEAR) Summative Assessment is the required outcome assessment in grades required outcome assessment in grades KK--3 for 3 for Colorado Reading First Colorado Reading First Cohort 2 Cohort 2 schools.schools.


Outcome Assessment Outcome Assessment BEAR Summative AssessmentBEAR Summative Assessment

BEAR has been determined to meetBEAR has been determined to meet the criteriathe criteria as as an outcome assessment identified by the Reading an outcome assessment identified by the Reading Assessment Committee (2002):Assessment Committee (2002):““Outcome measures must provide methods to Outcome measures must provide methods to document significant gain or improvement, as well document significant gain or improvement, as well as methods to document gradeas methods to document grade--level performance. level performance. The measures must also meet a very high The measures must also meet a very high standard for reliability and validity, because they standard for reliability and validity, because they may form the basis for relatively may form the basis for relatively ““high stakeshigh stakes””decisions about children, classrooms, and decisions about children, classrooms, and schools.schools.””


Outcome Assessment Outcome Assessment BEAR BEAR –– CSAP AlignmentCSAP AlignmentIn a preliminary study, BEAR was found to be In a preliminary study, BEAR was found to be

moderately correlated to the Third Grade moderately correlated to the Third Grade CSAP:CSAP:The correlation between BEAR Reading Basics The correlation between BEAR Reading Basics and the 2003 Third Grade CSAP is .74 (n=633)and the 2003 Third Grade CSAP is .74 (n=633)The correlation between BEAR Reading The correlation between BEAR Reading Comprehension and the 2003 Third Grade Comprehension and the 2003 Third Grade CSAP is .75 (n=622)CSAP is .75 (n=622)



BEAR BEAR –– CSAP AlignmentCSAP AlignmentComparison of Achievement by Performance CategoryComparison of Achievement by Performance CategorySpring 2003Spring 2003CSAP: ReadingCSAP: Reading BEAR: Reading BasicsBEAR: Reading Basics

Proficient and AboveProficient and Above 70%70% DevelopedDeveloped 71%71%Partially Proficient Partially Proficient 20%20% DevelopingDeveloping 18%18%

90%90% 89%89%Unsatisfactory Unsatisfactory 10%10% Not DevelopedNot Developed 11%11%

__________________________________________________________________________________________________________________________________

Spring 2003Spring 2003CSAP: ReadingCSAP: Reading BEAR: ComprehensionBEAR: Comprehension

Proficient and AboveProficient and Above 70%70% DevelopedDeveloped 36%36%Partially Proficient Partially Proficient 20%20% DevelopingDeveloping 56%56%

90%90% 92%92%Unsatisfactory Unsatisfactory 10%10% Not DevelopedNot Developed 8%8%

Updated 1/14/2003Updated 1/14/2003



BEAR is a group administered, locally scored BEAR is a group administered, locally scored assessment, designed to provide a assessment, designed to provide a comprehensive measure of student achievement comprehensive measure of student achievement in reading basics and reading comprehension.in reading basics and reading comprehension.BEAR is intended to be administered at the end BEAR is intended to be administered at the end of the school year to determine studentsof the school year to determine students’’progress in reading and to inform placement progress in reading and to inform placement and instruction for the next school year.and instruction for the next school year.



The BEAR Summative Assessment consists of The BEAR Summative Assessment consists of three subtests: Reading Basics, Comprehension three subtests: Reading Basics, Comprehension (Listening and Reading), and Language Arts.(Listening and Reading), and Language Arts.

For For Reading FirstReading First, Reading Basics and , Reading Basics and Comprehension subtests are REQUIRED.Comprehension subtests are REQUIRED.



All scoring is completed locally, with the All scoring is completed locally, with the assistance of Windowsassistance of Windows®®--based scoring and based scoring and reporting software. Scores offered include: raw reporting software. Scores offered include: raw score, percent correct, and instructional levels.score, percent correct, and instructional levels.Instructional levels are reported as: Developed, Instructional levels are reported as: Developed, Developing, and Not Developed.Developing, and Not Developed.



How long does it take to administer the How long does it take to administer the BEAR?BEAR?BEAR is intended to be an BEAR is intended to be an untimeduntimed test, but test, but takes approximately thirty minutes per subtest. takes approximately thirty minutes per subtest. An additional ten minutes per subtest should be An additional ten minutes per subtest should be scheduled for testing logistics.scheduled for testing logistics.The two required subtests should be The two required subtests should be administered in SEPARATE testing sessions.administered in SEPARATE testing sessions.


Outcome Assessment Outcome Assessment What is the administration window for What is the administration window for

the BEAR?the BEAR?Districts will be given a month (Districts will be given a month (appxappx. April 15. April 15--May May 15) within which to administer and score the 15) within which to administer and score the BEAR Summative Assessment.BEAR Summative Assessment.Within that window, schools will be requested to Within that window, schools will be requested to designate a ONE WEEK period for test designate a ONE WEEK period for test administration.administration.Schools requiring an earlier test administration Schools requiring an earlier test administration schedule will be required to submit a request for schedule will be required to submit a request for waiver to CDE.waiver to CDE.



The BEAR is designed to be scored locally, by The BEAR is designed to be scored locally, by hand. Scoring should be done by the classroom hand. Scoring should be done by the classroom teacher.teacher.

Hand scoring of the BEAR is facilitated by the Hand scoring of the BEAR is facilitated by the administration and scoring guides for each administration and scoring guides for each subtest. subtest.



Administration and Scoring GuidesAdministration and Scoring GuidesThe guides provide answer keys, scoring The guides provide answer keys, scoring rubrics, sample papers, and reproducible forms. rubrics, sample papers, and reproducible forms. The guides also provide instructions for The guides also provide instructions for interpreting the test scores. interpreting the test scores. The Summative Assessment includes one The Summative Assessment includes one Administration Guide and one Scoring Guide for Administration Guide and one Scoring Guide for each level.each level.



The BEAR Scoring and Reporting Software The BEAR Scoring and Reporting Software manages student test data, allows entry and manages student test data, allows entry and reporting of teacher comments, and generates reporting of teacher comments, and generates reports for parents, teachers, and reports for parents, teachers, and administrators. administrators. The software allows a teacher or administrator The software allows a teacher or administrator the option of entering student item responses the option of entering student item responses directly into the computer for automated directly into the computer for automated scoring. Users can also enter previously scoring. Users can also enter previously computed raw scores.computed raw scores.



Training for BEAR administration and scoringTraining for BEAR administration and scoring::A training session for District Assessment A training session for District Assessment Coordinators/Database Coordinators will be Coordinators/Database Coordinators will be scheduled for early spring 2006.scheduled for early spring 2006.The training will cover test administration, The training will cover test administration, scoring procedures, and data entry and scoring procedures, and data entry and reporting.reporting.



What are the reporting requirements?What are the reporting requirements?Results from the two required subtests of the Results from the two required subtests of the BEAR will be required for the student, grade, BEAR will be required for the student, grade, and school levels.and school levels.Results will be reported electronically.Results will be reported electronically.The BEAR Scoring and Reporting Software will The BEAR Scoring and Reporting Software will be used by the school to enter and compile be used by the school to enter and compile data, and to generate reports.data, and to generate reports.


BEAR Administration Rules BEAR Administration Rules (4/16/2004)(4/16/2004)

Refer to the Refer to the Administration Rules for Administration Rules for the BEAR the BEAR Reading FirstReading First Outcome Outcome Assessment: Students Tested and Not Assessment: Students Tested and Not Tested, Student Results Reported and Tested, Student Results Reported and Not ReportedNot Reported (4/16/2004)(4/16/2004)



1.1. All StudentsAll StudentsAll students who are able to take the All students who are able to take the assessment should take the assessment. All assessment should take the assessment. All students enrolled in the school should be students enrolled in the school should be accounted for. Even those students who accounted for. Even those students who have been determined not to be tested have been determined not to be tested should be entered into the scoring and should be entered into the scoring and reporting software, then declared reporting software, then declared ““Not Not TestedTested”” using the User Defined Field in the using the User Defined Field in the BEAR Scoring and Reporting Software.BEAR Scoring and Reporting Software.



2.2. Time Enrolled in SchoolTime Enrolled in SchoolAll students who are deemed eligible to take All students who are deemed eligible to take the BEAR should be tested, regardless of the BEAR should be tested, regardless of time in school.time in school.BEAR assessment results for students who BEAR assessment results for students who have been enrolled in the school for have been enrolled in the school for 6 6 months or longermonths or longer and are determined to be and are determined to be eligible for testing will be included in federal eligible for testing will be included in federal reporting and will be considered in decisions reporting and will be considered in decisions for continued funding from for continued funding from Reading FirstReading First..


BEAR Reporting BEAR Reporting (3/2/2004)(3/2/2004)

22Q. 22Q. What students are counted for the purpose of What students are counted for the purpose of making AYP determinations?making AYP determinations?

A.A. For schools: Students continuously enrolled in For schools: Students continuously enrolled in the school for a full academic year for whom the school for a full academic year for whom there is a CSAP, CSAPthere is a CSAP, CSAP--A, or A, or LecturaLectura assessment assessment available at that grade. available at that grade.

Full academic year is defined as "from one CSAP Full academic year is defined as "from one CSAP administration to the current CSAP administration to the current CSAP administration. For districts, a full academic administration. For districts, a full academic year is defined as "continuously enrolled in the year is defined as "continuously enrolled in the district from one CSAP administration to the district from one CSAP administration to the current administration.current administration.““



2.2. Time Enrolled in SchoolTime Enrolled in SchoolTime in school is defined as continuous Time in school is defined as continuous enrollment (uninterrupted by more than enrollment (uninterrupted by more than three weeks, not considering school three weeks, not considering school vacations), from one administration of the vacations), from one administration of the CRFCRF Outcome assessment to the next. Based Outcome assessment to the next. Based upon that criterion, enrollment is determined upon that criterion, enrollment is determined from May 1 of the previous school year. from May 1 of the previous school year. Note: May 1 is considered the opening date Note: May 1 is considered the opening date of the window for BEAR testing.of the window for BEAR testing.



2.2. Time Enrolled in SchoolTime Enrolled in SchoolBEAR assessment results for Kindergarten BEAR assessment results for Kindergarten students who have been enrolled in the students who have been enrolled in the school prior to October 1 (6 months to 12 school prior to October 1 (6 months to 12 months) must be considered in funding months) must be considered in funding decisions and for federal reporting, since no decisions and for federal reporting, since no students will have been enrolled for 12 students will have been enrolled for 12 months or greater.months or greater.


BEAR Administration Rules BEAR Administration Rules (4/16/2004)(4/16/2004)2.2. Time Enrolled in SchoolTime Enrolled in School

Student groups will be reported by time in Student groups will be reported by time in school, as follows:school, as follows:

0 to 6 months0 to 6 months6 months to 12 months6 months to 12 months12 months or greater12 months or greater

Based upon the above criteria and for coding Based upon the above criteria and for coding purposes only, NO Kindergarten students purposes only, NO Kindergarten students should be identified as should be identified as ““in school 12 months in school 12 months or greater.or greater.””For For Reading FirstReading First, there is not a reporting , there is not a reporting criterion for time in district.criterion for time in district.


BEAR Administration BEAR Administration (3/2/2004)(3/2/2004)Subgroups: Students with DisabilitiesSubgroups: Students with Disabilities

There is obviously not a There is obviously not a ““CSAPACSAPA”” version of the version of the BEAR,BEAR, therefore students with disabilities who therefore students with disabilities who are deemed unable to take the BEAR will be are deemed unable to take the BEAR will be exempted from the exempted from the CRFCRF Outcome assessment.Outcome assessment.Students who are exempted from the BEAR due to Students who are exempted from the BEAR due to disability will not be counted in the denominator disability will not be counted in the denominator of calculations, however it is expected that of calculations, however it is expected that schools will strive toward the 95% participation schools will strive toward the 95% participation rate of AYP.rate of AYP.School teams should consider whether School teams should consider whether accommodations are appropriate for students accommodations are appropriate for students with disabilities, on a casewith disabilities, on a case--byby--case basis. case basis.


BEAR Administration Rules BEAR Administration Rules (4/16/2004)(4/16/2004)3.3. Special Education and 504Special Education and 504

Special Education Students Not TestedSpecial Education Students Not Tested::There is obviously not a There is obviously not a ““CSAPACSAPA”” version of version of the BEAR; therefore, students with severe the BEAR; therefore, students with severe cognitive disabilities who are deemed unable cognitive disabilities who are deemed unable to take the BEAR by their IEP team will be to take the BEAR by their IEP team will be deferred from the taking deferred from the taking CRFCRF Outcome Outcome Assessment.Assessment.Students who are deferred from the BEAR Students who are deferred from the BEAR due to disability will not be counted in the due to disability will not be counted in the denominator of calculations, however it is denominator of calculations, however it is expected that schools will strive toward the expected that schools will strive toward the 95% participation rate for AYP. (Part of the 95% participation rate for AYP. (Part of the funding decision for Year 3 will be based funding decision for Year 3 will be based upon increases in the participation rate upon increases in the participation rate toward the 95% target).toward the 95% target).


BEAR Administration Rules BEAR Administration Rules (4/16/2004)(4/16/2004)SpEd Decision Flowchart (Jeffco)

Take BearSPED?Start No

Able to test?

Yes

Yes

Don’t Take Bear

No



Accommodations:Accommodations:The following categories of The following categories of accommodations ARE acceptable to accommodations ARE acceptable to use for BEAR:use for BEAR:PresentationPresentationResponseResponseSetting/GroupingSetting/GroupingSee the Administration Rules for specifically See the Administration Rules for specifically acceptable accommodations.acceptable accommodations.


BEAR Administration Rules BEAR Administration Rules (4/16/2004)(4/16/2004)Accommodations:Accommodations:TimingTiming

Because BEAR is unBecause BEAR is un--timed, flexibility in time timed, flexibility in time should be built in, and should not be considered should be built in, and should not be considered an accommodation.an accommodation.Any student who may need significantly more Any student who may need significantly more time to complete a test session should be time to complete a test session should be identified ahead of time whenever possible and identified ahead of time whenever possible and preferably the assessment should be preferably the assessment should be administered in a separate setting.administered in a separate setting.The BEAR Reading Basics and Comprehension The BEAR Reading Basics and Comprehension tests should be administered on separate days, tests should be administered on separate days, no more than twono more than two--three days apart for any three days apart for any student/group of students (except in cases of student/group of students (except in cases of extended absence or other extenuating extended absence or other extenuating circumstances).circumstances).


BEAR Administration BEAR Administration (3/2/2004)(3/2/2004)Subgroups:Subgroups:

Limited English ProficientLimited English Proficient“…“… the state is advocating that, when possible, the state is advocating that, when possible, limited English proficient students actively limited English proficient students actively participate in the state assessment program. The participate in the state assessment program. The state realizes that students who are still in the state realizes that students who are still in the process of acquiring English may not be able to process of acquiring English may not be able to adequately access the information contained in the adequately access the information contained in the English version of the assessments; however, English version of the assessments; however, currently, the state can provide this test only in currently, the state can provide this test only in Spanish at grades 3 and 4.Spanish at grades 3 and 4.””

20032003--04 CSAP Procedures Manual, p. 5804 CSAP Procedures Manual, p. 58


BEAR Administration BEAR Administration (3/2/2004)(3/2/2004)Subgroups:LimitedSubgroups:Limited English ProficientEnglish Proficient

Points to RememberPoints to RememberThe goal of The goal of Reading FirstReading First is English language is English language

literacy.literacy.The BEAR assessment does not have a The BEAR assessment does not have a Spanish language version.Spanish language version.The The CRFCRF team does not feel that an alternate team does not feel that an alternate Spanish assessment can/should be Spanish assessment can/should be mandated, therefore NEP, LEP students mandated, therefore NEP, LEP students deemed unable to take the BEAR will be deemed unable to take the BEAR will be exempted from the exempted from the CRFCRF Outcome assessment Outcome assessment for a limited time.for a limited time.



The participation/exemption and reporting The participation/exemption and reporting guidelines for ELL/Bilingual students has guidelines for ELL/Bilingual students has been more fully detailed in a document titled been more fully detailed in a document titled ““Guidelines for Participation of English Guidelines for Participation of English Language Learners in BEAR Outcome Language Learners in BEAR Outcome AssessmentAssessment”” (3/10/2004), and the (3/10/2004), and the accompanying accompanying ““BEAR ELL ChartBEAR ELL Chart”” (3/1/2004). (3/1/2004). Both may be found on the CDE Both may be found on the CDE Colorado Colorado Reading FirstReading First web site web site ((http://http://www.cde.state.co.us/cdecomp/rf_newww.cde.state.co.us/cdecomp/rf_news.htmws.htm).).



Take BearELL?Start No

Level 1 or 2 (A/B)?

Yes

No

Don’t Take Bear

Yes

ELL Decision Flowchart (Jeffco)


BEAR Reporting BEAR Reporting (3/2/2004)(3/2/2004)

The subgroups required by NCLB consist of:The subgroups required by NCLB consist of:Racial/Ethnic: Whites, Blacks, Hispanics, Racial/Ethnic: Whites, Blacks, Hispanics, Native American, AsianNative American, AsianEconomically Disadvantaged (students on free Economically Disadvantaged (students on free or reduced lunch)or reduced lunch)Students with Disabilities (students with Students with Disabilities (students with IEPsIEPs))Limited English Proficient (students classified Limited English Proficient (students classified as LEP or FEP who have not yet been as LEP or FEP who have not yet been reclassified)reclassified)

From the ABCFrom the ABC’’s of AYP (s of AYP (CDE, PBC, Draft, 3/31/03)CDE, PBC, Draft, 3/31/03)


BEAR Administration Rules BEAR Administration Rules (4/16/2004)(4/16/2004)5.5. Disaggregated GroupsDisaggregated Groups

The subgroups required for The subgroups required for Reading First Reading First reportingreporting consist of:consist of:Racial/Ethnic: White, African American, Hispanic, Racial/Ethnic: White, African American, Hispanic, Asian, and Native American Asian, and Native American Economically Disadvantaged (students on free or Economically Disadvantaged (students on free or reduced lunch)reduced lunch)Students with Disabilities Students with Disabilities Students with limited English proficiencyStudents with limited English proficiency

(From the (From the USDEUSDE’’ss Reading First Annual Reading First Annual Performance ReportPerformance Report ))Note: All required subgroup Note: All required subgroup disaggregationdisaggregation will will be automatically supported by the BEAR Scoring be automatically supported by the BEAR Scoring and Reporting software, and reporting will be and Reporting software, and reporting will be supported via the Reporting Spreadsheet supported via the Reporting Spreadsheet template.template.


Diagnostic AssessmentDiagnostic Assessment

Issue : How should the results of Issue : How should the results of early reading screenings be early reading screenings be interpreted and used?interpreted and used?In the past, children who scored below In the past, children who scored below the cutoffs on screening measures were the cutoffs on screening measures were usually referred immediately for further usually referred immediately for further diagnostic assessment to determine if diagnostic assessment to determine if they qualified for special education they qualified for special education programs.programs.

RathvonRathvon (2004)(2004)


Diagnostic AssessmentDiagnostic AssessmentIssue: How should the results of Issue: How should the results of

early reading screenings be early reading screenings be interpreted and used?interpreted and used?Now, however, there is an emerging Now, however, there is an emerging consensus that diagnostic assessments consensus that diagnostic assessments should be delayed until a period of should be delayed until a period of intervention has been provided. That is, intervention has been provided. That is, diagnosis should be based on the failure to diagnosis should be based on the failure to respond to researchrespond to research--validated treatments validated treatments rather than on an arbitrarily specified rather than on an arbitrarily specified discrepancy between cognitive ability and discrepancy between cognitive ability and reading achievement.reading achievement.




During the intervention period, which During the intervention period, which could extend across the early primary could extend across the early primary grades, childrengrades, children’’s responsiveness to s responsiveness to treatment would be frequently treatment would be frequently monitored with measures sensitive to monitored with measures sensitive to change, such as curriculumchange, such as curriculum--based oral based oral reading fluency assessments.reading fluency assessments.



TorgesonTorgeson (2002b) reported that all of (2002b) reported that all of the intervention programs resulted in a the intervention programs resulted in a significant reduction in the number of significant reduction in the number of children who would have been eligible children who would have been eligible for special services.for special services.



Instructional AssessmentsInstructional AssessmentsRecommendations and GuidelinesRecommendations and Guidelines

How much is too muchHow much is too much??More testing is not necessarily better testing.More testing is not necessarily better testing.Information from a variety of sources is Information from a variety of sources is important (body of evidence, triangulation) important (body of evidence, triangulation) for verificationfor verificationValidity, reliability, and rigor should be Validity, reliability, and rigor should be consideredconsidered


Instructional AssessmentsInstructional Assessments

Recommendations and GuidelinesRecommendations and Guidelines

Using informal assessmentsUsing informal assessmentsInformal assessments are not considered valid Informal assessments are not considered valid and reliable diagnostic SBRR assessmentsand reliable diagnostic SBRR assessmentsDRA is currently not aligned with SBRR.DRA is currently not aligned with SBRR.Use of an IRI might be helpful as part of a body Use of an IRI might be helpful as part of a body of evidence, but may not be an efficient use of of evidence, but may not be an efficient use of time within time within CRFCRF schools.schools.




Progress MonitoringProgress MonitoringCore reading assessments do not have the Core reading assessments do not have the specificity or rigor required of SBRR specificity or rigor required of SBRR assessments. They can provide meaningful assessments. They can provide meaningful information for progress monitoring if used in information for progress monitoring if used in conjunction with an SBRRconjunction with an SBRR--aligned assessment.aligned assessment.




Progress MonitoringProgress MonitoringRunning records are not SBRRRunning records are not SBRR--aligned aligned assessments for exclusive use in progress assessments for exclusive use in progress monitoring. If still using them, be clear about monitoring. If still using them, be clear about why and how.why and how.




Progress MonitoringProgress MonitoringAssessments without multiple versions are too Assessments without multiple versions are too limited for progress monitoring.limited for progress monitoring.


Reading FirstReading First AssessmentsAssessmentsIf you have questions or concerns about your If you have questions or concerns about your Reading FirstReading First Assessments, you may contact me Assessments, you may contact me

at: at: [email protected][email protected]

or or (303)866(303)866--69836983

Jim EckJim EckAssessment CoordinatorAssessment CoordinatorColorado Reading FirstColorado Reading First

Colorado Reading First - CDE...Colorado Reading First, 2004 Outcome Assessment What is the administration window for the BEAR? Districts will be given a month (appx. April 15-May 15)

Documents