1 Montana Comprehensive Assessment System Montana Office of Public Instruction Linda McCulloch, Superintendent May 2006
Jan 17, 2016
1
Montana Comprehensive Assessment System
Montana Office of Public Instruction
Linda McCulloch, Superintendent
May 2006
2
Welcome
Panelists– Sharon Houle, Measured Progress– Carol Taylor-Cann, Riverside Publishing– Bob Runkel, OPI—GAP Students– Lynn Hinch, OPI—English Language Proficiency– Donna O’Neill, OPI--NAEP Coordinator– Linda Peterson, OPI—Science Standards
– Moderator, Judy Snow, OPI--Assessment
3
Agenda
• MontCAS Overview– Judy Snow
• MontCAS—Updates on Other Assessments – Panelists
• MontCAS Issues and Details– Judy Snow
4
MontCAS Overview
• Assessment 101
• MontCAS Phases 1 and 2
5
Assessment 101
• Standardized Test• Norm-referenced test• Criterion-referenced
test• Performance levels• Alternate Assessment• LEP, ELL, ELP
• NCLB– No Child Left Behind– Adequate Yearly
Progress (AYP)
• Administrative Rules of Montana– ARM– Board of Public
Education (BPE)
6
Phase 1 Norm-referenced test (NRT)
—the Iowas—Riverside Publishing Company
Aligned to national standards Grades 4, 8, 11 Reading, language arts,
math, social studies, science Multiple choice Alternate assessment scales Funded by the state
Phase 2 Criterion-referenced test
(CRT)—Measured Progress Aligned to Montana Content
Standards Grades 3-8, and 10
(science in spring 2008) Reading and math Multiple choice, math short
answer and constructed response
Evidence based alternate Funded by NCLB
7
Other AssessmentsPanelists
8
Bob RunkelModified CRT
9
GAP Students
• In between CRT-Alternate and CRT• Modified performance standards• Feasibility Study
– Grade 5 students – Identified as special education students for the grade
4 CRT in spring 2006– Pilot math CRT based on modified performance
standards
• Tentative Window– September 11-19, 2006
10
Lynn Hinch English Language Proficiency Test
11
ELP
• English Language Proficiency– Required by NCLB– Grades K-12– Test Development through Mountain West
Consortium (MWAC)– Date TBA
12
NO CHILD LEFT BEHIND REQUIREMENTS
• Both Titles I and III require an annual assessment of English language proficiency of all K - 12 limited English proficient students
• Assessment must be aligned to state English language proficiency standards
• Assessment must report a separate score for listening, speaking, reading, and writing
13
TITLE III REQUIREMENTS
• States must establish Annual Measurable Achievement Objectives (AMAOs) that determine how LEP students are making progress in learning English, and whether students are attaining proficiency in English.
14
Status of English Language Proficiency Assessment
• Mountain West Assessment Consortium
• Test Development and Delivery
• RFP Process
• March '06 Request to the Department of Education for timeline extension
15
Donna O’NeillNAEP
16
NAEP
• National Assessment of Educational Progress– NAEP--Nation’s Report Card– Odd-numbered Years– Grades 4 and 8, Reading and Math– Required Sample
17
All About NAEP
• What is NAEP?
• Why is it important to Montana?
• Who takes the NAEP Assessment?
• What are the requirements for NAEP?
• New developments in Science & Writing
• What is the timeline for the NAEP Assessment?
18
NAEPNational Assessment
• The Nation’s Report Card• Measures academic achievement• Reports to the public what students know and can do:
– average scale scores– proficiency levels
• Formats vary by grade:– multiple choice – constructed-response
• Links performance to educational variables– Teacher background information– School characteristics
• NAEP is confidential
19
Montana
20
The Nation’s Report Card
21
Montana
22
Who?
• Random sample– 2,000 students per
• Subject• Grade
• Grades 4, 8 & 12
• Students with disabilities
• Limited-English proficient
23
NAEP Requirements
• Linked to Title I funding – Schools– Districts
• NAEP is designed – Minimal disruption of instruction– 90 to 120 minutes of each student’s time– Teachers are encouraged to observe– Field staff are many Montana teachers
24
New Developments
• Science Release– May
• Writing (8, 12)
• Pilot and Special Studies– National Indian Education Study (NIES)
• Grades 4, 8• NAEP data will be linked to survey data• Student, Teacher and Principal Questionnaires
25
NAEP Timeline
26
Key Dates
• Secure cooperation of districts and schools– Notify districts in early May– Notify schools of selection during May– Notify schools of assessment dates in June– Maintain communication with schools
throughout the process
27
Handouts
• Packet of Information– Facts About Montana Education– Historical information on results– NAEP Questions Tool– NAEP Data Explorer (researchers)
• http://nces.ed.gov/nationsreportcard/
28
Sharon HouleMeasured Progress
29
CRT Science Test
• Grades 4, 8, and 10
• Spring 2008
• Aligned to Revised Science Standards
30
Criterion-Referenced Test Where are we going?
Science--grades 4, 8 and 10• May 31, 2006
– National science content and bias reviews
• July 25– Montana science content and bias reviews
• Spring 2007– Field test
• Spring 2008– First Administration—CRT and CRT-Alternate
31
Linda Peterson
ACCREDITATION DIVISION
STANDARDS REVISION PROCESS
32
MontanaMontana StandardsStandards
(Intended)(Intended)
MontanaMontana StandardsStandards
(Intended)(Intended)
CurriculumCurriculum(Intended)(Intended)
CurriculumCurriculum(Intended)(Intended)
AssessmentAssessment(Assessed)(Assessed)(Learned)(Learned)
AssessmentAssessment(Assessed)(Assessed)(Learned)(Learned)
ProfessionalProfessionalDevelopmentDevelopmentProfessionalProfessionalDevelopmentDevelopment
InstructionInstruction(Enacted)(Enacted)
InstructionInstruction(Enacted)(Enacted)
5YCEP Continuous Improvement Process
5YCEP Continuous Improvement Process
5YC
EP
Co
nti
nu
ou
s Im
pro
vem
ent
Pro
cess
5YC
EP
Co
ntin
uo
us Im
pro
vemen
t Pro
cessM
on
tan
a S
urv
eys
of
the
En
acte
d C
urr
icu
lum
(M
SE
C)
– A
lig
nm
ent
Ass
essm
ent
Pro
cess
The Five-Year Comprehensive Education Plan (5YCEP) Building a Continuous Improvement Process for Montana Schools
33
Montana Board of Public Education
CHAPTER 54
CONTENT AND PERFORMANCE STANDARDS
10.54.2503 STANDARDS REVIEW SCHEDULE (1) Montana's content and performance standards shall be reviewed and revised on a five-year cycle beginning July 1, 2005.
Standards
CurriculumInstruction
Assessment
34
Montana Board of Public Education
Standards Revision 2005-2010
Purpose• Assure Montana citizens that its public schools are
providing all children of our great state with challenging academic expectations
• Revised standards clearly and consistently identify what students should know, understand and be able to do
• Revised standards provide a framework to help
guide local curriculum and instruction
35
Montana Board of Public EducationCriteria to Guide Standards Revision
Standards will be:
• academic in nature and content specific.
• challenging and rigorous.
• clear, understandable and free of jargon.
• measurable.
• address diversity, specifically fulfilling the commitment to implementing 20-1-501, Indian Education for All.
36
Montana Standards Review Schedule
Cycle I Science July 2005 – September 2006
Cycle II MathematicsCommunication Arts
TBD
Cycle III Social StudiesHealth EnhancementArtsTechnology
TBD
Cycle IV School CounselingLibrary MediaCareer and Technology EducationWorkplace CompetenciesWorld Languages
TBD
37
K-12 Science Standards
Montana Standards Framework
• Content Standards
• Benchmarks - Grades 4 and 8 and upon graduation
• Performance Descriptors
38
K-12 Science Standards
Performance Descriptors
• Advanced – superior performance• Proficient – solid academic competency• Nearing Proficiency – partial mastery of
prerequisite knowledge and skills• Novice – beginning to attain prerequisite
knowledge and skills
39
K-12 Science Standards
Content Standard 1
Students design, conduct, evaluate, and communicate processes and results of scientific investigations, and demonstrate thinking skills associated with this procedural knowledge
Benchmarks• Grade 4 (3) - describe and
communicate the results of scientific investigations
• Grade 8 (3) - review, communicate and defend results of investigations
• Upon Graduation – review evidence, communicate and defend results, and recognize that the results of a scientific investigation are always open to revision by further investigation
40
Performance DescriptorsContent Standard 1
Proficient• Grade 4 – with direction, completes a simple investigation with
identified variables, using appropriate tools and communicates results
• Grade 8 – identifies and communicates testable questions, plans and conducts experimental investigations and communicates results
• Upon Graduation – generates testable questions, constructs a plan for a controlled investigation, makes logical inferences based on observations, accurately interprets data by identifying the strengths and weaknesses in an investigation design, and communicates results.
41
K-12 Science Standards
Content Standard 3
Students demonstrate knowledge of characteristics, structures, and function of living things, the process and diversity of life, and how lining organisms interact with each other and their environment, and demonstrate thinking skills associated with this knowledge.
Benchmarks• Grade 4 – create and use a
classification system to group a variety of plants and animals according to their similarities and differences, (e.g., American Indian medicinal plants, American Indian dwellings)
• Grade 8 – create and use a basic classification scheme to identify plants and animals, (e.g., classification scheme based on American Indian use of indigenous plants for medicinal, food and other uses)
• Upon Graduation – recognize, generate and apply biological classification schemes to infer and discuss the degree of divergence using ecosystems, (e.g., American Indian food collections systems)
42
Performance DescriptorsContent Standard 3
Proficient• Grade 4 – identifies attributes of biotic (living) things and abiotic
(non-living) objects, including classification based on similarities and differences, basic structure and function, processes of each system
• Grade 8 – identifies and classifies biotic things and abiotic objects through application of common classification schemes; identifies the interdependence of life and the environment and explains how characteristics of living things change because of the environment
• Upon Graduation – organizes, classifies, and describes interactions of the biotic and abiotic parts of the biosphere as well as the natural history of interactions of life on Earth and uses these skills to solve related novel (to the student) problems
43
K-12 Science Standards Proposed Time Line
• March – May Gather Public Comments
• May 11 Present Update to BPE
• May 15 Presentation and Discussion with MACIE
• May – June Writing Team Work Sessions
• June Prepare draft standards for notice of hearing
44
K-12 Science Standards Proposed Time Line
• July Present to BPE draft Notice of Hearing and proposed time line
• August Notice of Hearing posted
• October Public Hearing
• November Final BPE Action Anticipated
• 2007 Implementation, Outreach, and Dissemination
45
K-12 Science Standards Revisions
Linda Vrooman Peterson
Accreditation Division Administrator
Office of Public Instruction
(406) 444-5726
Draft K-12 Science Standards link:http://www.opi.mt.gov/pdf/standards/ScienceStandardsMarch06.pdf
46
Assessment Issues and DetailsStandardized
Administration
Accurate Data
Technical Adequacy
Impact on Instruction
47
Standard Administration = Accurate + Reliable Data
•Participation •Training•Test Security•Special Populations
•Accommodations•Alternate Assessments
48
Participation All Means All!
• Phase 1 Options– No
accommodations– If in IEP, 504, or
LEP plan• Standard or non
standard accommodations
• Alternate Assessment Scale
• Combination
• Phase 2 Options– No
accommodations– Standard
accommodations– If in IEP or 504
plan• Non standard
accommodations• CRT-Alternate
49
Training so that . . .
• everyone understands the procedures for administration.
• the test is administered in a comparable way in all locations across the state.
• quality control procedures are utilized when returning test materials.
50
Test Security
All test items and responses to those items in MontCAS are secure materials and may not be copied or duplicated in any way; CRT materials may not be retained in the school after testing is completed.
It is very important to review test security information with test administrators during test administration training.
51
OPI Test Security • Any concern about breaches in test security or
noncompliance with test administration procedures must be reported immediately to the principal and system test coordinator and to the State Assessment Director.
• OPI Guidelines and Procedures for Test Security– This OPI publication outlines procedures for reporting
testing irregularities and should be made available to system superintendents, principals, and test administrators.
52
Standard accommodations do not change what we expect students to know and be able to do.
Test accommodations are based on individual student needs and classroom assessment practice; they are not for the use by an entire classroom or category of students.
They should not give students unfair advantages; rather they are meant to remove barriers that may exist due to a student’s learning style or disability.
Accommodations
53
Standard and Non StandardAccommodationsStandard • Do not change the
intent/content of the assessment
• Phase 1: Only by IEP, 504, or LEP plan
• Phase 2: By IEP, 504, LEP, or regular routine in the classroom
Non Standard• Change the
content/intent of the assessment
• Phase 1: Only by IEP,504, or LEP plan
• Phase 2: Only by IEP or 504 plan
54
Accurate Data• the information collected is of high quality
demographic informationenrollment informationdisability informationdecisions related to the use of
accommodations and the alternate assessment (CRT-Alternate)
55
2006 Data Check Points
OPI CRT Testing Cycle Enrollment Count• March 6, 2006 Window Opens• April 13, 2006 Data due to County Superintendent• April 20, 2006 Data due to OPI
Measured Progress CRT Online Demographic Data Verification• April 24 – May 19, 2006
OPI Comparability Study—June 2006
OPI Iowa Tests (NRT) Disaggregated Data Review• July 24 - August 4 , 2006
56
Technical Quality and Expertise
• Test Development
• Technical Advisory Committee (TAC)
• Technical Manuals
• CRT and CRT-Alternate Standard Setting
57
Montana Performance Levels
• Advanced• Proficient
• Nearing Proficiency• Novice
58
Standard Setting
• Standard setting is the process of determining cut points for performance levels for the CRT and the CRT-Alternate.
• Panels of Montana educators
• Facilitated by Measured Progress
Standard Setting
59
2006 Standard Setting Details
• For more information: http://www.opi.mt.gov/assessment/Phase2.html#TD
• To apply: – http://iservices.measuredprogress.org
• Travel, meals (those not provided during the sessions), and lodging reimbursements
• Renewal units and honoraria
60
2006 Standard SettingWhere and When
Red Lion Colonial Inn in Helena– June 21-22
• CRT, Reading and Math, grades 4, 8, and 10
– June 20-22• CRT-Alternate, Reading and Math, grades 3-8 and
10
– July 26-27• CRT, Reading and Math, grades, 3, 5, 6, and 7
61
Impact on Instruction
• Reporting
• Data Interpretation
• Tools
62
Reporting and Impact on Instruction
• Phase 1—NRT– NPR– NCE– Stanine– Performance Level
• Phase 2—CRT– Scaled Score
• Performance Level for each content area– Raw Score
• Number of points for each standard
63
Iowa Tests Interpretive Guide
2005
http://www.opi.mt.gov/PDF/Assessment/NRT/05InterpGuide.pdf
• Relationship of stanines, percentile ranks, and normal curve equivalents (p. 5)
• Comparison of NRT Scores (p. 6)
• Performance Profile (p. 12)
• Individual Customized Skills Report (p. 16)
64
CRT Results Interpretation
• 2005 Interpretive Guidehttp://www.opi.mt.gov/PDF/Assessment/CRT/05CRTCRTaltInterpGuide.pdf
– Roster and Item-Level Report (p. 6)
• Released Items– Items that scored are released.– http://www.opi.mt.gov/Assessment/index.html
• iAnalyze– A secure website where data gathered from the CRT
assessments are reported
65
Judy SnowState Assessment DirectorOffice of Public [email protected]
Assessment link:http://www.opi.mt.gov/Assessment/index.html JUMP link:http://www.opi.mt.gov/Assessment/JUMP.html