An Overview of the Race-to- the-Top Assessment Consortia Joe Willhoft, Executive Director SMARTER Balanced Assessment Consortium NATD Annual Meeting April 8, 2011 New Orleans
Dec 30, 2015
An Overview of the Race-to-the-Top Assessment Consortia
Joe Willhoft, Executive DirectorSMARTER Balanced Assessment Consortium
NATD Annual Meeting April 8, 2011New Orleans
22Apr. 8, 2011
• Council of Chief State School Officers (CCSSO), National Governors Association (NGA), and Achieve, Inc. agree to partner on a common standards project
• Summer 2009: College and career readiness standards developed in English/Language Arts and Mathematics
• K-12 learning progressions developed leading to college and career readiness in high school
• Multiple rounds of feedback from states, teachers, researchers, higher education, and the general public
• June 2, 2010: Final Common Core State Standards (CCSS) released
33Apr. 8, 2011
Aligned with college and work expectationsFocused and coherentInclude rigorous content and application of
knowledge through high-order skillsBuild upon strengths and lessons of current state
standardsInternationally benchmarked so that all students are
prepared to succeed in our global economy and society
Based on evidence and researchState led – coordinated by NGA Center and CCSSO
44Apr. 8, 2011
• The US Department of Education’s Race-to-the-Top competition awarded extra points to a state’s application if the state provided evidence of adopting, or moving toward adoption of “a” common core of standards
• When adopting the standards, states may not remove standards, but may add to them as long as the added state-specific standards comprise no more than 15% of the total
55Apr. 8, 2011
• As of April 1, 2011, 43 states including the District of Columbia have adopted the Common Core State Standards
• The eight states that have not yet adopted the standards: Alaska, Minnesota, Montana, Nebraska, North
Dakota, Texas, Virginia, Washington
66Apr. 8, 2011
For more information about the CCSS, go to...
www.corestandards.org
77Apr. 8, 2011
The MOSAIC Formative Assessment Consortium• In addition to the points awarded in RTTT
applications for adoption of the Common Core, states get points if they are part of an assessment consortium for formative (not summative) assessments.
• In Oct/Nov 2009, Wisconsin, Nebraska, Missouri, Iowa, and Kansas establish a formative assessment consortium, called MOSAIC.
• About 20-25 states sign MOUs to join the MOSAIC consortium
88Apr. 8, 2011
The SMARTER Summative Assessment Consortium• In fall of 2009 the US Dept of Ed signals it will divert
$360M from RTTT for summative assessments aligned to a common core, and begins planning meetings
• December 2009: Oregon, Idaho, Washington, Utah, Hawaii confer and agree to collaborate on an online, adaptive summative assessment that would include performance items
• The “Summative Multi-state Assessment Resources for Teachers and Educational Researchers” (SMARTER); Web site is launched; weekly planning calls begin
99Apr. 8, 2011
SMARTER and MOSAIC Merge• About 20 states are engaged in both SMARTER
and MOSAIC
• January 2010: MOSAIC states want their formative and interim tools to align to a common summative scale –
• MOSAIC elects to merge with SMARTER and adopt the SMARTER name; MOUs are merged
1010Apr. 8, 2011
SMARTER merges with Balanced Assessment System
• About 20 states (leadership from New England Common Assessment Program and West Virginia) work with Linda Darling-Hammond on a consortium that would include a balance of stand-alone tests and extensive performance events and projects
• March 2010: The Balanced Assessment Consortium and SMARTER join to work on a common proposal as the “SMARTER Balanced Assessment Consortium”
1111Apr. 8, 2011
PARCC has an overlapping history• Fall 2009 conversations among chiefs in Florida,
Massachusetts, and nine or ten other states to develop a common assessment with a design similar to that of Florida, with a level of rigor at least at the level of Massachusetts.
• Early engagement with Achieve, Inc. as a development partner
• Commitment to a “through course” design, with focus on summative results
1212Apr. 8, 2011
• Fall 2009, declares $350M for summative assessments• USED meetings, with invited papers and testimony
about what a multi-state assessment program should look like: • Nov. 12-13 (Boston)• Nov. 17-18 (Atlanta)• Dec. 1-2 (Denver)
• NRC Board on Testing and Assessment:• 2 conferences on new directions for state
assessments: Dec. 10-11 and Apr. 6-7• Through March, no indication on what the assessment
grant will look like
1313Apr. 8, 2011
• April 7: USED release of “Notice Inviting Applications” (NIA) for the Race to the Top Assessment Program
• Applications due in 11 weeks
• No more than two awards up to $150M each for summative ELA/Math assessments in grades 3-8 and high school, with possible supplement in $10M pieces
• No more than one high school course assessments; $30M
• Grants for development only; preparing assessments that can be given in 2014-15
1414Apr. 8, 2011
• “Applicant” is a single state acting on behalf of the consortium; consortium must have at least 15 states with at least 5 governing states
• Two types of membership: Governing state (part of only one consortium, part of decision-making) or advisory (can be in both consortia, less involved in decisions)
• Member states submit an MOU signed by Governor, Chief School Officer, President of State Bd of Ed, Chief Procurement Officer
• Membership condition: States must adopt the consortium’s common core by Dec. 31, 2011 or drop
• Acquire services of a Project Management Partner
1515Apr. 8, 2011
• Assess acquisition of and progress toward “college and career readiness (students able to take credit-bearing courses in English/math upon entry to college)
• Have common, comparable scores across all consortium states
• Provide achievement and growth information that can be used for teacher and principal evaluation and professional development
• Assess all students, except those with “significant cognitive disabilities”
• Be administered online, with timely results• Use multiple measures
Federal Register /Vol. 75, No. 68 / Friday, April 9, 2010 pp. 18171-85
1616Apr. 8, 2011
• 31 states – 17 Governing, 14 Advisory• WestEd was as interim Project Mgt Partner• Substantial foundation support from Sandler,
Hewlett, and Gates foundations for WestEd contract and convening of consortium states; CCSSO provided fiscal management
• Requested $150 million and a $10M supplemental
• Obtained support from higher ed institutions (IHEs) representing > 75% of 2009 college-enrolled seniors
1717Apr. 8, 2011
• Two comprehensive assessment awards to the only two bidders:SMARTER Balanced Assessment Consortium (SBAC)
Partnership for the Assessment of Readiness for College and Careers (PARCC)
• Both awarded their entire request:SBAC ~ $160M PARCC ~$170M
• No award for high school course exams (one unsuccessful bidder)
• Supplemental $16M to SBAC and PARCC; to support states’ implementation of Common Core
1818Apr. 8, 2011
Fiscal Agent: Washington State
17 Governing States 12 Advisory StatesCT, HI, ID, KS, ME, MI, MO, MT, NC, NH, NM, NV, OR, UT, VT, WA, WI, WV
AL, CO, DE, IA, KY, ND, NJ, OH, PA, SC, SD, WY
Membership a/o 3-31-11
19Apr 8, 2011
30 States in the SMARTER Balanced Consortium(a/o March 31, 2011)
20Apr 8, 2011
20
Governing Board State Participating State
25 States in the PARCC Consortium(a/o March 31, 2011)
2121Apr. 8, 2011
Gov Adv Gov A/P Gov ParticConnecticut G Alabama A/P Arizona GHawaii G Colorado A/P Arkansas GIdaho G Delaware A/P California PIowa A Kentucky A/P District of Columbia GKansas G New Jersey A/P Florida GMaine G North Dakota A/P Georgia GMichigan G Ohio A/P Illinois GMissouri G Pennsylvania A/P Indiana GMontana G South Carolina A/P Louisiana GNevada G 0 9 Maryland GHew Hampshire G Massachusetts GNew Mexico G Mississippi PNorth Carolina G New York GOregon G Oklahoma GSouth Dakota A Rhode Island GUtah G Tennessee GVermont G 14 2Washington GWest Virginia GWisconsin GWyoming ATOTAL 18 3
PARCC Only
AlaskaMinnesotaNebraska
TexasVirginia
5
Not in Either
SBAC OnlyIn both
Consortia
Consortia Designs
2323Apr. 8, 2011
An eligible applicant’s application must include...“...(a) An executive summary of the eligible applicant’s proposed
project;
(b) A theory of action that describes in detail the causal relationships between specific actions or strategies in the eligible applicant’s proposed project and its desired outcomes for the proposed project, including improvements in student achievement and college- and career-readiness;
(c) A plan for designing and developing the proposed assessment system;
(d) A plan for research and evaluation of the proposed assessment system;
(e) A plan for implementing the proposed assessment system; and
(f) A project management plan (including a workplan and timeline)...”
Federal Register; April 9, 2010 (p. 18174)
2424Apr. 8, 2011
• Grounded in a thoughtful, standards-based curriculum and represent an integrated system
• Assessments produce evidence of student performance
• Teachers involved in development and scoring of assessments
• System is state-led with transparent governance
• Assessments are structured to continuously improve teaching and learning
• Useful information on multiple measures that is educative for all stakeholders
• Adhering to established professional standards
2525Apr. 8, 2011
• State policies and practices that support consortium goals• Clear and rigorous content and skill targets for college
and career readiness• Effective communication of policies and standards• Supports and resources for teachers• Technology supports for learning and assessment systems• Rigorous summative assessments• Interim assessments and formative tools providing
actionable information• Teachers engagement in design, scoring, reporting of
assessments• Assessment information designed to improve learning
26Apr. 8, 2011
The SMARTER Balanced Theory of Action
All students leave high
school college and
career ready
Summative adaptive
assessments are benchmarked to
college and career readiness
Technology supports
innovative and
comprehensive
assessments
Technology provides
increased access to learning
State policies and practices
support increased
expectations
Common Core State
Standards specify K-12 expectations
for college and career
readiness
Clear communication of
expectations to stakeholders
Professional
capacity-buildingPD and other
supports for teachers to instruct on the CCSS
Teachers design and
score assessment items and
tasks
Interim assessments are used as progress
checks
Teachers use formative tools and
practices to improve
instruction
2727Apr. 8, 2011
How do we get from here... ...to here?
All students leave high
school college and career ready
Common Core State Standards specify K-
12 expectation
s for college and
career readiness ...and what can an assessment
system do to help?
28Apr. 8, 2011
The SMARTER Balanced Theory of Action
All students leave high
school college and career ready
Adaptive summative assessments
benchmarked to college & career
readinessCommon
Core State Standards specify K-
12 expectation
s for college and
career readiness
2929Apr. 8, 2011
Summative assessments using online computer adaptive technologies• Efficiently provide accurate measurement of all students, across
the spectrum of knowledge and skills, with shorter tests• Incorporate adaptive precision into performance tasks and events• Will assess full range of CCSS in English language arts and
mathematics; will include a variety of item types• Describe both current achievement and growth across time,
showing progress toward college- and career-readiness• Scores can be reliably used for state-to-state comparability, with
standards set against research-based benchmarks • The option of giving the summative tests twice a year.
30Apr. 8, 2011
The SMARTER Balanced Theory of Action
All students leave high
school college and career ready
Adaptive summative assessments
benchmarked to college & career
readinessCommon
Core State Standards specify K-
12 expectation
s for college and
career readiness
Interim assessments
that are flexible and open
3131Apr. 8, 2011
Optional interim assessments• Are aligned to and reported on the same scale as the summative
assessments• Help identify specific needs of each student, so teachers can provide
appropriate, targeted instructional assistance• Incorporate significant involvement of teachers in item and task
design and scoring• Are non-secure and fully accessible for use in instruction and
professional development activities • Provide students and teachers with clear examples of the expected
performance on common standards.
32Apr. 8, 2011
The SMARTER Balanced Theory of Action
All students leave high
school college and career ready
Adaptive summative assessments
benchmarked to college & career
readinessCommon
Core State Standards specify K-
12 expectation
s for college and
career readiness
Teachers can access formative tools and practices
to improve instructio
n
Interim assessments
that are flexible and open
3333Apr. 8, 2011
Web-based formative assessment resources• Online resources on assessment literacy, aligning assessments
to CCSS, and formative assessment guides• Training for local development of item and tasks and design and
use of scoring guides • Support of best practices through online learning modules • Comprehensive information portal, providing:
educator access to information about student progress toward college- and career-readiness
exchange of student performance history across districts and states
34Apr. 8, 2011
The SMARTER Balanced Theory of Action
All students leave high
school college and career ready
Adaptive summative assessments
benchmarked to college & career
readinessCommon
Core State Standards specify K-
12 expectation
s for college and
career readiness
Teachers can accessformative tools and practices
to improve instructio
n
Interim assessments
that are flexible and open
Optional Interim assessment system — no stakes
Summative assessment for accountability
Last 12 weeks of year*
DIGITAL CLEARINGHOUSE of formative tools, processes and exemplars; released items and tasks; model curriculum units; educator training; professional development tools and resources; scorer training modules; and teacher collaboration tools.
Scope, sequence, number, and timing of interim assessments locally determined
2 PERFORMANCETASKS Each:
• Reading/Writing• Math
COMPUTERADAPTIVE TESTS w/
Re-take Option
The SMARTER Balanced Design
* Windows may be adjusted based on results from the research agenda and final implementation decisions.
English Language Arts and Mathematics, Grades 3 – 8 and High School
Computer Adaptive Tests and Performance Tasks
INTERIM ASSESSMENT
Computer Adaptive Tests and Performance Tasks
INTERIM ASSESSMENT
(Source: ETS K-12 Center for Assessment and Performance Management)
Through-courseASSESSMENT4
• Speaking• Listening
25%
Through-courseASSESSMENT 1• ELA• Math
50%
Through-courseASSESSMENT 2• ELA• Math
90%
END OF YEARCOMPREHENSIVE
ASSESSMENT
75%
Through-courseASSESSMENT 3• ELA• Math
PARTNERSHIP RESOURCE CENTER: Digital library of released items, formative assessments, model curriculum frameworks, curriculum resources, student and educator tutorials and practice tests, scoring training modules, and professional development materials
Summative assessment for accountability
Required, but not used tor accountability
Apr 19, 2023
English Language Arts and Mathematics, Grades 3 - 11
The PARCC Design
37
Jamal Abedi
UC Davis/CRESST
Randy Bennett
ETS
Derek Briggs
University of Colorado
Greg Cizek
University of North Carolina
David Conley
University of Oregon
Linda Darling-Hammond
Stanford University
Brian Gong
The Center for Assessment
Ed Haertel
Stanford University
Joan Herman
UCLA/CRESST
Jim Pellegrino
University of Illinois, Chicago
W. James Popham
UCLA, Emeritus
Joe Ryan
Arizona State University
Martha Thurlow
University of Minnesota/NCEO
38
Henry BraunBoston College
Bob BrennanUniversity of Iowa
Derek BriggsUniversity of Colorado
Wayne CameraCollege Board
Linda CookRetired, ETS
Ronald HambletonUniversity of Massachusetts
Gerunda HughesHoward University
Huynh HuynhUniversity of South Carolina
Michael Kolen University of Iowa
Suzanne LaneUniversity of Pittsburgh
Robert LuechtUniversity of North Carolina, Greensboro
Jim PellegrinoUniversity of Illinois at Chicago
Barbara PlakeUniversity of Nebraska, Lincoln
Rachel QuenemoenNational Center for Educational Outcomes
Laurie WiseHuman Resources Research Organization (HumRRO)
Challenges
4040Apr. 8, 2011
Multiple testing occasions per year in grades 3-8 and high school• Infrastructure readiness• Student readiness / Opportunity-to-learn issues• Rapidly-changing platform technologies• Software interoperability
Maintaining interoperability across consortia• 2 CCSS consortia; 2 “1%” consortia; 2 ELD
consortia; 2 science consortia(?)• Standardization vs. Innovation
4141Apr. 8, 2011
Development of growth scales• “Progress toward college / career readiness• Within grade or out-of-level testing?• Comparability across consortia
Standard setting for college/career readiness• What to use for empirical criteria and database?
Does college ready = career ready?
Reliable and Feasible Scoring of New Item Types• Reliance on AI scoring
4242Apr. 8, 2011
Who does what in 2014?• RTTA is for development only; consortia end in 2014• Within grade or out-of-level testing?• Comparability across consortia
Research needs: Bridge studies (old-to-new) and Comparability studies (across states)
Development and Use of Common Protocols• Common accommodations• Loss of local control?
4343Apr. 8, 2011
Ensuring that clear communications are well understood• Across multiple states• Down to the school/classroom level
Maintaining coherent and effective governance• Perils of being too top heavy• Risks if too disconnected from leadership
4444Apr. 8, 2011
...the SMARTER Balanced Assessment Consortium can be found online at
www.k12.wa.us/SMARTER