Making Inferences About Impact on Student Growth
Dr. Patricia Reeves, Associate Professor Educational Leadership & ResearchWestern Michigan University – February 2014Achievement Centered Leadership (ACL) Grant Project: Cohort 1 – Session 1
ACL:
Sess
ion1
:DD
IM.R
eeve
s2.2
7.14
1
The Pending Legislation on Growth• There will be a State Growth and/or Value Added Model
• There will be criteria for mandatory State assessments used for teacher, school, and district effectiveness ratings
• Districts will need to augment the State Growth and Value-Added rating system with a Local Growth Model
• The point of growth models is to make inferences about how educators, schools, and districts are influencing learning and learning rates
ACL:
Sess
ion1
:DD
IM.R
eeve
s2.2
7.14
2
Growth Rating Recap: State Growth Rating + Local Growth Rating
Under the proposed legislation: •25% of evaluation based on growth starting 2014-15 •In 2017-18 growth becomes 50%
•State will provide growth ratings on State assessments •Districts can develop growth ratings based on a local growth model
ACL:
Sess
ion1
:DD
IM.R
eeve
s2.2
7.14
3
Building upon your District implementation plan for School ADvance: Growth Ratings
ACL:
Sess
ion1
:DD
IM.R
eeve
s2.2
7.14
4
Building upon your District implementation plan for School ADvance: Growth Ratings
ACL:
Sess
ion1
:DD
IM.R
eeve
s2.2
7.14
5
Building upon your District implementation plan for School ADvance: Growth Ratings
ACL:
Sess
ion1
:DD
IM.R
eeve
s2.2
7.14
6
How do you know students are successful in your schools?1. How do you define success?
a. What is the district definition?
b. What is the school level definition?
2. What measures help you track how well students are doing?
3. How do those measures align with your definitions of success? 7
ACL:
Sess
ion1
:DD
IM.R
eeve
s2.2
7.14
Three Decision Points for a Value Added or Growth Model
ACL:
Sess
ion1
:DD
IM.R
eeve
s2.2
7.14
8
Influences on Student Learning
ACL:
Sess
ion1
:DD
IM.R
eeve
s2.2
7.14
9
Assessing School, Teacher, and Administrator Influence on Growth
ACL:
Sess
ion1
:DD
IM.R
eeve
s2.2
7.14
10
Developing a Local Growth Model
1.What indicators and measures have you been using to evaluate teachers’ and administrators’ performance?
2.How well aligned are those indicators with your school improvement plan and the district improvement plan?
3.How well aligned are they with your definition of student success?
ACL:
Sess
ion1
:DD
IM.R
eeve
s2.2
7.14
11
Regardless of students’ entering achievement levels, growth is:
KEEPING THEM MOVING UP AND EXPANDING: Goal is to either maintain or accelerate growth rates if at, or above, target achievement levels to stay ahead of a success track (e.g. hitting 3rd, 7th/8th, and 11th Grade targets) and to branch out.
ACL:
Sess
ion1
:DD
IM.R
eeve
s2.2
7.14
12
MAKING SURE THEY KEEP UP: Goal is to maintain or accelerate the growth rates if at, or above, target achievement levels to stay on a success track (e.g. hitting 3rd, 7th/8th, and 11th Grade targets).
MOVING THEM UP: Goal is to accelerate growth rates until these students are also on target to reach achievement targets by certain grades in order to get on a success track (e.g. hitting 3rd, 7th/8th, and 11th Grade targets).
Higher
Middle
Lower
The Point of Growth Models• How are we doing at ensuring that our higher academic
performers are continuing to MOVE UP and branch out*?
• How are we doing at ensuring that our middle academic performers (the majority of our students) are continuing to either KEEP UP or MOVE UP and branch out*?
• How are we doing at ensuring that our lowest performers are on track to CATCH UP?
*Branch out means to expand development in other academic and/or non-academic indicators of student success, e.g. the arts, community service, athletics, and non-core curricular subjects (languages, logic, philosophy, social sciences, etc.)
ACL:
Sess
ion1
:DD
IM.R
eeve
s2.2
7.14
13
Performance Status(descriptive stat models)
Value-Added Models or VAM (linear stat models)
Growth to Proficiency Models (curvi-linear stat models)
Compares performance status of each student to fixed performance targets (e.g. proficiency level or X amount of pre-post growth).
Ignores differences in student growth rates and differences in student background.
Teachers rated on # or % of students achieving targets – based on either proficiency levels or growth levels.
Teacher information shows where each student performed in relation to performance targets for that year.
Compares growth of students to their academic peers and then compares groups of students to other groups of students by teacher.
Often controls for differences in student background.
Teachers rated on teacher to teacher comparisons of student growth by academic peer group or aggregate groups.
Teacher information shows how each teacher’s class grew compared to other teachers’ classes for that year, either by whole comparison group or sub-groups (e.g. low SES, ESL, etc.)
Compares growth of each student to growth of that student’s academic history and academic peers (i.e. students with the same performance history).
Does not need to control for differences in student background because teachers not compared to one another.Teachers rated on how they are moving students toward growth targets; i.e. catching up, keeping up, moving up.
Teacher information shows how each student is growing compared to growth targets for their academic peer group based on rate of growth needed to reach performance targets over several grades.
3 Common approaches for creating growth ratings
14AC
L:Se
ssio
n1:D
DIM
.Ree
ves2
.27.
14
The Logic of aLocal Growth Model
Projected Achievement
Actual Achievement
Value-added
Growth
Individual Student Past Performance
Instruction
15
ACL:
Sess
ion1
:DD
IM.R
eeve
s2.2
7.14
Value-Added Ratings
Growth & Value Added
Changes in student achievement and other success indicators across time
Estimations of Teacher and school influence
16AC
L:Se
ssio
n1:D
DIM
.Ree
ves2
.27.
14
Value-Added Models: How they work
Pre-TestYEAR 1
Post-TestYEAR 2
Actual student scale score
Predicted student scale score
Student scale score
Based on observationally similar students
VALUE ADDED
17AC
L:Se
ssio
n1:D
DIM
.Ree
ves2
.27.
14
Visualization with a SGP ModelEach student plotted against a backdrop of typical growth and status
18AC
L:Se
ssio
n1:D
DIM
.Ree
ves2
.27.
14
Individual Student Monitoring and RTI with a SGP Model
19AC
L:Se
ssio
n1:D
DIM
.Ree
ves2
.27.
14
Higher Than Expected
Setting Growth Standards
Value-added
GrowthGrowth
Thresholds
Expected Growth
Lower Than Expected
Unsatisfactory Growth
Evaluation
Categories
Highly Effective
Effective
Needs Improvement
Unsatisfactory
20AC
L:Se
ssio
n1:D
DIM
.Ree
ves2
.27.
14
Teacher Evaluation Example with a SGP Model
LGMs can create a single comparable growth metric across all tests.
21AC
L:Se
ssio
n1:D
DIM
.Ree
ves2
.27.
14
Growth to proficiency SGM Sets yearly targets that will put low-achievers on pace to meet proficient and narrow achievement gaps
Growth targets (based on proficient by grade 7)
22AC
L:Se
ssio
n1:D
DIM
.Ree
ves2
.27.
14
Processing the Challenge of Rating Individual Teachers and Administrators on Student Growth1. With your district team, discuss what measures you are
using now to determine how teachers are influencing student growth
2. How are you analyzing the data to determine a value-added rating?
3. Where do you think you need to go from here?
4. What conversations will you have back in your district?
23AC
L:Se
ssio
n1:D
DIM
.Ree
ves2
.27.
14
A tale of 2 teachers
One has an Effective RatingThe other,
a Highly Effective Rating
ACL:
Sess
ion1
:DD
IM.R
eeve
s2.2
7.14
24
Reminder: Three Purposes for Performance Evaluation
1. Achieve Organizational Goals, i.e. Student Outcomes
2. Guide the learning, growth, and development of personnel, e.g. teachers and leaders
3. Make decisions about hiring, placement, retention, promotion, and compensation AC
L:Se
ssio
n1:D
DIM
.Ree
ves2
.27.
14
25
Three Purposes for Performance Evaluation
1. Achieve Organizational Goals, i.e. Student Outcomes
2. Guide the learning, growth, and development of personnel, e.g. teachers and leaders
3. Make decisions about hiring, placement, retention, promotion, and compensation
26
Reminders:1. We can follow student success by identifying
success indicators and measuring growth toward established performance standards and beyond
2. We can grow student success by analyzing growth rates and intervening where/how needed with evidence based practices (strategies)
3. We can grow teacher, administrator, school and district performance by assessing both practice and results (i.e. student growth) and setting improvement goals accordingly
27
ACL:
Sess
ion1
:DD
IM.R
eeve
s2.2
7.14
Final Reminder: Growth Models work best when all growth goals are aligned across the school and district and based on definitions (indicators) of student success
ACL:
Sess
ion1
:DD
IM.R
eeve
s2.2
7.14
28
Teachers Administrators
Superintendent Board