Using EVAAS to Improve Student Performance Donna Albaugh Rachel McBroom Heather Stewart Region 4 PD Leads NCDPI
Jan 21, 2016
Using EVAAS to Improve Student
Performance
Donna AlbaughRachel McBroomHeather Stewart
Region 4 PD LeadsNCDPI
Outcomes
• Overview of EVAAS Growth Model
• Interpreting Value-Added Reports
• Interpreting Diagnostic Report
• Helping teachers use their individual data
Note: Slides marked with * are from SAS.
Underlying EVAAS Philosophy• All students deserve opportunities to make appropriate academic
progress every year.
• There is no “one size fits all” way of educating students who enter a class at different levels of academic achievement.
• Adjustments to instruction should be based on the academic attainment of students, not on socio-economic factors.
• Given reliable information on past effectiveness, educators can make appropriate adjustments to improve student opportunities.
• "What teachers know and can do is the most important influence on what students learn." (National Commission on Teaching and America's Future, 1996)
• One of the most important things educators can know is with whom they are effective and where they need to develop new skills..
* SAS
Achievement and Poverty
* SAS
Academic Growth and Poverty
No one is doomed to failure.* SAS
Benefits for Principals
• Gain a consolidated view of student progress and teacher effectiveness, as well as the impact of instruction and performance.
• Bring clarity to strategic planning and function as a catalyst for conversations that must take place to ensure that all students reach their potential.
• Understand and leverage the strengths of effective teachers.
• Use the valuable resource of effective teaching to benefit as many students as possible.
2012-13 Changes in Reporting
2011-12 2012-13
Above
Not Detectably Different
Below
Exceeds Expected Growth
Meets Expected Growth
Does Not Meet Expected Growth
Value-Added Reporting
Predictions
Projections
North Carolina uses two different models in EVAAS
1. The Univariate Response Model (URM) is used for EOGs in grades 5 and 8 science, EOCs in high school, and CTE Post-Assessments; URM will be used for the Common Exams (predicts)
2. The Multivariate Response Model (MRM) is used for EOGs in grades 3 – 8 mathematics and English Language Arts (projects)
Predict/Project
The URM uses a student’s prior test scores to predict where a student will be positioned in the statewide distribution of students who take a certain assessment
The MRM uses a student’s prior test scores to project where students will be positioned in the statewide distribution of students who take a certain assessment
In both cases, the basic methodology is the same: using a student’s prior test scores to consider their performance on an assessment he or she will take at the end of the school year
Predictive VA Model –Univariate Response Model (Predicts)• For reports where testing is not sequential
(gr. 5 and 8 science, EOCs in high school, CTE post-assessments)
• Common Exams
• Students must have three previous test scores for a predictive model
• Effect is difference between predicted and observed scores
Value-Added Reports – Multivariate Model (Projections)
Value-Added Reporting
Value-Added Reporting
The NCE Base is by definition set at 50.0, and it represents the average attainment level of students in the grade and subject, statewide.
If the school mean is greater, the average student in the school is performing at a higher achievement level than the average student in the state.
District Diagnostic Reports
• Use to identify patterns or trends of progress among students expected to score at different achievement levels
Diagnostic Report
Diagnostic Reports – the whiskers
Diagnostic Reports Looking for Patterns
Student Pattern Report
Student Pattern Report – Key Questions
Different experience?
Different strategies?
Different needs?
Number of hours?
Student Pattern Report – Key Questions
Different experience?
Different strategies?
Different needs?
Number of hours?
Rerun the report with new criteria.
YES!
Student Pattern Report – Next Steps
16 Students who attended for 40+ hours
All 31 Students in the Program
Less Informed Conclusion: We need to change the selection criteria for this program.
More Informed Conclusion: We need to adjust the recommended hours for participants.
Custom Student Report HANDOUT
Using Data to Drive Crucial ConversationsWhat’s a Crucial Conversation? And Who Cares?
A crucial conversation is a discussion between two or more people where stakes are high, opinions vary, and emotions run strong. When we face crucial conversations, we can do one of three things: We can avoid them, we can face them and handle them poorly, or we can face them and handle them well. Ironically, the more crucial the conversation, the less likely we are to handle it well. •www.wikisummaries.org
Frequently Asked Questions
How is the school composite determined?
In accordance with current State Board of Education policy, the school composite only includes assessments that are part of the State Testing Program.
• Included: EOCs and EOGs
• Not Included: CTE Post-Assessments and Common Exams
Many educators feel strongly that the CTE Post-Assessments and Common Exams should be included – the State Board of Education will consider this recommendation
Frequently Asked Questions
How will the Common Exams be entered into EVAAS?
The data will be processed and transferred in the same way as data from the State Testing Program: districts scan their answer sheets; DPI does quality checks and audits; DPI moves the data to SAS through a secure FTP
A major difference for all assessments (EOCs, EOGs, CTE Post-Assessments, and Common Exams) is that student-teacher links will come from the EVAAS roster verification process, not files pulled from NCWISE
Frequently Asked Questions
Why is there no growth data for individual students?
All growth models depend on having a large enough sample of student test scores to produce valid and reliable measures of growth
The ABC Growth Model is not valid or reliable when used to measure growth at the student-level; the error around any estimate is larger than the entire scale for an assessment
EVAAS will only produce growth values in which we can all have confidence, hence, there will be no growth values for individual students
Frequently Asked Questions
How can we have growth measures for assessments that have never been given before?
Even if an assessment has never been given before, a first-time administration will produce a mean score, and 50% of students will do better than the mean and 50% of students will do worse than the mean
Using percentiles and NCEs, EVAAS is always considering a student’s position in the statewide distribution, not the scale or raw score on the assessment
Frequently Asked Questions
How can we have growth measures for assessments that have never been given before?
For assessments that are completely new (like the Common Exams), SAS performs an analysis of data from first semester to determine the prediction model. After the second semester administration, all of the data are used in the model to produce value-added scores
Frequently Asked Questions
Why are there large numbers of teachers “not meeting expected growth” in some content areas and not others?
There are larger numbers of students taking some state assessments compared to others. Additionally, evidence from across the country shows that ELA teachers are less likely to have either an extremely positive or extremely negative impact than teachers of other content areas
The distributions currently in EVAAS are based on one-year of data; they tend to normalize as we add more data to the system
Questions?