Southern
Regional
Education
Board
HSTW
1
Southwest Ohio2014 Data Workshops
Using the HSTW Assessment and MMGW Survey Data
to Take Action
High Schools & Middle Schools
Mike Ross HSTW/MMGW Ohio Data Consultant
Susanne Davis HSTW/MMGW Ohio Assessment Coordinator
Southern
Regional
Education
Board
HSTW
2
Workshop Objectives
· Understand components of the HSTW and MMGW Assessment Reports
· Identify gaps in student achievement and instructional practices
· Identify new challenges· Link key Assessment data to the Site
Review Process (SRP)· Explore how Assessment data may
provide critical baseline or “readiness” levels for implementing common core state standards
Southern
Regional
Education
Board
HSTW
3
HSTW Key Practices
· High Expectations· Program of Study· Academic Studies· Career/Technical
Studies· Work-Based
Learning
· Teachers Working Together
· Students Actively Engaged
· Guidance· Extra Help· Culture of
Continuous Improvement
HSTW
Southern
Regional
Education
Board
HSTW
4
MMGW Key Practices
· Aligned academic core
· All students matter· High expectations
and extra help· Engaging students· Teachers working
together
· Support from parents
· Qualified teachers· Use of data· Use of technology
for learning· Strong leadership
Southern
Regional
Education
Board
HSTW
5
Why Are Data Important?
· Replace hunches with facts· Enable us to tackle root causes instead of
symptoms· Enable us to determine if we are
accurately tracking effects· Assist in understanding how school and
classroom practices impact achievement
Southern
Regional
Education
Board
HSTW
6
Data Can’t Help When…
· The data are not valid or reliable· The appropriate questions are not asked
after reviewing the data· Data analysis is not used for making wise
decisions· There is no commitment by teachers and
administrators to change school and classroom practices when data indicate the need for change
Make conscious efforts to develop a data-rich culture at your school!
Southern
Regional
Education
Board
HSTWMultiple Measures
Effort
Perceptions
Student Achievement
Demographics
School Processes
Based on Victoria Bernhardt’s Multiple Measures Model
Southern
Regional
Education
Board
HSTWCategories of Data
StudentAchievement
“Measures of student learning help us understandhow students are performing and what studentsknow as a result of instruction.
Based on Victoria Bernhardt’s Multiple Measures Model
Southern
Regional
Education
Board
HSTWCategories of Data
SchoolProcesses
“…programs, practices, and instructional strategies…that produce school and classroom results.”
Based on Victoria Bernhardt’s Multiple Measures Model
Southern
Regional
Education
Board
HSTWCategories of Data
Perceptions
“A particular view, judgment, or appraisal formed in themind about a particular matter...a belief stronger than impression and less strong than positive knowledge.”
Based on Victoria Bernhardt’s Multiple Measures Model
Southern
Regional
Education
Board
HSTWCategories of Data
Demographics
“Statistical characteristics of human populations…builds the context of the school …for which change is planned and takes place.”
Based on Victoria Bernhardt’s Multiple Measures Model
Southern
Regional
Education
Board
HSTWA Complete Picture
HSTW Assessment
State Test Scores/AYP
Course Failures
Discipline Referrals
AP/IB Enrollment
Industry Certification
Exams
SAT/ACT Scores
Southern
Regional
Education
Board
HSTW
13
Comparisonsof Assessments to OAA/OGT
HSTW Assessment
· Based on NAEP Standards
· 3 areas· Administered to
12th graders· Content through
grade 12· Multiple choice
OGT/OAA
· Based on Ohio standards
· 2 to 5 areas· Administered (initially)
to 3rd through 10th graders
· OGT – through gr. 10· Multiple choice &
constructed response
HSTW
Southern
Regional
Education
Board
HSTW
14
Comparisonsof Assessments to OAA/OGT
HSTW Assessment
· Reports Readiness Standards - Generalize to high school and college success
· Scale scores· Cut scores/
Proficiency Levels· Random Sample· Others?
OGT/OAA
· Not predictive of college success
· OGT tied to graduation
· Scale scores· Cut scores/
Proficiency Levels· All students· Others?
HSTW
Southern
Regional
Education
Board
HSTW
15
2014 Assessment & SurveysOhio Participation
· 2014 HSTW Assessment· 125 Ohio sites (41 SWO)· 6,314 Ohio 12th grade students (2,286 SWO)· 4,580 Ohio teachers (1,754 SWO)
· 2014 MMGW Surveys· 95 Ohio sites (31 SWO)· 12,118 Ohio 8th grade students (5,518 SWO)· 1,789 Ohio teachers (653 SWO)
Southern
Regional
Education
Board
HSTW
16
Approaching the Report
· Before analyzing data, ask yourself “who participated in this assessment?”
· Random sample or all students?· Career/technical or all students?· Who completed the assessment?
• Pages 62, 71, 86, 97 of High School assessment report
· What is your H.S. category? – A, B, C, D
Refer to top left of each page and
descriptions on page iv.
Southern
Regional
Education
Board
HSTW
17
High School Standard Setting – Readiness Standards
Reading Mathematics Science
Basic 250-271 257-291 258-285
Students performing at this level or above are able to enter postsecondary studies without needing additional preparation and/or are able to pass most employer exams for entry-level jobs.
Southern
Regional
Education
Board
HSTW
1818
2014 HSTW Assessment Results -All Ohio Sites
Percentage of Students Meeting Readiness Goals
60% 63% 60%65% 63% 63%
85% 85% 85%
59%57% 55%
0%
20%
40%
60%
80%
100%
Reading Mathematics Science
Ohio 2014 Ohio 2012 All Sites 2014 HSTW Goal
Source: 2014 HSTW Assessment – Ohio Composite, pg.1
HSTW Goal = 85%
Southern
Regional
Education
Board
HSTW
19
2014 HSTW Assessment Results -All Ohio Sites
Percentage of Students at Each Performance Level
40%
37%
40%
38%
39%
25%
19%
21%
27%
4%
4%
8%
Science
Math
Reading
Below Basic Basic Proficient Advanced
60%
63%
60%
Source: 2014 HSTW Assessment – Ohio Composite, pg.1
Southern
Regional
Education
Board
HSTW2014 Ohio Achievement Assessment 8th Grade Preliminary Results
Reading Mathematics Science0%
20%
40%
60%
80%
100%
87%80%
68%
86%
77%
69%75% 75% 75%
Percentage of Students at Proficient or Above
2014 State Std
Source: ODE website
State Standard
= 75%
Southern
Regional
Education
Board
HSTW2014 Ohio Achievement Assessment 8th Grade Preliminary Results
Science
Math
Reading
5%
6%
3%
27%
13%
9%
33%
41%
32%
16%
25%
32%
20%
14%
24%
Percentage of Students at Each Proficiency Level
Limited Basic Proficient Accelerated Advanced
Source: Ohio Department of Education
Southern
Regional
Education
Board
HSTW
22
Main Summary Reports
· Now that we know who participated and how Ohio students performed, how did your school do overall?
· H.S. - Executive Summary, pp. 1-2
· M.S. – refer to your OAA data
2014 All Ohio HSTW Sites Composite, p. 1
2014 All Ohio Career Centers Composite, p. 1
Southern
Regional
Education
Board
HSTW
Key Indicators of Student Achievementor Power Indicators (report page 2)
• 12 Student and 6 Teacher Indicators• Associated with very high mean
scores• ETS has found these indicators to be
statistically predictive of student achievement
25
2014 All Ohio HSTW Sites Composite, p. 2
Southern
Regional
Education
Board
HSTW
28
Activity Time
High School - Utilizing the Executive Summary & Analyzing Differences
· Report pp. 1-12· HS Handout #1
Middle School – Reviewing State Achievement Data
· State Report/Data· MS Handout #1
HSTW
Southern
Regional
Education
Board
HSTW
29
Activity Processing
· What value do these data provide?
· How can you use this executive summary and other features in the reports in planning at your school?
Southern
Regional
Education
Board
HSTW
30
Utilizing the High School Overview
andMiddle School Indices
Southern
Regional
Education
Board
HSTW
31
More Information
· Now that we’ve gotten a picture of overall performance, where can I get some more details?· H.S. - Executive Summary – pp. 1-2· H.S. - Overview – pp. 3-24· M.S. - Indices – pp. 6-8
Southern
Regional
Education
Board
HSTW
32
Semantics
· Indicator· Index or indices (pl.)
· Common meanings and distinctions
Southern
Regional
Education
Board
HSTW
33
“Our Definition”
· Indicator – a student experience(s)
and/or improvement strategy(ies) that are associated with higher levels of student achievement
Southern
Regional
Education
Board
HSTW
34
Definition
· Examples of Indicators Used a graphing calculator in
math assignments at least weekly.
Prepared a written report of science lab results at least monthly.
Used computer skills to complete an assignment or project in C/T classes at least weekly.
Southern
Regional
Education
Board
HSTW
35
Definition
· Index – A scale that combines multiple
items into a single value.
Each index is built around a common theme which HSTW or MMGW believes is a key element of student achievement.
Southern
Regional
Education
Board
HSTW
36
…be mindful
· “Although we cannot prove a causal connection between the indices and student test performance, it can be said that faithful adherence to the Key Practices is associated with better student achievement.”
Southern
Regional
Education
Board
HSTWUnderstanding the Indices
· Based on the HSTW Key Practices· Example – Literacy Across the Curriculum
· A series of indicators (and frequency) are identified for each practice· Example – Students often revised their
essays or other written work several times to improve their quality
· The number of indicators experienced determines the level of emphasis· Example – Intensive = 8-10 indicators
Source: 2014 HSTW Assessment – Ohio Composite, pg.13
High School Indicator Data(source: 2014 All Ohio HSTW Sites Composite, p. 26)
The MMGW Indices2014 MMGW Student Survey Ohio Composite, p. 7
2014 MMGW Student Survey Ohio Composite, p.17
2014 MMGW Student Survey Ohio Composite, p.18
2014 MMGW Student Survey Ohio Composite, p.19
Southern
Regional
Education
Board
HSTW
44
Activity TimeA More Detailed Picture of Your
School’s Performance
High School – Utilizing the Overview & the HSTW Indices
· Report pp. 1 & 13-23· HS Handout #2
Middle School –The MMGW Indices· Report pp. 6-7· MS Handout #2
HSTW
Southern
Regional
Education
Board
HSTW
45
Share
· On what three categories of indicators did most of your students report an intensive emphasis?
· What actions did you take that may account for these results?
· What challenges did you identify by looking at these data?
Southern
Regional
Education
Board
HSTW
46
The Importance of Leadership and School
Improvement
Southern
Regional
Education
Board
HSTW
47
Key Practice:Continuous Improvement
Use student achievement and program evaluation data to
continuously improve school culture, organization,
management, curriculum and instruction to advance student
learning.
HSTW
Southern
Regional
Education
Board
HSTWImprovement Cycle
Southern
Regional
Education
Board
HSTW
40% 42% 45%43% 43%47%
54% 56% 55%
Goals and PrioritiesClear
Demanding yetsupportive
Teachers continuallylearning
2014 2012 High-Scoring Sites
49
Teachers’ Perception on Continuous School ImprovementPercentage Who Strongly Agree
Source: 2014 HSTW Assessment – Ohio Composite, p.57
Southern
Regional
Education
Board
HSTW
40% 40%32%
39% 39%33%
53%47%
43%
Teachers and adminwork as team
Use datacontinuously
Intense Emphasis
2014 2012 High-Scoring Sites
50
Teachers’ Perception on Continuous School ImprovementPercentage Who Strongly Agree
Source: 2014 HSTW Assessment – Ohio Composite, p.57
2014 MMGW Teacher Survey Ohio Composite, p. 4
Southern
Regional
Education
Board
HSTW
52
Activity Time
High School – Leadership & Continuous Improvement
· Report pp. 187, 198, 201, 210-213
· Handout #3
Middle School – Leadership & Continuous Improvement
· Teacher Survey Report pp. 4 &16-19
· Handout #3
HSTW
Southern
Regional
Education
Board
HSTW
Engaging Instruction
…a key to ensuring rigorous and challenging learning experiences
Southern
Regional
Education
Board
HSTWKey Practice
· Engage students in academic and career/technical classrooms in rigorous and challenging assignments using research-based instructional strategies and technology.
Southern
Regional
Education
Board
HSTWFour Types of EngagementMMGW Student Report, pp. 33-40
· Intellectual - students work with new concepts, explain their reasoning, defend their conclusions and explore alternative strategies. Students have confidence in their ability to succeed and academic success is important for future goals.
· Emotional – students have opportunities to choose projects or areas of further study related to their interests and goals. Students who are engaged emotionally are able to relate what they are learning to their own life.
Southern
Regional
Education
Board
HSTWFour Types of EngagementMMGW Student Report, pp. 33-40
· Social – Student work in teams in class, participate in extracurricular activities, have friends at school, feel a sense of loyalty and belonging to the school, and believe in the legitimacy of school. Social engagement motivates students to stay in school.
· Behavioral - Students go to class prepared and actively participate, seek assistance when needed, and take challenging classes.
Southern
Regional
Education
Board
HSTW
SREB Literacy Goals
· Students will read the equivalent of 25 books per year across the curriculum.
· Students will write weekly in all classes.· Students will use reading and writing
strategies to help them understand and use the content of all classes.
· Students will write investigative research papers in all classes.
· Students will be taught as if they were in honors language arts classes.
Southern
Regional
Education
Board
HSTWActions for Engaging Students in Research-Based Instructional Strategies
· Project- or Problem-based learning· Cooperative learning· Student-designed research· Integrated, interdisciplinary studies· Integrating technology· Effective direct instruction
Southern
Regional
Education
Board
HSTWLiteracy Across the Curriculum or Engaging Literacy –Refer to p. 24 M.S and p.14 H.S. for complete lists
EXAMPLES OF INDICATORS· They often revised their essays or other written work
several times to improve their quality. (Similar)· They completed short-writing assignments of one to three
pages for which they received a grade in their English classes at least monthly. (Similar)
· They read an assigned book and demonstrated understanding of the significance of the main ideas at least monthly. (Similar)
· They stood before the class and made an oral presentation on a project or assignment to meet specific quality requirements at least once a semester. (Similar)
Southern
Regional
Education
Board
HSTWLiteracy Across the Curriculum – H.S.
Source: 2014 HSTW Assessment – Ohio Composite, p.1
24% 25%
36%
Percentage of Students Experiencing an Intensive Emphasis on Literacy
Across the Curriculum
Ohio 2014 Ohio 2012 High-Scoring Sites 2014
Southern
Regional
Education
Board
HSTWEngaging Literacy Across the Curriculum – M.S.
Source: 2014 MMGW Student Survey – Ohio Composite
36%43%
34%
Percentage of Students Experiencing an Intensive Emphasis on Literacy
Across the Curriculum
Ohio 2014 Ohio 2012 All Sites 2014
Southern
Regional
Education
Board
HSTWLiteracy Across the Curriculum – H.S.
51%56% 52%
64% 67% 65%71% 70% 68%
Reading Mathematics Science
Percentage of Students Meeting ReadinessGoals by Level of Emphasis on Literacy
Across the Curriculum
Low
Moderate
Intensive
Source: 2014 HSTW Assessment – Ohio Composite, p.14
Southern
Regional
Education
Board
HSTWLiteracy in All Classes
· Choose texts for students that are demanding enough to help develop strong reading skills
· Work with students directly in mastering texts - this is how students learn how to learn in each subject
· Students need more experiences in reading complex materials, analyzing texts, discussing what they read, and writing reports based on their reading.
Southern
Regional
Education
Board
HSTW
64
Activity Time - Literacy
High School – Engaging Instruction
· Report pp. 47-48, 77-84, 104 & 132
· HS Handout #4
Career Centers –
Quality CT; Integrating Academics into CT ; Work-Based Learning· Report pp. see
workbook handout for report pages
· CC Handout #5
Middle School – Engaging Classroom Practices
· Report pp. 22-24, 25-27, 28-30
· MS Handout #4
HSTW
Southern
Regional
Education
Board
HSTWEngaging Numeracy Across the Curriculum
–Refer to p. 27-29 M.S and p.15 H.S. for complete lists
EXAMPLES OF INDICATORS· They used math in classes other than mathematics at
least monthly. (Similar)· Their mathematics teachers sometimes or often
showed how mathematics concepts are used to solve problems in real-life situations. (Similar)
· They often developed and analyzed tables, charts and graphs in their school work. (Similar)
· They solved mathematics problems other than those found in the textbook at least monthly. (Similar)
· They used a graphing calculator to complete
mathematics assignments at least weekly. (Similar)
Southern
Regional
Education
Board
HSTWNumeracy Across the Curriculum – H.S.
38% 40%
53%
Percentage of Students Experiencing an Intensive Emphasis on Numeracy Across
the Curriculum
Ohio 2014 Ohio 2012 High-Scoring
Source: 2014 HSTW Assessment – Ohio Composite, p.1
Southern
Regional
Education
Board
HSTWEngaging Numeracy Across the Curriculum – M.S.
43%
31%38%
Percentage of Students Experiencing an Intensive Emphasis on Numeracy Across
the Curriculum
Ohio 2014 Ohio 2012 All Sites
Source: 2014 MMGW Student Survey – Ohio Composite
Southern
Regional
Education
Board
HSTWNumeracy Across the Curriculum – H.S.
39% 43%38%
60% 62% 59%70% 73% 70%
Reading Mathematics Science
Percentage of Students Meeting ReadinessGoals by Level of Emphasis on Numeracy
Across the Curriculum
Low
Moderate
Intensive
Source: 2014 HSTW Assessment – Ohio Composite, p.15
Southern
Regional
Education
Board
HSTW9 Actions to Build Students’ Math Abilities
· Develop students’ fluency in the language of mathematics.
· Help students find a purpose for mathematics.
· Teach students to organize and classify mathematics information.
· Help students gain a thorough understanding of fractions, decimals and percentages.
· Use multiple representations of information.
Southern
Regional
Education
Board
HSTW9 Actions to Build Students’ Math Abilities (contd.)
· Develop students’ understanding of appropriate applications of technology in mathematics.
· Emphasize the role of mathematics in the workplace and real world.
· Help students find a problem-solving strategy.
· Foster students’ confidence in and appreciation for mathematics.
Southern
Regional
Education
Board
HSTW
71
Activity Time - Numeracy
High School – Engaging Instruction
· Report pp. 48-49, 92-95, 131
· HS Handout #4
Career Centers –
Quality CT; Integrating Academics into CT ; Work-Based Learning
· Report pp. see workbook handout for report pages
· CC Handout #5
Middle School – Engaging Classroom Practices
· Report pp. 22-24, 25-27, 28-30
· MS Handout #4
HSTW
Southern
Regional
Education
Board
HSTWEngaging Science Curriculum and Instruction–Refer to p. 30-32 M.S and p.16 H.S. for complete lists
EXAMPLES OF INDICATORS· Their science teachers often showed how scientific
concepts are used to solve problems in real-life situations. (Similar)
· They used science equipment to do science activities in a classroom or laboratory at least weekly. (Similar)
· They used computers or technology to do science activities at least monthly. (Similar)
· They prepared a written report of their lab results at least monthly. (Similar)
· They participated in a classroom discussion relating science to everyday life at least monthly. (Similar)
· They worked with other students in their class on a challenging science assignment or project at least monthly. (Similar)
Southern
Regional
Education
Board
HSTWEngaging Science – H.S.
15% 16%21%
Percentage of Students Experiencing an Intensive Emphasis on Engaging
Science
Ohio 2014 Ohio 2012 High-Scoring Sites
Source: 2014 HSTW Assessment – Ohio Composite, p.1
Southern
Regional
Education
Board
HSTWEngaging Science – M.S.
33%
22%
31%
Percentage of Students Experiencing an Intensive Emphasis on Engaging
Science
Ohio 2014 Ohio 2012 All Sites
Source: 2014 MMGW Student Survey – Ohio Composite
Southern
Regional
Education
Board
HSTWEngaging Science – H.S.
49% 52% 48%
65% 68% 65%68% 68% 67%
Reading Mathematics Science
Percentage of Students MeetingReadiness Goals by Level of
Emphasis on Engaging Science
Low Moderate Intensive
Source: 2014 HSTW Assessment – Ohio Composite, p.16
Southern
Regional
Education
Board
HSTWHow can you improve the quality of instruction?
· Focus on improving professional practice· Common planning time
• Integrating• Common unit planning, syllabi, exams, etc.
· Peer observations· Demonstration classrooms· Professional development and follow-up· Team teaching· Teacher evaluation; annual plans for
improvement
Southern
Regional
Education
Board
HSTW
77
Activity Time - Science
High School – Engaging Instruction
· Report pp. 49-50, 104-108· HS Handout #4
Career Centers –
Quality CT; Integrating Academics into CT ; Work-Based Learning
· Report pp. see workbook handout for report pages
· CC Handout #5
Middle School – Engaging Classroom Practices
· Report pp. 22-24, 25-27, 28-30
· MS Handout #4
HSTW
Southern
Regional
Education
Board
HSTWShare
· What does this type of data tell us about quality instruction in our schools?
· How can these data be used to impact student achievement?
· What actions can be taken to increase the quality of instruction?
Southern
Regional
Education
Board
HSTW
79
Monitoring and Updating School Improvement Plans
Southern
Regional
Education
Board
HSTW
80
Data Tools
Compare your site 2012 & 2014 vs. All Ohio Sites vs. All Network Sites
hstwohioregions.org“Assessments” Tab
2014 Data Tools – Ohio High School Sitesor
2014 Data Tools – Ohio Middle Grade Sites
Southern
Regional
Education
Board
HSTW
81
Improvement Plansand Processes
Site Action Plans (SAP)CCIPOIP
Site Review Process (SRP)
Work from one plan!
Southern
Regional
Education
Board
HSTWOhio Improvement Plan
Southern
Regional
Education
Board
HSTWSRP Seven Phase Process
83
Site Review Process
Phase 1
Introduction to SRP process
Phase 2
Prepare for the SRP
Review Data
Phase 3Plan On-site
ReviewComplete DOP
Worksheets
Phase 4
Complete the On-site review and
share exit summary
Phase 5
Complete the final SRP report
including SIP
Phase 6
Present completed SRP
report/SIP plan to staff
Phase 7
Implement and update SIP
Southern
Regional
Education
Board
HSTW
84
Next Steps:How to Use the Assessment Report
· Show the “positives”· Identify the challenges
· Use state assessment & HSTW/MGA report findings that explain why
· What actions can you take to change that?
· Walk through the report· Identify strengths and weaknesses· Identify improvement actions
· Develop plan to share findings with the faculty and other stakeholders
Southern
Regional
Education
Board
HSTW
85
Next Steps: Identify and Prioritize Actions
· List areas in need of improvement· Rank items based on impact on student
achievement and completion rates.
· Identify short-term goals · Implementation in the first year· Have one item in each of the four areas of
structural, instructional, support and leadership change
· Identify long-term goals· Implementation in years two and three· Have one item in each of the four areas of
structural, instructional, support and leadership change
Southern
Regional
Education
Board
HSTW
86
Essentials for Moving Forward
· Keep the improvement plan “alive”· Keep staff informed and involved· Establish necessary structures and
processes (committees or focus teams, regular meeting schedules, agendas, and meeting notes)
· Use resources from SREB, state, and other sites
· Collect evidence to determine the effectiveness of improvement efforts
· Make “mid-point” corrections and revise improvement plan at least once a year
· Develop teacher leaders to make this happen!
Southern
Regional
Education
Board
HSTW
87
Website information and workshop documents:
www.hstwohioregions.org
click “Assessments”
www.sreb.orgclick “SREB Programs”
HSTW or MMGW
Southern
Regional
Education
Board
HSTW
88
Contact Information
Susanne Davis, HSTW SW [email protected](513) 677-5641
Mike Ross, HSTW SW [email protected](937) 996-4694
Missy Kramer, HSTW SW [email protected](513) 619-2330
Fran Cowart, SREB - [email protected](404) 879-5611
Jordan Hertl, SREB - [email protected](404) 879-5575
Southern
Regional
Education
Board
HSTW
89
Wrapping it all up--Remember…
All schools want to improve but few want to change. The fact remains
that to improve, one MUST change.
Remember that if nothing changes,nothing changes!
HSTW