Utilizing Student Academic Behavior Data to Improve Student Success Kira J. Holt Principal Consultant, ACT ©2012 by ACT, Inc. All rights reserved.

Utilizing Student Academic Behavior Data to Improve Student Success

Kira J. HoltPrincipal Consultant, ACT

©2012 by ACT, Inc. All rights reserved.

Workshop AgendaIntroduction

College and Career Readiness Challenge

Why Measure Academic Behaviors?

What is ENGAGE and What Does It Measure?

Administering ENGAGE

ENGAGE Reports

ENGAGE Resources

Summary and Conclusion

ACT’s M I S S I O NHelping people achieve education and workplace success

ACT’s V A L U E SExcellenceDiversity

LeadershipEmpowerment

LearningSustainability

Workshop AgendaIntroduction

College and Career Readiness Challenge

Why Measure Academic Behaviors?

What is ENGAGE and What Does It Measure?

Administering ENGAGE

ENGAGE Reports

ENGAGE Resources

Summary and Conclusion

The College and Career Readiness ChallengeOn the one hand, as reported in Ready for College and Ready for Work: Same or Different (2006), college and career readiness skills have converged.

On the other hand, in 2012 only 25% of graduating seniors who completed the ACT met all four of ACT’s College and Career Readiness Benchmarks.

ACT’s College Readiness Benchmarks

The ACT Benchmark Score indicated a 50% chance of obtaining a “B” or a 75% chance of obtaining a “C” in corresponding credit-bearing college courses.

College Readiness Benchmark Score

Test College Course EXPLORE (8/9) PLAN The ACT

English English Comp. 13/14 15 18

Math Algebra 17 /18 19 22

Reading Social Sciences 15/16 17 21

Science Biology 20/20 21 24

MEASURING STUDENT PROGRESS TOWARD READINESS IMPROVING COURSE RIGOR

SUPPORTING SOLUTIONS

PLANNING SCHOOL IMPROVEMENT

EXPLORE

8th and 9th grade curriculum-based educational and career planning

program

PLAN

10th grade curriculum-based educational and career planning

program

The ACT

11th and 12th grade curriculum-based assessment

for learning outcomes

ENGAGE

Middle and high school assessment that measures all

factors of academic success

QualityCore

Research-driven solutions for

strengthening curriculum

CoreWork DiagnosticsOnline service to diagnose and improve

content and practice areas

Core Practice AuditFramework for evaluating current

practices

ACT’s College and Career Readiness System

Workshop AgendaIntroduction

College and Career Readiness Challenge

Why Measure Academic Behaviors?

What is ENGAGE and What Does It Measure?

Administering ENGAGE

ENGAGE Reports

ENGAGE Resources

Summary and Conclusion

Why Measure Academic Behaviors?

• Identify at-risk students early

• Diagnose student strengths and needs

• Identify appropriate interventions

• Assess the effectiveness of interventions

What the Research Says… “current

research across disciplines

shows that non-academic aspects of

school are also significant

contributors to both school and

student success.”

(Page 2)

“While assessments of

academic achievement provide early

indication of risk, academic

behaviors are also important for persistence and

success.” (Page 1)

What the Research Says…

What the Research Says…

“As a leading indicator, aggregate academic behavior assessment results help educators know that their programs are on the path to success long before improvements in lagging indicators, such as academic readiness or graduation rates, can be documented.”

Enhancing College and Career Readiness and Success: The Role of Academic Behaviors, p.12

Research Summary

Student Academic Success

School Connectedness

Student Academic

Achievement

Student Academic Behaviors

Workshop AgendaIntroduction

College and Career Readiness Challenge

Why Measure Academic Behaviors?

What is ENGAGE and What Does It Measure?

Administering ENGAGE

ENGAGE Reports

ENGAGE Resources

Summary and Conclusion

What is ENGAGE?• measures behaviors and non-cognitive

attributes related to academic success and persistence

• helps educators evaluate students’ self-reported psychosocial attributes, determine their levels of academic risk, and identify interventions to help them succeed

What Does ENGAGE Measure?Domain ENGAGE Grades 6-9 ENGAGE Grades 10-12

Motivation (Getting work done)

Academic Discipline Commitment to

School Optimism

Academic Discipline General

Determination Goal Striving Commitment to

College Communication Skills Study Skills

Social Engagement(Getting along)

Family Attitude toward Education

Family Involvement Relationships w/

School Personnel School Safety Climate

Social Activity Social Connection

Self-Regulation(Keeping your cool)

Managing Feelings Thinking Before

Acting Orderly Conduct

Academic Self-Confidence

Steadiness

What Does ENGAGE Measure?Motivation

Definition: personal characteristics that help students succeed academically by focusing and maintaining energies on goal-directed activities

Grades 6-9 Examples: Academic Discipline,Commitment to School, and Optimism

Grades 10-12 Examples: Academic Discipline,Commitment to College, Study Skills, Etc.

What Does Engage Measure?Social Engagement

Definition: includes interpersonal factors that influence students’ successful integration into their environment

Grades 6-9 Examples: Family Attitudes toward Education, Family Involvement, Relationships

with School Personnel, and School Safety Climate

Grades 10-12 Examples: Social Activity and Social Connection

What Does ENGAGE MEASURE?SELF REGULATION

Definition: includes cognitive and affective processes used to monitor, regulate, and control behavior related to learning

Grades 6-9 Examples: Managing Feelings, OrderlyConduct, Thinking Before Acting

Grades 10-12 Examples: Academic Self-Confidence and Steadiness

Percentage of students accurately identified as having a 9th-grade GPA < 2.0

Selection Method Hit RateRandom 24%EXPLORE Composite only 69%ENGAGE Grades 6-9 only 80%EXPLORE & ENGAGE 83%

©2012 by ACT, Inc. All rights reserved.

Percentages of Students Accurately Identified for Dropping Out

Selection Method Hit RateRandom 17%EXPLORE Composite only 37%ENGAGE Grades 6-9 Grad. Index only 42%

EXPLORE & ENGAGE Grad. Index 52%

©2012 by ACT, Inc. All rights reserved.

What the Research Says about ENGAGE

©2012 by ACT, Inc. All rights reserved.

Workshop AgendaIntroduction

College and Career Readiness Challenge

Why Measure Academic Behaviors?

What is ENGAGE and What Does It Measure?

Administering ENGAGE

ENGAGE Reports

ENGAGE Resources

Summary and Conclusion

• Create Local Items:

• Reserve the computer lab and ensue the Internet Explorer is 8 or above. Chrome, Firefox, and Safari are viable alternatives.

• Make sure spam filters do not block e-mails from [email protected].

• Administer ENGAGE which takes approximately 30 minutes, but it is an untimed inventory.

• Complete the inventory in one sitting: students may take a break if necessary.

Administering ENGAGE ENGAGE Login Page

Workshop AgendaIntroduction

College and Career Readiness Challenge

Why Measure Academic Behaviors?

What is ENGAGE and What Does It Measure?

Administering ENGAGE

ENGAGE Reports

ENGAGE Resources

Summary and Conclusion

• Two Reports for Each Student•Student Report•Advisor Report

• An Aggregate School Report

• A School Roster Report

Utilizing ENGAGE Results: Report Types

Utilizing ENGAGE Results: Accessing Reports

ENGAGE Login Page

Profile of scores

Success Indices(only on Advisor & Roster Reports)

Interpretive feedback, sorted from strengths to needs

Sample ENGAGE Advisor Report

©2012 by ACT, Inc. All rights reserved.

• What do my scores mean?

• What actions should I consider?

• A message for parents

Utilizing ENGAGE Results: The Big Picture

Utilizing ENGAGE Results: The Roster ReportStudent ID Last Name Grade Level Prior grades

Academic Success Index

PercentileGraduation Index

Percentile

Academic Discipline Percentile

Commitment to School Percentile

Family Attitude toward Education

Percentile12345Sample 1 9th grade Below D's 1 2 3 99 9912346Sample 2 9th grade Mostly C's 12 6 13 99 4912347Sample 3 9th grade Mostly C's 34 26 50 34 2412348Sample 4 9th grade Mostly B's 26 24 27 99 2112349Sample 5 9th grade Mostly C's 21 5 21 40 6812350Sample 6 9th grade Mostly A's 85 89 46 30 9912351Sample 7 9th grade Mostly A's 81 56 88 99 5812352Sample 8 9th grade Mostly C's 13 22 46 34 9912353Sample 9 9th grade Mostly C's 12 4 24 99 9912354Sample 10 9th grade Mostly C's 12 8 15 50 6812355Sample 11 9th grade Mostly C's 7 40 2812356Sample 12 9th grade Mostly A's 71 47 42 50 6812357Sample 13 9th grade Mostly C's 16 2 24 63 4912358Sample 14 9th grade Mostly B's 37 17 42 40 5812359Sample 15 9th grade Mostly C's 17 14 24 17 2412360Sample 16 9th grade Mostly B's 74 99 6812361Sample 17 9th grade Mostly B's . . .12362Sample 18 9th grade Mostly A's 77 65 21 34 9912363Sample 19 9th grade Mostly C's 15 6 15 30 1512364Sample 20 9th grade Below D's 1 1 3 30 112365Sample 21 9th grade Mostly B's 77 89 85 99 9912366Sample 22 9th grade Mostly B's 24 4 39 50 2812367Sample 23 Mostly B's 52 31 29 15 1512368Sample 24 Mostly B's 63 75 54 34 4412369Sample 25 9th grade Mostly C's 6 28 2 2 4912370Sample 26 9th grade Mostly B's 37 34 12 40 3812371Sample 27 9th grade Mostly B's 65 70 66 99 9912372Sample 28 9th grade Mostly B's 56 65 66 99 3812373Sample 29 9th grade Mostly C's 7 2 27 30 912374Sample 30 8th grade Mostly C's 39 20 .12375Sample 31 9th grade Mostly A's 99 99 85 99 99

12376Sample 32 9th grade -- 16 25 19 7 612377Sample 33 9th grade Mostly C's 9 11 7 12 15

Workshop AgendaIntroduction

College and Career Readiness Challenge

Why Measure Academic Behaviors?

What is ENGAGE and What Does It Measure?

Administering ENGAGE

ENGAGE Reports

ENGAGE Resources

Summary and Conclusion

ENGAGE Resources

ENGAGE provides an innovative, easy-to-use, low stress, high impact method of

•

• Identifying at-risk students;

• justifying individual and school/district wide interventions; and

• helping all students reach their full academic potential and schools achieve their mission.

In sum...

Thank you for your time.

Kira J. HoltPrincipal Consultant, Southwest Region

ACT, Inc.P: 512.320.1850

[email protected]