Holistic Course Placement and Whole-Student Model of Early Intervention Kevin Li, Dean of Instruction Sara Schupack, Director of Developmental Education Lawrence Buonaguidi, Quality Assurance Coordinator Steve Robbins, Director of Research Innovation, Academic and Workforce Success The Innovations 2013 Conference March 12, 2013; Dallas, Texas
31
Embed
Holistic Course Placement and Whole-Student Model of Early Intervention
Holistic Course Placement and Whole-Student Model of Early Intervention Kevin Li, Dean of Instruction Sara Schupack, Director of Developmental Education Lawrence Buonaguidi, Quality Assurance Coordinator Steve Robbins, Director of Research Innovation, Academic and Workforce Success - PowerPoint PPT Presentation
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Holistic Course Placement and Whole-Student Model of Early
Intervention
Kevin Li, Dean of InstructionSara Schupack, Director of Developmental EducationLawrence Buonaguidi, Quality Assurance Coordinator
Steve Robbins, Director of Research Innovation, Academic and Workforce Success
The Innovations 2013 ConferenceMarch 12, 2013; Dallas, Texas
Part 1: Holistic Course Placement
ETS/CCC Collaboration Objectives• Combine COMPASS and Psychosocial Measure to accelerate
student progress and college completion– Students placed within the upper-band and combined with
high psychosocial scores can enroll in the next level course• Allow more individualized, actionable plans for students tied to
curricular goals• Non-cognitive survey and in-class behavioral ratings will be used
to trigger Early Alert and intervention strategies • Create supplementary services tied to Early Alert and to
behavioral monitoring
• Two new studies from the Community College Research Center (CCRC) at Teachers College, Columbia University suggest that a significant number of students may be accelerated if provided proper supports
• Placement tests alone do not yield strong predictions of how students will perform in college
• Success requires intervening with at-risk students to ensure completion and success
Background
An Alternative Measure Based on Grounded Research: Non-Cognitive Assessment
• A meta-analysis was conducted as an attempt to integrate the psychological and educational literatures regarding the postsecondary outcomes of achievement and retention (Robbins, S., Lauver, K., Le, H., Davis, D., Langley, R., & Carlstrom, A., 2004)
• Examined the relationship between psychosocial and study skill factors (PSFs) across 109 studies
• Results indicated relationships between retention and academic goals, academic self-efficacy, and academic-related skills
An Alternative Measure Based on Grounded Research: Non-Cognitive Assessment
• Respect the whole student (examine cognitive, non-cognitive, behavioral factors)
• Focus on the unique individuality of students• Focus on factors educators can control and empower students
to modify: such as motivation, social connectedness, as opposed to socio-economic, situational factors
• CCC found that behavioral monitoring and intervention was key to student success
[HAND-OUT for group exercise 1]
Group interaction 1: 5 min.
• Form a 3-4 person group• Look over the hand-outs• Discuss what “non-cognitive” or “psycho-social skills” mean to
you and how they are manifested.• How would you assess, monitor, and address these
components at your campus?• Report out
[HAND-OUT for group exercise 1]
Domain Facet Definition Example Items
Academic SkillsBehaviors, beliefs, and skills that directly facilitate academic success.
OrganizationBeing planful in one’s actions, including effective use of time management and organizational tools and strategies.
I make a schedule for getting my school work done.I take due dates seriously.
Behavioral Compliance Complying with expected, academic-related behaviors that directly impact academic success.
I attend almost all of my classes.I complete the reading that is assigned to me.
Academic Self-Efficacy Belief in one’s ability to perform and achieve in an academic setting.
I'm confident that I will succeed in my courses this semester.I can do well in college if I apply myself.
MotivationCommitment to, drive toward, and perceived importance of academic success.
Degree Goals Clear goal commitment and plan of action to achieve post secondary goals
One of my life goals is to graduate college.The benefit of a college education outweighs the cost.
Institutional Commitment Attitudes toward and attachment to college or the institution.
This is the right school for me.I’m proud to say I attend this school.
Energy & Determination Propensity to work hard in attaining academic goals. Nothing can stop me from graduating from college.I push myself very hard to succeed.
Self-Management Able to anticipate and respond to pressure and stress related to college life.
Stress Sensitivity Tendency to react to daily hassles academic stress I get stressed out easily when things don't go my way.I am easily frustrated.
Coping Using adaptive strategies to deal with academic stressors or pressures.
When feeling stressed about the amount of homework I have, I try to get organized to get on top of my homework.When worried about a test, I try to get prepared for the test regardless.
Test Anxiety Negative reactions to the pressures and experience of academic testing.
When taking a test, I think about what happens if I don't do well.Before a test, my stomach gets upset.
Social SupportPerceived availability of resources to support academic success.
Connectedness A general sense of belonging and engagement in college life.
I feel connected to my peers.People understand me.
Institutional Support The ability to identify and use resources and supports that lead to success in college.
If I don't understand something in class, I ask the instructor for help.I know how to find out what's expected of me in classes.
Instrumental Support External factors that could enable or impede academic success.
Family pressures make it hard for me to commit to school.People close to me support me going to college.
Background Information: Communicate key student information from both SuccessNavigator and SIS to faculty/advisor.
Risk Indices: Separate indices for both classroom and enrollment success. Based on background, cognitive , and psycho-social information and supported by statistical relationships with success.
Domain Scores: Four general areas of student strengths and weaknesses. Scores are presented normatively.
Feedback: Determine by more specific “facet” scores (see next page).
Action Plans: Suggested interaction with programs and services on campus.
Background Information: Communicate key student information from both SuccessNavigator and SIS to faculty/advisor.
Risk Indices: Separate indices for both classroom and enrollment success. Based on background, cognitive , and psycho-social information and supported by statistical relationships with success.
Domain Scores: Four general areas of student strengths and weaknesses. Scores are presented normatively.
Feedback: Determine by more specific “facet” scores (see next page).
Action Plans: Suggested interaction with programs and services on campus.
Background Information: Communicate key student information from both SuccessNavigator and SIS to faculty/advisor.
Risk Indices: Separate indices for both classroom and enrollment success. Based on background, cognitive , and psycho-social information and supported by statistical relationships with success.
Domain Scores: Four general areas of student strengths and weaknesses. Scores are presented normatively.
Feedback: Determine by more specific “facet” scores (see next page).
Action Plans: Suggested interaction with programs and services on campus.
The “Success Navigator” - Purpose
• Low Stakes designed to assess the dispositions and noncognitive skills of incoming college students
• Can be used with or without academic markers (SAT, HS GPA, course placement score)
• Three primary intended uses:– Identifying likelihood of persistence to degree and
0 - 20 NA Conference Required21 - 29 NA Foundational Studies 130-37 >= .8 Foundational Studies 238 - 42 NA Foundational Studies 243-49 >= .8 Reading 99
50 – 57 NA Reading 09958-64 >= .8 Reading 12565 - 72 NA Reading 12573-79 >= .8 Reading Not Required
80 and above Reading Not Required
• More accurate placement• Acceleration: Less time spent in developmental education• Help us get to know our incoming students early and
efficiently• More nuanced understanding of the whole student in a
meaningful way• Provide data to help develop placement and intervention
programs
Expected Outcomes of Holistic Placement
Part 2: Whole-Person Model of Early Alert
Whole-person model: 3 domains of assessment
Whole-student/Whole-person Early Alert Model
Math Readiness
Effort Level
High Medium Low
High .96 .91 --
Medium .94 .77 .82
Low .86 .61 .38
Note. N = 411. Success rates were not calculated for cells with insufficient sample sizes (denoted by a dash).
Research shows that behavioral monitoring and intervention are key to student success
Group interaction 2: 10 minutes• Form a group of 3-4• Look over the hand-outs.• Think about your students and your programs: What are ways
that you could best support the whole student on your campus? What collaborations across departments and units could you encourage?
• What are different ways that you can reach out to at-risk students?
• What are different ways that you can monitor students’ behaviors in order to intervene with a robust support program?
• Report out
Use the Whole Student Model to Drive Intervention Strategies
• Map support services to individual students’ needs based on non-cognitive scores
• Feedback loop to share non-cognitive report with students to arrive at action plans for students
• Consider other strategies to further student success, such as supplementary instruction, early alert and support other new and existing initiatives
[HAND-OUT for group exercise 2]
Comprehensive Early Interventions• Model and coach at-risk students toward a set of desirable behaviors.• Based on previous research (Li et al., 2012), low-level dev ed math students can
increase the likelihood of passing the course by 30% if they exhibit desirable in-class behaviors:– Active participation in groupwork (Student is actively engaged during group
work; helps other students with assignments; does his/her fair share of the work, etc)
– Active participation in lecture (Student's alert, attentive during class, asks/answers questions, etc)
– Attendance (Attends class, stays for whole period, etc)– Completion of homework assignments (Student completes assignments
thoroughly and completely, turns assignments in on time)
[HAND-OUT for group exercise 2]
Interventions: Yes we can!
• Three basic implications on early interventions:• As educators, we can motivate our students, shape and
reinforce their intentionality and academic determination. (Surrounding non-cognitive assessment)– Outward behaviors are indicators of inwards states such as
motivation and engagement– We have the opportunity and capability to coach our
students towards a set of desirable classroom/academic behaviors.
[HAND-OUT for group exercise 2]
Comprehensive Early Interventions• Map support services to non-cognitive psychological constructs:
Academic Skills, Self Management, Motivation, Social SupportNon-Cognitive construct Facet
Psycho-social Skills Refer Student to Existing natural fit support services at CCC
Academic Skills (Behaviors, beliefs and skills that facilitate success)
Social support(Perceived availability of resources)
•Connectedness•Institutional Support•Instrumental Support
Help seeking New student orientations, advisors, service referrals
Peer support Student clubs and organizations
Recommendations: The 3 Pillars of Success
• Place students more accurately using multiple measures, including non-cognitive skills for possible acceleration
• Smarter, strategic interventions: coordinate efforts to support the whole student• Institutional commitment: Faculty, advisors, and staff collaborate in early alert
program• Generate fine-grained reporting to empower frontline workers and drive further
research and program development
Behavioral Factors
Instruction and Institutional Resources
Academic Readiness
ReferencesACT, Inc. (2002). Creating seamless educational transitions for urban African American and Hispanic students. Iowa City, IA: Author.Scott-Clayton, J. (2012). Do High-Stakes Placement Exams Predict College Success? Predicting Success in College: The Importance of
Placement Tests and High School Transcripts. CCRC Working Paper No. 41. New York: Community College Research Center, Teachers College, Columbia University.
ACT, Inc. (2004). Schools involving parents in early postsecondary planning. Iowa City, IA: Author.ACT, Inc. (2008). Student Readiness Inventory user’s guide. Retrieved from ACT, Inc. website:
http://www.act.org/sri/pdf/UserGuide.pdf
ACT, Inc. (2008). What we know about college success: Using ACT data to inform educational issues. Iowa City, IA: Author.
ACT, Inc. (2008). Student Readiness Inventory Toolshop. Retrieved from ACT, Inc. website: http://www.act.org/sri/studentguide/toolshop.html
Allen, J. & Robbins, S. (2010). Effects of interest-major congruence, motivation, and academic performance on timely degree attainment. Journal of Counseling Psychology, 57, 23-35.
Allen, J., Robbins, S., Casillas, A., & Oh, I.-S. (2008). Third-year college retention and transfer: effects of academic performance, motivation, and social connectedness. Research in Higher Education, 49(7), 647-664.
American Association of Community Colleges (AACC). (2009). Community college fast facts. Retrieved July 10, 2009 from http://www2.aacc.nche.edu/research/index.htm
Belfield, C.R., Crosta, P.M. (2012). Predicting Success in College: The Importance of Placement Tests and High School Transcripts. CCRC Working Paper No. 42. New York: Community College Research Center, Teachers College, Columbia University.
References (cont.)Berkner, L., & Choy, S. (2008). Descriptive summary of 2003-04 beginning postsecondary students: Three years later. Washington,
DC: U.S. Department of Education, Institute of Education Sciences.
Bragg, D. D. (2001). Community college access, mission and outcomes: Considering intriguing intersections and challenges. Peabody Journal of Education. 76(1), 93-116.
Community College Research Center. (2002a). Accountability and learning outcomes in community colleges. Retrieved July 7, 2008, from http://ccrc.tc.columbia.edu/Seminar.asp?uid=8
Community College Research Center. (2002b). Accountability and learning outcomes in community colleges. Retrieved July 7, 2008, 2008, from http://ccrc.tc.columbia.edu/Seminar.asp?uid=8
Habley, W. & McClanahan, R. (2004). What Works in student retention – All survey colleges. Iowa City, IA: ACT, Inc.Harris, S. (1991) Can’t you guys read? Cartoons on academia. New Brunswick, NJ: Rutgers University Press.Herzog, K. & Miller, M. P. (Eds.). (1985). Scholarship: More great cartoons from the Kappan. Bloomington, IN: Phi Beta Kappa
International Inc.Le, H., Casillas, A., Robbins, S., & Langley, R. (2005). Motivational and skills, social, and self-management predictors of college
outcomes: Constructing the Student Readiness Inventory. Educational and Psychological Measurement, 65, 482-508.Li, K, & Cordes, C. (2010). Individualizing the Delivery of Math Developmental Education. Talk presented at the 34th Annual National
Association of Developmental Education Conference, Columbus, OH.Lotkowski, V., Robbins, S., & Noeth, R. (2004). The role of academic and non-academic factors in improving college retention.
Retrieved from ACT, Inc. website: www.act.org/research/policy/index.html. McCabe, R.H. (2000). No one to waste. Washington, DC: Community College Press.NCES (2004). Remedial education at degree-granting postsecondary institutions in fall 2000 (NCES 2004-010). NISOD (2012). Assessing College-readiness assessments. Webinar delivered on 3/28/2012.
References (cont.)Pascarella, E. T., & Terenzini, P. T. (2005). How college affects students: A third decade of research. San Francisco: Jossey-Bass.Peterson, C. H., Casillas, A., & Robbins, S. B. (2006). The Student Readiness Inventory and the Big Five: Examining social
desirability and college academic performance. Personality and Individual Difference, 41, 663-673.Porchea, S., Allen, J., Robbins, S., Phelps, R. (in press). Predictors of long-term enrollment and degree outcomes for community
college students: Integrating academic, psychosocial, socio-demographic, and situational factors. Journal of Higher Education.
President’s Commission on Higher Education. (1947). Organizing higher education. Washington, DC: Author.Robbins, S. B., Allen, J. Casillas, A., Akamigbo, A., Saltonstall, M., Campbell, R., Mahoney, E. & Gore, P.A. (2009). Associations
of resource and service utilization, risk level, and college outcomes. Research in Higher Education, 50(1), 101-118. Robbins, S., Allen, J., Casillas, A., Peterson, C., & Le, H. (2006). Unraveling the differential effects of motivational and skills, social,
and self-management measures from traditional predictors of college outcomes. Journal of Educational Psychology, 98, 598-616.
Robbins, S. B., Lauver, K., Le, H., David, D., Langley, R., & Carlstrom, A. (2004). Do psychosocial and study skill factors predict college outcomes? A meta-analysis. Psychological Bulletin, 130, 261-288.
Robbins, S., Oh, I., Le, H., & Button, C. (2009). Intervention effects on college performance and retention as mediated by motivational, emotional, and social control factors: Integrated meta-analytic path analyses. Journal of Applied Psychology, 94, 1163-1184.
Swail, W. S. (2004, January 23). Legislation to improve graduation rates could have the opposite effect. The Chronicle of Higher Education, B16.
Refining Course Placement Practices and Driving toward
Student Success using a Holistic Model
Correspondence regarding this presentationshould be addressed to: