How a Culture of Inquiry Can Help Catalyze Completion Efforts
Post on 24-Feb-2016
24 Views
Preview:
DESCRIPTION
Transcript
www.inquiry2improvement.com
Dr. Rob JohnstoneOhio Board of Regents
Charting the Course for Student CompletionColumbus, OH
December 3rd, 2013
How a Culture of Inquiry Can Help Catalyze Completion Efforts
www.inquiry2improvement.comNational Center for Inquiry & Improvement
Acknowledgements Much of the content in this presentation in conjunction with
national projects such as Completion by Design, the Aspen Prize for CC Excellence, and Bridging Research, Information & Cultures (BRIC)
Content has also been developed by and with a host of national partners, including:
• Community College Research Center (CCRC)• Jobs for the Future• JBL Associates• Public Agenda• The Research & Planning (RP) Group• WestEd
Infographics were primarily designed by Greg Stoup
An Applied Inquiry Framework for Student Completion – April 2013
www.inquiry2improvement.com
The Student Experience
An Applied Inquiry Framework for Student Completion
April 2013
www.inquiry2improvement.comNational Center for Inquiry & Improvement
5www.inquiry2improvement.com
A Culture of Inquiry & Action
A RESOURCE for INSTITUTIONAL change
April 2013
www.inquiry2improvement.comNational Center for Inquiry & Improvement 6
What is a Culture of Inquiry?
Institutional capacity for supporting open, honest and collaborative dialog focused on strengthening the institution and the outcomes of its students.
www.inquiry2improvement.comNational Center for Inquiry & Improvement 7
Culture of Inquiry: Features● Widespread access to user-friendly
information on student outcomes● Encouraging more people to ask a wider
collection of questions and use their evidence and conclusions to enhance decision making
● Reflective and dynamic discussions across constituency groups
www.inquiry2improvement.comNational Center for Inquiry & Improvement 8
Culture of Inquiry: More Features
● Continuous feedback so adjustments can be made along the way and processes can be adapted
● A sense of ownership over improving student outcomes – not blaming the student
● Using insight generated from inquiry to act at varying levels of the institution to create the conditions to improve outcomes
9www.inquiry2improvement.com
Applied Inquiry Framework for Student Completion
Developed for Completion by Design
A Resource for Institutional Change
www.inquiry2improvement.comNational Center for Inquiry & Improvement 10
An Applied Inquiry Framework for Student Completion
Stage 1 – Explore how to improve outcomes
Stage 2 – Gather meaningful evidence
Stage 3 – Discuss evidence broadly
Stage 4 – Use evidence to inform change
Stage 5 – Measure the impact of change
www.inquiry2improvement.comNational Center for Inquiry & Improvement 11
Explore how to improve student outcomes
Focus inquiry on designing approaches that improve student outcomes
STAGE 1
www.inquiry2improvement.comNational Center for Inquiry & Improvement
Key Questions
When was the last time you sat in a standing committee meeting on your campus that used evidence to explore a key student success outcome for more than 20 minutes?
What types of questions do we spend most of our organizational resources answering?
www.inquiry2improvement.comNational Center for Inquiry & Improvement 13
Gather meaningful evidence
Collect high-quality, meaningful evidence at the student support, classroom, program, and institutional levels
STAGE 2
www.inquiry2improvement.comNational Center for Inquiry & Improvement 14
When gathering evidence, make sure you are focusing on the right data…
www.inquiry2improvement.comNational Center for Inquiry & Improvement 15
Persistence Rate
Success Rate
1989 2008
20 year trend for California CCcourse success & persistence rates
What does that tell us about the usefulness of these metrics in setting institutional strategies?
www.inquiry2improvement.comNational Center for Inquiry & Improvement
Three Studies to Energize a Campus Conversation About Student Success
• Grades & Success in Sequenced Courses• Cohort Tracking in Developmental Education• Levels of Preparedness in GE Courses –
success of simultaneously enrolled students in Math / English courses
www.inquiry2improvement.comNational Center for Inquiry & Improvement 17
Study 1:Grades in Sequenced Courses
Note: Data tracked from 2006-07 to 2011-12
www.inquiry2improvement.comNational Center for Inquiry & Improvement
Grades in Sequenced Courses Backdrop• Course sequences analyzed in Mathematics,
English, Biology, Chemistry, Physics & Economics• Relationship between grade in 1st course in
sequence and success in 2nd course in sequence was examined.
• Enrollment data tracked from 2006-07 to 2011-12
• Cohort Ns ranged from 1,890 to 126,367
www.inquiry2improvement.comNational Center for Inquiry & Improvement
Segmentation of Students Taking Intermediate Algebra
19
A in Math 30810%
B in Math 30813%
C in Math 30812%
Unclear Status in
Math 308*3%
Math 310 Re-peater / Non
Success25%
Math 310 Attempt was First Math
Course29%
Taking Math 308 / 310 in
Same Semester8%
Other1%
www.inquiry2improvement.comNational Center for Inquiry & Improvement
Table for Intermediate Algebra
Status in Math 310 Count % of Total N
Success Rate in Math 310
A in Math 308 5,659 10% 81%
B in Math 308 7,726 13% 60%
C in Math 308 7,234 12% 37%
Unclear Status in Math 308* 1,675 3% 16%
Math 310 Repeater / Non Success 14,568 25% 41%Math 310 Attempt was First Math Course 16,812 29% 57%Taking Math 308 / 310 in Same Semester 4,505 8% 68%
Other 487 1% 44%
Total 58,666 100% 53%
20
A
B
C
The grade received by students in Math 306
Intermediate AlgebraPre-Algebra Introductory
Algebra
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Success Rate of those same students in Math 308
27%
51%
76%
Developmental Education Math Course Sequence
Math 310Math 306 Math 308
Introductory AlgebraMath 308
B
A
Transfer Level
Those placing directly into
Math 30860%
Math 1300s
A
B
C
The grade received by students in Math 308
Pre-Algebra Introductory Algebra
Intermediate Algebra
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Success Rate of those same students in Math 310
37%
60%
81%
Developmental Education Math Course Sequence
Math 306 Math 308 Math 310
Intermediate AlgebraMath 310
B
A
Transfer-Level
Those placing directly into
Math 310 57%
Math 1300s
A
B
C
The grade received by students in Math 310
Intermediate Algebra
Transfer-Level
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Success Rate of those same students in Math 1300s
44%
63%
81%
Intermediate Algebra – Transfer-Level Mathematics
MATH 310 MATH 1300s
Transfer-Level
MATH 1300s
B
AThose placing
directly into Math 1300s
66%
A
B
C
The grade received by students in Math 2413
Calculus I Calculus II
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Success Rate of those same students in Math 2414
50%
78%
90%
Calculus Course Sequence
MATH 2413 MATH 2414
Calculus II
MATH 2414
B
A Those placing directly into Math
2414
72%
A
B
C
The grade received by students in English 306
Two Levels below
Transfer
One Level below
Transfer
Transfer Level English
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Success Rate of those same students in English 307
49%
66%
77%
Developmental Education Writing Course Sequence
Those placing directly into English 307
ENGL 306 ENGL 307 ENGL 1301
One Level below
Transfer
B
A66%
A
B
C
The grade received by students in English 307
One Level below
Transfer
One Level below
Transfer
Two Levels below
Transfer
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Success Rate of those same students in English 1301
46%
61%
74%
Developmental Education Writing Course Sequence
Those placing directly into English 1301
ENGL 307ENGL 306
Transfer Level English
ENGL 1301
B
A71%
A
B
C
The grade received by students in English 1301
Composition I
Composition II
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Success Rate of those same students in English 1302
58%
77%
88%
College English Course Sequence
ENGL 1301 ENGL 1302
Composition II
ENGL 1302
B
A Those placing directly into English 1302
68%
A
B
C
The grade received by students in Biology 1406
Biology I Biology II
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Success Rate of those same students in Biology 1407
60%
76%
90%
College Biology Course Sequence
BIO 1406 BIO 1407
Biology II
BIO 1407
B
A Those placing directly into Biology 1407
70%
A
B
C
The grade received by students in Biology 2401
Anatomy Physiology
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Success Rate of those same students in Biology 2402
50%
72%
88%
College Biology Course Sequence
BIO 2401 BIO 2402
Physiology
BIO 2402
B
AThose placing directly into Biology 2402
67%
A
B
C
The grade received by students in Chemistry 1411
General Chemistry I
General Chemistry II
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Success Rate of those same students in Chemistry 1412
46%
70%
86%
College Chemistry Course Sequence
CHEM 1411 CHEM 1412
General Chemistry II
CHEM 1412
B
AThose placing directly into
Chemistry 141266%
A
B
C
The grade received by students in Physics 1401
General Physics I
General Physics II
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Success Rate of those same students in Physics 1402
87%94%
University Physics Course Sequence
PHYS 1401 PHYS 1402
General Physics II
PHYS 1402
B
A Those placing directly into Physics 1402
78%69%
A
B
C
The grade received by students in Economics 2301
Macro Economics
Micro Economics
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Success Rate of those same students in Economics 2302
74%81%
89%
Economics Course Sequence
ECON 2301 ECON 2302
Micro Economics
ECON 2302
B
A Those placing directly into
Economics 230271%
www.inquiry2improvement.comNational Center for Inquiry & Improvement 33
A Direct Look at Meaningful Analyses
from Four-Year Schools
www.inquiry2improvement.comNational Center for Inquiry & Improvement
Simple and Effective Analyses to Inform Student Progress and Completion Efforts
•Based on analyses from: Florida State University Georgia State University San Diego State University University of North Carolina – Greensboro University of Texas University of Southern California University of Wisconsin Eau Claire Virginia Commonwealth University
34
www.inquiry2improvement.comNational Center for Inquiry & Improvement
Simple and Effective Analyses to Inform Student Progress and Completion Efforts
1. Targeting at-risk populations (UNCG)2. Identifying At-Risk Students Using
Historical Data (GSU, FSU)3. Identifying At-Risk Students Using
Milestones (FSU, GSU)4. Tracking beyond Freshmen to
Sophomore Retention Rates (FSU)5. Tracking Second Year Students with
Sophomore Standing (GSU)
35
www.inquiry2improvement.comNational Center for Inquiry & Improvement
Simple and Effective Analyses to Inform Student Progress and Completion Efforts
6. Identifying Correlates of Success (VCU, UWEC, SDSU)
7. Identifying Correlates of Non-Success (USC)
8. Predictive Model for Retention (VCU, GSU)
9. Impact of Course Withdrawals (VCU)10. Additional Analyses (USC, FSU, GSU,
UWEC)
36
www.inquiry2improvement.comNational Center for Inquiry & Improvement 37
First Year to Second Year Retention, Georgia State University
3/30/10
Fall '00 Fall '01 Fall '02 Fall '03 Fall '04 Fall '05 Fall '06 Fall '07 Fall '080%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
80% 82% 81% 83% 80% 81% 82% 83% 83%
Retention
www.inquiry2improvement.comNational Center for Inquiry & Improvement 38
First Year Retention & Progression Rates - Georgia State University
3/30/10
Fall '00 Fall '01 Fall '02 Fall '03 Fall '04 Fall '05 Fall '06 Fall '07 Fall '080%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
80% 82% 81% 83% 80% 81% 82% 83% 83%
22%27% 28%
33% 35%39%
49%
62%67%
Retention Retained & Sophomore
www.inquiry2improvement.comNational Center for Inquiry & Improvement 39
Discuss evidence broadly
Engage a variety of campus stakeholders in evidence-based discussions about improvements in practice
STAGE 3
www.inquiry2improvement.comNational Center for Inquiry & Improvement 40
Data do not
speak for themselves.
www.inquiry2improvement.comNational Center for Inquiry & Improvement 41
The vital role of exploration
● In order to make data useful, ample time and space are needed to discuss and analyze the information and connect it back to the original research question.
● Answers are not always immediately apparent, so skilled facilitation may be needed to dig out the deeper meaning.
● Multiple perspectives and types of information are often needed to make sense of individual data points.
!
www.inquiry2improvement.comNational Center for Inquiry & Improvement 42
Study 3:•English & Math Preparedness &
Success in GE Courses –
Not in an English course
Engl 826
Engl 836
Engl 100
Engl 110
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
31%
48%
68%
75%Not in an English course
64%
67% 33%
Success Rate of those same students in Psychology 101
English course taking profile for students in Psychology 101
Note: Enrollments from Summer 2000 to Spring 2009; Success is defined as A/B/C/CR grade
Success in Psychology 101 for students simultaneously enrolled in an English course
English 826(Two Below)
English 836(One Below)
English 100(Transfer A)
English 110(Transfer B)
Not in a Math course
Funda-mentals
Beginning Algebra
Intermediate Algebra
Transfer Level
0%10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
43%
51%
63%
76%Not in an Math course
64%
63% 37%
Success Rate of those same students in Psychology 101
Math course taking profile for students in Psychology 101
Note: Enrollments from Summer 2000 to Spring 2009; Success is defined as A/B/C/CR grade
Success in Psychology 101 for students simultaneously enrolled in an Math course
Fundamentals Beginning Algebra
Intermediate Algebra
Transfer Level Math
0%10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
31%
48%
68%
75%
Note: Enrollments from Summer 2000 to Spring 2009; Success is defined as A/B/C/CR grade
Success in five highly enrolled GE courses by English enrollment level
Psychology 101 Speech 101 Economics 101 History 101 HSCI 101
54% 57%
73%
82%
27%
56%
73%79%
48%43%
51%
64%60%
74%
82%88%
English 826(Two Below)
English 836(One Below)
English 100(Transfer A)
English 110(Transfer B)
0%10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
48%
58%
68%
76%
Note: Enrollments from Summer 2000 to Spring 2009; Success is defined as A/B/C/CR grade
Success in five highly enrolled GE courses by English enrollment level
Fitness 334 Accounting 101 Music 202 Biology 250 Sociology 101
37%
50%
66%
73%
31%
64%
74%80%
40%
60%65%
50%
59%
74%
English 826(Two Below)
English 836(One Below)
English 100(Transfer A)
English 110(Transfer B)
40%
N/A
www.inquiry2improvement.comNational Center for Inquiry & Improvement 47
Use evidence to inform change
Implement changes in practice and policy based on analyses and discussion of college evidence
STAGE 4
www.inquiry2improvement.comNational Center for Inquiry & Improvement 48
Use evidence to guide innovation
● In this context, research and applied inquiry are fundamentally interventionist in nature.
● We are not seeking absolute truths; rather we are looking for patterns of evidence that inform action-oriented decisions.
● Failure can be seen as an opportunity for learning, especially when outcomes are shared and used to inform further improvements in practice.
The process of inquiry is not a search for an absolute truth
Domain of possible solutions
We answer the questions that eliminate dead end solutions
And what do we do when the evidence is ambiguous?
Gregory M Stoup, Cañada College
What to do when you reach the limits of your research and yet still face multiple choices?
Trust your expertise & choose !
www.inquiry2improvement.comNational Center for Inquiry & Improvement 50
Measure the impact of change
Evaluate the impact of practice changes on student performance
STAGE 5
www.inquiry2improvement.com
Exploring the Preventing Loss, Creating Momentum Framework
www.inquiry2improvement.comNational Center for Inquiry & Improvement
Loss & Momentum Framework
CONNECTION
Interest to Application
ENTRYEnrollment to Completion of
Gatekeeper Courses
PROGRESSEntry into Course of Study to 75% Requirements
Completed
COMPLETIONComplete Course
of Study to Credential with Labor Market
Value
POLICIES
PRACTICES
PROGRAMS
PROCESSES
www.inquiry2improvement.comNational Center for Inquiry & Improvement
PROGRESSENTRY COMPLETIONCONNECTION
Students never apply to college
Students delay entry into college
College counseling patterns that lead to:- under enrollment- little program-specific guidance- missed financial aid opportunities
Unstructured programs / too many choices
Extended onramps delay entry to programs of study
Students fail to enroll/pass Gatekeeper courses
Poor work-school balance
Part-time enrollment forcing long completion times
Progress not monitored / feedback givenLife events / “Stop out or drop out”
Transfer without credentialStudents accumulate credits (& debt) not aligned with completion
Never complete college level math
Credential doesn’t support needed wage & aren’t stackable
Completion by Design Framing ModelSome Known Loss Points
Poor academic preparation
www.inquiry2improvement.comNational Center for Inquiry & Improvement
PROGRESSENTRY COMPLETIONCONNECTION
foster college-going norms in High School
expand awareness of college programs and requirements
dual enrollment & AP credit
Completion by Design Framing ModelMomentum Strategies
take placement test in high school
educational planning in high school
aggressive financial aid support
accelerate entry to POS
shorter, faster, cheaper course design
effective academic catch-up programs
mandatory intrusive advising focused on programs of study
programs to incentivize optimal attendance
student progress to completion monitored & feedback provided
accelerated competency-based programs
emergency aid for students
remove barriers to graduation
Learn & Earn and Career Pathway programs
incentives to transfer with credentials
mandatory intrusive advising toward certificates degrees & transfer
First Time Student
Successful Completion
www.inquiry2improvement.com
Final Thoughts
An Applied Inquiry Framework for Student Completion – September 2013
www.inquiry2improvement.comNational Center for Inquiry & Improvement 56
Culture of Inquiry: Why All the Fuss?● Because this ongoing work is challenging but
necessary!● Work needs to marry insight & evidence● Problems are large and recurring● No silver bullet / evident answers● Multiple solutions likely needed● Progress is not linear● Requires cross constituency interaction
www.inquiry2improvement.comNational Center for Inquiry & Improvement
Next Steps• What are some “quick win” opportunities
for demonstrating the value of a culture of inquiry to practitioners and stakeholders?
• What are some of the challenges to developing or further evolving a culture of inquiry?
Completion by Design? Completion by Accident?
www.inquiry2improvement.comNational Center for Inquiry & Improvement 58
Find Out More
• The National Center for Inquiry & Improvement websitewww.inquiry2improvement.com
• Dr. Rob Johnstone, Founder & Presidentrob@inquiry2improvement.com
• CBD Inquiry Guides: http://www.inquiry2improvement.com/publications-resources
top related