Data-Driven Student Success Programming in Residence Life - ACUHO-I 2013

Data-Driven Student Success Programming in Residence Life

Presented by: Ryan O’ConnellAssociate Director, Student Success Programs

June 2013

@CampusLabsCo #labgabwww.campuslabs.com/blog

WE LIVE AND BREATHE

dedicatedemployees

90+

CHANGED THE CAMPUS FOCUS FROM COLLECTING DATA TO CONNECTING DATA

700+Member campuses and

valued partnerships

Location: Buffalo, NY & Atlanta, GAm

an

y r

oo

ted

in

hig

her

ed

DATA

CAMPUS LABS SOLUTION

One system that collects and reports data, and brings data together for new insights.

Organize

planning,

program review

and

accreditation

Assess student

learning and

success

Elevate

teaching

and learning

Retain

students

through

early alert

Create

meaningful

involvement

experiences

This session will not be a product demo.

Rather, a

of residential life promising practices,

peppered with core competencies of

assessment.

CELEBRATION

Agenda

Intentional Program Design

Binghamton University Example

Learning Outcome Development

Assessing Outcomes

Student Completion Stats

NAU’s Non-Cognitive Approach

Continuous Assessment

NASPA/ACUHO-I Benchmark Data

Supported by data,

residential program design can impact your

students’ lives and your institution's retention

rate.

INTENTIONAL

Learning

Program Program Program Program Program Program

Learning

Student learning is not the

result of discrete experiences

but rather the product of

many different kinds of

experiences in and outside

the classroom over an

extended period of time.

Student Life

Classroom

Advising

Residential Life

College Experience

College Experience

Experience

College Experience

College Experience

College Experience

College Experience

College

ExperienceCollege

Experience

College ExperienceCollege Experience

College Experience

College

Experience

CAS & Learning Reconsidered 2

6 Domains:o Knowledge acquisition, construction, integration

and application

o Cognitive complexity

o Intrapersonal development

o Interpersonal competence

o Humanitarianism and civic engagement

o Practical competence

Dimensions within each domain provide further clarification and direction.

http://www.cas.edu/getpdf.cfm?PDF=D87A29DC-D1D6-D014-83AA8667902C480B

Other Sources

General education learning outcomes

Institutional mission statement

Accreditation standards

Other professional organizations

Strategies for Developing OutcomesStrategy 1: Mission Statements

Strategy 2: Professional Organizations

Strategy 3: Student Work

Strategy 4: An Epistemological and Ethnographic Process

Strategy 5: Deep and Surface Approach to Learning

Read Peggy Maki’s "Assessing for Learning: Building a

Sustainable Commitment across the Institution”

Chapter 3: Making Claims About Student Learning Within

Contexts for Learning

Process for Developing Own Framework

1. Research

2. Reflection

3. Collaboration

4. Consensus

Read Linda Suskie’s “Assessing Student Learning: A Common Sense Guide”

Chapter 8: Developing Learning Goals

Our Suggested Process

Gather documents

Assemble core group

to draft outcomes

Send draft to

leadership for

feedback and review

Use feedback to refine

and revise

Publicize and utilize outcome

framework

Continue to review

and revise

Jennifer Hapgood, Business

Manager and Assessment Specialist

Public Northeastern University

4-year Institution

Over 14,000 Students

10% Attrition after Freshman Year

Implemented Card Swiping System to Monitor Programming

Participation

Comprehensive Data Systems in Student Affairs to track Participation in Programs

Binghamton University

Pillars of Residential Life

Global

Assisting residents to

develop relationships

with others as well as

preparing them to

become informed

global citizens

Academic

Assisting residents to

successfully navigate

the collegiate

experience and

presenting them the

greatest chance of

being a successful

student

Pillars of Residential Life

Personal

Assisting individual

residents to further their

individual growth

Community

Development

EXERCISEWrite down no more than 7 learning domains that

you think are addressed through Residential Life

at your campus.

As a small group, share your domains and agree

on 4-6 domains along with a campus name.

Program Design

• Who does the outcome pertain to?Audience/Who

• What do you expect the audience to know/be able to do?Behavior/What

• Under what conditions or circumstances will the learning occur?Condition/How

• How much will be accomplished, how well will the behavior need to be performed, and to what level?

Degree/How much

ABCD Learning Outcomes

Learning outcome examples

1. After attending a financial aid session, students will be able to accurately fill out the FAFSA form.

2. As a result of three meetings with an academic coach, students will be able to compare study strategies that is appropriate for their learning style.

3. After participating in a leadership retreat, organization presidents will be able to differentiate between tasks to complete themselves versus those to delegate to their executive board before their first spring semester planning meeting.

Co-Curricular Mapping

Staff Training

• Module inserted into normal RA/RD training

processes

• Began emphasis on domains

• Review of each Hall’s programs and

demographic make-up prior the term

Event Promotion & Tracking

Scanfobs

Used at each Residential

Life Event

Very portable and easy

to use

Attendance added to B-

Engaged easily

EXERCISEList 5 programs that your RA staff could host that

map to the outcome domains that you resolved in

the previous exercise. As a group, discuss how

you might engage in this exercise on campus.

How do you train your RA’s to be good

programmers?

Assessing Learning Outcomes

Any process employed to gather data which

requires subjects to display their knowledge,

behavior, or thought processes.

Direct Methods Indirect Methods

Any process employed to gather data which asks subjects to reflect upon

their knowledge, behaviors, or thought

processes.

List three sections that should be listed on your

resume:

I can name all of the sections that should be

included when I create my resume.

Strongly agree

Moderately agree

Neither agree nor disagree

Moderately disagree

Strongly disagree

Direct or indirect?

As a result of three meetings with an academic

coach, students will be able to compare study

strategies that are appropriate for their learning

style.

Asking students to tell you how they are studying

and comparing that to the study strategies guide

Direct!

Direct or indirect?

As a result of three meetings with an academic

coach, students will be able to compare study

strategies that are appropriate for their learning

style.

At the conclusion of the meeting, asking students

to raise their hand if they know what study

strategy they will use (and tracking that)

Indirect!

Learning Surveys

• Examples from

Delaware

Ways to Gather Survey Self Report DataAs a result of ____, I can _____:Agreement: Strongly agree, Moderately agree, Neither agree nor disagree, Moderately disagree, Strongly disagree (another version removes the “moderately” qualifier and/or uses “neutral”)

To what extent did ____ help you_____:Extent (5 pt): A great deal, Considerably, Moderately, Slightly, Not at all

How would you rate your ability to _______:Proficiency: Beginner, Developing, Proficient, Advanced, Expert

What did you learn about _____?

Can you ____?Yes, No

Beginning

1

Developing

2

Accomplished

3

Exemplary

4

Score

Assessment of the

Situation

Staff did not identify the

key areas of concern in the

case study

Staff identified some of the

key areas of concern in the

case study

Staff identified the most

significant areas of

concern in the case study

Staff identified all areas of

concern in the case study

Approach with

Student(s)

Staff recommended an

approach with the

student(s) that is

inappropriate and/or

unprofessional

Staff recommended an

approach that was

acceptable, but could be

improved

Staff recommended an

approach that was

professional and suitable

for the situation

Staff recommended an

approach that was exactly

what is expected

Referrals to Campus

Resources

Staff did not make any

referrals to campus

resources

Staff acknowledged the

resident’s need for a

referral, but didn’t provide

any specific examples

Staff acknowledged

resident need for referral

and gave appropriate

examples

Staff acknowledged

resident need for referral,

provided appropriate ones,

and offered to personally

assist resident to obtain

services

Reporting

Staff did not recommend

reporting the situation to

the appropriate person and

did not write an incident

report

Staff only reported the

situation to the RD or

wrote an incident report –

not both.

Staff reported the situation

to the supervisor and

wrote an incident report

Staff informed the RD,

wrote an incident report,

and mentioned keeping

the RD continuously

informed of any follow up

Community

Education

Staff did not recommend

any community education

as a result of the scenario

Staff mentioned

community education but

did not provide examples.

Staff recommended

specific community

education components

Staff recommended a

variety of community

education components

Follow Up

Staff did not follow up with

the student(s) involved

after the initial contact

Follow up was done, but

not appropriately

Staff followed up with the

student(s) regarding the

initial concern.

Staff followed up with the

student regarding the initial

and also maintained a

positive relationship with

the student over time.

Comments: Overall

Average:

Break

Student Completion StatsNAU’s Non-Cognitive ApproachContinuous AssessmentACUHO-I Benchmark Data

First-year retention rate

http://www.higheredinfo.org/dbrowser

77%

Student Completion

6 year college graduation rate

55%

Student Completion

http://www.higheredinfo.org/dbrowser

Student CompletionPercentage of

students who

graduate from high

school in four years,

go directly to

college, are retained

to second year, and

graduate in six

years.

http://www.higheredinfo.org/dbrowser

Public Southwestern University

4-year Institution

Over 17,000 Students on the Flagstaff Campus and 1,100

Faculty

27% Attrition after Freshman Year

Goal to increase retention rate by 10%

Student academic services unit within Student Affairs

Northern Arizona University

A Non-Cognitive Approach

Non-cognitives

Non-cognitive attributes are those academically

and occupationally relevant skills and traits that

are not specifically intellectual or analytical in

nature. They include a range of personality and

motivational habits and attitudes that facilitate

functioning well in school. Non-cognitive traits,

skills, and characteristics include perseverance,

motivation, self-control, and other aspects of

conscientiousness (Borghans et al., 2008).

Literature

Student characteristics including psychosocial factors influence retention (Brown et al., 2008; Tinto 1975, 1993).

Influence of psychosocial factors on retention extend to two-year institutions (Napoli and Wortman, 1998)

Robbins, S., Lauver, K., Le, H., Davis, D., Langley, R., & Carlstrom, A. (2004) examined 109 studies and found that psychosocial and study skill factor (noncoginitives) were predictive beyond traditional factors.

The Student Strengths Inventory Framework

Risk Scale Definition Sample Item

Academic

Success

Academic Engagement The value an individual places on

academics and attentiveness to

school work.

I turn my homework in

on time.

Academic

Success

Academic Self-Efficacy An individual’s confidence in his or

her ability to achieve academically

and succeed in college.

I will excel in my

chosen major.

Retention Educational

Commitment

An individual’s dedication to college

and the value placed upon a college

degree.

I see value in

completing a college

education.

Retention/

Academic

Success

Resiliency An individual’s approach to

challenging situations and stressful

events.

I manage stress well.

Retention Social Comfort An individual’s comfort in social

situations and ability to communicate

with others.

I am comfortable in

groups.

Retention Campus Engagement Involvement in campus activities and

attachment to the college/university.

Being active in extra-

curricular activities in

college is important to

me.

Implement a readiness program to

increase

persistence and improve student

success.

NAU’s Goal

Institution Needs

Could identify some of their “at-risk” student groups

• First generation college students

• Ethnic minorities

• Low income students

• Lower SAT/ACT scoring students

BUT needed to know

• More about which students within those groups were

most in need.

• Which services might best help which students.

NAU needed to LEARN

MORE about

each INDIVIDUAL

STUDENT.

Strategic Intervention Model:Freshman Outreach Program

Implemented early in the first semester

Include groups known to be at risk

Collaborative and not duplicative

• academic advising and first year programs

• Summer bridge program and first year mentoring

• any key groups interfacing with new freshmen

Systematic

Whole student

Grounded in research

Program Model: Student Affairs

Prioritize Outreach

Connect Evaluate

Ethnic minority students

First generation college

students

Low income students

Low academicpreparedness

Predictive Model Scores

Meeting one-one

Review results

Refer resources

Track connections

Student perspectives

Track referrals and use

Primary Data Sort

Creating lists of students to be seen by each office

“Waterfall”– Summer Bridge Students (FG, LI)

– Native American Students

– Other Ethnic Minorities

– Other First Generation/Low Income

– Other Low Scoring (low ACT or HS Core GPA)

Secondary Data Sort

Non-cognitive Results– Low Index Scores

– Low Academic Scale Scores

In House Predictive Modeling Scores

– Likelihood of Leaving

Prioritizing Student Groups

One-on-One Meetings in Student Affairs

Systematic Outreach– Look up schedules and set times– Postcard home with appointment time– Postcard to campus with appointment time– Personal e-mail with appointment time– Appointment time on their web calendar– Reminder phone call 1-2 days before appointment– If miss scheduled meeting, protocol for rescheduling

at least twice more– Social Media: Facebook

The student’s peer group is the single most potent source of

influence on growth and development during the undergraduate years.

Astin, A. What Matters in College? Four Critical Years Revisited. (p. 398). San Francisco:

Jossey-Bass, 1993.

Purpose of Meeting

Above all else, connect with student

Acknowledge student’s strengths and identify campus activities to reinforce them

Match student’s needs to campus resources

Gather some data on SSI experience

– How did you hear about your appointment?

– How seriously did you take the SSI when you completed it at orientation?

Incoming Data Fall 2010 Outcomes

Total

Students

Fall 2010 % Ret'd F11

Ave HS Core

GPA Ave ACT Ave SAT

Census Cr

Hrs

Earned Cr

Hrs Term GPA

Good Acad.

Standing

Freshman

Outreach

SA Target Met 786 74% 3.32 23 1049 15.0 13.3 2.90 88%

SA Target Not

Met 1030 67% 3.22 23 1043 14.9 12.4 2.65 80%

GSSC Target

Met 195 77% 3.01 20 949 14.9 13.3 2.78 88%

GSSC Target

Not Met 117 69% 3.12 20 956 15.1 13.2 2.84 85%

Expected to Do

Well 644 78% 3.43 26 1169 15.2 13.9 3.10 90%

No Survey 460 64% 3.17 22 1047 14.6 11.7 2.65 80%

Assessment of Approach

Provide all students with program/service information, and

target needy students for higher-level contact

EARLYDirect outreach and referrals to needed services must happen

STRATEGICALLY

< Student Success

Advisor

< Student Success

Advisor

< RA

Erin Grisham

Executive Director, Educational Support Services

EXERCISEReview the non-cognitive report of your partner

and hold a mock intervention conversation.

Graphical Representation of Results

Percentile Ranks

Result Descriptions

Recommendation Statements

The Student Report

EXERCISEReview the non-cognitive report of your partner

and hold a mock intervention conversation.

How would you approach RA training?

How could you incorporate non-cognitives into

your programming efforts?

This information seems very personal, I’m not

trained in psychology/counseling.

Should I share the risk indices with students?

What if my conversation reveals an urgent

issue?

Common Concerns

Ongoing Data Collection

Integrate key questions into your ongoing surveys

to serve as triggers for success outreach

Option to target groups of students based on

known issues (e.g., social comfort, academic

engagement)

Micro Surveys

Fall

How does it feel to be away

from home?

How are things going with your

roommate?

Have you made friends?

Spring

How do you feel about your academic performance last fall?

How much do you like your major/studies?

Academic resources are available on campus to help you succeed in your classes. Which of the following reflects your use of these support services?

Creates a time-

appropriate data

point that informs

intervention

activity.

EXERCISEList assessment items that you could embed into

operational, satisfaction, and follow-up surveys

that would be good success indicators.

As a result of ____, I can _____:Agreement: Strongly agree, Moderately agree, Neither agree nor disagree, Moderately disagree, Strongly disagree (another version removes the “moderately” qualifier and/or uses “neutral”)

To what extent did ____ help you_____:Extent (5 pt): A great deal, Considerably, Moderately, Slightly, Not at all

How would you rate your ability to _______:Proficiency: Beginner, Developing, Proficient, Advanced, Expert

What did you learn about _____?

Can you ____?Yes, No

Recap

National Data

The most important housing considerations when deciding to attend this institution

www.naspaconsortium.org

How frequently did you interact with…

www.naspaconsortium.org

What are the most common ways you learn about programs and events offered by residence life?

www.naspaconsortium.org

Programmatic Engagement

www.naspaconsortium.org

Has your involvement in residence life programming and events provided you with skills and abilities you will use after college?

www.naspaconsortium.org

Which issues have you learned more about/become more aware since living on campus/in the residence halls?

www.naspaconsortium.org

HAVE A GREAT CONFERENCE!

campuslabs.com