Using Student Engagement to Stimulate Change on Campus John Hayek, Ph.D. Senior Associate Director.

Using Student Using Student Engagement to Stimulate Engagement to Stimulate Change on CampusChange on Campus

John Hayek, Ph.D.John Hayek, Ph.D.Senior Associate DirectorSenior Associate Director

Advance OrganizerAdvance Organizer

What kind of information about the student experience is compelling and useful for stimulating change and improvement on campus?

OverviewOverview

NSSE Primer

Ways to Stimulate Change

Institutional Examples

Open Discussion

Foundations of Student Foundations of Student EngagementEngagement

Assessing the Student Experience

1970s

Quality of Student Effort (Pace)

1980s

Student Involvement (Astin)

Social & Academic Integration (Tinto)

Good Practices in Undergraduate Education (Chickering & Gamson)

Learning and Development Model (Pascarella)

1990s

Student Engagement (Kuh)

National Survey ofNational Survey ofStudent EngagementStudent Engagement

Started in 1999 with 12 institutions – grown to 530+ in NSSE 2005

Over a half million students (first-year students and seniors) at 850 colleges and universities (2000-2004)

Focuses on promoting effective educational practice and institutional improvement

Web and paper versions; extensively tested to ensure validity and reliability

Assesses the extent to which student are engaged in educational practices related to high levels of learning and development

Using Student Engagement To Stimulate Change

1) Link to Mission & Accreditation

2) Share Information Widely3) Enhance Faculty

Development4) Benchmark (External &

Internal)5) Connect to Outcomes and

Other Campus Data6) Emphasize Effective

Educational Practices

1. Link to Mission & 1. Link to Mission & AccreditationAccreditation

AASCU Mission StatementsKey Words

DiversityTeaching and LearningGeneral EducationInformation TechnologyLiberal ArtsCommunityProductive CitizensScholarshipResearchAcademic ExcellenceHigh Quality EducationCritical ThinkingHealth and WellnessStudent DevelopmentComprehensiveGlobal

NSSE Areas of Focus

Academic & Social Experiences

Technology

Diversity

Higher Order Thinking

Reading and Writing

Time Usage

Enriching Educational Experiences

Quality of Relationships with Students, Faculty, and Staff

Arts, Wellness, & Spirituality

Civic Engagement

Campus Environment

Advising and Mentoring

Satisfaction

Personal and Educational Growth

1. Link to Mission & 1. Link to Mission & AccreditationAccreditation

NSSE Accreditation Toolkit

Links Between NSSE & Accreditation

Mapping NSSE to Accreditation Standards

Vignettes of Institutional Usage

2. Share Information 2. Share Information WidelyWidely

“NSSE is a great way to stimulate reflection and debate about what we do more and less well, and why. For us it’s proving an exciting and enlivening tool for self-reflection and self-improvement.”

Internal External

President Governing Boards

Faculty / Committees / Deans / Chairs

Accreditation

Students / Groups / Organizations Alumni

Service Learning Prospective Students

Enrollment Management / Admissions

Media

Student Affairs / Student Services Parents

First-year Experience Fund Raising

Advising State Policy Makers

Assessment & Institutional Research Performance Indicators

Michael McPherson, President of The Spencer Foundation(former President of Macalaster College)

2. Share Information Widely2. Share Information Widely

3. Enhance Faculty 3. Enhance Faculty DevelopmentDevelopment

Mini-Grant: Early Engagement of Mini-Grant: Early Engagement of First-Year StudentsFirst-Year Students

Vice President for Academic Affairs

Identified start-up resources (up to $2,500 per year for two years)

Help academic departments introduce students engagement initiatives

Designed to improve department’s engagement of its students during their first semester at the university

• Faculty Retreats & Workshops

• Scholarship of Teaching & Learning

• Course Evaluations

• Tip of the Week

• Incentive Programs

• FSSE / NSSE Gap Analysis

3. Enhance Faculty 3. Enhance Faculty DevelopmentDevelopmentFSSE / NSSE Gap MatrixFSSE / NSSE Gap Matrix

Mass Comm 4307 Importance x Engagement Matrix

Faculty Say 50% +

“Important” or Students Say

“Very Important” Never

(14) Participate in a community-based project

(18) Talk about career plans with instructor

Faculty Say 50% +

“Important” or Students Say

“Very Important” “More than 5 times”

(1) Ask questions during class

(9) Work with classmates on projects during class

(11) Put together ideas/concepts form different courses when completing assignments

(12) Put together ideas/concepts from different courses during class discussions

(24) Analyze basic elements of an idea, experience, or theory

(25) Synthesize/organize ideas, information, or experiences

(26) Make judgments about the value of information, arguments, or methods

(27) Apply theories/concepts to practical problems

50% +

Faculty Say Students Say

“Not Important” Never

(13) Tutor/teach other students

50% +

Faculty Say Students Say

“Not Important” “More than 5 times”

(10) Work with other students outside of class to prepare assignments

(22) Discuss ideas from class with others outside of classHit

HitMiss

Miss

4. Benchmark - External4. Benchmark - External

Variable

Bench-mark Class Mean Mean Sig a

Effect

Size b Mean Sig a

Effect

Size b

1. Academic and Intellectual Experiences

FY 1.96 2.50 2.69

SR 3.11 3.10 3.18

FY 2.38 2.23 *** .48 2.24 *** .50

SR 2.65 2.74 2.45

FY 2.96 2.49 ** .29 2.44 ** .32

SR 2.53 2.66 ** .27 2.68 *** .32

FY 3.20 3.04 3.05

SR 3.27 3.35 3.35

FY 2.82 2.73 2.73

SR 2.73 2.81 2.81

FY 1.77 1.89 1.99 * -.24

SR 1.82 2.10 2.03 * -.18

FY 2.45 2.37 2.33

SR 2.68 2.51 * .20 2.44 ** .28

FY 2.05 2.32 ** -.33 2.39 *** -.41

SR 2.54 2.71 * -.19 2.73 ** -.21

FY 2.57 2.44 2.47

SR 2.68 2.84 * -.20 2.86 * -.22

ACL

ACL

ACL

ACLWorked with classmates outside of class to prepare class assignments

OCCGRP

Put together ideas or concepts from different courses when completing assignments or during class discussions

INTIDEAS

Come to class without completing readings or assignments

CLUNPREP

Worked with other students on projects during class

CLASSGRP

Worked on a paper or project that required integrating ideas or information from various sources

INTEGRAT

Included diverse perspectives (different races, religions, genders, political beliefs, etc.) in class discussions or writing assignments

DIVCLASS

Made a class presentation CLPRESEN

Prepared two or more drafts of a paper or assignment before turning it in

REWROPAP

Nesseville State UniversityNSSE 2004 Means Comparison Report

Asked questions in class or contributed to class discussions

CLQUEST

In your experience at your institution during the current school year, about how often have you done each of the following? 1=never, 2=sometimes, 3=often, 4=very often

NSSE 2004Nesseville State

Nesseville State compared with:

Master's

a.

b.

c.

d.

e.

f.

g.

h.

i.

National Survey of Student Engagement

• Consortia

•ADP

• Peer Groups

• Aspirant Groups

• Special Analyses

4. Benchmark - External4. Benchmark - External

2003 IUB Benchmark Deciles

National 0 10 20 30 40 50 60 70 80 90 100 0 10 20 30 40 50 60 70 80 90 100

Level of Academic Challenge

45 49 50 51 52 53 55 56 57 60 68 40 53 54 55 56 57 58 59 60 63 74

Active and Collaborative Learning

30 36 37 39 40 41 43 44 46 48 60 38 45 47 48 49 50 51 52 54 55 66

Student-Faculty Interaction

23 31 32 34 35 36 38 40 41 45 74 28 36 38 40 42 43 45 47 50 54 70

Enriching Educational Experiences

40 48 51 53 56 58 59 61 64 67 80 31 41 42 44 46 48 51 53 55 59 75

Supportive Campus Environment

46 55 57 59 60 62 63 65 66 69 85 45 51 54 55 57 59 60 62 64 66 76

Res/Doc-Ext 0 10 20 30 40 50 60 70 80 90 100 0 10 20 30 40 50 60 70 80 90 100

Level of Academic Challenge

47 48 49 50 51 51 53 54 55 57 62 50 52 53 54 55 55 55 57 57 59 61

Active and Collaborative Learning

33 35 35 36 37 38 38 40 41 42 47 39 43 44 45 45 46 47 47 48 49 56

Student-Faculty Interaction

29 31 32 32 33 34 34 35 36 38 44 31 36 37 38 38 39 41 42 43 44 50

Enriching Educational Experiences

48 52 53 55 56 58 59 60 61 64 71 39 43 44 45 46 47 48 49 52 54 58

Supportive Campus Environment

50 54 55 56 57 58 59 59 60 61 72 45 48 50 51 52 53 54 55 57 58 69

First-Year Senior

Level of Academic Challenge

Nesseville ConsortiumCarnegie National

First-Year 52.5 52.4 51.8 53.4

Senior 56.3 55.6 54.9 57.0

Challenging intellectual and creative work is central to student learning and collegiate quality. Colleges and universities promote high levels of student achievement by emphasizing the importance of academic effort and setting high expectations for student performance

25

35

45

55

65

75

Ben

chm

ark

Scor

es

Nesseville 52.5 56.3

Consortium 52.4 55.6

Carnegie 51.8 54.9

National 53.4 57.0

First-Year Senior

Level of Academic Challenge Items:

Preparing for class (studying, reading, writing, rehearsing, etc. related to academic program)

Number of assigned textbooks, books, or book-length packs of course readings

Number of written papers or reports of 20 pages or more; number of written papers or reports of between 5 and 19 pages; and number of written papers or reports of fewer than 5 pages

Coursework emphasizing analysis of the basic elements of an idea, experience or theory

Coursework emphasizing synthesis and organizing of ideas, information, or experiences into new, more complex interpretations and relationships

Coursework emphasizing the making of judgments about the value of information, arguments, or methods

Coursework emphasizing application of theories or concepts to practical problems or in new situations

Working harder than you thought you could to meet an instructor's standards or expectations

Campus environment emphasizing time studying and on academic work

Benchmark Act.Pred

.Residua

lStandard Residual

Academic Challenge 56.0 52.4 3.5 1.3

Active Learning 43.5 41.4 2.1 .6

Stu-Fac Interaction 42.7 35.8 6.9 1.7

Enriching Experience 50.6 50.4 .2 0.0

Supportive Environment 70.8 63.3 7.5 2.0

4. Benchmark – Internal4. Benchmark – Internal

4. Benchmark – Internal4. Benchmark – Internal

30

35

40

45

50

55

60

Lowest Major

Average

Highest Major

Business Engineering Other Education Profes-sional

Arts &Humanities

SocialSciences

BiologicalSciences

Math &PhysicalSciences

Management

ChemicalEngineering

MechanicalEngineering

CriminalJustice

Kinesiology

PhysicalEducation

Elem./MiddleEducation

Pharmacy

Pre-Med

Theater orDrama

Speech

PoliticalScience

Sociology

Biochemistry

EnvironmentalScience

Chemistry

Math

InternationalBusiness

5. Connect to Outcomes & 5. Connect to Outcomes & Other Campus DataOther Campus Data

Retention Rates to Second Year by Engagement

79

84 8582 82

8488

86 8785

91 91 9188 89

9395

9289

9292

84

9193 93

0

10

20

30

40

50

60

70

80

90

100

Level of AcademicChallenge

Academic andCollaborative Learning

Student Interactionw ith Faculty

Enriching EducationalExperiences

Supportive CampusEnvironment

Very Low Low Average High Very High

NSSE BenchmarksGrad. Rate

SR

Satisfaction

FY

Satisfaction

SR

Academic Challenge .46 .29 .28

Active & Collaborative Learning

.09 .25 .23

Student Faculty Interaction .37 .25 .29

Enriching Educational Experiences

.48 .22 .23

Supportive Campus Environment

.26 .56 .60

5. Connect to Outcomes 5. Connect to Outcomes & Other Campus Data& Other Campus Data

5. Connect to Outcomes 5. Connect to Outcomes & Other Campus Data& Other Campus Data

In-house surveys

National surveys CIRP / CSS

YFCY

CSEQ / CSXQ

EBI Benchmarking surveys

Noel Levitz Student Satisfaction Inventory

ETS Major Field Tests ACT Collegiate Assessment of Academic

Proficiency

Institutional data such as GPA, financial aid, transcripts, retention, certification tests, etc.

6. Emphasize Effective 6. Emphasize Effective Educational PracticesEducational Practices

1) A “living” mission and a “lived” educational philosophy

2) An unshakeable focus on student learning

3) Clearly marked pathways to student success

4) Environments adapted for educational enrichment

5) An improvement-oriented campus culture

6) Shared responsibility for educational quality and student success

Based on higher than predicted graduation rates and student engagement

Institutional ExamplesInstitutional Examples

Many schools are stimulating change and improvement on campus by using student engagement data.

University of Missouri – St. University of Missouri – St. LouisLouis

Glen Hahn Cope

Provost and Vice Chancellor for Academic Affairs

Using NSSE Data to Stimulate ChangeAASCU 2005 San Diego Meeting

University of Missouri University of Missouri inin St. St. LouisLouis

Urban public research university

Recent leadership and vision changes

UM System Strategic Planning UMSL Action Planning

NSSE Participation (begins with 2000 pilot)

From “administrative use only” in 2000 to

2003, 2004 campus-wide forums From “denial of results” to colleges

verifying, faculty discussions, and Action Plan benchmarks

University and Campus University and Campus Activities Activities

University System-wide Efforts UM New Faculty Teaching Scholars President’s Academic Leadership Institute

Campus-wide Efforts Center for Teaching and Learning (est. 10/2000)

Faculty and TA programs, orientations Kuh presentations with academic and student

affairs leaders, early career faculty (2/02) Engagement concepts routinely used in

program names UMSL at 40:Campus Conversation Series 2003-

2004 November: Student engagement February: Engaged research April: Community Engagement February ‘05:Creating an Engaged University

Action Planning 2003-2004 Office of the Provost created August 2004 Reliance on faculty governance, input Benchmarks with NSSE data

College ActivitiesCollege Activities

College efforts – sampling Arts and Sciences appended with

permission 15 NSSE items to fall semester 2002 course evaluations

College of Business Administration includes all majors in capstone course in its sample

Honors College oversamples its freshmen and seniors

College efforts – discussion and action Presentations about NSSE invited by

COE, CoBA A & S Dean’s charge to departments Embracing undergraduate research

OutcomesOutcomes Increased campus-wide awareness (FSSE,

NSSE) Persistent conversations Increased communication Responsibility assumed at unit level

Increased acceptance of methodology, data

Interventions identified in colleges and departments

Increased response rates NSSE FSSE2003 38.0% 26.4%2004 47.5% 44.5%

Norfolk State University (NSU)

• University of opportunity

• Wide variety of programs for students seeking access to an affordable high-quality education

• Founded in 1935

• Located in the downtown Norfolk, Virginia

• Virginia’s largest public historically black university (HBCU)

• Seventh largest HBCU in the nation

• Approximately 6,000 culturally diverse students

Surveys of Student Engagement at NSU

• Spring 2002– NSSE (paper mode)

• Spring 2003– NSSE (web-based mode)– FSSE (web-based mode)

• Spring 2004– NSSE (local administration)

• Fall 2004– BCSS (paper mode)

• Spring 2005– NSSE (web+ mode)

Using NSSE for Quality Enhancement

• NSSE Data

• NSSE Process

• NSSE Concept

Using NSSE for Quality Enhancement: Data

• Set up internal and external benchmarks to assess and monitor NSU performance on NSSE benchmarks and individual items, salient for NSU

• Provide information for internal decision-making and strategic planning

• Engage faculty, administrators, and students in conversations to explore best educational practices

Using NSSE for Quality Enhancement: Data (Cont’d)

• Advance campus initiatives

• Articulate and affirm effective institutional practices and improve NSU self-image and community perception

• Triangulate internal reports and research projects

• Triangulate external reporting

Using NSSE for Quality Enhancement: Process

• Provide information for internal decision-making

• Advance campus initiatives

• Identify effective methods to administer other university-wide surveys.

Using NSSE for Quality Enhancement: Concept

• Begin developing a more comprehensive institutional concept of academic quality

• Attract faculty and administrators’ attention to best practices in the undergraduate education

Using NSSE for Quality Enhancement: Future Plans

• Assessment of new campus initiatives– First-Year Experience

– American democracy project (ADP)

• Reaffirmation of Accreditation– Southern Association of Colleges and Schools (SACS)

Compliance Certification Audit

– SACS Quality Enhancement Plan (QEP) Theme

Contact Information

Nuria M. Cuevas, Ph.D.

Associate Vice President for Academic Affairs andDirector, Institutional Effectiveness and Assessment

NORFOLK STATE UNIVERSITY

Phone: (757) 823-8408E-Mail: [email protected]

Web: www.nsu.edu/iea

Using NSSE DataUsing NSSE Data

Sharon Hahs

ProvostFebruary 6, 2005

A Metropolitan University in a Pastoral Setting

◆ In Illinois, 20 minutes from the St. Louis Arch

◆ Only public university in Southwestern Illinois

◆ Most populous region of downstate Illinois

◆ St. Louis metro area includes 2.7 million people

With Programs in:

◆ Arts and Sciences

◆ Business

◆ Dental Medicine

◆ Education

◆ Engineering

◆ Nursing

◆ Pharmacy

◆ 13,493 Students

◆ More than half of students receiving baccalaureate degrees started as transfer students.

◆ About 30 percent of undergraduates are new each fall, two-fifths of those as transfer students.

◆ On-Campus Housing for 2,900 Students◆ Most students live and work in a 60 mile radius and commute to classes.

10,811 Undergraduate

2,485 Graduate

198 Professional

Long Term Goals ◆ Revisited Every 5 to 10 Years

◆ Measured Annually

◆ Lead to Short Term Goals

Short Term Goals◆ One to Three Years Long

◆ Founded on evidence

◆ Measured and Evaluated

◆ Lead to Actionable Projects

SIUE’s Long-Term GoalsSIUE’s Long-Term Goals

1. Engaged Student and Capable Graduates

2. Innovative High Quality Programs3. Committed Faculty and Staff4. Harmonious Campus Climate5. Active Community Engagement6. Sound Physical and Financial

Assets7. Excellent Reputation

Measures for Long Term GoalsMeasures for Long Term Goals

◆ Multiple Measures

◆ Longitudinal Data

◆ External Comparative Data

◆ Intra-Institutional Data

Has content parallel to existing alumni and faculty surveys

Annual since 2000

Urban Consortium

Can be “cut” to components within SIUE

NSSE:

Uses of NSSE in Measuring Uses of NSSE in Measuring Long Term GoalsLong Term Goals

◆ Measures of Student Engagement

◆ Measures of Faculty and Staff Commitment to Educational Opportunity

◆ Measures of Harmonious Climate

◆ Measures of Community Engagement

◆ Measures of SIUE Reputation

Freshman Seminar ProposalFreshman Seminar ProposalApril 2002April 2002

• Piloted four options:– Honors Seminars– University Experience Course– Culture, Ideas, Values Course (CIV)– Learning Communities in Academic

Development

Common goals for freshman seminars—Freshman Seminar Proposal 2002

1--To assist new freshman in making the transition to college level work and expectations;

2--To orient students to the services and culture of the University;

3--To engage students in an intellectual community of students and faculty.

Review committee and use of Review committee and use of special studyspecial study

• Ad hoc subcommittee of the Curriculum Council, Faculty Senate– Literature Search– Focus Groups– NSSE study

• Summary Report—NSSE 2003 Special Course Oversample April 19, 2004

Of the questions that showed significance at the 0.1 level, four either directly or indirectly related to the objectives of the freshman seminar course as outlined in the proposal. These included the following questions:

1 A--Asked questions class or contributed to class discussion.

10 F--Attending campus events and activities (special speakers, cultural performances, etc.)

11 K--Understanding yourself

13--How would you evaluate your entire educational experience at this institution?

(Summary Report—NSSE 2003 Special Course Oversample April 19, 2004)

New Student Seminar Task Force Report and New Student Seminar Task Force Report and Recommendations (June, 2004)Recommendations (June, 2004)

• Recommendation 1: Adopt a freshman seminar requirement

• Recommendation 2: Include a freshman seminar in general education reform.

• Recommendation 3: Create committee for implementation and management.

Question 7. Which of the following have you done or do you plan to do before you graduate from your institution?h. Culminating senior experience (comprehensive

exam, capstone course, thesis, project, etc.)

• Response to 7.h.:– Yes—70%– No—25%– Undecided—5%

Senior Assignment is a graduation requirement

Activity--NSSE questions Activity--NSSE questions related to Learning Objectivesrelated to Learning Objectives • Compare the NSSE questions with the

SIUE Statement of Objectives – 1. Identify a NSSE question that could make

a difference.– 2. What objective does this question

measure and how does the question measure it?

– 3. How is this question actionable? What could be done to improve the score?

Committee on AssessmentCommittee on AssessmentAQIP Action Project Recommendations

• Student perception and understanding of the Senior Assignment as a culminating experience (NSSE questions 2c, 7g,h, 11j,m)

• Student perception of academic advising (NSSE questions 1o, 12)

• Quantitative reasoning (NSSE questions 2b,d,e, 11f)

• Communication (speaking and writing) (NSSE questions 1a,p,q, 4c,d,e, 11c,d)

• Expectations, relationships, and diversity (NSSE questions 8, 9)

Open DiscussionOpen Discussion

www.nsse.iub.edu