NSSE: A Window into Student and Institutional Performance George Kuh University of Maine Trustees
Post on 13-Jan-2016
30 Views
Preview:
DESCRIPTION
Transcript
NSSE:NSSE:A Window into Student A Window into Student
and Institutional and Institutional PerformancePerformance
George KuhGeorge KuhUniversity of Maine TrusteesUniversity of Maine Trustees
Farmington MEFarmington MEMarch 19, 2007March 19, 2007
Commission on Future of Higher EducationCommission on Future of Higher Education
1.1. Improve student academic preparation and Improve student academic preparation and make more financial aid availablemake more financial aid available
2.2. Simplify & restructure student financial aid Simplify & restructure student financial aid system using incentives to control costs system using incentives to control costs
3.3. Cultivate a "robust culture of Cultivate a "robust culture of accountability & transparency“: invent accountability & transparency“: invent new student learning measures, make new student learning measures, make comparable information public, focus comparable information public, focus accreditation on outcomesaccreditation on outcomes
4.4. Embrace continuous innovation and Embrace continuous innovation and quality improvement. quality improvement.
5.5. Target federal investments to critical areas such Target federal investments to critical areas such as math, science, and foreign languages. as math, science, and foreign languages.
6.6. Develop a national strategy for lifelong learningDevelop a national strategy for lifelong learning
Association of American Colleges and Universities
We value what we measureWe value what we measure
Wise decisions are Wise decisions are needed about needed about whatwhat to to measure in the context measure in the context of campus mission, of campus mission, values, and desired values, and desired outcomes. outcomes.
OverviewOverview
1.1. Why student engagement Why student engagement mattersmatters
2.2. What we’ve learned from NSSEWhat we’ve learned from NSSE
3.3. Unsolicited advice for trusteesUnsolicited advice for trustees
4.4. ConversationConversation
Student Success in CollegeStudent Success in College
Academic achievement, Academic achievement, engagement in engagement in educationally purposeful educationally purposeful activities, satisfaction, activities, satisfaction, acquisition of desired acquisition of desired knowledge, skills and knowledge, skills and competencies, competencies, persistence, attainment persistence, attainment of educational of educational objectives, and post-objectives, and post-college performancecollege performance
Pre-college Characteristics Pre-college Characteristics Associated with Student SuccessAssociated with Student Success
Academic preparationAcademic preparationAbility and college-level skillsAbility and college-level skillsFamily education and supportFamily education and supportFinancial wherewithalFinancial wherewithal
32.N162FG12
Postsecondary Education OPPORTUNITY Association for Institutional Research – May 16, 2006 – Chicago, Illinois
Early College Indicators of Early College Indicators of Persistence and SuccessPersistence and Success
Goal realizationGoal realization Psycho-social fitPsycho-social fit Credit hours completedCredit hours completed Academic and social supportAcademic and social support Involvement in the “right” kinds Involvement in the “right” kinds
of activitiesof activities
Factors That Threaten Persistence and Factors That Threaten Persistence and Graduation from College Graduation from College
academically underprepared for academically underprepared for college-level workcollege-level work
first-generation college studentfirst-generation college student gap between high school and college gap between high school and college 30+ hours working per week30+ hours working per week part-time enrollmentpart-time enrollment single parentsingle parent financially independentfinancially independent children at homechildren at home
What Really Matters in College: Student Engagement
Because individual effort and involvement are the critical determinants of impact, institutions should focus on the ways they can shape their academic, interpersonal, and extracurricular offerings to encourage student engagement.
Pascarella & Terenzini, Pascarella & Terenzini, How College How College Affects StudentsAffects Students, 2005, p. 602, 2005, p. 602
Foundations of Student EngagementFoundations of Student Engagement
Time on task Time on task (Tyler, 1930s)(Tyler, 1930s)
Quality of effort Quality of effort (Pace, 1960-70s)(Pace, 1960-70s)
Student involvement Student involvement (Astin, 1984)(Astin, 1984)
Social, academic integration Social, academic integration (Tinto,1987, 1993)(Tinto,1987, 1993)
Good practices in Good practices in undergraduate education undergraduate education (Chickering & Gamson, 1987)(Chickering & Gamson, 1987)
College impact College impact (Pascarella, 1985)(Pascarella, 1985)
Student engagement Student engagement (Kuh, 1991, (Kuh, 1991, 2005)2005)
Good Practices in Good Practices in Undergraduate EducationUndergraduate Education
(Chickering & Gamson, 1987; Pascarella & Terenzini, 2005(Chickering & Gamson, 1987; Pascarella & Terenzini, 2005))
Student-faculty contactStudent-faculty contact Active learningActive learning Prompt feedbackPrompt feedback Time on taskTime on task High expectationsHigh expectations Experiences with diversityExperiences with diversity Cooperation among studentsCooperation among students
The Student Engagement TrinityThe Student Engagement Trinity
What students What students dodo -- time and energy -- time and energy devoted to educationally purposeful devoted to educationally purposeful activitiesactivities
What institutions What institutions dodo -- using -- using effective educational practices to effective educational practices to induce students to do the right induce students to do the right thingsthings
Educationally effective institutions Educationally effective institutions channel student energy toward the channel student energy toward the right activitiesright activities
NSSE’s Survey InstrumentThe College Student ReportThe College Student Report
Student Behaviors
Institutional Actions & Requirements
Reactions to College
Student BackgroundInformation
Student Learning &
Development
National Survey of National Survey of Student EngagementStudent Engagement(pronounced “nessie”)
Community College Community College Survey of Student Survey of Student EngagementEngagement(pronounced “cessie”)
College student surveys that assess the extent to which students engage in educational practices associated with high levels of learning and development
Effective Educational PracticesEffective Educational Practices
Level of Level of Academic Academic ChallengeChallenge
Active & Active & Collaborative Collaborative
LearningLearning
EnrichingEnrichingEducational Educational ExperiencesExperiences
SupportiveSupportiveCampusCampus
EnvironmentEnvironment
StudentStudentFaculty Faculty
InteractionInteraction
NSSE Project ScopeNSSE Project Scope
1,000,000+ students from 1,100+ 1,000,000+ students from 1,100+ different schools different schools
80% of 4-yr U.S. undergraduate 80% of 4-yr U.S. undergraduate FTE FTE
50 states, Puerto Rico, Canada50 states, Puerto Rico, Canada
70+ consortia70+ consortia
Documenting Documenting Good PracticeGood Practice
Institutional Institutional ImprovementImprovement
Public Public AdvocacyAdvocacy
NSSE Core PurposesNSSE Core Purposes
Customized Institutional ReportCustomized Institutional Report
OverviewOverviewInstitutional dataInstitutional dataMeans and frequenciesMeans and frequencies1st year students and 1st year students and
seniorsseniorsComparisons by peers, Comparisons by peers,
Carnegie, nationalCarnegie, nationalNational benchmarks National benchmarks Data use tipsData use tipsCD with raw data, etc.CD with raw data, etc.And more!And more!
What have weWhat have welearned so far?learned so far?
Grades, persistence, Grades, persistence, student satisfaction, student satisfaction, and engagement go and engagement go hand in handhand in hand
Does institutional size matter to Does institutional size matter to engagement?engagement?
Yes, Yes, size matters.size matters.
Smaller Smaller isis generally generally betterbetter..
Benchmark Scores for All Students by Undergraduate Enrollment
Benchmark Scores for All Students by Undergraduate Enrollment Intervals
25
35
45
55
65
0 -1000
1001 -1500
1501 -2000
2001 -2500
2501 -3000
3001 -4000
4001 -5000
5001 -7500
7501 -10000
10001 -15000
15001 -20000
20001 -25000
25001 -highest
Enrollment Intervals
Level ofAcademicChallenge
Active andCollaborativeLearning
StudentInteractionswith FacultyMembersEnrichingEducationalExperiences
SupportiveCampusEnvironment
Academic Challenge, Active Learning, Student-Faculty Interaction by Enrollment
20
30
40
50
60
186 1238 1921 3060 6337 11343 29426Enrollment
Level of Academic Challenge
Active and Collaborative Learning
Student Interactions with Faculty Members
Student engagement varies Student engagement varies more more withinwithin than between than between institutions.institutions.
Supportive Campus Environment:Supportive Campus Environment: Seniors at Master's InstitutionsSeniors at Master's Institutions
0
20
40
60
80
100
1 2 3 4 5 6 7 8 9 10 11 12 13 14
Master's Institutions
Percentile 10
Percentile 50
Percentile 90
Academic Challenge by Institutional TypeAcademic Challenge by Institutional Type
Seniors
25
30
35
40
45
50
55
60
65
70
75
DocExt
DocInt
MABacLA
BacGen Nation
First-Year Students
25
30
35
40
45
50
55
60
65
70
75
DocExt
DocInt
MA BacLA
Bac Gen
Nation
Ben
chm
ark
Sco
res
Behold the Behold the compensatorycompensatory effects of engagementeffects of engagement
Association of American Colleges and Universities
Narrow Learning is Not EnoughNarrow Learning is Not Enough——The Essential Learning OutcomesThe Essential Learning Outcomes
Knowledge of Human Cultures Knowledge of Human Cultures and the and the Physical & Natural WorldPhysical & Natural World
Intellectual and Practical SkillsIntellectual and Practical Skills
Personal and Social ResponsibilityPersonal and Social Responsibility
Integrative LearningIntegrative Learning
Integrating ideas or information from various sources
Included diverse perspectives in class discussions/writing
Put together ideas from different courses
Discussed ideas with faculty members outside of class
Discussed ideas with others outside of class
Analyzing the basic elements of an idea, experience, or theory
Essential Learning Outcome: Integrative Learning
Synthesizing & organizing ideas, info., or experiences
Making judgments about the value of information
Applying theories to practical problems or in new situations
Examined the strengths and weaknesses of your own views
Tried to better understand someone else's views
Learned something that changed how you understand an issue
NSSE DEEP LEARNING SCALE
Essential Learning Outcome: Intellectual and Practical Skills
NSSE SELF-REPORTED GAINS Acquiring job or work-related knowledge and skills
Writing clearly and effectively
Speaking clearly and effectively
Thinking critically and analytically
Analyzing quantitative problems
Using computing and information technology
Working effectively with others
Solving complex real-world problems
NSSE SELF-REPORTED GAINS Voting in local, state, or national elections
Learning effectively on your own
Understanding yourself
Understanding people of other racial and ethnic backgrounds
Developing a personal code of values and ethics
Contributing to the welfare of your community
Developing a deepened sense of spirituality
Essential Learning Outcome: Personal & Social Responsibilities
Gains in Intellectual and Practical Skills by Deep Learning Quartiles
lowest second third highest
Deep Learning Quartiles
First-Year
Senior
Very much
Quite a bit
Some
Very little
Gains in Personal and Social Responsibilities by Deep Learning Quartiles
lowest second third highest
Deep Learning Quartiles
First-Year
Senior
Very much
Quite a bit
Some
Very little
Effective Educational PracticesEffective Educational Practices
First-Year Seminars and Experiences First-Year Seminars and Experiences Common Intellectual ExperiencesCommon Intellectual Experiences Learning CommunitiesLearning Communities Writing-Intensive CoursesWriting-Intensive Courses Collaborative Assignments and ProjectsCollaborative Assignments and Projects “ “Science as Science Is Done”; Science as Science Is Done”;
Undergraduate Research Undergraduate Research Diversity/Global LearningDiversity/Global Learning Service Learning, Community-Based Service Learning, Community-Based
Learning Learning InternshipsInternships Capstone Courses and ProjectsCapstone Courses and Projects
Effects of Learning Communities on EngagementEffects of Learning Communities on Engagement
Standardized Standardized Regression Regression CoefficientCoefficient Sig.Sig.
Y-Y-Standardized Standardized
Effect sizeEffect size
Standardized Standardized Regression Regression CoefficientCoefficient Sig.Sig.
Y-Y-Standardized Standardized
Effect sizeEffect size
Engagement ActivitiesEngagement Activities
Academic EffortsAcademic Efforts .16.16 ****** .32.32 .12.12 ****** .28.28
Higher Order ThinkingHigher Order Thinking .20.20 ****** .40.40 .15.15 ****** .35.35
Academic IntegrationAcademic Integration .19.19 ****** .39.39 .16.16 ****** .38.38
Active and Collaborative LearningActive and Collaborative Learning .26.26 ****** .53.53 .24.24 ****** .54.54
Interactions with FacultyInteractions with Faculty .30.30 ****** .60.60 .22.22 ****** .51.51
Diversity ExperiencesDiversity Experiences .21.21 ****** .41.41 .16.16 ****** .36.36
Perception of Campus EnvironmentPerception of Campus Environment
Quality of Academic AdvisingQuality of Academic Advising .12.12 ****** .23.23 .08.08 ****** .17.17
Supportive Campus EnvironmentSupportive Campus Environment .19.19 ****** .37.37 .14.14 ****** .32.32
SatisfactionSatisfaction .13.13 ****** .25.25 .10.10 ****** .23.23
Learning OutcomesLearning Outcomes
Gains in Personal and Social Gains in Personal and Social .24.24 ****** .48.48 .18.18 ****** .40.40
Gains in Practical CompetenceGains in Practical Competence .22.22 ****** .45.45 .16.16 ****** .36.36General Education GainsGeneral Education Gains .18.18 ****** .36.36 .11.11 ****** .24.24
First-yearFirst-year SeniorSenior
Diversity Diversity Diversity Diversity
Density Climate In Course- Diversity Density Climate In Course- Diversity
Dependent variable Index Diversity Work Press Index Diversity Work Press
Student Engagement
Academic challenge + + + Higher order thinking + + + + + +Active and Collaborative + + + + + +Diversity-related activities + + + + + + + +Supportive Campus Env.
Supportive Campus Env. + + Interpersonal - + Support for learning + + + + + +Satisfaction - + +Gains-Interpers. Dev.
Gains - Personal/social + + + + + + +Gains-Social Awareness
Gains-Understanding div. + + + + + + + +Gains -Cont. to community + + + + + +Gains-Understand self + + + + + +
First-year students Seniors
Faculty Time Devoted to Research and Faculty Time Devoted to Research and Probability of Students Doing ResearchProbability of Students Doing Research
0
0.13
0.25
0.38
0.50
Pro
bab
ilit
y o
f U
nd
erg
rad
uate
Researc
h
2.02 6.04 10.07 14.09 18.11
Faculty Research (hours)
Faculty View of Importance of Doing Research Faculty View of Importance of Doing Research with Students and Probability of Students Doing with Students and Probability of Students Doing
ResearchResearch
0.13
0.25
0.38
0.50
0
Pro
bab
ilit
y o
f U
nd
erg
rad
uate
Researc
h
1.87 2.25 2.62 2.99 3.37
Importance of Undergraduate Research
Faculty Priorities and Faculty Priorities and Student EngagementStudent Engagement
AVGAVG STUDENTSTUDENT
AVGAVG FACULTYFACULTY AcadAcad emicemic cchallhallengeenge
ActActiveive --ccollabollab
Diversity Diversity eexperiencesxperiences
StuStudentdent --ffacacultyulty
AAccadademicemic challchall engeenge emphemphasisasis
ActActiveive --collabcollab practicespractices
EmphEmph asisasis on diversityon diversity experiencesexperiences
EmphEmph asisasis on on hhigherigher orderorder thinkingthinking
ImpImportanceortance enrichingenriching ededucuc eexpxp erienceseriences
What to Make of This? What to Make of This?
1.1. When faculty members When faculty members emphasize certain educational emphasize certain educational practices, students engage in practices, students engage in them to a greater extent than them to a greater extent than their peers elsewhere.their peers elsewhere.
2.2. Good things go togetherGood things go together
Effective Educational PracticesEffective Educational Practices
First-Year Seminars and Experiences First-Year Seminars and Experiences Common Intellectual ExperiencesCommon Intellectual Experiences Learning CommunitiesLearning Communities Writing-Intensive CoursesWriting-Intensive Courses Collaborative Assignments and ProjectsCollaborative Assignments and Projects “ “Science as Science Is Done”; Science as Science Is Done”;
Undergraduate Research Undergraduate Research Diversity/Global LearningDiversity/Global Learning Service Learning, Community-Based Service Learning, Community-Based
Learning Learning InternshipsInternships Capstone Courses and ProjectsCapstone Courses and Projects
Effective Educational Practices Increase Effective Educational Practices Increase Odds That Students Will:Odds That Students Will:
Invest time and effort Invest time and effort Interact with faculty and peers Interact with faculty and peers
about substantive mattersabout substantive matters Experience diversityExperience diversity Get more frequent feedbackGet more frequent feedback Discover relevance of their Discover relevance of their
learning through real-world learning through real-world applicationsapplications
Institutional ReflectionInstitutional Reflection
Areas of Areas of EffectiveEffective
EducationalEducationalPracticePractice
Areas of Areas of Question or Question or ImprovementImprovement
PossibilitiesPossibilities
Merge NSSE data Merge NSSE data with school recordswith school records
Identify engagement Identify engagement patterns by student patterns by student characteristicscharacteristics
Predict retention, Predict retention, degree attainment, degree attainment, grades, other grades, other outcomesoutcomes
Track student Track student engagement year to engagement year to yearyear
Compare vs. peer, Compare vs. peer, aspirational, same-aspirational, same-region schools region schools
Program evaluation Program evaluation Accreditation Accreditation Consortium and Consortium and
system data sharingsystem data sharing Student outcomes Student outcomes
researchresearch Public reportingPublic reporting A national reporting A national reporting
template?!?template?!?
NSSE Accreditation ToolkitNSSE Accreditation Toolkit
• Links Between NSSE & Accreditation
• Mapping NSSE to Accreditation Standards
• Vignettes of Institutional Usage
NSSE Questions
1.l. l. Used an electronic medium (list-serv, chat group, Internet, etc.) to discuss or complete an assignment
1.m. Used e-mail to communicate with an instructor
10.g. Using computers in academic work
11.g. Using computing and information technology
SACS Standard
3.4.14 The institution’s use of technology enhances student learning, is appropriate for meeting the objectives of its programs, and ensures that students have access to and training in the use of technology.
Questions Trustees Should AskQuestions Trustees Should Ask
• To what extent does the To what extent does the institutional mission—espoused institutional mission—espoused and enacted—unmistakably and enacted—unmistakably emphasize student success? emphasize student success?
• How does the institution’s How does the institution’s spending plan enhance student spending plan enhance student learning and success? learning and success?
• What resources are available to What resources are available to support trying out good ideas? support trying out good ideas?
Put money where it will make a Put money where it will make a difference in student engagementdifference in student engagement
“…“…in professional in professional baseball it still baseball it still matters less how matters less how much you have much you have than how well you than how well you spend it”spend it”
Endowments & Undergraduate Enrollments Endowments & Undergraduate Enrollments of Selected Public DEEP Institutionsof Selected Public DEEP Institutions
Evergreen
$2,395,656
4,171 students
Fayetteville State
$8,465,541
5,029 students
UMF
$9,391,000
2,452 students
George Mason
$40,812,720
18,091 students
Source: US News America’s Best Colleges 2007
• What institutional and student-What institutional and student-performance indicators are used performance indicators are used to inform board decisions? to inform board decisions?
• Who is responsible for Who is responsible for maintaining an institutional focus maintaining an institutional focus on student success and reporting on student success and reporting these key indicators to the these key indicators to the board? board?
• Does the institution participate in Does the institution participate in NSSE and how are the results NSSE and how are the results being used (Bok, 2006)? being used (Bok, 2006)?
• Where do students and their Where do students and their learning fall among the board’s learning fall among the board’s interests and priorities? interests and priorities?
• Is a standing committee dedicated Is a standing committee dedicated to student learning and campus to student learning and campus life?life?
• Does the culture of the board Does the culture of the board encourage encourage or or discouragediscourage open, open, productive dialogue about matters productive dialogue about matters related to student success?related to student success?
•
Why stay the course?Why stay the course?
The good-to-great-transformations The good-to-great-transformations never happened in one fell swoop. never happened in one fell swoop. There was no single defining There was no single defining action, no grand program, no one action, no grand program, no one killer innovation, no solitary lucky killer innovation, no solitary lucky break, no miracle moment. break, no miracle moment. Sustainable transformations Sustainable transformations follow a predictable pattern of follow a predictable pattern of buildup and breakthroughbuildup and breakthrough……
(Collins, 2001, p. 186)
Last WordLast Word
A college or university cannot change A college or university cannot change the lineage of its students. Campus the lineage of its students. Campus cultures do not change easily or cultures do not change easily or willingly. But we can do far more to willingly. But we can do far more to shape the way students approach shape the way students approach college and what they do after they college and what they do after they arrive. arrive.
Do we have the Do we have the willwill to use promising to use promising policies and educational practices to policies and educational practices to increase the odds that more students increase the odds that more students “get ready,” “get in,” and “get “get ready,” “get in,” and “get through?”through?”
Questions & Questions & DiscussionDiscussion
top related