Improving Higher Education through Research on Students Anne Marie Delaney Director of Institutional Research Babson College.

Improving Higher Education through Research on Students

Anne Marie Delaney

Director of Institutional Research

Babson College

Introduction

This presentation offers ideas for developing a research program to:

• Enhance Effectiveness in Attracting Students

• Increase Institutional Understanding of Students

• Support Institutional Retention of Students and

• Evaluate Outcomes for Graduating Seniors and Alumni

Enhancing Effectiveness in Attracting Students with Admission Research

Inquiry, applicant and accepted student studies identify the institution’s competitive position and enhance the ability to attract high quality students.

Admission research literature offers ideas for designing admission studies.

Hossler and Gallagher (1987) – three stage model of college choice

Hossler, Braxton and Coopersmith (1989) - research important at each stage

• Research at the predisposition stage documents the importance of parental involvement.

• Research at the search stage highlights the role of admission tests and academic tracking.

• Research at the final stage can improve marketing and the selection of students.


Purpose: To increase the pool of applicants to the institution

Example: Inquiry Study of Babson College Fast Track MBA Program, 2007

Results:

• About one-half of the respondents were planning to apply to the School.

• Many cited personal satisfaction, knowledge & career advancement as goals.

• Several objected to the Graduate Management Admission Test (GMAT).

Recommendations:

• Maintain contact with those who inquire but do not apply in the near term.

• Promote the Program's ability to fulfil personal, academic and career goals.

• Evaluate whether the GMAT is appropriate for those with work experience.


Purpose: Understand students' enrolment decision and identify top competitors

Example: Fall 2007 Two-Year MBA Accepted Student Survey

Results:

• Most rated the School positively on promptness of replies and individual attention.

• Only a minority rated the admission catalogue and newspaper adv. very positively.

• Some non-enrolling students expressed concern about the quality of the students.

Recommendations:

• The Admission staff should continue to the very high level of personal service.

• Review and enhance the admission catalogue and newspaper/journal

advertisements.

• Increase opportunities for prospective students to meet with enrolled students.

Increasing Institutional Understanding of Students

Entering Freshmen

It is important to understand the characteristics, goals and aspirations of new students.

How well the institution meets expectations will affect retention and satisfaction.

The Cooperative Institutional Research Program Survey (CIRP) is a national survey for new freshmen that addresses:

• reasons for attending college;• reasons for choosing this college; • self-ratings of abilities; and• goals, values, expectations, career plans and aspirations.

Entering Freshmen

Example: Enhancing Support for Student Diversity through Research

Purpose: To understand how international and domestic freshmen differ with respect to:values and goals

Results: International students …

• put more emphasis on gaining a general education;• had lower verbal aptitude scores; and • were more committed to helping others and promoting racial understanding.

Recommendations:

• Design admission strategies to reflect international students' values.• Provide language support programs. • Help domestic students appreciate values and customs of other countries.


First Year Students

Understanding students' experience in the first year is critical to ensuring their success.

Highlights from the Literature

The global emergence of first-year programs reflects the higher education's realisation of the significance of the freshman year (Barefoot, 2000).

Empirical studies document the significant impact of first year college experiences on a range of outcomes - academic success, intellectual growth, retention and satisfaction.

The strongest predictors of 11 of 12 outcomes were the vitality of student interaction with faculty and with each other during the first year (Gerken and Volkwein, 2000).

Many challenges remain - low academic success rates; less academically challenging experiences than students expect; and inadequate attention on enhancing learning (Upcraft, Gardner & Barefoot 2005).


First Year Students

Your First College Year (YFCY) Survey is administered at the end of the first year and addresses several aspects of students' experience, including:

• perceived growth; • level of academic adjustment; • satisfaction with the academic program, student services and • satisfaction with overall college experience.

YFCY responses are linked to CIRP survey responses of entering freshmen, providing a basis for assessing changes in student characteristics, values and goals in the first year.


First Year Students

Example: The First Year in College: Understanding What Makes a Difference

Results:

• A positive social experience is crucial to student satisfaction with the first year.

• The first year experience differs by gender, with female students less comfortable.

• International and domestic students hold very distinct values and goals.

Recommendations:

• Initiate special outreach programs to ensure that female students feel integrated.

• Encourage students to develop new friendships and participate in student clubs.

• Promote opportunities for students to get to know and interact with faculty.

• Encourage students to embrace the opportunity to learn about other cultures.


First Year Students

Example: Why faculty-student interaction matters in the first year experience

Results:• Significant relationships were found between interaction with faculty and perceived growth in knowledge; academic adjustment and satisfaction with courses.

• Interaction with faculty significantly predicted academic performance, and satisfaction with faculty contact significantly predicted overall satisfaction.

Recommendations:

• Recruit and offer incentives for faculty to serve as advisors to first year students.

• Establish weekly seminars with faculty serving as chairs to lead discussions.

• Create informal social opportunities for students to get to know faculty.

• Create research assistantships for students to work closely with faculty.


Supporting Institutional Retention of Students

Perspective on Retention Research

Retention research, which has been conducted in the United States for some time, has more recently assumed prominence in Europe.

Rowley (2003) notes that recent emphasis on increased accessibility and diversity and changes in funding policies have placed retention on the agenda for UK universities.

There are two types of retention studies: a short-term analysis of the percent of students in each class returning each semester and a longitudinal study of a specific cohort.

A longitudinal study links student characteristics, admission criteria and college experiences to student performance and retention and offers more in-depth information.

Example: Designing Retention Research for Assessment and Enhanced Competitive Advantage

• The study is based on a six-year longitudinal file of 400 first-time, full-time freshmen who entered college in the 1998 fall semester.

• Data from the admission, financial aid and registrar computer system files were merged with (CIRP) Freshman Survey and the Class of 2002 Senior Survey.

Results:• Average high school grade, admission rating and first semester average college grade were significant predictors of graduation.

• Average high school grade, first semester average college grade and satisfaction with academic advising were significant predictors of final average college grade.

• Impact on intellectual self-confidence and satisfaction with faculty, business courses, and sense of community were significant predictors of overall satisfaction.


Example: Designing Retention Research for Assessment and Enhanced Competitive Advantage

Recommendations:

• Continue to emphasize the admission rating in acceptance decisions; it was an excellent predictor of graduation status and academic performance in college.

• Closely monitor freshmen’s academic performance in the first semester. It was the strongest predictor of graduation status and final grade point average.

• Enhance student satisfaction with faculty, the quality of business instruction and sense of community. These were significant predictors of overall satisfaction.


Evaluating Outcomes for Graduating Students

Graduating Seniors

• A graduating senior survey typically elicits students' evaluation of college in terms of perceived of growth in knowledge and satisfaction with programs and services. • In Expanding Students' Voice in Assessment Through Senior Survey Research, Delaney (2005) presents a model for designing senior surveys to impact policy.

• Review the institutional mission.

• Identify the goals of the undergraduate academic program.

• Define the major components of the undergraduate student life experience.

• Develop a means to evaluate academic success and satisfaction with student life.

• Design a statistical analysis plan to address planning and policy issues.

• Translate the results into recommendations for planning policy.

Graduating Seniors – Trend Study

Example: Expanding Students' Voice in Assessment through Senior Survey Research

Results:

• Perceived gains in-depth knowledge - strongest predictor of overall satisfaction.

• Satisfaction with social life -strongest predictor of choosing the same college.

Recommendations:

• Publicize students' high level of satisfaction with faculty and business courses.

• Promote the College's success in developing students' leadership and teamwork.

• Strengthen the emphasis on writing in the curriculum.

• Craft strategies to improve students' low satisfaction with campus social life.


Graduating Seniors – Comparative Study

• Assessing Undergraduate Education from Graduating Seniors' Perspective: Peer Institutions Provide the Context presents a comparative senior survey research study.

• The study was based on survey responses of 244 graduating seniors from the primary institution and about 1,500 students from 39 peer colleges and universities in

the U.S.

• Based on institutional type and U.S. News and World Report rankings, the peer institutions were classified as: first tier colleges; first tier universities; first and second

tier colleges; second, third, and fourth tier colleges; and a diverse tier group.

• This research demonstrates how a customised survey, with some items common to a consortium senior survey, was used to compare the satisfaction and evaluation

of graduating seniors at the study institution with that of their peers.


Graduating Seniors

Example: Assessing Undergraduate Education from Graduating Seniors' Perspective: Peer Institutions Provide the Context

Results: Comparative analyses identified:

• business courses, faculty, career preparation and impact on students' technological skill as strengths and

• students' knowledge of arts and foreign languages and campus social life as areas for improvement at the primary institution. Recommendations:

• Publicise students’ satisfaction with courses, faculty, and career preparation.

• Use these results as baseline data for future curriculum evaluation.

• Continue efforts to enhance campus social life.


Assessing Program Effectiveness with Alumni Studies

• Williford and Moden (1989) noted that alumni offer a unique contribution in assessing the quality of their education tempered by their experiences since graduation.

• Lee Harvey (2000) recommends alumni studies to assess how well higher education is preparing graduates with the personal and intellectual skills needed in the workplace.

• A consideration in designing an alumni study is whether the major focus should be on:

• evaluating the educational program or on • monitoring graduates' professional development.

• Recent classes should be selected if the primary purpose is to evaluate the curriculum.

Earlier classes should be chosen if the focus is on graduates’ development.

Example: Voices of Experience: Renewing Higher Education with Alumni Studies

• Model for assessing how well higher education prepares graduates for changing professional and labour market realities.

• Findings are based on a study of alumni from a highly innovative MBA Program. Results: While alumni valued the innovative curriculum, they recommended:

• increased focus on functional skills; • more balance between team work and individual work; and• expansion of the program's area of specialisation.

Recommendations:• Strengthen the emphasis on functional skills. • Intensify the rigour of technical and financial courses.• Expand the school's reputation in finance as well as in entrepreneurship.


Example: Ideas to Enhance Higher Education’s Impact on Graduates’ Lives: Alumni Recommendations

• Based on a survey of 522 alumni, the study presents a model for assessing higher education's effectiveness at the undergraduate level.

Results:• Alumni reported high satisfaction with the academic program and career preparation, but lower satisfaction with social life, sense of community and student voice in policies.

• Satisfaction with sense of community and career preparation and perceived growth in the ability to acquire new knowledge, communicate orally and understand others significantly predicted overall satisfaction.

Recommendations:

• Endeavour to create and nurture a strong sense of community on campus.

• Encourage students to participate in intercollegiate athletics and other activities.

• Publicise how well the College develops abilities critical to success in business.


Improving Higher Education through Research on Students

Conclusion

• Administrators and researchers should share a perspective on institutional research.

• While the practice of institutional research involves the production of data, the art is contextualizing the data and converting it into meaningful information (Olsen, 2000) .

• Studies should be based on sound methodology; focused on policy and be action-oriented for decision-making.

• Research has shown that assuming a pro-active role by formulating recommendations and conducting follow-up studies enhances the potential for impacting policy.

Thank you

Improving Higher Education through Research on Students Anne Marie Delaney Director of Institutional Research Babson College.

Documents

stage research

admission research purpose

prospective students

selection of students

enrolled students

admission research inquiry

admission research literature

research important