Research Project

Katherine Murphy - Leah Pasquesi Allison Schipma

Loyola University Chicago College of Arts and Sciences Deans Office

Evaluation Plan

1

Katherine Murphy, Leah Pasquesi & Allison Schipma

Table of Contents Evaluation Plan 3 Theoretical Framework 4 Contextual Relevance 5 Mission 5 History and Purpose 5 Staff Structure 7 Key Goals 8 Stakeholders 9 Program Resources 9 Logic Model Description 10 Assumptions 11 External Factors 12 Purpose of Evaluation 12 Evaluation Approach 13 Quantitative Approach 14 Research Design 14 Population 15 Survey Instrument 16 Pilot Test 18 Qualitative Approach 22 Focus Group Participants 23 Focus Group Procedure 23 Interview Participants 24 Interview Protocol 25 Focus Group Implementation 26 Interview Implementation 26 Analysis of Data 27 Coding Process 28 Reliability 29 Limitations 29 Final Report 30 Timeline 30 Budget 32 Next Steps 32 References 34 Appendix A- College of Arts and Sciences Organizational Chart 35 Appendix B- Advisor Case Load Assignments 36 Appendix C- Academic Advising Learning Outcomes and Responsibilities 37 Appendix D- Deans Office Mission 39 Appendix E- Logic Model 40 Appendix F- Survey Instrument 41 Appendix G- Survey Design Considerations 49 Appendix H- Focus Group Email Invitation 50 Appendix I- Focus Group Consent Form 51 Appendix J- Focus Group Protocol 52 Appendix K- Interview Consent Form 54 Appendix L- Interview Protocol 55 Appendix M- Timeline 58 Appendix N- Budget 59 Appendix O- Evaluation Poster Presentation 60

Evaluation Plan for the College of Arts and Sciences Deans Office Academic Advising

Staff at Loyola University Chicago

The Academic Advising staff within the College of Arts and Sciences (CAS) Deans

Office at Loyola University Chicago (LUC) seeks to provide support to juniors, seniors and

transfer students in pursuit of their academic, social and personal goals. As a whole unit, the

CAS Deans Office seeks to encourage students to acquire, experience, and reflect on knowledge

in the Jesuit tradition of a transformative education. The CAS advising staff works to achieve

this through their commitment to a holistic advising philosophy. Through a variety of services

provided including one-on-one advising, drop in advising, email correspondence, and

workshops, the CAS advisors aim to teach self-awareness and responsibility to the students they

advise as students make the transition from college student to college graduate.

This evaluation project describes the context of the CAS advising office and its functions,

focusing specifically on one-on-one advising services. Additionally, it highlights the theoretical

framework for academic advising while also identifying stakeholders, resources, desired learning

outcomes, and the responsibilities of the CAS advisors, all of which provide necessary context to

inform the overall approach. This information is central to the development of the mixed method

evaluation plan that the CAS advising team plans to implement after this semester. The plan

discusses the rationale for this evaluation while also outlining the specific assessment procedures

and analysis plans. The evaluation approach has been developed in consideration of both the

students perspectives on their level of satisfaction of one-on-one advising services as well as the

perspective of the CAS advising team. The plan has been designed in an effort to remain both

clear and useful in anticipation of its implementation.

3

Theoretical Framework

The Council for the Advancement of Standards in Higher Education states that The

mission of Academic Advising Programs (AAP) is to assist students as they define, plan, and

achieve their educational goals. The AAP must advocate for student success and persistence

(2013, p. 5). The National Academic Advising Association (NACADA): The Global

Community for Academic Advising (2003) created a task force to help define the concept of

academic advising in an effort to guide professionals in the field. From this research, several

themes emerge as elements of academic advising including a dynamic relationship between

student and advisor, cultivation of goals and decision-making skills, and influence beyond the

students class schedule and four-year plan (NACADA, 2003).

Often, academic advisors are seen as those who schedule students into courses however,

advising takes a more developmental role in the students college career (NACADA, 2003). The

Council for the Advancement of Standards in Higher Education (2013) guidelines indicate that

AAPs should help students cultivate meaning in their own lives (p. 2). This goal should be

outlined in learning outcomes within each AAP.

LUCs College of Arts and Sciences AAP outlines several learning outcomes that should

be met by students, including: personal growth, the ability to effectively identify resources,

integration of curriculum into academic career and personal goals, as well as an understanding of

the importance of outside of the classroom experiences. The outcomes follow the Council for the

Advancement of Standards in Higher Education guidelines in that they are wholly

developmental; they speak to students academic, career, personal, and social development

(Council for the Advancement of Standards, 2013). The CAS advising team collaborated to

4

outline these outcomes and circulated them as an internal document. This document has been

included in the Appendices under Appendix C.

According to the Council for the Advancement of Standards in Higher Education

guidelines (2013), AAPs should regularly provide evidence that learning outcomes are being

met, thus an evaluation is needed to assess both student and advisor perceptions of outcome

achievement. This evaluation will address the effectiveness of the College of Arts and Sciences

AAP, particularly as it relates to one-on-one advising. In the context of this evaluation,

effectiveness is measured by the achievement of learning outcomes.

Contextual Relevance

When forming the assessment plan and preparing for its implementation, it is necessary

to keep in mind the context of the office and the goals and expectations of staff. Through the

process of identifying key stakeholders, our work was consistently grounded in the mission of

the advising staff. The mission is articulated below as well as in Appendix C.

Mission

The mission of Academic Advising in the College of Arts and Sciences Deans Office is

to assist students in the developmental process of pursuing and achieving their academic, social,

and personal goals. As a holistic advising office at LUC, we strive to teach students self-

awareness and responsibility while transitioning from college student to college graduate.

History and Purpose

While the focus of this evaluation will be on the CAS advising team, it is important to

situate the advising team within the larger structure of the Deans Office. The CAS academic

advising team operates within the CAS Deans Office. CAS as a whole is the oldest of LUCs

10 colleges and schools, has been established for over 140 years and includes disciplines within

5

the humanities, natural sciences, and social sciences. The mission of the Deans Office is

grounded in Jesuit tradition. The mission is articulated below and has also been provided in

Appendix D.

Through the University's Core Curriculum and its many majors and minors, the College

of Arts and Sciences is dedicated to the Jesuit tradition of a Transformative Education in

the disciplines encompassed by the humanities, natural sciences, and social sciences. We

encourage our students to acquire, experience, and reflect on knowledge, and then decide

what it means for them in terms of individual choice, action, commitment, and service to

others.

The CAS advising team has always played a crucial role in the operations of the Deans

Office as advisors are those within the office that work most directly with students. While

advising support has always been a main function of the advising staff within the Deans Office,

the operations of this team have shifted at various points to accommodate students needs and to

provide more extensive support. For instance, in the past few years, the team has developed a

workshop series entitled CAS Around Campus which is designed to provide additional support

for students. Some of the workshop offered include: Graduate School Preparation, Making

the Most of Your Engaged Learning Requirement, and Registration Nuts & Bolts, a

workshop designed to provide students who have recently transferred to LUC with additional

support with registration beyond what was provided at orientation. Additionally, CAS advisors

began offering CAS Around Campus weekly drop-in advising hours to be more accessible and

to provide advising support at multiple locations around campus.

Before highlighting the current staff structure (Appendix A), it is helpful to understand

the general history of advising in CAS. Advising of junior and senior students, including those

6

who transferred to LUC as juniors and seniors, has typically occurred within the Deans Office;

however, for a period of approximately five years, some advising responsibilities shifted. In

2004, the Office of New Student Advising was created in an effort to provide advising to all first

year students across the university in the same space. In 2005, the Office of New Student

Advising expanded to 10 advisors, becoming the Office of University Advising. This office

began advising freshman, sophomore and junior students, including all first year, sophomore,

and junior CAS students. While no formal advising team existed within CAS, senior students

were advised by the senior class coordinator within the CAS Deans office. In 2009-2010 a two-

year model was adapted across all colleges which moved advising of all junior and senior

students back into the individual schools and colleges. Thus, CAS began advising their junior

and senior students within the Deans Office. First and second year students were to be advised

by a separate office, the Office of First and Second Year Advising. During this transition, there

were three CAS Assistant Deans and four advisors. Soon after, one of the Assistant Deans left

and a fifth advisor was added. This same structure still exists in the CAS office, though there is

now a graduate assistant who assists with advising. Additionally, as of Fall 2013, transfer

students who enter LUC with more than twenty credit hours are also advised by CAS,

specifically by the graduate assistant.

Staff Structure

There are approximately 3,500 students who are advised within CAS Deans Office. As

mentioned previously, the CAS academic advising staff is housed within the Deans Office. The

five full-time academic advisors report to Joyce Knight, Assistant Dean of Advising. Each of the

full-time advisors has a caseload of approximately 550 junior and senior students which includes

students who transfer to LUC as junior and seniors in CAS. Advisor caseload assignments are

7

determined by students last names. A breakdown of the advising structure is provided in

Appendix B. The Graduate Assistant, who works 20 hours in the office per week and manages a

caseload of approximately 90 students, reports to Lester Manzano, Assistant Dean of Student

Academic Affairs and the Water Tower Campus. Currently, Leah Pasquesi, one of the members

of the evaluation team, holds this position as Graduate Assistant. In addition to their roles as

Assistant Deans, Knight and Manzano manage a caseload of approximately 200 students. Both

of the Assistant Deans report to the CAS Dean; Father Thomas Regan currently serves as the

interim Dean. Though not directly members of the advising team, several other individuals are a

part of the Deans Office, including three Associate Deans, front desk staff comprised of student

workers and office assistants, as well as other individuals whose names and titles are listed on

the organizational chart provided in Appendix A.

Key Goals

The CAS advising team has created the following key learning outcomes (Appendix C)

designed to inform its advising model:

Assist students with their personal growth in terms of developing communication,

decision-making, and problem solving skills;

Assist students with effectively identifying resources;

Assist students in understanding the relationship between their academic experiences and

their academic, career and personal goals;

Assist students in understanding the importance of including experiences outside of the

classroom into their educational plans.

With these goals in mind, each advisor aims to approach their appointments with a

developmental approach as opposed to a prescriptive approach. This means that instead of

8

having a student bring the advisor a problem and have the advisor fix it in the meeting, the

advisor will work with the student to help the student understand the process and have a sense of

responsibility in developing their academic plan.

Stakeholders

Our most relevant stakeholders are the CAS Dean, Assistant Deans and the academic

advising team. We have identified these folks as being the most relevant stakeholders as they are

most invested in advising services within CAS. In meeting with the advising team on several

occasions, they appear to be very committed and invested in this proposed assessment. As a

member of this evaluation team as well as the advising team, Leah met with the advising staff to

evaluate and update Academic Advising Learning Outcomes and Responsibilities. The finalized

list of Academic Advising Learning Outcomes and Responsibilities has been provided in

Appendix C.

Program Resources

For the sake of this evaluation, we will focus on program resources that relate to one-on-

one advising appointments. One-on-one advising appointments are currently scheduled for one

half hour. Full-time advisors leave space in their calendars for six appointments per day,

allocating up to an additional 15 minutes for preparation for each session. Advisors must also

make time for necessary follow-up to appointments including potential emails and adding notes

on students academic records through Loyolas Online Connection to University Service

(LOCUS), a system to access student account information. Front desk staff also play an

important role in the one-on-one advising process as they are responsible for scheduling

appointments and greeting students.

9

Logic Model Description

In order to ensure that the evaluation was founded in core concepts that speak to the

needs of the CAS Deans Office, we created a logic model (Appendix E) to help guide the

direction of the designed qualitative and quantitative research methods. When referencing the

model, it is necessary to connect every output, input, and goal back to the learning outcomes that

were identified by the staff within the CAS Deans Office. Though the CAS Deans Office

provides a variety of resources to students, our evaluation focuses on the specific goals

experiences of one-on-one advising appointments that are offered to junior, senior, and transfer

students within CAS. The model outlines the input basics of one-on-one advising appointments

in the first column by listing the three main components of the foundation for these

appointments; financial inputs, physical inputs, as well as inputs that take into consideration time

of each participant. Moving into the next column, the model speaks to the outputs that the inputs

have generated.

These outcomes are separated into two genres, activities and participation. Activities

considered within this model include the various advisor job functions such as one-on-one

advising appointments, graduation audits, workshop facilitation, informational orientation

sessions, and various staff development opportunities. The staff development opportunities

include weekly staff meetings as well as an advisor manual that guides their advising

practices. The participation section of the logic model names the various participants within the

evaluation. Participants of the evaluation for the CAS Deans Office include students who are

requesting the one-on-one advising appointments. The student population that the department

serves are those students whose credits identify them as junior, senior, or CAS students who

10

transfer to LUC with at least 20 credits. The participants also include office staff as well as

campus partners who may refer students to the department for an appointment.

The last piece of the logic model speaks to the short-term, medium-term, and long-term

goals of the evaluation. Each goals section is directly related to the specific learning outcomes

that were developed by the CAS Deans Office. The purpose of connecting these goals to the

learning outcomes specified by the department is to ensure that the advisors are meeting the

intended standards of the departments within their one-on-one appointments. The short- term

goals will address the immediate satisfaction of the students after their one-on-one advising

appointments. By focusing on the students satisfaction immediately after the one-on-one

advising appointments, the department is able to change any immediate needs or answer any

questions the students may have.

The medium-term and long-term goals focus on students long-term development of the

skills needed to identify campus resources and connect their current academic commitments to

their future goals and aspirations. The medium-term goals section will focus on how well

students feel prepared to identify their own, individual short and long-term goals, the students

knowledge of policies and procedures, the students ability to articulate their major or intended

field as well as the students desire to participate in undergraduate research opportunities,

community service, or study abroad experiences. The long-term goals portion focuses on the

ways in which their current academic experiences connect with their experiences after

graduation.

Assumptions

Some critical pieces of information to attend to in the logic models creation are the

assumptions being made. The effectiveness of the evaluation is centered on whether or not the

11

students are actually going to utilize their advisors for the one-on-one advising appointments. If

students are not utilizing their advisors for that intended purpose then our data may be affected

negatively. In addition to the previously stated assumption, the model assumes that the students

are able to identify that their advisors can assist them with outlined learning outcomes of the

department. A large portion of the evaluation is based on the assumption that the advisors are

seen as key figures who can be trusted when discussing the students future academic goals,

career aspirations and personal needs.

External Factors

There are three main external factors that will affect the evaluation we have created: the

financial support of the university on behalf of the support for the advising staff; the relationship

between the advisor and the student; and the follow through of the student. In terms of the

universitys support financially for the advisors, the advisors may need to spend extra time in the

office completing the interviews. Additionally, when looking at the advisor-student relationship,

we will need to attend to the comfort level that the student has with their advisor to discuss

sensitive topics such as their current academic record, future goals or difficulties they are

facing. Finally, we will need to acknowledge the follow through of the student in regards to

scheduling their follow up appointments or even attending their initial appointment once it is set

up. We will also need to focus on the honesty and comfortability of the students within their

group dialogues about their one-on-one advising experiences.

Purpose of Evaluation

The CAS Assistant Deans have identified the need to evaluate academic advising services

as no formal evaluation of services has been previously done. In consultation with the CAS

advising team, it was established that there is a need to specifically evaluate the effectiveness of

12

one-on-one advising appointments as this is the primary service provided. Because advising

sessions have two key populations involved, the advisor and the student, we feel it is important

to approach this from both the students and advisors perspectives. Therefore, our research

question seeks to understand students perceptions regarding the effectiveness of their advising

appointments in terms of how well sessions are meeting their needs. Additionally, we would like

to explore how one-on-one advising appointments meet the learning outcomes from the

perspective of the CAS advising staff.

Evaluation Approach

As previously stated, this evaluation plan is vital, as little to no assessment of advising

services has been done in this office up until this point. The evaluation will aid the advising

team in assessing their current model which may require some future adjustments that may lead

to further evaluations. We have the opportunity to approach the research from two ends of the

spectrum, both internally and externally. One of the members of this evaluation team, Leah, is a

Graduate Assistant in the office and thus is able to provide the team with a deep understanding of

the office and a thorough sense of how the advising team operates. Leah is also part of the

advising team as she advises a small caseload of CAS students. Another member of the

evaluation team, Ally Schipma, is a Graduate Intern, working in the office for 10 hours a week

specifically on assessment initiatives. Her role will help guide and facilitate our research from

both from an internal perspective and an external perspective as she is part of the office but does

not advise students. As part of her role, Ally researches existing data and benchmarks against

other institutions, which provide continuous insight for our plan. The third member of the

evaluation team, Katherine Murphy, does not work in the office and thus will be able to hold the

evaluation team accountable for the validity of the questions that are guiding our research. She

13

will also provide a fresh, external perspective on data obtained from the evaluation of current

practices in CAS.

Quantitative Approach

In order to measure the degree to which CAS learning outcomes are being met within the

context of one-on-one advising appointments, we will implement a post-appointment survey that

students will be asked to complete before leaving the CAS office. Since the CAS office

officially advises junior, senior, and transfer students, this survey will only be given to these

students. Additionally, this survey will be given only to students who make appointments, not

students who attend drop-in advising hours.

Research Design

We will be using a one-shot case study design in this evaluation. A one-shot case study

design is best for this evaluation because CAS has never engaged in formal evaluation and

therefore does not have any assessment data on record. This assessment will help to establish a

baseline of data and give the office a starting point for more evaluation in the future. A control

group is not available to study as all students receive advising through one-on-one advising, thus

making a comparison design ineffective (Creswell, 2009). Additionally, since we are examining

one specific intervention, a case study is most appropriate.

Because we are looking to survey those who come in for one-on-one advising support,

we will be doing convenience sampling. Convenience sampling is most appropriate as our data

will only reflect the sentiments and experiences of those who come in for one-on-one advising

support and thus, those who are accessible at the time of data collection. Though we are

interested in the general feedback and satisfaction level of CAS students who utilize one-on-one

advising, it is important to consider the varying needs of juniors, seniors and transfer

14

students. Therefore we will be stratifying our sample by class level in order to capture the

specific needs and experiences of different groups within our population. We recognize that this

sample may not be reflective of the overall population of students advised within CAS as it is

based on who is coming in to utilize advising services. As previously articulated, convenience

sampling is most effective in this sense as we are only interested at this time in the feedback of

students who utilize one-on-one advising services rather than attempting to capture a sample that

reflects the population in terms of seniors, juniors and transfer students. Additionally, we will

collect surveys throughout the spring semester to identify the varying appointment types and

content that may arise as a result of the timeline of an academic semester. For instance,

appointments that occur during the month February may be focused on summer course selection

whereas appointments in the month of March may be more focused on issues related to study

abroad course approvals.

Population

The population that we are intending to study consists of students who utilize CAS

academic advising. This population includes primarily junior and senior students, but may also

include non-degree seeking students as well as students who have transferred to Loyola with at

least 20 credits. It is important to note that we are specifically surveying students who utilize

one-on-one advising services, hence why our surveying will take place after a student comes in

for an appointment. After a student completes the survey, their name will be logged into a

database. By placing their name in a database, we will avoid surveying the same student twice,

but their name will not be connected in any way to the survey they complete, thereby

maintaining anonymity. We determined that it would be beneficial to survey students who have

been assigned to a CAS advisor for at least one semester, to ensure that they have properly

15

acclimated to a new advising office and have had the opportunity to meet with their academic

advisor.

Survey Instrument

The survey, which can be found in Appendix F, has a variety of questions that provide

the participants an opportunity to reflect on three main aspects of their academic advising

appointment including:

1. The process in which students set up advising appointments

2. The actual content of their one-on-one advising appointment

3. Timeframe to graduation and registration process, and

4. How the content of their discussion connects to their future needs and goals

This survey was built on the foundation of the learning outcomes set forth by the CAS Academic

Advising Office which are grounded in the advising mission and also connect to the mission of

the Deans Office.

The survey is broken down into four separate pages with each page covering a different

component of the students academic advising experience. We anticipate students needing seven

to ten minutes to complete the survey, but this will be verified in the pilot test, which will be

further discussed. We have formatted the survey with various types of questions including items

on a Likert- scale and multiple choice options. We chose to have varying Likert- scale options

so that we were able to stay specific and relevant to each question, while keeping the students

engaged throughout their time completing the survey.

The first section of the survey includes four questions that provide a framework of how

the student went about scheduling their appointment, their comfort level with reaching out

initially to the office, and the ease of finding an appointment that fit into their schedule. This

16

section is important because it provides the office data about the initial perception students have

regarding the accessibility of the office and the services that they offer. Building a positive

initial rapport is necessary in order for the office to maintain positive constructive contact with

the students at LUC.

The second section of the survey asks intentional questions about the students

satisfaction with the content of their one-on-one advising appointment and how comfortable the

student felt moving forward with the knowledge they gained about campus resources and how to

connect with them. As an office, this information will be valuable in further developing advising

appointment content and providing advisors with helpful feedback on how to structure their one-

on-one advising appointments to best serve the students needs.

The third aspect of the survey focuses on the intended graduation date of the student and

the students familiarity with the registration process. The survey focuses on the intended

graduation date of the student so that whomever is evaluating the data collected will be able to

classify the responder of the survey as either a junior or senior without making the student self-

identify. Since there is a large transfer student population, we wanted to provide the opportunity

for students to not feel pressured to identify with one class, but keep in mind their graduation

goal. We also wanted to include questions that elicit information as to how comfortable students

feel with the registration process. Since this office provides students direct support with the

registration process, we wanted to include that feedback so the advisors would be able to gauge

how comfortable students are with the registration process. If results indicate that students are

not comfortable with the registration process, this will provide the advising team with useful

information and will prompt necessary considerations of how to provide more direct support

with this process. Additionally, the follow-up question asks students to further explain why they

17

did not feel comfortable with the process which may also provide further context and useful

information.

The final piece of the survey relates directly to the specified learning outcomes that the

CAS Academic Advising Office has developed as a team. These questions all explore students

level of reflection on their values, interests, strengths, and potential challenges. When

understanding the approach to advising that the office takes, it is useful to acknowledge the

emphasis on developmental advising rather than prescriptive advising. Not only does this

advising philosophy inform the work that advisors do, but also provides the framework for the

development of learning outcomes the team created.

As stated by King (2005) developmental advising emphasizes the relationships between

students and their campus environment, focuses on the student as a whole person, and meets

students where they are at in terms of their own development. Developmental advising strives to

provide students with the support and opportunities for efficacy building in recognizing what

their academic requirements are and different ways in which the student can fulfill

them. Likewise, it means allowing the student to make any final decisions about their schedules

and take the initiative to seek out different avenues. The learning outcomes emphasize students

reflection on the various ways in which their academics connect back to their values, interests,

strengths, and potential challenges throughout their time at LUC

Pilot Test

The survey will be pilot-tested with a group of students in order to ensure that questions

and directions are relevant and easily understood, the language of the survey is accessible, and

that the questions yield usable data. We will be inviting our student employees to take the pilot

test. We will ask them to provide us with their feedback upon completing the survey. Therefore,

18

this will be done in person when the students are in the office for their shifts. They will provide

us with the feedback directly in person on an individual basis. We will pilot test our survey

towards the end of the fall semester.

Administration

The survey will be computer-based and will be accessible via a Google Forms link that

will be provided to students at the conclusion of their appointments. In the CAS Deans Office,

there are two computers available in the mailroom of the office. The front desk staff will help

administer the surveys, and will set students up on one of the computers and will ensure that

students are able to access the survey properly. It is helpful to note that these computers are in a

separate space from the offices of advisors. Students will walk down the hallway in order to

complete their survey. We acknowledge that there still may be an issue of confidentiality with

our participants responses; however, we believe that having students take the surveys in a space

that is separate from where their advising appointments take place and also having front desk

staff administer the survey will help minimize any potential pressure the students may feel when

completing the survey.

The main contact for the survey in CAS will be one of our evaluation team members,

Leah Pasquesi. We recognize that having Leah, someone who advises a small caseload of

students, as the main contact may also present issues around honesty and comfort when it comes

to collecting and coding data; however, we believe that having her as a member of both the

advising team and this assessment team is helpful in ensuring that the goals of this plan are being

achieved and also that data are properly coded in a way that will be useful and practical for the

advising team. It is also important to note in regards to confidentiality that students will not be

asked to report their names nor will they be asked to report their advisors name on the survey.

19

When the student checks in for their appointment, the front desk staff will again inform

the student that they have the opportunity to take a survey at the end of their appointment. At the

conclusion of the appointment, the front desk staff will escort the student to the computer bank

and open the survey for the student. On the first page of the survey, the student will be able to

give consent to take the survey. Once the student completes the survey, they will be free to leave

the CAS office. If the student chooses not to complete the survey, they may simply walk away

from the computer bank.

We anticipate a high level of response as the surveys will be given directly following the

appointment. We would like to avoid sending surveys to students at a later date, or even later in

the day as this may lower our response rate. Additionally, if we were to send out surveys at a

later date or time, this may also affect our results as students may only feel inclined to fill out the

survey if they were not pleased with the service they were seeking.

We will encourage the CAS advising team to build an extra fifteen minutes into their

advising sessions to afford students with the time to complete the survey. Instead of scheduling

thirty minute appointments, the front desk staff will schedule students appointments for forty-

five minutes. Students will be made aware of this time block when they schedule their

appointments.

Statistical Analysis

To analyze the survey data, we will use SPSS, a statistical analysis software available for

free use to LUC students and staff. The demographic data will be used to perform frequency

tests to show the class standing (first year, sophomore, junior, senior) of students who received

one-on-one advising and completed the survey immediately afterward. Students will self-

identify class standing using categorical labels. While class standing is officially based on credit

20

hours, most students identify themselves as junior, or senior, based on their years of

attendance at an institution. Additionally, many students are often unaware of the exact number

of credit hours they have completed. This data will be presented in a simple pie chart which is

easy to read and interpret for practitioners using the data. As there is no control group,

comparisons will not be made between control and experimental groups, but will be made across

class standing.

The survey is largely composed of Likert-type scales that measure agreement to several

statements related to the achievement of learning outcomes (i.e. ability to reflect upon values,

ability to identify resources). In presenting data from the Likert scales, we will use standard

deviations to show the statistical mean of agreement as it pertains to the achievement of

designated CAS learning outcomes. We will also use cross-tabulation to show relationships

among nominal variables, such as class standing, and ordinal variables such as level of

achievement of developmental outcomes. Finally, we will use one-way ANOVA tests to

examine continuous variables across categorical variables (Schuh, 2006). For example, we may

examine level of satisfaction across student class standing. This data will be presented in a table

format and will be helpful in training the advising team to work with different student class

levels and student type (e.g. transfer student).

Limitations

While surveys can provided a great amount of quantitative data, they come with

limitations that we will have to consider in the implementation of this plan. First, there is a

limitation with our sampling process. Because only students who make appointments with the

office will be taking the survey, we are already limiting the amount of responses. Additionally,

students may feel pressured to take the survey because it will be administered directly after their

21

appointment. One way to combat this will be to reinforce the idea that students must provide

consent and ensure the student understands this.

Another limitation of the quantitative process is connected to data analysis. While we

have knowledge of statistical tests, it will be important to have data and conclusions checked for

accuracy and understanding. It might be helpful to reach out to colleagues on campus for

this. Additionally, conclusions from quantitative data might be better explained by qualitative

data. Thus, comparing conclusions between the two data sets will be helpful in ensuring that the

survey is yielding the same understanding as the focus group questions.

Qualitative Plan

In addition to a quantitative survey, we will employ a qualitative research method

involving both advisor and student populations. Combining these methods will help ensure

triangulation and will help us develop accurate and reliable findings, from the data (Martinson

& OBrien in Wholey et al, 2010). While findings from the survey can provide an outlook on the

offices effectiveness, they are not necessarily enough to paint a complete picture as to the state

of the CAS advising team.

The qualitative approach will assist us in exploring both aspects of our research question

as it aims to capture students experiences with one-on-one advising through several focus

groups, as well as the advisors perspectives of one-on-one appointments through semi-

structured interviews. The focus groups will provide us with insight into the unique experiences

and perspectives of students, providing us with a more comprehensive understanding of how

different students experience one-on-one advising. Additionally, our qualitative approach will

provide us with a better understanding of advisors perspectives on the effectiveness of their

advising appointments and their own ability to meet the learning outcomes identified by the

22

advising team. Through our focus groups and semi-structured interviews, we are able to gain

better insight into students experiences as qualitative methods look for a range of ideas or

feelings people have and help us to understand differences in perspectives between groups or

categories of people (Schuh et al., 2009, p. 129). As Schuh (2009) acknowledges, these

qualitative approaches provide us with the opportunities to ask for clarification, details and to

follow-up on topics covered in the survey.

Focus Group Participants

All students who take the survey will have the opportunity to express interest in

participating in a focus group (see Quantitative Procedures). Of the students who indicat interest

in participating in a focus group, we will use stratified purposeful sampling to create focus

groups around student type: junior, senior, and transfer (Schuh, 2009 p. 90). Stratified

purposeful sampling allows us to coordinate groups that represent different categories of

individuals (Schuh, 2009 p. 90). By doing so, we build credibility into our qualitative method

(Schuh, 2009 p. 90). Students will be invited to participate in the focus group via email

(Appendix H). To ensure that groups are adequately represented, we will continue to reach out to

populations until our goal number of participants is met.

Focus Group Procedures

When looking to create the structure of our focus group (See Focus Group Protocol

Appendix J), we are aiming to create a setting that promotes open dialogue. We will have

participants sign a consent form (Appendix I) so they are familiar with the expectations of the

group dialogue. We chose focus groups when deciding the best procedure for our students

because we wanted to create an environment in which students felt comfortable, familiar, and

23

supported by their peers. By being surrounded by fellow students, as opposed to a one-on-one

interview with a professional staff member, we are hoping that students will be encouraged to

participate in honest and organic dialogue. To fulfill the facilitator role, we will be having our

group member Katherine step in. Since Katherine is not affiliated with the College of Arts and

Sciences Deans Office in any way visually, we believe that students will feel more comfortable

in the aspect of privacy and confidentiality that we are promising.

We are anticipating the focus group dialogue taking 45 minutes to 60 minutes to

complete. We are looking to have three separate focus groups that hold students from each of the

three major student populations that the College of Arts and Sciences Deans Office works

with. As previously stated, the office works with transfer, junior and senior students; so when

we are building our focus groups we will be mindful to keep students from each of those

populations together (as identified by the students). By grouping each student into their

respective groups we are hoping to again provide a sense of comfort that will allow for open and

honest dialogue through shared experiences. During our focus group dialogue we will be

covering two main aspects of the students one-on-one advising experience: the students overall

sentiments about advising appointments and their feedback about the content of advising

appointments.

Interview Participants

Since the CAS Academic Advisors are a very small community of five full-time advisors,

we will attempt to interview all academic advisors (a census) currently employed in the office

(Schuh, 2009 p. 56). Because we need to ask probing, open-ended questions, we decided it is

better to interview advisors individually in a semi-structured format (Wholey et al, 2007 p. 367).

24

Since advisors are employees of the office conducting the survey, they will be strongly

encouraged to participate in the interview, but will of course have the opportunity to decline

participation. While we recognize the ethical implications of strongly encouraging participation,

making efforts to improve the office is a condition of employment. Interview participants must

still give informed consent and will be given the option to pass on any question in the

interview (Appendix K).

Interview Protocol

For the interview protocol portion of our assessment, we will be using a semi-structured

interview process because they are most effective when using with probing, open-ended

questions (Shuh, 2009). We have chosen to do private individual interviews so that we are

respecting the confidentiality of the advisors and giving them the opportunity to be open and

candid with their thoughts and opinions. By providing a safe space for the advisors to speak

freely, we believe that we will receive organic and valid answers from each participant. We are

anticipating that each interview will take between 45 minutes to 60 minutes to complete in

full. We are hoping that the dialogue will be more discussion-based than interview

structured. The reason that we are striving for this dynamic is once again focused on creating a

comfortable and open space that promotes honest dialogue. To create a comfortable dialogue, we

will have the external evaluator, Katherine, conduct the interviews. Since Katherine is not

affiliated with CAS, she will be able to create a comfortable, unbiased space for advisors.

During our advisor interviews we will be focusing on four main pieces of one-on-one

advising appointments from the advisors role:

1. The role of the advisors within the appointment

25

2. The advising mission and how that relates to how they advise

3. The learning outcomes of advising appointments and how they are met, and

4. The relationship between the advisor and students

By covering these areas we are hoping to capture feedback from the advisors on all aspects of the

advisors experiences with one-on-one advising appointments.

Focus Group Implementation

All student participants will be asked to report to a room on campus that is not housed

within the CAS. Depending upon availability, this may be a classroom or a conference room.

Either the CAS intern or the external evaluator (Katherine) will moderate the focus group. The

evaluation team will also train CAS student workers as note takers. These students will also sign

a confidentiality agreement. Note takers will summarize their notes with the group after each

segment to ensure clarity and validity.

Focus groups will be held once per month (for a total of three months) to serve two

purposes: 1) to capture students who had a variety of academic advising appointments and 2) to

allow time for processing and coding data. Each session will last approximately one hour and

will consist of 10-12 participants. The sessions will be audio-recorded to preserve the rich

transcript data. However, all data will be coded and reported so that identifying information is

not revealed.

Interview Implementation

We will be implementing a semi-structured interview process for all academic advisors.

This format was chosen as the best option for qualitative data collection because we need to

26

conduct a formative program evaluation and want one-on-one interviews with key program

managers, staff, and frontline service personnel (Adams in Wholey et al, 2010 p.367). We feel

that advisors may not feel as comfortable being open and honest if they are in a focus group with

their co-workers.

Academic advisors will be asked to report to a conference room located outside the CAS.

We hope that physical separation from the office will encourage honesty and provide a

comfortable space for the staff. The external evaluator will conduct the interviews so that the

staff feels free to speak candidly and honestly about their experience in CAS. Interviews will

occur toward the middle of the semester, around week seven or eight, as it is a slower time for

advisors. The interviews will last 45 minutes to one hour. Interviews will be audio-recorded to

preserve the rich data, however all coded and reported data will not reveal any identifying

information. Throughout the interview, the interviewer will remind the advisor of this and

encourage honest answers. Additionally, the interviewer will summarize the main points of each

segment of answers to ensure clarity and validity. Since CAS is such a small office, we will

make sure to use these summary statements as opposed to verbatim statements to avoid

identification of sources.

Analysis of Data

Because we will be audio taping our interview and focus groups we will be transcribing

both of these. In preparation for the interview and focus groups, we will be utilizing our logic

model to develop coding schemas that will help us prepare for our analysis. We utilized our

logic model when developing coding schemas so there is consistency when comparing and

considering qualitative and quantitative data. We will use descriptive coding. As Rogers &

27

Goodrick in Wholey et al (2010) write, Descriptive coding...involves tagging text to identify the

topic or issues contained in that text, (p. 438). This is most appropriate as this approach will

assist us in managing the volume of data, making it easier to retrieve and aggregate data as it

relates to particular topics. Additionally, this will assist us in identifying sub-categories that

connect to the broader coding categories. Because we need to code both our focus groups and

interviews, we felt that descriptive coding would be a helpful approach in tagging several

sources. As Wholey, Hatry, and Newcomer (2010) identify, this will allow us to tag serially,

working through each transcript separately but with some consistency. The CAS intern and

external evaluator will work together to code the data. The CAS intern will be able to shed light

on the meaning of terminology and phrases used in the sessions, while the external evaluator will

be to maintain the integrity of the coding process as an outside source.

Coding Process

Because we have multiple sources of data, it is important to first organize and prepare our

data. Audio tapes from both the interview and focus group will need to be transcribed. As our

current budget stands, we will not be hiring outside help with transcribing nor do we currently

have access to a transcribing machine. To reference the budget, see Appendix N. If the budget

allows, we may end up using a transcribing machine and hire a professional. As such, it will be

helpful for us to transcribe the interview and focus groups ourselves so we can review data

throughout the process. We will make and keep a logbook as way to keep track of our analysis

as we go, which will also serve as an audit trail (Rogers & Goodrick in Wholey et al, 2010).

We will read through all information to get an overall sense of our data. In identifying

themes or categories, we will do focused coding as we review the data to see if topics can be

combined to create a new topic or to determine whether macro topics can be identified. We will

28

create categories or themes for topics in order to identify relationships between the data, topics,

and categories. With focused coding, we will be able to use themes in our data to draw

conclusions and make inferences about academic advising and its role in helping students meet

learning outcome goals. Once the codes are developed, we will create a coding rubric to aid in

the coding process and to ensure reliability.

Validity and Reliability

Our qualitative approach allows for triangulation of data and will assist us in overcoming

potential biases. Additionally, as there are three of us who will be analyzing this data, there will

be multiple investigators collecting data that can prevent potential bias from occurring. This

checking of coding will increase inter-rater reliability (Rogers & Goodrick in Wholey et al,

2010). We also feel it is important to do some member checking throughout the semi-structured

interview and focus group processes. Member checking will involve providing summary

statements to our participants to ensure we are accurately capturing their voices and ideas

(Rogers & Goodrick in Wholey et al, 2010).

Limitations

We acknowledge that in having some stake in the evaluation project that there is a

possibility for bias when writing questions, facilitating, and interpreting our results. We feel that

having three evaluators assisting with the process will be in our favor as we can hold each other

accountable. Similarly, we intend to actively hold one another accountable, check assumptions,

and maximize our interpretations of data as it relates to this assessment, particularly in Leahs

case as she is a member of the advising team.

We also acknowledge that descriptive coding can take considerable time and hard work

in order to remain consistent and stay focused on our evaluation questions while also being

29

attentive to unexpected findings. Again, playing to our strength of having three evaluators will

be a helpful way to ensure that we are being collectively attentive to inter-rater reliability and

validity.

Final Report

Through our final report, we intend to incorporate interpretations and identified themes

that emerge during data analysis. The report will serve primarily as a place to share themes that

were revealed from the data while also making sure to capture the multiple perspectives that

emerge from students as well as advisors. When it is important to capture the voices or specific

language of participants, we will use direct quotations. In these instances, we will ensure that the

quotations are not identifying in any way. Paraphrasing quotations that might be identifying will

be a way to ensure confidentiality. We will be sure to vary the length of quotations in an effort

to both maximize space in our final report and also to be stylistically appealing. We also plan on

employing many of the writing strategies suggested by Creswell (2009) including intertwining

quotations with the authors interpretations, and using first-person through the collective we to

capture the voices of the authors. We will also be intentional in acknowledging our positionality

and how our social identities and roles inform and sometimes influence our ability to serve as

evaluators. Finally, we will present text in tabular form, particularly through the usage of a table

to display the different codes used.

Timeline

To implement this evaluation project in an efficient manner, we have created a timeline

(see Appendix M) to track implementation segments. A full cycle of the evaluation will take one

full semester. In the future, however, the timeline may be shortened to do smaller evaluation

30

check-ins as needed. For the purposes of this project, the evaluation will take place in the

Spring 2015 semester.

Beginning in January 2015, information about the evaluation project will be disseminated

to the CAS Deans Office staff. Administrative roles will be assigned to full-time and student

staff members managing the front desk, while advisors will learn about their role in interviews

and encouraging students to participate. The students will begin taking the survey starting the

second week of classes (January 20). We are choosing to bypass the first week of classes as that

is a busy time in the office, and it would be challenging to manage the evaluation process while

also attending to the extremely high volume of students. While many students meet with

advisors during the first week, these appointments are less developmental and are intended to

troubleshoot registration and enrollment problems only.

As mentioned previously, students will be invited to take the survey when they sign up

for an advising appointment. After a three-week survey period, we will extend a focus group

invitation email to all students who participated in the survey process. We will hold the focus

group during the fourth week. This process will be repeated three times throughout the

semester. The student evaluation piece will be complete by May 1. Data from the survey and

focus groups will be coded and analyzed during the month of May by the CAS graduate assistant

and intern, Leah Pasquesi and Ally Schipma. During the summer the CAS Deans Office team

will have time to look over the report and create a plan for next steps.

We will hold interviews with advisors during the sixth and seventh week of the semester

(approximately February 16-27). Traditionally, these weeks are a slower period for

advisors. Since the interviews will be conducted during the business day, we want to be sure that

we are respectful of advisors schedules and caseload. Katherine will code and analyze the

31

interview data throughout the month of March and April, in order to supply it to the CAS Deans

Office before the end of the school year.

Budget

We have included a budget to provide the CAS advising time with an estimate for the

cost of implementing this plan. The budget (Appendix P) covers the projected costs for

implementation of this plan in the Spring 2015. Further plans for evaluation would need

additional budgetary considerations. The budget for the evaluation plan is rather small as the

plan was developed in a way to be cost effective. Where most money is intended to be spent is

for the focus group, in order to provide incentives for participation as they require students to

give up 45 minutes to an hour of their time. As such, pizza and soda will be provided. Since the

CAS advising team operates within LUC, the office has access to several resources that help to

significantly cut costs. Included in this are audio recording devices as well as spaces to conduct

interviews and focus groups that can be reserved through LUC. Additionally, SPSS (the

statistical analysis software) is available for free use by LUC students, faculty, and staff, and,

will be used to analyze the survey results.

Next Steps

This plan will be next be proposed to the CAS team in an upcoming staff meeting. While

an interest has been shown in an assessment for the office, the team will need to consider how it

wants to move forward with assessment implementation. While focus groups and advisor

interviews will yield helpful insights, they require time and manpower to be implemented. It

may first be more realistic to implement the survey immediately, while postponing focus groups

and interviews or re-formatting the process to better suit the needs of the office (if the proposed

timeline is not feasible).

32

Additionally, the CAS office may benefit from other types of assessment in the

future. While the goal of this evaluation is to assess the effectiveness of one-on-one advising

appointments, it is only targeted to those students who participate in one-on-one advising. For

example, it might be a good idea to design an evaluation assessing other types of advising (i.e.

drop-in, orientation) or students who do not engage with the office on a regular basis. Since the

office does not have any assessment data on file, there is a need to gather more information about

all advising interventions conducted in CAS.

33

References

Council for the Advancement of Standards in Higher Education. (2013). Academic advising

programs: CAS standards and guidelines. Retrieved from:

http://standards.cas.edu/getpdf.cfm?PDF=E864D2C4-D655-8F74-

2E647CDECD29B7D0

Creswell, J.W. (2009). Research design: Qualitative, quantitative, and mixed method approaches

(3rd Edition). Thousand Oaks, CA: Sage Publications.

NACADA. (2003). Paper presented to the Task force on defining academic advising. Retrieved

from NACADA Clearinghouse of Academic Advising Resources Web site:

http://www.nacada.ksu.edu/Resources/Clearinghouse/View- Articles/Definitions-of-

academic-advising.aspx

King, M. C. (2005). Developmental academic advising. Retrieved from NACADA Clearinghouse

of Academic Advising Resources Web

Schuh, J.H. & Associates (2009). Assessment methods for student affairs. San Francisco, CA:

Jossey-Bass

Wholey, J.S., Hatry, H.P., & Newcomer, K.E. (Eds.) (2010). Handbook of practical program

evaluation (Third Edition). Jossey-Bass: San Francisco.

34

http://standards.cas.edu/getpdf.cfm?PDF=E864D2C4-D655-8F74-

Thomas J. Regan, S.J.Thomas J. Regan, S.J.Thomas J. Regan, S.J.Thomas J. Regan, S.J. Interim Dean

Kathleen AndradeKathleen AndradeKathleen AndradeKathleen Andrade

Manager of

Operations

Arthur LurigioArthur LurigioArthur LurigioArthur Lurigio

Senior Associate

Dean, Faculty Affairs

Elizabeth EllisElizabeth EllisElizabeth EllisElizabeth Ellis

Business Manager

Adam PatricoskiAdam PatricoskiAdam PatricoskiAdam Patricoski

Academic Advisor

Megan DalyMegan DalyMegan DalyMegan Daly

Office Assistant

AsimAsimAsimAsim

GangopadhyayaGangopadhyayaGangopadhyayaGangopadhyaya

Associate Dean, Resources & Planning

Jacqueline LongJacqueline LongJacqueline LongJacqueline Long

Associate Dean, Academic Affairs

Eileen KearnsEileen KearnsEileen KearnsEileen Kearns

Executive Assistant

to the Dean

Joyce KnightJoyce KnightJoyce KnightJoyce Knight

Assistant Dean, Advising

Rachel WikeRachel WikeRachel WikeRachel Wike

Academic Advisor

TBATBATBATBA

Academic Advisor

Blake ChamblissBlake ChamblissBlake ChamblissBlake Chambliss

Academic Advisor

Marilyn MuiMarilyn MuiMarilyn MuiMarilyn Mui

Office Assistant

Maria LettiereMaria LettiereMaria LettiereMaria Lettiere

Assistant Director, Communications,

Academic Affairs, &

Summer Sessions

Lauren SanchezLauren SanchezLauren SanchezLauren Sanchez

Senior Academic

Advisor

Lester ManzanoLester ManzanoLester ManzanoLester Manzano

Assistant Dean, Student

Academic Affairs

Marianne WolfeMarianne WolfeMarianne WolfeMarianne Wolfe

Administrative

Assistant, WTC

ConcettaConcettaConcettaConcetta

DAgostinoDAgostinoDAgostinoDAgostino

Office Assistant

Laurie BucholzLaurie BucholzLaurie BucholzLaurie Bucholz

Office Assistant

COLLEGE of ARTS & SCIENCESOffice of the Dean

Leah PasquesiLeah PasquesiLeah PasquesiLeah Pasquesi

Graduate Assistant, Student

Academic Affairs

Kurt PetersonKurt PetersonKurt PetersonKurt Peterson Development Officer

8/13/2014

Appendix A: CAS Organizational Chart 35

Appendix B Advisor Caseload Assignments

College of Arts and Sciences Primary Academic Advisors

CAS Deans Office at LSC: 773.508.3510 (Sullivan Center 235) CAS Deans Office at WTC: 312.915.6520 (Lewis Towers 930)

Updated: November 17, 2014

First-Years/Sophomores A-Z First and Second Year Advising (FSYA) Sullivan

Center 260 (FSYA)

Transfers (20-44 hours) A-Z Leah Pasquesi, Graduate Assistant Sullivan

Center 235

LSC Juniors/Seniors (55+ hours) / LSC Transfers (45+ hours) A-C Rachel Wike, Academic Advisor Sullivan

Center 235 (CAS Deans Office)

D-H Blake Chambliss, Academic Advisor I-Mg Adam Patricoski, Academic Advisor Mh-Ron

Lauren Yurman, Academic Advisor

Roo-Wel

Lauren Sanchez, Senior Academic Advisor

Wem-Z

Joyce Knight, Assistant Dean

WTC* Juniors/Seniors (55+ hours) / WTC* Transfers (45+ hours) A-Z Lester Manzano, Assistant Dean

*WTC departments include: Criminal Justice & Criminology; Computer Science (fourmajors: Communication Networks & Security, Computer Science, Information Technology, Software Engineering); and Economics.

Sullivan Center 235 (CAS Deans Office) or Lewis Towers 930 (CAS-WTC Deans Office)

St. Joseph College Seminary Juniors/Seniors/Transfers and Non-Degree-Seeking Students A-Z Joyce Knight, Assistant Dean Sullivan

Center 235 (CAS Deans Office)

36

COLLEGE OF ARTS AND SCIENCES OFFICE OF THE DEAN SULLIVAN CENTER, SUITE 235 1032 W. SHERIDAN ROAD CHICAGO, IL 60660 773.508.3500 LUC.edu/cas

COLLEGE OF ARTS AND SCIENCES

Academic Advising Learning Outcomes and Responsibilities

Our Mission The mission of Academic Advising in the College of Arts and Sciences Deans Office is to assist students in the developmental process of pursuing and achieving their academic, social, and personal goals. As a holistic advising office at Loyola University Chicago, we strive to teach students self-awareness and responsibility while transitioning from college student to college graduate.

Ongoing interaction and communication among students, their college, their academic advisor and faculty advisor are essential for academic success and also aid in the development of the following learning outcomes.

College of Arts and Sciences Academic Advising Learning Outcomes for Students Personal Growth: Develop communication, decision-making, and problem-solving skills Reflect upon and articulate how the Jesuit education has shaped your life goals Reflect on your values, interests, strengths, and challenges Define and clarify your short-term and long-term goals Articulate your goals during advising sessions Describe problems you face by reflecting on what caused them, what can be done to resolve

them, what you learned from them, and how to avoid them in the future Feel empowered to accomplish goals and take initiative in completing necessary tasks

Resource Identification: Learn to locate and effectively use University and College information and resources that help you achieve your goals Seek to become acquainted with the academic structure of the University, the College of Arts and

Sciences, and academic programs and requirements Become knowledgeable about College policies and procedures and utilize university

resources and services Identify Web sites, campus offices, and faculty or staff you can consult with questions

Curriculum Integration: Understand the relationship between your academic experiences and your academic, career, and personal goals Research, identify, and pursue integration of co-curricular learning and long-term goals Articulate how your major field of study helps you achieve your goals Schedule courses so you can graduate in a timely manner based on your educational plan Connect your educational plan to your career goals Identify personal strengths and areas for growth

Engaged Learning: Understand the importance of including experiences outside of the classroom in your educational plan Utilize a faculty advisor in your major(s) to gain information and experiences beyond the

classroom (internships, job shadowing, research opportunities) Participate in undergraduate research, off-campus volunteering, learning abroad, and/or

internship(s) Reflect and discuss how participating in these activities helps you achieve your goals

Appendix C: Academic Advising Learning Outcomes & Responsibilities37



Academic Advisor and Advisee Responsibilities Academic Advisors and advisees both play important roles in the advising process.

Your Academic Advisor will... Protect and maintain the integrity of the Loyola University Chicago degree by enforcing all

University, College, and department policies and requirements Treat you as an individual with your own experiences, values, and backgrounds. The College of

Arts and Sciences is a safe-space environment, and all students are welcome to speak to theiracademic advisor freely and with honesty.

Understand and effectively communicate the curriculum, graduation requirements, andUniversity and College policies and procedures in order to provide helpful and appropriateadvisement

Assist you with defining and making educational plans consistent with your goals, abilities, andinterests

Be accessible and approachable Respond to all e-mail and phone inquiries within 48 hours of receipt of a message, if practical Make appropriate referrals to campus resources Offer recommendations and discuss strategies for academic, personal, and social

success Encourage you to develop the skills that will foster personal responsibility Maintain confidentiality (will not discuss issues with parents, family members, and non-University

persons without your written consent)

As a student, you are expected to Accept responsibility for your decisions and actions (or inactions) that affect your educational

progress and goals Display respect for your academic advisor and have a positive attitude toward the advising

process Attend advising meetings as scheduled (If you are running late, going to be more than 15

minutes late, or if you cannot attend your appointment, please call the CAS Deans Officeat 773.508.3500, or 312.915.6520 for WTC-based appointments.)

Be aware of important dates and deadlines (e.g., last day to withdraw a course) that areaccessible from the academic calendar www.luc.edu/academics/schedules on theLoyola Web site

Check your Loyola e-mail account regularly (do not have it forwarded to another e-mail account),read messages from university offices, and respond appropriately

Plan ahead and seek assistance from your academic advisor. You should make appointments inadvance as appointment availability is limited at busier times of the semester (e.g., first week ofclasses, registration).

Come to advising appointments prepared to assume responsibility for degree planning Ask questions if you do not understand an issue or have a specific concern Keep a personal record of your progress toward meeting academic goals and requirements Learn the remaining requirements needed to graduate and finalize graduation plans Learn the strategies necessary for transitioning from college student to college graduate

38

http://luc.edu/academics/schedules/index.shtml



CAS Deans Office Mission Statement

Through the University's Core Curriculum and its many majors and minors, the College of Arts and Sciences is dedicated to the Jesuit tradition of a Transformative Education in the disciplines encompassed by the humanities, natural sciences, and social sciences. We encourage our students to acquire, experience, and reflect on knowledge, and then decide what it means for them in terms of individual choice, action, commitment, and service to others.

Appendix D: Dean's Office Mission39

12/3/2014 PostAdvisingAppointmentSurvey

https://docs.google.com/forms/d/1i2zUs2_1rKqSG3SPM927PfLP48MKSpMrhvNW9j22Uo/printform 1/8

PostAdvisingAppointmentSurveyWeaskthatyoucompletethisshortsurveyforusafteryouradvisingappointment.ThepurposeofthissurveyistomeasuretheeffectivenessofyouroneononeadvisingexperienceswithinCollegeofArtsandSciences.Yourresponsesarecriticalinhelpingourdepartmenttoprovidemeaningfulandeffectiveappointments.Theresponsesyouprovidewillremainanonymous.Weaskthatyoureflectonyourmostrecentoneononeadvisingappointmentandconsiderthefollowing:

AppointmentsetupexperienceYourgoalsforyourspecificoneononeadvisingappointmentYourabilitytoconnectwithhelpfulcampusresourcesTheregistrationprocessYourpersonalvalues,interests,strengths,&challenges

Wethankyouinadvanceforyourcarefulreflectionandconsideration!

* Required

1. Pleasecheckallthatapply.Iama:Markonlyoneoval.

Freshman

Sophomore

Junior

Senior

Transfer

2. Iintendtograduate...*Markonlyoneoval.

SpringorSummer2015

Fall2015

Spring2016orlater

3. PleaseselectyourmajorCheckallthatapply.Checkallthatapply.

AfricanStudies&theAfricanDiaspora

Anthropology

Sociology&Anthropology

Bioinformatics

Biology

Biochemistry

Appendix F: Survey Instrument41



ClassicalCivilization

Greek

Latin

CommunicationNetworks&Security

ComputerScience

InformationTechnology

Mathematics&ComputerScience

Physics&ComputerScience

SoftwareEngineering

CriminalJustice&Criminology

Economics

English

English,CreativeWriting

Dance

ArtHistory

StudioArt:Ceramics

StudioArt:Drawing&Painting

StudioArt:Photography

VisualCommunication

Music

Music:JazzStudies

Music:SacredMusic

Theatre

ForensicScience

History

HumanServices

InternationalStudies

Mathematics

MathematicsEducation

Statistics

TheoreticalPhysics&AppliedMathematics

French

Italian

Spanish

Philosophy

Philosophy:SocialJustice

Biophysics

42



Physics

PoliticalScience

Psychology

Sociology

Theology

ReligiousStudies

Women'sStudies&GenderStudies

4. Doyouhaveapreprofessionalgoal?*Markonlyoneoval.

Yes

No

5. Whatisyourpreprofessionalgoal?Checkallthatapply.

Prehealth

Prelaw

Other:

SettingupaCASAppointmentPleaseanswerthequestionsbelowregardingyourmostrecentexperiencewithsettingupanacademicadvisingappointment.

6. Didyoucall,email,orstopintosetupyouradvisingappointment?*Markonlyoneoval.

Call

Email

Stopin

43



7. Whatwasyourreasonforschedulinganadvisingappointment?*Checkallthatapply.

Schedulingclasses

Registration

Transfercredit

Studyabroad/courseapprovals

Ontracktograduationcheck

Academicperformanceinoneormoreclasses

Major/minorchanges

Postcollegeplans

Graduationrequirementscheck

Other:

8. HowcomfortabledidyoufeelinreachingouttotheCollegeofArtsandSciencestoscheduleanappointmentwithyouradvisor?*Markonlyoneoval.

1 2 3 4 5

NotAtAllComfortable VeryComfortable

9. Pleaseratetheeaseofcreatinganappointmentthatfitintoyourschedule.*Markonlyoneoval.

1 2 3 4 5

Difficult Easy

ArticulateYourGoalsandNavigateYourResourcesPleaseanswerthequestionsbelowregardingyourmostrecentacademicadvisingappointmentexperience.

10. Myadvisorcreatedthespaceformetoclearlyarticulatemygoals.*Indicatewhetheryoucameintoyouradvisingappointmentwithaspecificgoalortaskandwhetherornot,basedoffofthat,youstructuredthemeetingaroundthatneed.Markonlyoneoval.

1 2 3 4 5

Myadvisorsettheagenda Isettheagenda

44



11. MyadvisorandIdiscussedmygoals/needsformyadvisingappointment.*Markonlyoneoval.

1 2 3 4 5

StronglyDisagree StronglyAgree

12. MyadvisorandIwerefocusedonmyissue/concernduringtheappointment.Markonlyoneoval.

1 2 3 4 5

StronglyDisagree StronglyAgree

13. Myadvisorprovidedusefulinformationaboutcampusresourcesrelatedtomygoals/needsduringourappointment.*(websites,campusoffices,facultyandstaffaroundcampus)Markonlyoneoval.

Yes

No

NotApplicabletomyappointmentneeds

14. Basedonthisadvisingsession,Icannowidentifyacampusresourcethatmightbehelpfultomeinthefuture.Markonlyoneoval.

Yes

No

NotApplicabletomyappointmentneeds

15. Basedonthisadvisingsession,Ifeelcomfortablereachingouttoconnectwiththiscampusresource.Markonlyoneoval.

1 2 3 4 5

ExtremelyUncomfortable ExtremelyComfortable

TheRegistrationProcessPleaseanswerthequestionsbelowregardingyourintendedgraduationdateandyourfamiliaritywiththeregistrationprocess.

45



16. IunderstandwhatIneedtocompleteinordertograduatebymyintendedgraduationdate.*Markonlyoneoval.

Yes

No

Istillfeelslightlyunsureaboutwhatisrequiredofme

17. Iunderstandtheclassregistrationprocess*Markonlyoneoval.

1 2 3 4 5

Donotunderstandtheprocess

Fullyunderstandtheprocess

18. Ifeelcomfortableregisteringforclassesonmyown.*Markonlyoneoval.

1 2 3 4 5

ExtremelyUncomfortable ExtremelyComfortable

Reflectingonyourvalues,interests,strengths,andchallenges.Pleaseanswerthequestionsbelowregardingyourmostrecentacademicadvisingexperience.

19. Myadvisingsessionallowedmetoreflectonmyvalues.Markonlyoneoval.

Yes,Iwasabletoreflectonmyvaluesduringmyadvisingappointment.

No,Iwasnotabletoreflectonmyvaluesduringmyadvisingappointment.

Notapplicabletomyadvisingappointment.

20. Ihadopportunitytoreflectonmyinterestsduringmyadvisingappointment.Markonlyoneoval.

Yes,Iwasabletoreflectonmyinterestsduringmyadvisingappointment.

No,Iwasnotabletoreflectonmyinterestsduringmyadvisingappointment.


46



21. Myadvisorcreatedspaceformetoreflectonmystrengthsduringmyadvisingappointment.Markonlyoneoval.

Yes,Iwasabletoreflectonmystrengthsduringmyadvisingappointment.

No,Iwasnotabletoreflectonmystrengthsduringmyadvisingappointment.


22. Iwasabletoreflectonmypotentialchallengesduringmyadvisingappointment.Markonlyoneoval.

Yes,Iwasabletoreflectonmypotentialchallengesinmyadvisingappointment.

No,Iwasnotabletoreflectonmypotentialchallengesduringmyadvisingappointment.

NotApplicabletomyadvisingappointment.

Aftermyadvisingappointment,Ifeel...

23. Encouragedtoreflectonhowmyvaluesrelatetomygoalsinthefuture.Markonlyoneoval.

Yes,Ifeelencouraged.

No,Idonotfeelencouraged.

Iamunsureofmyfeelings.

24. Encouragedtoreflectonhowmyinterestsrelatetomygoalsinthefuture.Markonlyoneoval.



Iamunsureaboutmyfeelings.

25. Encouragedtoreflectonhowmystrengthsrelatetomygoalsinthefuture.Markonlyoneoval.




26. Encouragedtoreflectonpotentialchallengesthatrelatetomygoalsinthefuture.Markonlyoneoval.




47



Poweredby

Thankyou!TheCollegeofArtsandSciencesAcademicAdvisingOfficewouldliketothankyoufortakingthetimetocompletethissurvey.Yourfeedbackisgreatlyappreciated!

48

https://www.google.com/forms/about/?utm_source=product&utm_medium=forms_logo&utm_campaign=forms

Appendix G Survey Design Considerations

1. Were the directions clear? 2. Were any items difficult to read due to length of sentences, word choice, or terminology? 3. What did each question mean to you? 4. Do you feel the questions and organization of the survey flowed? 5. How long did it take you to complete the survey? 6. Did you have any trouble accessing the survey or navigating it once it was accessed? 7. Do you have any concern with the length of the survey? 8. Do you have any other concerns or general feedback? 9. Do you have any suggestions for making the overall survey experience or survey

questions easier to understand and to complete?

Adapted from Schuh, 2009

49

Appendix H Focus Group Email Invitation

Hello ______________, As a valued member of the College of Arts and Sciences student community, we invite you to participate in a focus group that will help us to evaluate and improve our advising program. The focus group will be held on DATE, at TIME in LOCATION. We would like your feedback and reflections on the following:

Overall sentiment of one-on-one advising appointments Content of one-on-one advising appointments

The focus group will be led by a facilitator who is not a member of the CAS advising staff. All content from the focus group meeting will be void of names or other identifying information. The information you provide will be used to understand the state of academic advising and make improvements to the office and advising practices as needed. If you would like to participate, please RSVP to COORDINATOR by DATE. If you are interested in participating, but cannot do so at the stated time and date above, please email COORDINATOR so that we can reach out to you in the future. If you have any questions about the focus group, please contact:

Joyce Knight, M.Div. Assistant Dean for Advising [email protected]

50

mailto:[email protected]

Appendix I Focus Group Consent Form

Project Title: CAS: Deans Office Academic Advising Evaluation Evaluator: Katherine Murphy Introduction: You are about to take part in a focus group to help the College of Arts and Sciences Deans Office at Loyola University Chicago gain a better understanding of your experience with the one-on-one advising appointments that are offered. Please read through the following information carefully. If you have any questions regarding the information below, please ask the moderator prior to deciding whether to participate. Procedures: Once you agree to participate in the focus group, you will be asked a series of questions pertaining to your perspective on the one-on-one advising experience. There is no right or wrong answer and we ask that you respond honestly and openly. Please know that you are not required to respond to any of the questions and should only respond based on your comfort level. Confidentiality: By participating in the focus group, your name will be omitted from any responses you provide. Any information provided will be compiled in a report and basic themes will be shared with the CAS Deans Office.. We will use the information yielded from these focus groups to understand the state of academic advising and make improvements to the office and advising practices as needed. Voluntary Participation: Participation in this focus group is voluntary. At any point in the focus group, you are free to withdraw from participation. Please know that you are not required to stay for the entire time. Statement of Consent By signing below, you acknowledge that you have read the information above and agree to participate in the focus group. Additionally, you have had a chance to ask any questions you may have about the process. A copy of this document will be provided for your records.

_______________________________________ _________________ Participant's Signature Date

_______________________________________ _________________ Evaluators Signature Date

51

Appendix J Focus Group Protocol

Introduction Hello, my name is __________ and I will be facilitating this focus group today. I would like to start off by thanking you for taking the time to participate in this focus group. For the next hour or so we will be talking about your experiences with advising in the College of Arts and Sciences: Deans Office. Purpose of Focus Group You all are here because you are a student in the College of Arts and Sciences who has utilized one-on-one advising services. We have invited you to share details about your experience in your advising appointments. I will have questions to help guide our conversation, though we would like to keep this an

Research Project

Documents

evaluation approach

appendix n budget

purpose of evaluation

sciences cas deans office

evaluation project

appendix f survey instrument

appendix e logic model

oneonone advising services